Angie Chang: Welcome back to ELEVATE Virtual Conference. My name’s Angie Chang, founder of Girl Geek X, and with us today we have Dr. Sylvia Martin, the Chief Nursing Officer at Kaiser Permanente. She spent over a decade as a hospital administrator at Stanford Children’s Health and earned her doctorate in nursing practice from Yale University. She has earned both her MS and BS from the University of Alabama Huntsville. We welcome today Dr. Sylvia Martin.
Dr. Sylvia Martin: Welcome to this girl geek presentation on mastering effective interviewing skills and situational interviews in the professional setting. Allow me to introduce myself. I am Dr. Sylvia Martin. I am a Chief Nursing Officer with over 25 years of healthcare experience and many years of hiring team members at all levels. I received my doctorate degree in nursing from Yale University, completing my dissertation on moral courage. In my free time, I enjoy spending it with my family or volunteering in the community.
Dr. Sylvia Martin: In my current role, I conduct situational interviews because I found them to be effective in helping me identify the desired characteristics and strengths of those I’m interviewing.
Dr. Sylvia Martin: I’d like to start out by establishing a shared understanding of the purpose of interviews, and that is making an assessment of your skills, qualification, critical thinking, and even your potential. From an interviewer’s perspective, we’re seeking the best candidate for the organization, for the department, for the team, and the role. From an interviewee’s perspective, this is your opportunity to shine, to showcase your skills, your ability to make a constructive impact, and you want to market what you bring to the table and even brag about what you’ve accomplished. This is your time to really put it out there for people to see how great you are and why they need to hire you.
Dr. Sylvia Martin: One of the most important things you want to be able to do in any interview is build a rapport and connection with those who are interviewing you. Most commonly, you’ll either be interviewing with one individual or you’ll have a panel that you’ll be interviewing with, meaning that you’ll have anywhere from three to 10 people that may be there to hear what you have to say in the interview.
Dr. Sylvia Martin: I usually conduct panel interviews. This allows different perspectives to have input on the hiring decision, which I really appreciate. Now, some of the ways that you can incorporate into your interactions in the interview to help build this rapport and connection is starting with a warm greeting. Use active listening skills. Your body language matters if you need to take notes so you can use the interviewer’s name when you speak to them.
Dr. Sylvia Martin: You want to share stories and communications that will highlight your characteristics and your strengths and what you have to bring to the team that will help everyone succeed. Give them that energy enthusiasm that will help inspire them around the opportunity to work with you. Find common ground. Cautiously use humor when that can be appropriate. Ask about their role.
Dr. Sylvia Martin: Showing curiosity is a great way to engage and connect with people and expressing gratitude. It’s always a connecting formality to say thank you and be gracious about the time people are spending to get to know you and to find out why you would be the best person for the role.
Dr. Sylvia Martin: It’s also important to communicate confidence and establish credibility in the short time you have in your interview. I call this commanding the room. To command the room in your panel interview, it will demonstrate confidence and that you’re present and you can communicate effectively. You want to come in with a brief, impactful introduction that highlights your background and skills and your energy and enthusiasm for the opportunity in front of you and the role. This can be done with a confident, clear tone, concise statements, confident body language and posture, making appropriate eye contact, and avoiding fidgeting and body movements that may distract from what you’re trying to communicate.
Dr. Sylvia Martin: Remember, commanding the room is not about dominating the conversation, but rather demonstrating your competence, professionalism, and your ability to engage effectively with a diverse panel. Practice and preparation will go a long way in boosting your confidence and helping you make a lasting impression. When done with grace, it will get everyone’s attention and make them curious to know what is different about you, and that’s a good thing because you’re about to tell them.
Dr. Sylvia Martin: There are different types of interviews that can be conducted when you go in for an interview for a new job. There are behavioral interviews, situational interviews, competency-based, and traditional interviews. For the focus of this presentation, we are going to talk about situational interviews, where an assessment is made for problem solving, critical thinking, emotional intelligence, self-awareness, and decision-making. What else can we learn in situational interviews?
Dr. Sylvia Martin: The author Janneke Oostrom wrote an article on situational interviewing and in her research found that there was considerable similarity between what an interviewee says they would do and their actual behaviors and corresponding work situations. What does that mean? It connects the intentions to actual behaviors, and situational interviewing will highlight the ability of the interviewee to correctly decipher situational demands.
Dr. Sylvia Martin: When I ask a question, I can see are they able to line up what I’m asking with what that situation would or does demand in their answer, either from experience or from hypothesis, right? Showing your confidence in the ability to influence in a positive way to find the desired outcomes or learn from a situation, and then an assessment of authentic characteristics versus an interview persona. Say, where a person might come to an interview and only share ideal responses. Situational interviewing will give you a more accurate reflection of the person’s real life responses to give you an authentic look at who that person is and what they have to bring to your team.
Dr. Sylvia Martin: Now, when you’re being asked questions in a situational interview, the best evidence-based methods to respond is by using the star method. STAR is an acronym for situation, task, action, and result. It is much easier to share a focused answer providing the interviewer with a digestible but compelling narrative. Sometimes people may tend to provide too much detail and their answers get too long, thus their main points get sort of lost in the dialogue. You want to focus on one or two sentences for each letter of the acronym.
Dr. Sylvia Martin: Choose a few strong, versatile example stories that you’ll practice and have ready, making sure that they are still authentic in your experience and responses in the past or your intentions in how you know you would behave to respond to these situations. Your situation is going to describe the context and the challenges that you faced. Your task explains what you needed to achieve and why it was strategically important. The action is outline the steps that you took to develop and execute the plan and your results, highlighting the positive outcomes such as enhanced customer satisfaction. Or clearly explaining what you learned from the situation and what you would do differently next time if it didn’t go as well as you’d hoped.
Dr. Sylvia Martin: It’s important to be prepared for possible questions in a situational interview. On this slide I’m sharing some of the more common questions you may encounter. You would want to have your STAR method response prepared or at least formulated in your mind for these or similar questions. I’m going to demonstrate a couple of questions and responses to show you what this would look like.
Dr. Sylvia Martin: The question is, “Tell me about a time when you were overwhelmed at work and your manager asked you to take on an additional task or initiative.” My situation, I had several complex projects requiring almost all of my time. They were engaging and allowed me to collaborate with other teams. The task one of my team members received a well-deserved promotion, meaning I needed to hire a replacement team member.
Dr. Sylvia Martin: The action, I acknowledged my limited time and capacity and talked with my manager and the colleague to gain support with the hiring process. The result, my colleague agreed to complete the screening and first round interviews, allowing me the time I needed to wind down my projects, then take over the final round of the interviews, completing the process in a timely manner without delaying the completion of my projects.
Dr. Sylvia Martin: A second scenario would be “Tell me about a time when you had to handle a difficult team member. How did you address the situation?” In my previous role as a project manager, I was leading a cross-functional team responsible for developing a new product. One of my team members consistently missed deadlines and had a negative attitude, and both were affecting team morale. The success of the project depended on timely deliveries from every team member. It was crucial that I addressed the performance issues and improve the collaboration within the team.
Dr. Sylvia Martin: First, I scheduled a private meeting with the employee to discuss the challenges and concerns. I wanted to understand the root cause of the behavior and address any underlying issues. During our conversation, I discovered that the employee felt completely overwhelmed with his workload and felt he didn’t have enough support.
Dr. Sylvia Martin: Next, I decided to work very closely with the employee to develop a clear action plan. I allocated specific tasks that aligned with their strengths and expertise, and I provided additional resources to help manage the workload. I also started regular check-ins to monitor progress and offer guidance. To improve the team morale, I organized team building activities and open communication and collaboration. During team meetings, I highlighted individual and team achievements, celebrating milestones and success, which positively reinforced a culture of appreciation and recognition.
Dr. Sylvia Martin: As a result, the employee’s performance significantly improved. They began meeting deadlines consistently with a transformation in their attitude, becoming more positive and engaged. Team morale boosted with this increased communication and a stronger sense of unity. The project met its deadlines on time with positive feedback from both internal stakeholders and customers.
Dr. Sylvia Martin: This experience taught me the importance of proactive communication, empathy, and tailored support from my team members. It reinforced my belief that addressing performance issues head on, having difficult conversations and fostering a positive team environment can lead to a remarkable improvement and successful outcomes.
Dr. Sylvia Martin: These are just a couple of examples of how you can move through the STAR method to answer situational interview questions. Before we close out this presentation, I’ll share just a few other tips for interview success. You want to make sure that you’ve researched the company, that you know how the company is doing in the market in a SWOT sort of format and where this company stands so that you can speak in an informed way around a vision for your role and how you can contribute, and that’ll help you tailor your message for what you want to communicate.
Dr. Sylvia Martin: This will give a why to your vision as well for the work that you can do at this company. You want to emphasize mutual benefits, and you want to connect your past work experience and education to what you will bring that will make you successful in this role. Give a good foundation to why they should want you there on their team, why they need you there on their team.
Dr. Sylvia Martin: Demonstrate a future oriented mindset and a proactive attitude. You want to break down your thought processes, show that you’re aligned with the company goals, mission and vision, and provide tangible examples from your experience so that they’re able to see the authenticity in all of that. You want to be a good listener and ask for feedback and be passionate and energetic and gracious and express gratitude.
Dr. Sylvia Martin: These are all things that will help you build that rapport and connection and inspire the team that’s interviewing you to help you be successful in snagging that opportunity that’s in front of you for this job. Thank you so much for your time and attention today in attending this presentation, and I want to wish you all the best in your future interviews. Have a good day.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Angie Chang: I’m the founder of Girl Geek X. With us today we have Rachel Rogers, who is the vice president of product marketing at Bentley Systems. She’s been at Bentley Systems for over a decade. Prior to Bentley was a director at Autodesk and Intergraph, where she began her career as a marketing writer. She has her BSBA from the University of Alabama and Huntsville. We have with us today as well, Natalie Plummer, the director of diversity, equity, and inclusion at Bentley Systems, who will be moderating the Q&A. Please pop your questions into the chat, Rachel and Natalie would love to answer them. Welcome, Rachel.
Rachel Rogers: Thank you so much, Angie. Good morning from sunny San Francisco Bay Area. It’s actually going to be a beautiful day today, and I’m excited to be here, and to share some of the best advice I’ve received. As Angie just said, I’ve been in this career for the last 30 years, over three decades, and I would like to share some of the best advice, and some of the things that I’ve learned myself through trial and error over those last three decades.
Rachel Rogers: As Angie just mentioned, to share a little bit about myself, I started my career at Intergraph in Huntsville, Alabama in the nineties as a marketing writer. Then I worked in the AEC division, which supported architecture, engineering, construction. Needless to say, at that time it was definitely a male-dominated team. In the mid two thousands, I moved across country from Huntsville to the Bay Area and worked at another CAD company, Autodesk, but this time on the sales side, to really help balance out my career and learn a different aspect of the business.
Rachel Rogers: This time I was one of a handful of women on the team that supported the infrastructure division and software industry. 11 years ago I joined Bentley, the third leading infrastructure software company in the world. We’re not the third, we’re one of the major ones, but I’ve always been in infrastructure. And I’ve worked from home for the last decade leading a global team, and I’m happy to report that now I work with so many talented, exceptional women, but we’re still in a male-dominated tech field. All of these adventures over the last 30 years have really led me to where I’m today.
Rachel Rogers: I hope that today I can share some of the insight that I’ve learned that’s going to be helpful for you. I’ve narrowed it down because, trust me, I’ve learned a lot in those three decades. I’ve narrowed it down, all that plethora of advice that I’ve received to what I think is the top five. Let’s get started with the countdown.
Rachel Rogers: Number five, or actually this could be number one too, but number five, life is not fair. I grew up in the deep south with a brother that was two years older than me. At that time, boys had the freedom to do almost anything they wanted, whereas girls did not. They could play sports, ride their bikes around the neighborhoods, stay out all day, disappear for hours without anybody know, avoid housework. You name it, they could get involved with it.
Rachel Rogers: I would often complain to my mother, “That is not fair.” And she would always say, “Who said life was fair?” It really pissed me off every single time, but she was right. As we always say, mothers are always right in the long run.
Rachel Rogers: Life is not fair, but even as adults, we can have a hard time not looking at others and not thinking that life isn’t fair. We have a tendency to focus on what’s happening to other people around us instead of focusing on how we can grow ourselves and our own careers.
Rachel Rogers: Even today I often have, and throughout my management, I’ve always had conversations with team members discussing why other colleagues were promoted instead of them, or how was somebody else chosen to do that project instead of me? Why does that person have a higher title than I do when I have more years on the job? Or the title is different.
Rachel Rogers: It’s really easy to get caught up into that. Why not me? What is it about me? We cannot control what happens to other people unless we’re in charge of making those decisions. Then often, even when we are in charge of making them, we’re still doing things on the recommendations of the people above us.
Rachel Rogers: If you spend your time worrying about your colleague’s projects, their opportunities, their career instead of yourself and your own professional growth in career, chances are you’re always going to be frustrated. Now, I’m not saying it’s easy. I’m one of the first ones that have learned this lesson.
Rachel Rogers: A lot of things are out of our control in the corporate world. Things like teams being reorganized around you, other people being promoted, not knowing about new opportunities. It’s happened to me many times, and I know I’ve reacted badly several times myself, but the best advice that I’ve received over the years has been that opportunities may not happen for you at the same pace as others.
Rachel Rogers: We each have our own unique path, and that’s the only thing that we can influence. And once you realize that you can only influence what you can control, you’re in a better place. Like how you react to things going on around you, how you advocate for yourself, and how you focus on your own career to grow your own successes. Life may not be fair, but it can be a lot easier to deal with when we focus on ourselves and not others.
Rachel Rogers: Number four, because we’re going to do kind of a David Letterman count. So number four, find your voice. I just mentioned that you need to advocate for yourself to support your own career and find your own successes. Finding your voice can be really challenging as a woman in a male-dominated industry. Or even as a person that may not like to speak in front of others, or that may not be something they’re comfortable with.
Rachel Rogers: For many years, I was the only woman on all-male teams. Remember all those engineers I just talked about? I was also one of the youngest, most times. So I would get really frustrated over the fact that guys just simply had no problems just talking over me in meetings, not listening to my ideas, just shrugging it off. It truly was like that FedEx commercial from years ago where it’s an office setting.
Rachel Rogers: A woman says, “We can use FedEx for shipping to save money in our budget.” And no one listens to her. And then a man says the exact same thing and everybody says, “Oh my gosh, that’s an incredible idea.” They heard him, they didn’t hear her.
Rachel Rogers: It’s not as funny in real life as it is in that FedEx commercial. Not that it really was funny. But men are comfortable leading conversations and they really usually don’t have a problem interrupting. Whereas women are much less prone to doing so. We’re less prone to leading it, we’re less prone to interrupting conversation. We sit back and wait to be included, oftentimes.
Rachel Rogers: You can’t wait for someone to ask you for your opinion or share your ideas. You have to speak up. You have to find your voice. You have to learn to share your ideas and your opinions, and really establish yourself as a valuable member of the team.
Rachel Rogers: Once you build your own credibility and showcase your skills and knowledge, then people around you are going to start listening more. They’re going to start asking questions of you. And they’re going to want your opinion, and they’re going to see your value, so not only are you building your own confidence in what you have to offer, but you’re also building a really strong brand for yourself.
Rachel Rogers: Make sure that you share your successes, your wins, and not only to your team, but to your manager. It’s the only way to make sure that your contributions are going to be known across the organization. You’ve got to speak for yourself so that you know you’re being seen and heard.
Rachel Rogers: You’ve also got to let people know, you need to make sure that your managers and others know what your wishes are for your career. You’re responsible for your own career growth. Let’s make sure that people know what your wins are, and make sure people know, especially your management, where you want to go with your career. Advocate for yourself, find your own voice.
Rachel Rogers: Number three, building relationships. At the beginning of my career, I used to think that I would simply grow my career based on all my hard work, my successful projects, my proven leadership, my other contributions. I was really naive, because, hopefully this isn’t too old of a phrase, but if it takes a village to raise a child, it really takes a network in support of others to help your career.
Rachel Rogers: You need a team of people inside your own organization helping support you, and helping find opportunities for you to shine, to grow, to build your own career. It’s difficult to do that just by yourself as a solo-person. You need to have that network supporting you, but building a network doesn’t just happen. I mean, you really have to invest the time and effort into connecting with people and nurturing your network. It’s difficult. It takes time.
Rachel Rogers: Your network, it can include a lot of people, like your current colleagues, past colleagues, managers, even managers from other companies you continue to stay connected to, industry groups that we can connect to, and a variety of social platforms now, alumni, other online connections. All these things are really important. These groups of people can really help pull together to help progress your career, let you know about new career opportunities within your company.
Rachel Rogers: They can help be influential in your own promotions or opportunities, new opportunities for you. And really help provide insight and support when you need it. Remember to invest the time in building them. It’s really easy to get caught up in, and I’m guilty of this as well. We get caught up in our daily job, all the urgent tasks, but we need to find the time to build those relationships so we have a support system in place when we need it.
Rachel Rogers: Remember, build your network, but just as another critical, find a mentor or a coach to help guide you. Look around your own organization, reach out to somebody you admire and ask if they have time to mentor you. I found that most people really are willing to help and invest time into giving back. I know that I really enjoy mentoring others and giving back, and for the most part others do as well. Find a mentor, it’s really going to be helpful in your career.
Rachel Rogers: Number two, and one of my personal mantras, is trust your instincts. Listen to your gut or your intuition. It’s telling you the right thing to do. Another lesson I’ve learned over the years is when I ignore that voice inside of me, things do not go well. I have so many examples of ignoring my intuition, and trusting other people’s opinion, and it not working out well for me or for my team.
Rachel Rogers: Perfect example is, years ago, we needed to hire a new product manager for the team. And even though the position was several layers below mine, I always like to interview candidates at the end to make sure that not only do they have the skills, but will they be a good fit for the overall team?
Rachel Rogers: Several colleagues and the manager interviewed the candidate. Everybody thought they had the right skills, they had the knowledge, they had the experience in the industry, they thought they would be really successful in the position. That person bubbled up to the top. I talked to them, but after I interviewed them, I really didn’t think that they were going to be a good fit personality-wise, and I thought that it could have a negative impact over the overall health of the team, but I wanted to support the manager and the team that interviewed them, so I ignored my gut and we made an offer and hired the person.
Rachel Rogers: Well, six months later, it was clear that the new colleague was very disruptive to the team. Production was down, frustration was really high, and then I had to handle the situation. It could have all been avoided if I just trusted my instinct. That little voice inside of you knows what’s best for you, learn to listen to it and to trust yourself. You know what’s best for you.
Rachel Rogers: Number one, and the most important of all, know that you can do anything. After you’ve learned to trust yourself, your intuition, you need to realize that you can do anything. Early in my career when I doubted myself, I would tell myself over and over, you can do anything. It was my mantra. I wrote it down in meetings when I was unsure of myself. I would write it down in the notes I was taking, or doodling.
Rachel Rogers: When I would feel overwhelmed by new projects, or opportunities, or having to build new teams or new companies or all the things that you’re asked to do. When you start feeling intimidated and overwhelmed by new challenges, just tell yourself, I can do anything. Have faith and belief in yourself that that’s the first step to your success, is knowing that you can do it.
Rachel Rogers: It’s really easy to get caught up in that noise, that others are smarter than you, they’re more educated, they’re more whatever, but knowing that you can do anything helps put you at ease. It gives you that confidence. It’s your greatest gift to know that you’re in charge of your life, and you can accomplish anything. Keep telling yourself that over and over until you believe it, because you can. You can do anything.
Rachel Rogers: Don’t forget. Focus on what you can control. Advocate for yourself. Build that network for support. Trust yourself and the intuition, and keep telling yourself you can do anything.
Rachel Rogers: With that, I’d like to welcome Natalie Plummer. She’s our director of diversity, equity, and inclusion to join me. And we’ve got a couple of minutes that we can ask any questions that we may have.
Natalie Plummer: I think I’m just going to go through some of the things, the questions and some of the points you raised. People can continue to put questions in the chat. I’m kind of monitoring the chat to see any questions.
Natalie Plummer: Here’s a question I want to ask you. One of the things that you said is people need to build a network. Here’s my question and it has a bunch of sub-parts because I could pick your brain all day. How do you build that network? Is it a mentor? Is it through an advocate?
Natalie Plummer: There’s a difference, and does it matter whether that advocate or mentor is a man or a woman? And does it matter if they’re even in your field?
Rachel Rogers: Oh, that is a great question. A, I don’t think it does. Many good things. I do not think it does matter that it’s in your field. Honestly, think it’s better to not be in your field because you get a different perspective. Does it matter if it’s a male or a female? Absolutely not. What matters to me, it’s somebody you admire, that you will listen to, that they, you know, will have insight. So they’ve been through things. They can coach you, you can help learn from their wisdom.
Rachel Rogers: I do not think that you have to have a woman to help you. I mean, it’s great because they can understand a lot of things, but I’ve had a lot of great male mentors. It helps me. Of course I to, I didn’t have that many women around the first 15, 20 years, so, I did. I had a lot of great guys that helped advocate for my career and helped do things. I don’t think that that matters. I think it’s whoever you’re comfortable with.
Rachel Rogers: I do want to go back, and I think you said, what’s the difference? Your network, totally different from your coach or your mentor. And by the way, an executive coach, a great investment, one of the best things I’ve ever done. I think a mentor is great. That network is, the people that you’ve worked with, that you build, they’re all over so they can help you. Whether they’re in your current company or in other companies. Because those are the ones you’re going to look to when, maybe it’s time for you to start a new career.
Rachel Rogers: Maybe you’re looking for something else. Maybe you just want a new opportunity within your same company, but you don’t know where to start. You want to broaden, like I did. I went from always being in marketing to being to the sales side. Maybe you want to broaden your experience. Having that network will let you know new opportunities and help advise you and what’s the best direction to go,
Natalie Plummer: Then logistically, how do you go about doing it? Is it something you carve out time for at the end of your day every day? Is it a weekly event? Is it monthly? Is it going to networking events once a week? Is it going on LinkedIn and seeing people you admire, and reaching out from them? Logistically, how do you recommend it when you have a busy day and you have your life afterwards?
Rachel Rogers: I’ll have to say, I’m not the best at it. Because as I said, this was something that I learned. I was very naïve. I thought for years and years that everything’s just going to happen based on my contributions.
Rachel Rogers: I do think, yes, you have to make time for it. You do need to schedule as a part of your day. I mean the LinkedIn, the social, doing those kinds of things. Obviously you can do that after hours and be able to build your own network, but I think building your network within the company, absolutely.
Rachel Rogers: You need to schedule meetings with people. You need to ask if they have the time to chitchat with you, if they need to ask for their advice. People appreciate that too. I think making those connections, and building them and nurturing them, just like friendships, you have to nurture relationships with people.
Rachel Rogers: Having a relationship at work is not unlike having a relationship with friends, that you need to put the time in and nurture that relationship. I’m guilty of not doing that as well. We all get busy in all of our demands, and our tasks, and our deadlines. But we really need to do it. It’s going to make you feel better as well. You need that personal time. You need time to connect with people, and not just heads down doing your job.
Natalie Plummer: It’s almost part of your job, as far as just expanding your network. That’s part of it as well.
Rachel Rogers: It’s definitely part of your own personal growth. As we talked about before, one of the things that you really need to learn, and it’s hard to learn is that you’re responsible for your own career. We think that other people are, oh, I’m going to promote them, or I’m going to do that. I’m just going to sit.
Rachel Rogers: No, you have to advocate for that, and then building that network helps. Because then people know what you have accomplished, and what you’re able to do. You may have opportunities in other areas, that’s certainly happened for me. Someone else advocating for me in a totally different area because we had a relationship, or we’d work together, or they knew … I would’ve never done that if I hadn’t been volunteered for different teams, or done this or that, or reached out across organizations to build networks.
Rachel Rogers: That’s one way that you can help do that, is to definitely reach out in your own organization and talk to others.
Natalie Plummer: All right. Going on to your next slide, you mentioned trust your instincts. Now, if you’re a younger person coming into this field, or even if you’re a person who just doubts themselves a little bit, how do you know what’s your instincts versus what’s your fear? That kind of negative voice in your head?
Natalie Plummer: How do you distinguish between the two when you’re entering a new field and maybe you don’t trust yourself as much as you should?
Rachel Rogers: I think that’s a good question. We’re now at 10:20, so I don’t know if Angie, if we’re out of time or if we can just … You definitely have to, that’s a great question, about learning to listen to yourself. Not listening to the negative thoughts, but listening to the positive thoughts and trusting your gut.
Rachel Rogers: It does take experience and time. I think that we’re out of time and I totally appreciate it. I can answer other things. And Angie, I just want to say thank you for giving us, Natalie and I have the opportunity to talk today.
Angie Chang: Thank you both for joining us today. I know we will see some of your faces at the virtual booth later, so everyone who has questions for them, please hang on to them and connect with them on LinkedIn and add in the virtual booth. See you in the next session. Thank you.
Natalie Plummer: Thank you so much.
Rachel Rogers: Thank you.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Angie Chang: With us today, we have Karen, who is a director of engineering at JLL Technologies, where she leads a team of talented engineers, engineering leaders across eight products with an ability to drive innovation and strategic business goals. Prior to JLLT, she was a software engineer at Intuit and Yahoo. And as a seasoned engineering leader, we’re excited to hear about how her experience in web development, machine learning, and data informed her guidance on influencing others and decoding unspoken rules. Excited for this talk. Welcome, Karen.
Karen Lo: Thank you so much, Angie, for the very warm welcome. Hi, everyone. Thank you for tuning in today. I know you’ve got a lot of things that you could be doing, so I appreciate your time today. I will be speaking about Rewriting the Leadership Manual: A Playbook on Influencing for Non-Influencers.
Karen Lo: For this talk, I was inspired by the other woman on my team and wanted to really write down the different things that I learned over the years and how I’ve applied them through every position I’ve had outside of just my management position that have helped me over the years in providing me with an influencing style, whereas I’m typically somebody who’s not really that open spoken or talking about my own promotions, things like that. This is my approach for how I’ve been influencing, and let me go to the next slide.
Karen Lo: Here’s an overview. It will consist of three parts. Part one, learn the rules to break the rules, part two, de-weaponizing incompetence, and part three, legitimizing invisible work. I’ll dive deeply into each of these parts, and please feel free to type in any questions you have so I can answer them later.
Karen Lo: Part one, learn the rules to break the rules. No one likes to be solutioned at. I know that I don’t like to, especially when sometimes I’m just trying to rant about something, whether it’s about a process that’s really annoying or something that seems to be broken from my point of view. And sometimes I just want somebody to listen to me without providing me with solutions.
Karen Lo: I think that this applies to several people, especially when you join a new team, right? You don’t want to be the person who thinks that they know everything already before they’ve actually understood the nuances of each problem. When you listen and observe, which is step one, you are listening and validating that a problem or challenge exists. This in itself is empowerment. If you validate to somebody that, “Hey, I hear you and I totally understand your pain,” sometimes you may not have to agree with it, but just by saying you hear them, that provides a lot of trust that the person will have in you that, hey, you’re actually trying to listen to the problems that they have.
Karen Lo: A couple of examples of how you might see this conversation play out in real life is sometimes you might say something like, “Hey, I empathize with you feeling frustrated with the engineering team’s technical setbacks, and that is causing you to lose trust in our ability to deliver.” Sometimes you’re talking to a product manager or sometimes even a client, and we have engineering setbacks all the time, but it doesn’t mean that people are happy with it. And so you saying that instead of being very defensive about, oh, we have to do this or we have to do that, just acknowledging that goes really far.
Karen Lo: Another example is, “Oh, I thought the ticketing system was really cumbersome and unnecessary, but I now see it was put in place because people kept pinging your team for help and updates without sufficient details.” We all know that context switching is incredibly expensive. And so if a team has put a ticketing process in place and it is annoying to go through, sometimes you just want to see, hey, how did we get there in the first place?
Karen Lo: This is step one, first, listening and observing. Once you’ve listened and you feel comfortable enough to say something, step two is offering intentional support. This is very different than offering support. How many of you have ever found yourself in a position where you yourself have said or someone has said to you, “Hey, let me know if you need anything.” That kind of puts the receiving end in a position where they’re like, “Well, I have to put in extra work to reach out to this person to tell them what I need.”
Karen Lo: A couple of examples for what you can use instead that’ll be more effective are, “Hey, would you mind if I reached out to my leadership team about this? I think they’ll be able to help, and I’ll CC you in the email.” Or, “How about I take this off your plate?” Or, “I know a person who can help. Let me start a group chat.” Or, “I’ve faced a similar issue. Let me send you the documentation on how to fix it.” Each of these things are very intentional in you actually recommending a path forward and providing them with just a yes/no instead of, “Hey, tell me whatever you want under the sun,” because that is probably not going to happen and they’re probably not going to ask you for help.
Karen Lo: Part two, once you’ve listened and you’ve gained some credibility, the second part is to de-weaponize incompetence. I know this has many charged words, de-weaponizing and incompetence, but how many of you have ever been in a position where you’ve done something because you were either faster at it, you could do it faster, someone else asked you to do it because you were better at it, or you were maybe somebody who did it more thoroughly? Sometimes people are like, “Hey, can you do this? You tend to be a lot better at it than I am.”
Karen Lo: These are signs that I’m just sharing on how to be mindful of when you see this happening. Because while not all requests are inherently bad, sometimes you’re asked to perform a task that you may not really feel like you should be the one doing, but then you find yourself gaslighting yourself by saying, “Oh, well, that engineer or that manager is too busy. I guess I’m the only one who could do it.” Or for example, am I being asked…
Karen Lo: These are the different ways, I guess, to pinpoint if this is happening to you. First one is, am I being asked to do something purely because I’m better or faster at it? Am I being asked to undermine a process for someone else’s convenience? Is someone asking for help but expecting me to do the bulk of the work? Or is someone using urgency or impending deadlines to convince me to do something that could otherwise be done by someone else?
Karen Lo: Step one, identifying when you’re being put in a position where someone may be using incompetence to get you to do something, whether or not it’s intentional or not, right? Some people, that’s just the way that they operate and it’s not really meant to be hurting you or anything, but you need to identify if you think you’re being hurt by it and being put in a position where you’re doing work that you think could be done by other people or that person themselves. So remember, we’re all data-driven people here, so no need to use data to gaslight yourself too.
Karen Lo: Once you’ve identified if you’re in that position, step two is mitigating. Once you have that in mind, you’re like, “Okay, what can I do now?” You control the precedent that you set. You give yourself a choice. In a lot of these positions, when I’ve been part of situations like this, I typically think to myself, “Oh, I don’t have a choice,” or, “Oh, I have to do it.” Which sometimes you accept that you have to do it, but sometimes you don’t.
Karen Lo: A couple of ways that you might be able to mitigate the situation is you can say something like, “Hey, I appreciate that you think I’m someone who’s much better at this, but let’s spend some time for me to watch and provide some feedback as you go through the steps so you also become proficient.” This puts them in the position where they’re actually doing the work and you’re just providing feedback so that they become better at it. And that way in the future, it’s less likely that they will say something like, “Hey, can you just do this? You’re a lot better at it.”
Karen Lo: Second way is, “I understand the urgency of this request. However, I want you to be aware that if we bypass the existing processes to release sooner, we risk breaking core functionality and need to perform an emergency rollback. But I’m happy to oblige if you are willing to accept that risk.” Instead of saying, “Hey, I didn’t have a choice. That director asked me to release this right now and bypass going through QA or going through our regular things.”
Karen Lo: At the end of the day, you’re the one who signed off on doing this thing and you’re the one who actually executed it. Instead of saying that you don’t have a choice, you reflect this back at the requester. And when you reflect the responsibility back at the requester, they need to provide you with an official sign-off on the thing that they’re asking you to do because then you will truly be giving them the option to do it or not and not feeling like you yourself had to break rules in order to accomplish something for someone else.
Karen Lo: One more example is, “Hey, let’s discuss with our manager PM to see what the urgency and priority is. I have work on my plate that will be dropped if I need to jump on the task you’re requesting help with.” This is probably a very common scenario in which you might get pinged by somebody else, or maybe your PM themselves will ask you, “Hey, can you jump on this very urgent bug or deliver on this feature by the end of the week?”
Karen Lo: Well, you can reflect that back to them and say, “All right, I will do it. But just to let you know, I’m working on this thing that we discussed at the beginning of the sprint, and that for sure will slip. Is that the choice you want me to make?” And that way they have to think about what the prioritization is and you do not drop it and then have repercussions at the end when maybe you ended up not delivering on the initial work that you had signed up for at the beginning of the sprint. All right, so that was part two.
Karen Lo: Part three is legitimizing invisible work. What exactly is invisible work? I think a lot of us have done this type of work before. Invisible work is generally very habitual work that we do. Maybe it’s setting up meetings, writing documentation, taking notes to send for a meeting afterwards, remembering people’s birthdays, and celebrating milestones. I don’t know if that sounds familiar to you, but those are a lot of things that I personally have done that felt invisible.
Karen Lo: The first step to legitimize that type of thing is we want to cascade the recognition. Instead of you saying, “Hey, I’m doing all this stuff, guys. I should be recognized for it.” It’s much more about being intentional about who you recognize, how you recognize them, and just being present in being aware that someone did something that maybe went above and beyond, or they did exactly what was within their job description, but they did it really well.
Karen Lo: When you create a feedback loop that takes away pressure to self-promote, you promote other people to start doing the same thing. An example is maybe you’re sending an email to a person that you enjoyed working with, or sorry, maybe you really enjoyed working with somebody. You send an email to their manager, your manager, letting them know about this positive experience you had and how the person assigned helped you accomplish your tasks. Stuff like this that’s day- to-day, they generally are not recognized, but I don’t think there’s any harm in just saying, “Hey, thanks so much for helping me with this request. I know you guys are swamped with a lot of things, but just by the way, I don’t know if you even realize how important it was for me, but this helped me unblock my deployment. And normally, I’m waiting two weeks back and forth.” Something like that, it goes a long way.
Karen Lo: Another way you can do this is privately pinging a person in leadership or maybe even a peer to suggest that they provide recognition for this person because you think that it’s important that they know that this person either helped to go through an architectural review for the first time when the company is trying to make that more standardized. Maybe they did something really well. Maybe they performed their on-call duties really well and the incident management team would love to know that.
Karen Lo: You are just directing the people who might care about it and you’re surfacing what you want to talk about and recognize them for, but you’re not doing it yourself. You’re asking someone else to do it. And generally they’ll probably say, “Oh yeah, I would love to recognize this person.” Or even just give a quick shout-out or send an email just to let broadly more people know about the work that this person did.
Karen Lo: And step two is setting up a framework. Like I mentioned before, invisible work is generally habitual work. And habitual work is the prime candidate for automation and structure because we know that if you’re going to be repeating the same thing over and over, why not make life a little bit easier for not just yourself but everyone else?
Karen Lo: First thing that goes into this is to set up the framework. What does it look like if you were to repeat this task and be able to share it across not just your team, but maybe other teams as well? Because remember, a framework is something that is repeatable structure-wise, but not all the steps in a framework need to be the same. So they can always be adapted to fit any other use case.
Karen Lo: Some prime examples of how you might set up, or what you might set up a framework for are scrum leader responsibilities, maybe meeting scheduling, team engagement activities, documenting, metric reporting. All of these things you can put together, hey, as a scrum leader, every week you have stand-ups that you run, grooming sessions, sprint planning, retros, sprint review. These are all very straightforward things that every time you’re a scrum leader, you’re probably going to be doing these things.
Karen Lo: Something like team engagement activities and planning, why don’t you just have everybody put in their birthdays and then maybe integrate with a Slack bot so that it reminds everyone when it’s their birthdays. Things like that are prime examples of how you can set up a framework. And of course, there’s many other ways you can do it, but these are just some examples for you to look at.
Karen Lo: The last step of this playbook for part three is to democratize that. Now that you have the invisible work lined out step by step, it is something that you don’t have to do by yourself anymore. Once you have that together, you can then set up a meeting and say, “Hey everyone, here’s a framework I put together for what you would do as a scrum leader. It would be really great if we each owned, every one of us owned the scrum process because this is for us and this is for our team. I’ve laid it out really simply so that you just need to follow these things and then this is the way that you lead them. And I can definitely provide some feedback along the way. I’m not going anywhere, but I want this to become something that is a shared responsibility.” And like I mentioned earlier, automated birthday milestone reminders.
Karen Lo: Third one is to have explicit driver expectations outlined on who does what and when documented and frequently referenced. And I say frequently referenced because we all know we write documentation that goes into the abyss and no one ever reads it ever again. The framework type of documentation is what I would generally expect drivers to frequently reference until they forget about what is expected of me and they just know, and then they share it with anyone else who joins the team because then those people will also be taking on the responsibility.
Karen Lo: And finally, we want to incorporate team citizenship as a value and form of recognition. You don’t have to be a manager to do this, but if you are in a management or leadership position, it is critical that you are recognizing citizenship. Because it’s so easy for us to get bogged down by the day-to-day things of we’re coding or we’re going to meetings about things and we’re creating pull requests. And we know what our work is. But the part that’s important is the way that we interact with each other. That doesn’t always get intentionally highlighted.
Karen Lo: Intentionally highlighting, “Hey, thanks so much for replying back to me so quickly.” Or, oh, so-and-so, I was running into an issue and they were like, “Why don’t we hop on a chat and I’ll walk you through this thing?” And it just goes a really long way to providing a very healthy culture in which people start to be a lot more mindful about the things that they recognize each other for. And then they’re more aware of what they’re doing so that they can share the love with everybody else. And eventually you create a culture in which people recognize each other without needing to prompt for, “Hey, you should probably recognize that person,” into your culture.”
Karen Lo: That’s pretty much in a nutshell three parts for how I have learned to influence over the years. I hope that this has provided everyone with at least some nuggets of knowledge. Again, thank you so much for joining me today. I know you guys are busy, so have a great rest of your day.
Angie Chang: Thank you, Karen. I know you wanted to share your slides. If you could do that on LinkedIn or whatever social network that you prefer, we can reshare it and people can save it and look at it for themselves. You can always replay this as well. Thank you so much for joining us, and we’ll be hopping to our next session.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
As we kick off Latine/X Heritage Month (September 15 – October 15, 2023), we’re spotlighting and celebrating some of the inspiring Latine/x women who’ve joined the Girl Geek X community as SPEAKERS and THOUGHT LEADERS at our various Girl Geek Dinners and ELEVATE Virtual Conferences over the past 15 years.
Aubrey is The Mathpath (Math Nerd + Empath), Senior Director of People Operations & Strategic Programs at Culture Amp, and a startup investor and advisor.
Through all her work, she seeks to question, reimagine, and redesign the systems and practices that surround us to ensure that all people can access equitable opportunities and build a better world. Her work is undergirded by her training in social scientific methods and grounded in the fundamental dignity and value of every person.
Her professional expertise covers a broad range of equitable enterprise operations, from talent lifecycle programs and accessible product development to event design and communications & media. She is the inventor of the balanced teams approach to building proportional representation and a culture of belonging in the workplace, as well as the Balanced Teams Diversity Assessment in the Atlassian Team Playbook. She works to open source these methods for all practitioners and business leaders, and releases thought leadership and tools to create positive change at aubreyblanche.com.
She is an advisor to a variety of groups seeking to build a more just world, including Aleria Research and Joonko. Her work has been featured in Wired, the Wall Street Journal, the Australian Financial Review, USA Today, Re/Code, First Round Review, and more. She also has previous academic affiliations with Stanford and Northwestern, and an appointment at the Equity by Design Lab at the Stanford Graduate School of Business.
Despite the accolades listed here, she asks that you engage with her work to judge her competence: traditional proxies of merit and/or competence help reinforce the systems that keep incredible people from the opportunities they deserve.
“When you take a diversity and inclusion job, in most organizations, you give up all of your power but still have all of the responsibility. So what I’m suggesting is that you go into a place in an organization where you have great power, and then take great responsibility.
Let’s say you’re a Director of Marketing. You’re responsible for hiring and promotions, compensation of your people, the culture in your organization, you probably have control of a budget, and you have influence over how others in the organization act and think about these issues. You can simply demand that the hiring processes in your organization are fair and that they’re audited. You can insist on pay equity audits to make sure that people are compensated commensurate with their value. For the culture, you can enforce standards of behavior and respect for other people. And you can influence, just by your behavior, the way that other leaders in your organization can show up as allies.
So when I say don’t get a diversity and inclusion job, I’m not telling you to give up on creating systemic change. What I’m recommending is that you go from influencing people to bring equity and justice in the world to actually bringing equity and justice into the world yourself.“
Cindy is the author of “Lean Customer Development: Building Products Your Customers Will Buy” and Director of UX at PowerPoint (Microsoft.) Previously, she was the Director of Customer Research at GitHub, and also served as Director of User Experience for Yammer (a Microsoft company).
She has over a dozen years’ experience leading design, product management, user research, and customer development for startups, and used that background to drive intrapreneurial change within Microsoft.
Cindy has spoken at Girl Geek Dinners at Yammer, GitHub, and she joined us virtually for our inaugural ELEVATE virtual conference in 2018, with an evergreen talk titled: “The Customer Is Not Always Right.”
Here’s one of the top takeaways from Cindy’s ELEVATE talk:
“A lot of us, when we’re interacting with customers and hear a demand for features, it’s hard to ask why. But it’s useful to take that step back. I like to announce it as such, and say, ‘Just a second. I want to be sure I understand something. It sounds like you’re asking for this feature. Just to be sure I understand, if we had already built it, what would it allow you to do? Essentially, how would it make your life better if you had this thing?’
When you ask people some polite version of ‘how would it make your life better?’ a lot of times you get a non-answer. You’ll get an answer like, ‘Well, it would just be nice to have,’ or ‘your competitor has it.’
Once in a while, you might hear: ‘Oh, you know, it would take me half the time to sort my data. Oh, I wouldn’t have to waste head count on this position. We could start coding tomorrow.’ When people have a story, that’s something worth doing.”
Citlalli is a Director of Engineering (Network Security) at Palo Alto Networks. She has also worked as Director of Engineering at Splunk, where she oversaw the Enterprise Security team. Citlalli previously served as Director of Cloud Security Engineering at Palo Alto Networks, where teams develop the backend of the Public Cloud Security service that protects enterprises as they unleash the power of the cloud.
Citlalli has navigated her teams through M&A integrations while successfully building highly distributed API-based SaaS security platforms.
Earlier in her career, she developed software for CirroSecure, Cisco, Apple and The Central Bank of Mexico. Citlalli holds a BS in Computer Systems Engineering from Tecnolgico de Monterrey in Mexico, and an MS in Information Security Technology and Management from Carnegie Mellon University in Pittsburgh PA. She’s also an advisor at Techmmunity.
Our favorite words of wisdom from Citlalli: “One of my values is transparency, so as a leader, I would rather know the good, the bad, and the ugly upfront, because then I can do something about it. During interviews, I’m very transparent. I’ll say: ‘I think you’re a great fit. I really want you to work for us, but you’re going to face this, this, this, and that,’ and even in the questions ask them, ‘How have you dealt with this type of situation?’ So, ‘Tell me the worst mistake you’ve made and how you came out of it.’ And you can tell when people have done it before or learned something, and that reflects their own transparency.”
As a pioneering woman in tech, Diane began her tech career at Hewlett-Packard as a Senior Software Engineer in the 80’s. Diane is a proven, respected technology executive, currently serving as Vice President of Technology at Amazon, and she previously oversaw Amazon’s AWS Commerce Platform in a VP role.
Prior to joining Amazon, she was Vice President/GM of Platform Services at VMware. She was also Vice President of Engineering for the Online Services Division at Citrix. She has worked as an Executive Consultant with several start-ups where she functioned as the Vice President of Engineering, and served as VP of Product Development at Intuit. She is an active member in various women and minority forums.
Our favorite excerpt from her talk: “In my career, I’ve never really met anyone who would admit to being a bad decision maker.We all think we’re good decision makers. Maybe that’s true. I mean, we’ve all probably made bad decisions in the past. But in my opinion, the worst thing is no decision. There’s a lot of analysis paralysis. Probably one of the most important things around making a decision… is to make a decision. You can’t just pore over the information over and over again.
Part of what I do is make sure I really understand the opportunity. Determine the expected return on investment — is the return sufficient to out weight other possibilities? What is the opportunity cost for investing in this instead of something else? You could be doing something really cool, but completely miss out on another opportunity. You have to really understand and think about what that means to you. Solicit different opinions. Have a sounding board, share your thought process, get feedback. Then step back, look at the facts, and make the decision. People get caught up in the should I, or shouldn’t I, and you just have to be bold sometimes and make those decisions.”
Geysa is the Vice President of Product Management at ServiceNow. Previously, she served as Senior Director of Product Management, Customer Experience at AppDynamics. She also worked at Adobe and Get Satisfaction as a Director of Product Management.
Originally from Brazil, Geysa graduated in the top 5% of her class from Universidade Federal da Bahia with a degree in Computer Science. She started her career as a programmer, but later went back to school to get her MBA in Marketing and Finance and then switched careers to move from development to Product Management. Through a series of big and small companies, Geysa found herself at AppDynamics to rejoin their CEO David Wadhwani who she worked closely with at Adobe, to build AppDynamics Customer Experience.
Jessica (she/her) is a Latina developer, educator and advocate in tech. Most recently, she served as a developer relations engineer at Google, connecting with developers and creating resources. She is on the board of Girl Developer It and a Women Techmakers ambassador. Jessica worked with at-risk youth and adults facing mental health challenges before pivoting to working in tech.
Our favorite advice from Jessica: Next time you get stumped during a whiteboarding exercise, don’t panic. You’re not going to say “I don’t know,” and you don’t have to apologize or feel bad for not instantly knowing the answer or the next step. Instead, try: “Hmmmm. This is interesting.”
Saying “this is interesting” buys you time.
Now you can think about it, or discuss why you find it interesting. You can think through it out loud, or turn it into a collaborative discussion without getting flustered.
Don’t let getting stumped on a whiteboard — which isn’t even part of the actual job — ruin an otherwise great interview!
Jomayra has experience working with early and growth-stage companies both as an investor and an operator. She is currently a Partner at Reach Capital, and sits on the Board of Directors at both LatinxVC and WorkWhile. Previously, she was a Principal at Cowboy Ventures, and prior to that, she spent nearly 3 years as an investor at Emerson Collective.
As an early hire on the investing team, she played an important role in creating internal processes, building key investment theses, and helping to grow the team. During her time there, she worked on a diverse range of investments, including traditional venture investments and buyouts, and developed a special interest in companies tackling issues related to the Future of Work. She also worked at BloomBoard, an early-stage education technology company, where she focused on customer success and growth.
She is incredibly passionate about partnering with entrepreneurs to help grow and scale their companies.
“We’re seeing a flipping of the whole employment model on its head, which is the ability to not even rely on the concept of an employer to generate income. Self-employment isn’t new, but what is new are platforms that help to enable new types of self-employment. So if you’re a writer, you no longer have to rely on large publishers to monetize your writing. You can use Substack. If you are an educator and you want to teach about art or poetry or creative writing, you can use Outschool and generate either supplemental income, or something that actually generates a majority of your income and have that optionality on your own. We’re moving into a world where you have more ownership over your career than ever before. And with the rise of options, the rise of data, and the rise of having access to communities that can help you, we have the ability to be more conscious workers.”
#8 – Lili Gangas – Kapor Center Chief Technology Community Officer
Lili is the Chief Technology Community Officer at the Kapor Center for Social Impact. In her role, Lili helps catalyze Oakland’s emergence as a social impact hub of tech done right – where tech, diverse talent, and action driven partnerships can tackle pressing social and economic inequities of our communities head-on.
Lili advises inclusive tech entrepreneurship ecosystem building activities Oakland initiatives such as Oakland Startup Network, TechHire Oakland, Latinx in Tech, Kapor Center Innovation Lab.
Before coming to the Kapor Center, Lili was an Associate Principal at Accenture Technology Lab’s Open Innovation team, based out of Silicon Valley, building bridges between startups and commercial clients. She was also a founding member of the Innovation Services team at Booz Allen specializing in crowdsourcing, prize challenges, and open data solutions at the federal level. Before that, Lili could be found in the lab working on software and hardware solutions for the aerospace industry as a Senior Multi-Disciplined Software Engineer at Raytheon Space and Airborne Systems.
Lili is a proud immigrant from Bolivia who believes in fostering inclusive tech ecosystems for all. She’s been an active Startup Weekend organizer – helping launch Women’s Edition, Impact Edition, and Latinx in Tech Editions. She also helped organize the first ever TEDxOakland. She is an advisor to tech focused nonprofits such as AI-4-All.org, 1Degree.org and Dreamwakers.org.
Lili holds an MBA from New York University Stern School of Business, a B.S. in Electrical Engineering from the University of Southern California and Systems Engineering Certification from UCLA Extension.
In her 2019 Girl Geek X talk, “Tech Stayers & Leavers,” Lili shared some poignant advice for managers and execs who want to stop women from leaving their companies:
“If you’re a C-Suite exec at a tech company, or you’re a manager, there are ways that you can directly really help create a more level playing field for everybody in your workplace, and ultimately, women, we really just want to have equal pay. We cannot believe that we’re in 2019 and we still have issues that we’re still being underpaid. Specifically, Latinas in the US are significantly underpaid. They’re about 56 cents on the dollar compared to a white male.
Second, improving company leadership is critical. Without having the C-Suite, the CEO, and also the managers across the different angles being able to advocate and really create and put forward new policies… this is going to continue. We have to lead by example.
Promotion is also important. This is an area where a lot of women that were surveyed, specifically expressed that this is why they were leaving, in addition to wanting to have a better work/life balance. If you’re not finding the opportunity internally, you’re going to leave. But sometimes if your job at the moment is providing you a great work/life flexibility, it’s harder to make that change. Sometimes our careers start plateauing, but we have to be mindful that there are other opportunities and options. Ultimately, we just want to have a much more positive and respectful work environment.”
#9 – Luiza Pena – Cadence Lead Application Engineer
Luiza is a Lead Formal Verification Engineer at Cadence Design Systems in Belo Horizonte, Brazil. She is working in the Semiconductor industry by driving usage success and business across several top notch US-based companies remotely. She believes women can boost their career through good strategies and overcome the networking challenges of remote work. With this mindset, she has worked with mentorship and career counseling volunteer projects for STEM women inside and outside Brazil.
Our fave quote from Luiza’s talk: “You need to build self-awareness by understanding your position and where you can go in the company – what are your strengths and where can you get with these skills that you can stand out? And you always need to work on your personal branding so people see what you’re doing and the results that you’re getting. And you need to understand who are the key people that are going to help you amplify your impact and take the next step in your career. This is stakeholder’s awareness. Be intentional and pro-actively connect with the people who can help your career.“
#10 – Maria Lucena – Fidelity Investments Director of Architecture
Maria is a Director of Architecture at Fidelity Investments, and has been in Software Development for over a decade. She started as a freelance web developer in 2009, with only a Web Development Diploma.
Once she had a solid portfolio, she landed a compelling opportunity at Santander Bank in Boston. After working with Java, Oracle, Microsoft SQL servers, and Angular 1, Maria was ready for her next adventure.
In 2015, she was going to school part-time for her Associate’s in IT and landed a job at Fidelity Investments. Here’s where Maria’s career took off. She is a full-stack engineer, but her best work has been backend work, which is where she prefers to spend her days. In the last two years, she has applied for two patents with a college at Fidelity, one of which has been accepted.
Rocio is a Senior Engineering Manager at Github. She leads the Actions Compute team, ensuring the massive capacity required to power Actions runners. Prior to this role, she was engineering manager and tech lead for inner source and open source at Intuit. She is a true community builder, and works to connect and collaborate with software engineers to deliver amazing end-to-end solutions.
She is also currently Co-Chair for Grace Hopper Conference’ Open Source Day.
Outside of work, Rocio is the Co-founder of “Emar”, a small business with a mission to connect US small businesses with technology needs to software engineering interns in Peru.
Sandra is an internationally recognized business leader, currently serving as GM/VP/CMO at Microsoft. Prior to joining Microsoft, she built an impressive career at Intel spanning nearly 16 years and leading multiple departments. Most recently, she held the role of Vice President for Intel Sports and Media, responsible for partnering with the sports and media industry to provide the future fans and consumers with the next generation of immersive media experiences. Her team at Intel was focused on leading the business, marketing, and market development efforts of Intel Sports and Intel Studios.
She also previously worked within Intel’s New Technology Group, leading and managing the Fashion wearable business. In this role, she was a vocal advocate for the convergence between fashion and technology. Earlier in her Intel career, she held various roles within corporate marketing, including director of new business marketing and director of consumer marketing. In the latter role, she led Intel’s brand repositioning work to earn an Intel Achievement Award in 2010. She also earned industry honors.
Before joining Intel in 2005, she worked at Adobe Systems Inc., Macromedia, Computer Associates International Inc. and several other technology companies where she earned a reputation for transforming and growing businesses. She holds a bachelor’s of science degree in economics and textiles and clothing from the University of California at Davis. In addition, she attended the Stanford Intel Accelerator Program. As part of contributing to the community, she is focused on building the next generation of women leaders and is a vocal advocate for equality.
Since 2018, Sandra has co-chaired the World Economic Forum’s Global Future Council on AR/VR. She has been recognized as one of the “Most Powerful Latinas” (ALPFA), Most Powerful Women in Tech” (National Diversity Council), “Top Women in Media” (Cynopsis), “Top 10 Latina Executives” (LatinaStyle), “Top 100 Most Influential Latina” (Latino Leaders) , “Most Influential and Notable Hispanic Professionals in Information Technology” (HiTec) and “Game Changer” (Sports Business Journal).
Sandra has spoken at several Girl Geek X events going back as far as 2015, including our 2019 ELEVATE International Women’s Day Virtual Conference, where she shared poignant advice on being yourself in her keynote talk: “Being Unapologetically You.“
One of our favorite takeaways from Sandra’s talks: “If I could go back and advise my younger self, my advice would be… be your unapologetic you.“
In being yourself, in refusing to assimilate or mold yourself to your surroundings, once you stop trying so hard to fit it… you discover what you’re capable of. In the process, you gain confidence, and you find your voice.
“As a Latina, when you’re born, the culture tells you never to challenge seniority. But challenging seniority in a corporate setting is really about intellectual curiosity, and trying to do what’s right for the business. And so, I have found the confidence and the voice to have those conversations by being my unapologetic self.”
Honorable Mentions– Latine/X Women We’d Love to Hear From!
The above list is just a snapshot of the dozens of remarkable Latine/X leaders and innovators who’ve joined us at Girl Geek X / Bay Area Girl Geek Dinners over the past 15+ years. There are many more we’d love to invite back to speak again, or feature at a Girl Geek event for the first time!
We also accept speaker suggestions/nominations, so if there’s an awesome Latine/X woman or non-binary individual you think we should invite to speak at a future Girl Geek X event, head on over to our LinkedIn post and tag her in the comments!
Plus We Have Bay Area Volunteer Opportunities for Latine/X Month!
On Monday, October 2, 2023 (1:00pm-2:30pm), Girl Geek X Community Volunteers will read books to 2-3 elementary school classes that celebrate Latine/x culture in a 90-minute volunteer shift. Books and sample questions to guide conversations are provided by the nonprofit Oakland Public Education Fund.
Volunteers do not need to identify as Latine/x to participate, and those who do identify as such are encouraged to participate and share about their culture with students.
Fall essentials: Whether you want to buy from women-owned companies, or are looking for a gift for a professional woman to benefits from productivity software or the gift of mentorship, look no further than the Girl Geek X Gift Guide to support women-owned companies and women!
The Best Productivity Software Multitudes for Engineering Teams
Want to ship quality code, faster? And want to do it in a human-centric way?
Multitudes supports happier, higher-performing engineering teams. The AI coach product spots delivery risks including what work is blocked, who’s at risk of burnout, and more; they then guide teams to take action with recommendations and nudges in Slack. Multitudes is founded by CEOLauren Peate.
Save 10% with “GIRLGEEKX10” for your first 6 months after the free one-month trialof Multitudes when you list Girl Geek X as the referrer to get early access!
Getting Women Engineers Hired To Startups The New Club Retreats and Recruiting for Women Engineers
Looking for community and opportunity? Women engineers need to look no further.
The New Club connects women in engineering with retreats in New York and San Francisco. The retreats will be October 20-22, 2023 in Hudson Valley, NY and November 3-5, 2023 in Mendocino County, CA. The New Club is founded by CEO Laura Du.
The Best Mentorship Program Career Accelerator for Women for $1,245.00 $1,182.75 at WEST
If you have wanted to bottle up and gift a mentorship experience for a professional woman, here’s your chance!
WEST supports ambitious women technologists through 1:1 and group mentorship. Get hands-on support as you explore your career options, grow your influence, improve your communication, and more through our Career Accelerators
Girl Geek X’s highly-anticipated ELEVATE Conference and Career Fair on September 6, 2023 hosted over 1.5k mid-to-senior women in tech around the world online for inspiration, connection, and learning.
Thank you to our inspiring speakers & sponsors for helping make ELEVATE conference an incredible experience. Check out our sponsor’s remote/flexible jobs – they are actively hiring! Please spread the word and help a girl geek find her next role in tech!
Here are the most popular talks from September 6th’s ELEVATE 2023! You can watch (or re-watch) them at the links below:
Mentors joined from companies like Google, Airbnb, Amazon, Autodesk, Twilio, Fastly, Okta, Bayer, Alphawave Semi, Anthropic, Kohl’s, Riot Games and more. Mentors ranged from CTO to engineering managers, VPs to product managers and engineers.
BENTLEY SYSTEMS (Nasdaq: BSY) is the infrastructure engineering software company. We provide innovative software to advance the world’s infrastructure – sustaining both the global economy and environment. Our industry-leading software solutions are used by professionals, and organizations of every size, for the design, construction, and operations of roads and bridges, rail and transit, water and wastewater, public works and utilities, buildings and campuses, mining, and industrial facilities.
GRAMMARLY is trusted by more than 30 million people and 50,000 professional teams worldwide every day to help them ideate, compose, revise, and comprehend communications. Combining advanced machine learning with human expertise, break new ground in natural language processing (NLP) and AI to offer unmatched communication assistance to individuals and enterprises.
VANNEVAR LABS combines top software engineering talent with decades of mission experience to get state of the art technology to the people that keep us safe. Vannevar Decrypt is a foreign text workflow platform built for national security.
We’re an independent, not-for-profit, locally governed health plan company – meaning we live and work alongside our Tennessee business customers and plan members. Our 6,000 employees have built our strong reputation for integrity, excellent service and community leadership. We are part of the BlueCross BlueShield Association, a nationwide association of health care plans, so plan members have access to the same quality health benefits while traveling or living out of state that they have while in Tennessee.
UCLA Information Technology Services is largest provider of technology services to the university. We partner with many academic research and administrative units throughout UCLA to enable their mission, enhance their effectiveness and allow them to leverage cost effective IT infrastructure, serving more than 60,000 students, faculty and staff through enterprise applications, information management, collaboration solutions, computing platforms, storage, data center facilities and the campus-wide voice and data networks.
Autodesk makes technology that millions of people use to design and make the world around us. Our solutions are used to create all kinds of amazing things—from the greenest buildings to the cleanest cars, from the smartest factories to the biggest blockbusters. And our platform helps our customers combine technologies to solve their challenges, whatever their industry. Together, we help our customers turn their ideas into new realities that shape a better future for everyone.
Cadence is a pivotal leader in electronic systems design, building upon more than 30 years of computational software expertise. The company applies its underlying Intelligent System Design strategy to deliver software, hardware, and IP that turn design concepts into reality. Cadence customers are the world’s most innovative companies, delivering extraordinary electronic products from chips to boards to complete systems for the most dynamic market applications, including hyperscale computing, 5G communications, automotive, mobile, aerospace, consumer, industrial, and healthcare. For eight years in a row, Fortune magazine has named Cadence one of the 100 Best Companies to Work For.
At USDS, we are mission-driven professionals who are passionate about applying our work and lived experiences to public service. We come from a range of cultural, geographical, and ethnic backgrounds, and we represent a myriad of intersecting identities, just like the people we serve. USDS operates on a tour-of-service model with a maximum term of four years. While most people serve for one or two years, we’ll consider some shorter tours as well. Time commitments are not binding.
To learn about the developer, product, design, data science, and acquisition skills we’re hiring for, see “How we work”. We are always hiring for a number of different roles, so if you are an expert in your field and are interested in working at USDS, you should apply here: www.usds.gov/apply
Like GPS for your code, CodeSee is on a mission to help you build the applications you love without the guesswork. Instantly visualize a map of your app’s services, directories, and file dependencies. CodeSee ensures your team stays code synced.
Don’t miss CodeSee CEO and Co-Founder Shanea Leven speaking at ELEVATE 2023 Conference & Career Fair online – it’s FREE to register & attend!
Dematic is transforming global supply chains through software to embrace trends in digitalization and e-commerce. We design, build, and support intelligent, automated solutions for a wide variety of industries – general merchandise, grocery, food and beverage, healthcare, apparel, and many more. With engineering centers, manufacturing facilities, and service centers located in more than 35 countries, the Dematic global network of over 11,000 employees has helped achieve nearly 8,000 worldwide customer installations for many of the world’s leading brands.
Over 100 girl geeks joined networking and lightning talks from women working in engineering, product, and design at the sold-out Grammarly Girl Geek Dinnerat Grammarly’s office in downtown San Francisco, California on August 29, 2023.
Grammarly women shared lightning talks about building GrammarlyGO, Grammarly’s new contextually aware generative AI communication assistant that allows you to instantly compose, rewrite, ideate, and reply. Grammarly is hiring!
Transcript of Grammarly Girl Geek Dinner – Lightning Talks:
Angie Chang: Is [this] your first Girl Geek Dinner? Wow, that’s a lot. How many of you have been to more than five Girl Geek Dinners? Yay! So good to see everyone. My name’s Angie Chang, in case you didn’t know, and you could tell by the t-shirt, I am the Girl Geek X Founder, and started Girl Geek Dinners in the Bay Area 15 years ago, so we’ve been doing events like this at hot tech startups up and down from San Francisco to San Jose, and I’m in the East Bay, so I wish there was more events over there as well. Tell your employers they need to have one of these showing off their amazing women in tech and product.
Girl Geek X founder Angie Chang welcomes the sold-out crowd to Grammarly Girl Geek Dinner on August 29, 2023 in San Francsico! (Watch on YouTube)
Angie Chang: I want to say thank you so much to everyone at Grammarly for helping put this event together. They have been so amazing and supportive and they’re definitely hiring, so please talk to someone here that has Grammarly on their shirt. They’re very friendly, so I’m going to say thank you for coming and hopefully you’ve made a lot of good connections. I know I’ve seen a lot of people talking to each other and I hope you have LinkedIn with each other or Facebook or whatever people are using these days, and continue to stay in touch.
Angie Chang: A lot of us are in this industry working to keep women in tech and I think that involves all of us together, so thank you. Keep coming back to events! Keep giving each other job leads! Keep poking other girl geek to get in the car ride together to get to that event after work when we’re all tired! Thank you for coming! I hope you learn something, make a new friend, and have a good night!
Charlandra Rachal: Thanks, Angie. I’m super excited to kick things off and host this fireside chat with director Heidi Williams, who’s been very involved in building our generative AI features for enterprise. Heidi, welcome!
Heidi Williams: Hi. Thanks for having me! Great to see you all here. It’s awesome. Full crowd!
Charlandra Rachal: Yeah. For those who aren’t super familiar with Grammarly, can you give us a quick overview of our company and our product?
Heidi Williams: Sure. I like to make a joke that either people have never heard of Grammarly or they love it! I know I talked to a few folks already that love it, but for folks who aren’t familiar, we are an AI-enabled writing assistance that helps with your communication wherever you write, and I do mean everywhere. Our mission is to improve lives by improving communication. Earlier this year, we also launched our first generative AI product to help you with even more writing and communication assistance beyond just revision, but also getting into ideation and brainstorming and composition and comprehension. It’s been really fun to see the product evolve in the time that I’ve been here.
Charlandra Rachal: I hear that you just celebrated three years here, so woo woo! Three years! Can you tell us what brought you here and what really keeps you here?
Grammarly Technical Sourcer Charlandra Rachal and Director of Engineering Heidi Williams welcome the audience at Grammarly Girl Geek Dinner. (Watch on YouTube)
Heidi Williams: When I was speaking about the mission, improving lives by improving communication, I do feel like I got to a point in my career, I’m a little farther along maybe than some of you, that I really wanted to work on something impactful and I feel like Grammarly more than any other place, it resonated with me that improving lives by improving communication is so real. It’s not a fake slogan because communication is what makes us uniquely human.
Heidi Williams: I was excited about the idea that we’re not just a platform to help you communicate more effectively, but also to help educate you along the way, especially thinking about things, like insensitive language or bias, there’s an opportunity to help educate people about the possible impact of their words that they may not even know is having a negative impact on someone, and so I got really inspired about the mission.
Heidi Williams: I’m also a word nerd, so that part was really fun as well. I think what keeps me here, is that everyone is so excited about the mission and the people. I think our values are amazing. We really live by our values, we hire and fire by them.
Heidi Williams: The last thing I’ll say is, we’re an amazing size company, where there’s still interesting problems to solve, but we’re small enough that people can really take the initiative if they see a problem that needs to be solved, or they want to advocate for something to change in some way, they’re really empowered to do that. I love being at that size company and our values really help us be successful doing that as well.
Charlandra Rachal: Nice. I like you mentioned initiative and impact. Do you have any sharing stories that you can share that where you seeing either yourself or someone else really make impact?
Heidi Williams: I have three examples if you’ll bear with me for a minute, but I see it all over the place, and it’s not just in the product. It’s about our organization, our culture. There’s an engineer on my team, her name is Lena, and she recognized on the product side that engineers were struggling with a certain pattern of ‘how do I reliably save settings about the individual, about their team and their organization about specific features. Then if I have all of these settings, how do I combine them and know which setting to apply at any time?’
Heidi Williams: She interviewed a bunch of engineers, realized it really was a problem for folks, and then proposed a new project called the Settings Registry, and advocated for it to be on our roadmap. It’s been exciting that she could spot an opportunity and a challenge for our developers and really advocate for that. That’s exciting!
Heidi Williams: The second one, I actually led an initiative where I noticed that I love our hiring process, but I noticed that we had one particular gap, which was that we didn’t necessarily have an interview where we asked people about their experiences. We ask about their knowledge, but we don’t ask, ‘what is the proudest thing that you ever built and tell me how it was designed and what did you learn?’
Heidi Williams: I noticed similarly that we weren’t necessarily getting the accept rates from underrepresented groups that I thought we should be getting, and advocated that this might give people an opportunity to talk about themselves, and for folks who aren’t used to bragging about themselves, that might not come out in a normal interview, but if you give them an opportunity to talk about themselves, then they can actually show off how good they are at stuff, which is exciting.
Heidi Williams: That pilot was successful, showed that we greatly increase the accept rates for folks from underrepresented groups to a really high degree, and now we’ve rolled that out as an interview across the engineering organization, so really proud of that.
Heidi Williams: The last one I’ll mention related to culture, Bhavana, who you’ll hear from later, identified an opportunity that folks were looking for mentorship inside of our women in tech group, and so she started a pilot with a few other folks to introduce an internal mentorship program for women in tech and we’re kicking that off in September.
Charlandra Rachal: I love that. Yes. I feel like the last two really spoke to me, especially being in recruiting so I love that a lot. Now, Grammarly continues to expand in its enterprise space. How do you drive value for Grammarly business with generative AI?
Heidi Williams: It was very exciting to see our generative AI product come out. A little bit of context: the part of the product that I’ve worked on is Grammarly Business, which is our B2B product for teams and organizations.
Heidi Williams: As we all know, communication is not a one person sport. There’s a team dynamic, there are team norms, there’s organizational knowledge that are part of the communication that you have at work. We looked at opportunities for how to incorporate organizational knowledge.
Heidi Williams: We have a feature called Knowledge Share that helps you define terms, definitions related links, key people, and then we can use that as part of the generative AI output to help you have something that knows something about your organization instead of maybe a more generic response.
Heidi Williams: We did things like that and then incorporated some of our Grammarly business features like style guides and brand tones, which help you speak with a consistent voice, and brand tones in particular, you can have a response from our generative AI product, and then choose ‘make it sound on brand to my company’.
Heidi Williams: That was a way that we could really make the information, both the information and the tone be tailored to your organization.
Charlandra Rachal: Nice. Well, I heard that there was some quick turnaround times. Can you tell us more about that?
Heidi Williams: It was definitely felt like this huge opportunity, this huge moment where a lot of folks are talking about generative AI and it’s an area (LLMs) we’ve been investigating for a long time and understanding what their capabilities and limitations were and whatnot, and so I think we really rallied as an engineering organization, and I think the way that we were able to turn things around quickly really came from our leadership approach, which is the idea that we really want to empower teams to make the best possible decisions on the ground.
Heidi Williams: The way to do that is to help with transparency and sharing context around ‘what are the business needs, what are the product needs, what are our customer needs, what problem are we solving for the user?’ Let me give you all of that information, all of that context. At the end of the day, if you need to choose, should this be a radio button or a dropdown or this should work this way or connect with that system, you can make that decision because you have all of that information. Really trying to be transparent and share context so that people are empowered to make decisions on the ground and not feel like they’re stuck with somebody else making decisions and kind of blocking them from things.
Charlandra Rachal: I hear you that you mentioned customer feedback. Do you have any feedback that you’re able to share with us?
Heidi Williams: Sure. You’ll hear more about it in one of the talks today. We did run a survey after launching GrammarlyGO and wanted to know how are people using the product and what’s working and what’s not working. Through that feedback, one of the themes that we heard was that ‘it didn’t sound like me’.
Heidi Williams: We started investigating – ‘how do you tailor the output to sound authentic to you?’ And it sounds, I see a lot of head nods, that resonates and what not. We invested in an area called My Voice and figuring out how to have your own voice profile and use that for all of the responses that are generated, so it’s more likely to sound like you than not and saves you an extra step for trying to even interpret what your own voice is. We can actually help you with that, so you’ll hear more about that when Jen talks about it.
Charlandra Rachal: Great. Well, I know this is one question that I know a lot of people probably want to ask but probably wouldn’t ask, but what would you say really sets us apart from our competitors?
Heidi Williams: Yeah, I was talking to someone ahead of time who asked this question, I’m like, oh, you’ll have to wait. <laughs> Great question. There’s one thing. First of all, I mentioned earlier, we work everywhere and that is one difference from some of the other products that are out there. We work in every writing surface, desktop web, and so we can be right in line for where you’re already doing your thinking, your writing your communication, so that’s certainly one.
Heidi Williams: The two I wanted to really call out, which I think are kind of reinforced by our engineering culture, is our important focus on security, trust and privacy, and also responsible AI. Because at the foundation of everything we do, we really want our customers and users to trust us with their writing and to feel like we can do things to make personalized experiences and what not, and so, what’s interesting to me, I feel like more than any engineering organization I’ve ever been at, because we are so mission-aligned, we recognize we have this huge responsibility to our users to be thoughtful about their data and their privacy, their security.
Heidi Williams: I feel like we care a lot about security maybe earlier than most engineering teams where at the very end before you ship security goes, ‘oh, not yet!’ And you’re like, ‘oh, I can’t’. The whole idea that engineers will advocate for, am I doing this right from the beginning, and wanting to make sure that’s so they’re proactive about asking for feedback about security and privacy. Or even there was a scenario where we had an idea about a feature and people are like, ‘That feels like it might invade privacy. Can we talk about that before we launch it?’
Heidi Williams: I really loved that people could bring that up and that we’re all trying to achieve the same thing, and so it’s a very fair question and let’s make sure we’re holding that to high regard.
Heidi Williams: Then on the responsible AI side, I think we’re so lucky to have an incredible team of linguists who can help us beyond what other competitors can do who don’t have a team of linguists where we can help sort of filter things like the inputs to generative AI to make sure that people are not asking for something harmful, but also that whatever they type in, they’re not getting harmful responses, which are either insensitive or inflammatory or traumatizing in some way.
Heidi Williams: I love the fact that we have the capabilities of being able to create these filters and create a safe environment for people to use these large language models, which have who-knows-what in them. Love that we are actually able to do that. We’ve also been able to build that not just through humans, but figuring out how to build automation and testing and all through the development process help you understand that you’re not going to create a feature that unintentionally create some sort of biased output or something like that, and so just tremendous examples over our long history in this of finding ways to make sure that we are building a product that is responsible and then also keeps everybody safe, secure, and all their information private as well.
Charlandra Rachal: Nice. Well that was fascinating, right, everybody? Alright, so we are going to dive deeper now to exactly how our generative AI features were built. As a heads up, we are going to ask for questions at the end and I’ll bring up all of the speakers including Heidi herself. For now, welcome Jennifer van Dam, who’s a senior product manager here!
Jennifer van Dam: Hey everyone. I’m Jennifer van Dam, product manager here at Grammarly. I’ve been here for three years and I worked on our features like emotional intelligence, tone detection, tone rewrites, inclusive language, and most recently I helped build out our generative AI product, GrammarlyGO, which I’ll be talking about today, so super excited to take you all through the journey.
Grammarly Senior Product Manager Jennifer van Dam talks about building the generative AI product GrammarlyGO from zero to one. (Watch on YouTube)
Jennifer van Dam: First off, I want to give a huge shout out to my fellow girl geek PMs that helped build GrammarlyGO together with me. We were a team of three PMs leading multiple product efforts. Specifically, my product focus was on the UX and also on the zero to one stage, so figuring out the UX framework and the zero to one building process. That’s what I’ll dive in deeper today. First off, I wanted to start with a refresh of Grammarly before GrammarlyGO.
Jennifer van Dam: What Grammarly has been focused on for many, many years is helping make your communication more effective by proof writing and proof reading and editing your writing. Anywhere you write, let’s say you’re writing an email, a message, a Google doc, Grammarly will read the text that you have written already and make sure it’s correct and clear and delivered in a way that you want to come across. But, we have a big mission of improving lives by improving communication, so we fully were aware that this is a small part of communication that we want to help with, and we’ve had many dreams beyond proof writing and editing.
Jennifer van Dam: One big user problem we always heard, for example, was the ‘blank page problem’. For years, we’ve heard that our users really struggle with the inception stage of communication – the writing, getting those initial ideas on paper – and it was a huge productivity blocker. That’s just an example from user problems we’ve been hearing for years, and we always dreamt about solving it, and we were super excited with this recent technological leap that with generative AI – now we have the technology to solve all those user problems we always dreamt about.
Jennifer van Dam: That’s how we built GrammarlyGO. We went from proofreading and editing towards helping solve composition, brainstorming, and all these new use cases, which was really, really complex, because we went from a decade of in-depth expertise of rewriting, towards composition and brainstorming, and we had a pretty aggressive timeline as well. This was super, super challenging.
Jennifer van Dam: What made it really challenging? First of all, it was zero to one. We had no prior experience how this would land with our users and there was no data we could rely on, so we had to make a really risky decisions because we went from a proven product concept with product-market-fit towards a huge uncertainty and risk area, which it was really exciting, but super, super challenging. How can we predict how it will be received with the absence of data?
Jennifer van Dam: Essentially we really had to take on a beginner’s mindset to solve these new use cases and almost operate like a startup again to build this new product from scratch, but we’re also an established company – pretty big – and we have millions of users, 30 million daily active users that have a super high bar of our product. We were building zero to one moving fast, but also had a very high bar we wanted to meet for our users in terms of quality, responsible AI, and security that we wanted to deliver.
Jennifer van Dam: How do you solve such a huge, huge problem? What we did was, let’s just start with the earliest draft possible, and get it out – get it out to users. What we did is we created this really highly-engaged alpha community, and we built very early prototype, and we shifted and we asked for continuous feedback, and it was really, really engaged community that would give us feedback super fast and inform next iterations. We really focused on the core experience before we wanted to invest in any type of polish or any type of design polish, we made a commitment – let’s not focus on that. Let’s figure out the UX framework.
Jennifer van Dam: We had a big challenge. How do we create a UX where someone can brainstorm and compose something from scratch? What is intuitive? What will land with our users? To give you an example of the fidelity of prototypes, what we did is we started in grayscale, because we made a commitment to figure out the framework, before deeply investing into building something out, because we weren’t sure if this is the version to commit to.
Jennifer van Dam: This turned out to be a great idea because we did end up throwing away a couple of prototypes, and the third prototype was the one that we felt landed the most and that we committed to building out and refining, which was of course a huge process as well and took us a lot of time. Sarah will actually be giving a fascinating talk later about all the design and brand work that went into polishing this prototype, so I won’t go too deep into that.
Jennifer van Dam: What was really cool about this prototyping stage is, the user empathy led to innovation. We came up with things that we didn’t necessarily plan from the start. One thing we kept on hearing when we were asking for feedback on the UX and was it intuitive to compose and brainstorm?
Jennifer van Dam: A lot of the feedback we were getting was ‘it just doesn’t really sound like me’. And that made people drop off. They would compose an email or a document, but it didn’t sound like something they would write or want to use, so this was a huge risk of people dropping off and also, it wasn’t of the quality we wanted to meet. This led us to come up with the voice feature that Heidi talked about with before.
Jennifer van Dam: This is a classic example – it wasn’t on our roadmap from the start, but it really, being in tune with the user made us come up with this feature. I remember when we launched our first basic version of it, how excited everyone was, and that made us realize how important voices in generative AI – and it led us to much deeply invest in this area, so we’re keeping investing in this and also it helped actually become an important competitive differentiation.
Jennifer van Dam: To take it even further, we would also hear from users, okay, now it sounds like me, but in this situation it doesn’t sound how I want to sound, which was also a really hard problem. We heard this a lot in the email reply use case, and what we came up with is harmonizing your voice preference with your audience as well. Let’s say, I prefer to sound casual maybe 80% of the time, but I got this super formal email, it would be a little bit awkward if I replied casually there.
Jennifer van Dam: We also created a model that looks at the context of your communication and your audience, and harmonize that with your voice preference so it doesn’t diverge too much, but lands somewhere in the middle. This was an awesome, awesome project and Bhava is going to do a much deeper dive into replies after this.
Jennifer van Dam: Looking back, this was a huge product, and when I reflect back on what were the things that made it successful, I think first of all the team was really, really important when we started this project because building in a zero to one, very high ambiguity, it’s not for everyone. It can be quite chaotic.
Jennifer van Dam: We started with a very small team that was comfortable with iterations and ambiguity and okay throwing away work for the sake of learning. We intentionally kept this team very, very small until we resolved the main ambiguities, we started to scale up the team a little bit or slowly.
Jennifer van Dam: We were very intentional about the initial zero to one stage, and then scaling the team, and we had a lot of high alignment and energy because of this, because the people on the team were excited about these problems.
Jennifer van Dam: We also learned that prototypes are huge to align leadership, because it’s easy to get stuck in discussions, discussing strategy or design flows, but there’s nothing like proving it with real concepts and real user feedback and real prototypes. And then also our transparency principle really helped. We had a ton of cross-functional collaborators and of course it’s inevitable zero to one, there’s going to be all these changes and all these teams are relying on you.
Jennifer van Dam: We were super, super transparent with changes and reasoning and this really helped us creatively problem solve. In the case when there were changes, we would come together and this basically set up a place for innovation with cross-functional collaboration as well.
Jennifer van Dam: What’s next for Grammarly? At Grammarly, we believe that AI is here to augment your intelligence. That is really our product philosophy. We believe that AI is not here to take over your life or dictate you, but it’s here as a superpower, to help you communicate more effectively.
Jennifer van Dam: This is the product philosophy we’ve taken building GrammarlyGO, and this is our philosophy with all our next products and features that we’ll be launching. I can’t share too much about it, but I can share that this is the philosophy we take in building the next features that we’ll be releasing. Thank you. Alright, next up is Bhavana who’s going to be talking about the fascinating project called Quick Replies.
Grammarly Machine Learning Engineer Bhavana Ramachandra talks about engineering the generative AI product GrammarlyGO at Grammarly Girl Geek Dinner. (Watch on YouTube)
Bhavana Ramachandra: Thanks Jen for that awesome overview. Jen spoke about how Grammarly expanded into the user’s writing journey, and we’re going to take a small detour into one of the features that we built, which was Quick Reply, or replying quickly to emails. My name is Bhavana, I’m an ML engineer at Grammarly. I’ve been here for about three years. I was one of the many engineering geeks on this project. There are a couple of folks here in the audience today – Jenny’s here, Yichen is here, yeah, wanted to give a shout out to the team.
Bhavana Ramachandra: Today, I’ll be really talking about foundations in motion and with respect to the quick reply feature. Jen touched upon this previously. We have invested quite a bit in understanding what our users want in terms of their writing, and we were looking at expanding into this user journey, so really we built a lot of fundamental understanding over the years that helped us accelerate into the new product areas that we wanted to go into.
Bhavana Ramachandra: Another shout out is that, the team that worked on Quick Reply, all the point people, all the cross-functional people, were women. We have an analytical linguist, computational linguist, ML engineer, senior PM, and interestingly this has not been my first project when we were all women but yeah. Four foundations deriving from two projects that I have worked on coming into this one. One was Tone. Jen mentioned she has been working on Tone as well.
Bhavana Ramachandra: I’ve been here for three years. She’s been here for three years. Tone was our first project together, so Tone was one of them as well as Recap, which is our investment in 2022 to really go beyond the writing phase and to also to the reading phase to help users read faster so that we can help them write better. With the respect to tone, this was the first version of this. We also have tone rewrites, but this one helps users identify the top three tones in their text so that they can reflect on if that’s exactly how they want to sound.
Bhavana Ramachandra: Zooming into what was the fundamental understanding we built in each of these projects. The first one I’ll cover is Tone, and the three areas we invested is product definition, quality and our AI. For product definition, some of you might be thinking like, ‘Hey, this sounds like sentiment analysis and that is a pretty well solved problem’ but really our product team tries to think about what is the user value of sentiment. If you look at user text, honestly, eight out of 10 times users sound positive. That is not helpful to know.
Bhavana Ramachandra: What the product team did was actually define 50 tones over different aspects of your writing that’s actually helpful for you to know. Do you sound optimistic? Do you sound direct? Do you sound confident? Do you sound worried? Do you sound concerned? The product team really came up with a wide range of tones. In terms of quality, we iterated quite a bit over it and during this phase we actually came up with three levels of hierarchy.
Bhavana Ramachandra: When you have 50 tones, especially if you’re building models for 50 tones, it’s a bit hard – one to get data and to make sure you’re iterating over quality of all of ’em. The way we tackle this is, we define three levels. We have the tone at the really granular level, we have the sentiment at the highest level, but we also came up with tone groups that was maybe around, eight tone groups, and that helped us identify quality at different levels. Then, we really try to nail quality in terms of what is the user values.
Bhavana Ramachandra: Now as an ML engineer, I like to see quality always improving, but is it really worth it to invest in taking one tone from 90 to 92% or is it better for us to improve on a certain tone group that is really valuable to our users?
Bhavana Ramachandra: That’s the kind of trade-off that we had to make and then we really derive over time. I also want to mention our AI is one of our biggest tenants as Heidi mentioned. In this case, this feature was one of the first few pieces to pilot our sensitivity process. The REI manager today was during this process shaping up our formal sensitivity process. We’ve always done it and I think she was making that a very formal process.
Bhavana Ramachandra: Apart from that, we also wanted to make sure that any tone suggestions we make – because we have a varying level of quality, we did want to understand – what are the sensitive cases, and what is our risk of quality with respect to sensitivity. That’s something that we understood during this project as well.
Bhavana Ramachandra: The second one is Recap, which was our comprehension project that we worked on in 2022. Here we were going beyond the writing journey into the reading journey of the user. We invested a lot in understanding the user problem. We had many, many discussions about certain areas that surprised me that I’ll get into. There were also technical challenges because now we needed to again look at the context outside of the text that you’re writing. Where is it? Where are we getting this context from? And then we had a whole new set of ML problems, which is exciting for me.
Bhavana Ramachandra: For the user problem, I wanted to touch on two things. Delight versus value. We wanted to provide summaries and so we identified emails and we wanted to provide summaries as well as to to-do items. But does it really make sense in all use cases? For example, if you have a one line email, it doesn’t make sense to summarize that.
Bhavana Ramachandra: Or, if you have a social promotional email that says, sign up now, that sounds like a task, but all of us know that’s not really a to-do item for any of us. These are the kind of gotchas that we were like, ‘oh, we have a model, but is it actually useful in all cases’ or ‘how long should a summary for a really long email be’ versus ‘a one paragraph email’ be? These are the kind of things that we iterated over quite a bit. And also understanding the context and intent of the user.
Bhavana Ramachandra: Imagine you have an email, an announcement to your entire organization. If you’re a manager versus if you are an engineer versus if you are in design, you might have different takeaways from that email. Trying to understand a bit more of what is that context and what is the intent of the user.
Bhavana Ramachandra: Coming down to the ML problems itself, like I said, there were two things that we were trying to provide – summarization – as well as – task extraction – or to-do list, but because we were talking about delight versus value, context and intent, we also invested in a couple of different areas, including signature detection, intent understanding, and email taxonomy. Trying to understand what is the context of the user – that was more of email taxonomy and intent understanding, and signature detection really helped us. When you look at emails sometimes especially short emails, if the signature is longer than the email, then the model sometimes gets tripped up.
Bhavana Ramachandra: This is true for generative AI as well because yeah, for many different reasons, sometimes models are not perfect, so it helps to help them along the way, and signature detection was one of those areas.
Bhavana Ramachandra: In all of these areas, we spend time annotating our own data sets because email is a space where data sets are not as public, so this was one where we had to understand what data sets existed, what were the things that we were trying to build. As Heidi said, we have a big internal team of analytical linguists, and they help us identify the data, identify what our guidelines are, and go get us annotations that we can build models with, and these were all the areas that we collaborated with them on.
Bhavana Ramachandra: Putting all of that together, from the Tone project, we knew tone was something that our users cared about that we wanted to bring into this feature. You’ll see it says, ‘Jason sounds caring’, but that’s not like models don’t know to think about that. That’s something we have to prompt them to think about.
Bhavana Ramachandra: All of the tone taxonomy that I spoke about, the 50 tones, that made it into the prompt as well. In terms of the recap project, we really built a reply user – like, who are the users? You might get a hundred emails, but you probably reply to 10. What are these reply use cases is something that we had built an understanding that came into this project as well. That really helped us understand quality for launch.
Bhavana Ramachandra: As Jen said, we were not trying to polish, but we were trying to aim for user value. That meant, are we comfortable with the quality for launch? We know that we’re going to iterate over it, but for launch, does this look good? It’s something we try to understand.
Bhavana Ramachandra: Then, on the client side, a lot of the logic, that we built for the earlier project, got repackaged and reused for this one as well. We were using a new protocol, we were using, so it wasn’t just copy paste, but repackage. And then our AI as always, because it’s a generative AI output, we want to be sure that any output that we’re sharing with our users does not have bias, and some high risk scenarios. That’s something as well that we made sure this feature and the output of this feature goes through.
Bhavana Ramachandra: This was one of the few features we built for launch, but it did get a couple of different shoutouts. I know WSJ called out, we had a lot of users sending us, this was awesome. I specifically wanted to, we had a segment on NBC where Courtney Naples, who is our director of language research, spoke about this and the host in fact called out the feature and mentioned how the output of GrammarlyGO sounds like him, versus OpenAI does not. And yeah, that was a really nice moment for us to see. That’s it. Next off, we have Sarah who will be talking to us through all the explorations that the brand design team did for launch as well as our product.
Grammarly Brand Designer Sarah Jacczak talks about designing the generative AI product GrammarlyGO at Grammarly Girl Geek Dinner. (Watch on YouTube)
Sarah Jacczak: Thanks so much, Bhavana. Hi everyone, my name is Sarah and I’m a brand designer at Grammarly. I’ve also worked here for three years and I’m so excited to share the brand design team’s work and show some of the behind the scenes process of the GrammarlyGO launch.
Sarah Jacczak: To start off, I want to intro the go-to-market design team. The team consisted of product brand designers, motion designers, content designers, brand writers, design researchers, and design operations. This was a complex launch and we were designing something completely new, and there were a lot of moving pieces and constant changes. On top of that, we needed to move fast. Having a team with a wide range of expertise, it allowed us to work quickly and collaboratively, and we were able to impact areas across product, brand, and marketing for this launch.
Sarah Jacczak: I want to give a special shout out to the brand and content designers and brand writers. I’ll be sharing some of their incredible work on the GrammarlyGO identity and campaign later on. To give a quick overview of the scope of work, the brand design team worked on in-product systems, a new brand identity that included a new logo and color palette, and a go-to-market campaign toolkit, which included guidance on how to design and write about Grammarly’s generative AI features.
Sarah Jacczak: To do this work, we had to consider how users will interact with this new experience and how we would differentiate Grammarly go from competitors. This required close collaboration with product and engineering teams as well.
Sarah Jacczak: When designing GrammarlyGO, one problem we identified early on was, we needed a way for users to access this new experience. We knew that users were familiar with clicking on the Grammarly icon to open the Assistant Panel and accept writing suggestions, but integrating GrammarlyGO features with this existing UI was not an option for the launch and it was something we would have to address in the future.
Sarah Jacczak: For the launch, we needed to keep these two experiences separate. and we decided to add a second entry point into the Grammarly widget, which would open the GrammarlyGO experience.
Sarah Jacczak: Here are some early explorations of the GrammarlyGO entry point. So on the left we tried two different button designs for the desktop app and browser extension, and we consider it a badge treatment on the desktop app, which has floating widget. The benefit here is that on desktop, the widget wouldn’t be much larger, so it wouldn’t interfere more with text fields.
Sarah Jacczak: However, the visual treatment, it felt kind of like a notification and because of its small size, we were worried it wouldn’t attract much attention. And so we moved on to another exploration. On the right is another exploration where we considered having multiple inline buttons with different icons, so there would be a new unique icon for composed reply and rewrite features, but when prototyping this design, we found that it was a little too cumbersome, and so we decided to simplify it down to one icon for all GrammarlyGO features. And this is what we launched with a single light bulb icon to open the GrammarlyGO assistant window.
Sarah Jacczak: Having one icon as the entry point gave us room to surface prompts that have unique icons. You can see on the example on the right, we have the improve it icon with the pencil, and this prompt appears when a user highlights their text and it gives them a quick and easy way to generate another version of their writing.
Sarah Jacczak: While we were designing how users would access GrammarlyGO, we’re also designing icons. We started exploring icons before we had a name, but we knew it needed to be unique and it would live next to the G icon. We explored a wide variety of approaches. Some were more literal and represented generative AI, like writing and pencils and sparkles and magic, and other explorations were more focused on abstract representations of speed and ideation. But yeah, we could have kept going and going, and this is not even all the explorations, but because of the tight timeline, we had to make a decision.
Sarah Jacczak: We went with the light bulb because we felt it was effective in conveying the new ideation capabilities of GrammarlyGO. We also saw an opportunity to design new product iconography for prompts. These icons would accompany the suggested prompts that appear based on a user’s writing.
Sarah Jacczak: Early testing showed that prompt writing is challenging, and so we prioritize these suggested prompts that are based on a user’s context, and we wanted to make this experience more visual and more delightful. Again, icon explorations range from abstract to literal, but we saw that these icons needed to convey meaning, and also support the prompt compi so we move forward with the literal direction.
Sarah Jacczak: Another discovery was that operating within the new limited color palette was challenging and it didn’t quite feel unified with the existing UI, so we looked to Grammarly’s tone detector iconography, and these emojis, they would appear in the same UI as the prompt icons, so it made sense to create a cohesive experience here.
Sarah Jacczak: We referenced the colors and styling of these emoji to create the foundation for the new prompt icons and here the prompt icons that we designed for launch. You can see they’re literal in that they depict the meaning of the prompt in a simple way, and keeping them simple also ensured that they could scale and be legible at small sizes. We also selected colors and subtle gradients that felt cohesive with the existing emoji icons. This resulted in an icon set that feels warm, friendly, and is hopefully fun to interact with.
Sarah Jacczak: We also needed to consider scalability. There would be hundreds of prompts and we wouldn’t be able to design an icon for each prompt, so we grouped them into categories. Each category has an icon, and within that category are prompts that share that icon. For example, any prompts about writing or composition, we’ll use the pen and paper icon and any prompts about ideation, we’ll use the light bulb and so on. We also identified which prompts we feel would be used frequently and created unique icons for those to add variety and more delight.
Sarah Jacczak: While some of the team was working on iconography and content design in the product, others were working on the identity and go-to-market campaign. Here are some of those explorations – a variety of logos were explored and taglines, as well as graphics for the campaign, and some visual explorations use gradient orbs while others focused on movement and transformation by using over layers of shapes and lines. And for the go-to-market campaign, we created a new tagline as well – ‘go beyond words’. It’s active and it conveys Grammarly’s ability to assist users beyond their writing.
Sarah Jacczak: We also designed a new logo that incorporates a bolder G with a circle forming the O as a nod to the classic Grammarly button. For the GrammarlyGO identity and campaign, the brand design team landed on a concept that uses overlapping shapes to convey transformation and the iterative process where one idea is built on the next. The softness of the gradients also speak to the human qualities, and they’re juxtaposed with hard edges to represent technology, and these overlapping shapes were further brought to life with animation.
Sarah Jacczak: The team also worked on a design toolkit, and this toolkit was shared across the company. The toolkit included logo, color palette, illustrations, photography, motion guidelines, and a library of product examples to be used across the campaign. A style and verbal direction guide was also created to ensure how we speak about GrammarlyGO is consistent. The brand writers provided headline examples based on themes. There was headlines about creativity, such as let your ideas take shape, headlines about productivity, such as ‘discover new ways to get things done’, and headlines about trust like ‘AI innovation with integrity at its center’.
Sarah Jacczak: This campaign was pretty large. We had a lot of requests and a lot of marketing channels to design for, but because the brand and brand writers and brand designers collaborated and built these systems and guidelines, we were able to move quickly and create consistency despite many people working on the campaign production.
Sarah Jacczak: Here are just a few examples of the work created for the campaign. The team created a series of demo videos and animated gifs that show product functionality, and these were used across marketing and PR. The team also worked on onboarding emails, landing pages, in product onboarding, blog posts, ads, and social assets,
Sarah Jacczak: To get a further sense of the scope of work, here’s some numbers from the naming and identity work. Over 500 names were considered, 188 Jira tickets were completed, over 105 taglines were explored, 45 videos explored, 39 product examples designed and animated, and over 230 logos were explored. And so, while these numbers don’t tell the full story, and we had challenges along the way, the team was able to overcome this and collaboratively design a new experience and produce a successful launch in a short amount of time. Thank you.
Charlandra Rachal: Thanks Sarah. To all the speakers that put together this incredible presentation, I learned a lot and I work here, so I hope you guys really enjoyed all of that. Let’s welcome back all of our speakers for Q&A. There is someone who is going to be in the audience with a mic, and I see it first hand already, so we will get a mic right over to you.
Audience Member: As mentioned, it was uncharted territory. I was curious how you went about ideating the first project. Was it based on existing user information you had? Was it academic papers? How’d you go about it?
Jennifer van Dam: That’s referencing my talk, so happy to talk more about it. What I mean with uncharted territory is the solution. We knew it was a problem – we’ve been hearing for years that since we started, we hear from our users, they struggle with these communication problems. What was the uncharted territory is the solution and delivering the product in a way that lends and resonate with our users.
Jennifer van Dam: The approach we decided to take is directly into the prototyping stage because we felt it was really important to connect the text and the product to the user. Let’s say, you want to compose an email, we can design and show you concepts, but we need that moment of you writing your text and seeing the output. That’s why we jumped right into the prototyping stage as our way to research the solutions and the design approach.
Charlandra Rachal: There was another hand right here…
Audience Member: Hello, my name is Kate, and probably question also to Jennifer because it was on one of your slides. When talking about prototyping, you were speaking about more empathy and I took a screenshot. Let me see how it looked like there.
Audience Member: ‘Deeper user empathy.’ Can you please elaborate a little bit more on that, how it worked? How did you do it while you were still prototyping, please?
Jennifer van Dam: Yeah, deep user empathy. What I really meant with that was to understand and dive into the types of things our users are trying to achieve. What are the types of use cases that, here’s a prototype, did you use it for rewrites? Did you use it for emails? What were those things?
Jennifer van Dam: We did so many sessions talking to people and getting their feedback to get empathy and then of course we had questions, but then the feedback we got was, ‘oh, but it didn’t sound like me’. This is what I mean by deep user empathy is really getting into the mindset of empathizing what is working and what is missing. That really helped us inform iterations and changes and new features or scrapping features.
Audience Member: Thank you. I would assume that the launch of ChatGPT definitely affected Grammarly. What would be the key learnings? What were the key learnings for you as product leaders from getting the LLMS viral and thanks for the presentations. My name is Maria.
Heidi Williams: One of the things which I think was interesting – we’ve been doing AI for a long time, we’ve used a lot of different technologies, whether it’s rule-based or machine learning, or all sorts of different technologies, exploring LLMs on our own.
Heidi Williams: The biggest thing about ChatGPT that ended up, was actually the discussion in the world about how to use AI as an augmentation tool. Before, you would have to convince people AI was okay and trusted and then all of a sudden overnight everyone’s like, ‘of course you trust it. Look at this’.
Heidi Williams: Now all of a sudden we don’t have to waste time talking about should you use AI? Now it’s about how can we be a trusted partner on how best to use AI and help you be more effective and help you succeed in your job or in your life. It has changed the conversation of ‘should I’ to ‘how should I’, and that’s been interesting and amazing that we can now focus on just solving real problems as opposed to convincing people they have a problem that AI can help with.
Audience Member: Thank you so much. First of all, want to say huge shout out for Girls Geek and Grammarly for putting together. Thank you. That’s a great event. I’m a huge fan of Grammarly. Go now. It finally sounds like Shakespeare in my emails and not like a broken machine. I want to ask you this question. I think Heidi and Bhavana, that’s questions for you. You said that one of the feedback was, ‘it doesn’t sound like me’.
Audience Member: Are you using LLMs to train data which your users are input? And if so, how do you also prevent some data security in terms of, for example, I’m putting something in my email as a product manager about revenue or about some specific of the product, which haven’t been on the market, and I’m always a little bit worried where this data is coming from. Yeah, I think it’s a good question. One is LLMS for make me sounds like it’s me. And the second one is the data privacy. Thank you.
Heidi Williams: I can talk about at least part of it and then if you have things to add as well. Because Grammarly has been around for so long, and that we are a trusted source, we were able to negotiate a really amazing contract with our LLM provider, which means that they don’t train on any data that we send to them. Not everybody could negotiate that, but because we’re Grammarly and we’ve been around so long and have such a big user base, we were able to do that.
Heidi Williams: I feel like that was a huge thing that is very differentiated from just using whatever’s on the market is that they’re not training on any of the data that we send. From that perspective of the data privacy, but if you want to talk more about the my voice and about how we do that and how do we then, if either of you want to add…
Bhavana Ramachandra: I’m going to pass to Jen.
Jennifer van Dam: Could you repeat the question?
Audience Member: Sure. How you make, if we’re not sending data to any LLMs, how you make it sound more like me, for example, GrammarlyGO, always suggesting me to be more assertive, which I think I’m already too much and then I’m like, no, no, no, let’s make it more positive. Yeah, how this happened, how it sounds more like me is like data being trained.
Jennifer van Dam: We look at context and communication patterns, so it doesn’t necessarily train on your data per se, but on the patterns of your communication and the context. That’s how we understand your voice profile across.
Bhavana Ramachandra: To add to that – we understand what your tones you prefer or you use are, but we don’t actually pass that on to LLMs. We had our tone detectors since 2019. We’ve been telling users how they sound for a bit now. We use that information to really update the writing rather than train the LLMs itself with your data.
Charlandra Rachal: I feel like I’ve been neglecting this side over here, so right here in the front.
Audience Member: More of a quick technical question. Do you list all of the tones that you detect for and measure somewhere publicly, or is that behind closed doors?
Jennifer van Dam: We have a homepage that lists a lot, but not all, of our tones. We feel it’s too competitive to reveal 50 plus. But yeah, you definitely can find information about a lot of our tones that we support with tone detection.
Bhavana Ramachandra: Maybe this is a challenge. Can you write enough with Grammarly to find all of them?
Audience Member: The tables have turned. I wanted to direct a question to the second speaker after Jennifer van Dam. Yes, you, um, why does my voice sound like this? Ah ha ha I see what you did there. The question I have more specifically was, when tone is being cited as a suggestion, when you write a sentence and it connotes that, ‘oh, your tone is serious and neutral’, and when you add a word or two and it changes the tone entirely, I’m curious, what quantitative scales do you use behind the scenes to make those on the spot judgements? You mentioned your team had a lot of linguists on it, and I was hoping you could expand on that because that has been an object of curiosity of mine for a while, possibly. Thank you.
Bhavana Ramachandra: Yeah, I’m just going to Jen about… Yeah, I think for, in terms of how do we decide, in fact, when we started looking at rewriting for tone, I think our initial exploration had just neutral and new tone versus in certain cases we were actually able to provide three levels, friend, friendlier, friendliest, but really depended on how much data we’d have. If you can actually, you can be neutral, but you can’t be more neutral. It really depended on the tone and how much data we have.
Bhavana Ramachandra: I think this is a part where our linguist really helped us really dive deep into this and look into each of the tone that we have, get data for each one of these tones. I think for our tone retype explorations, we started with the tones where we had most understanding and first started with two levels and then moved on to three.
Audience Member: Thank you. Hi, my name is Leanne and big fan of Grammarly. My question for whoever thinks that they’re best equipped to answer this is, could you tell us a bit more about how Grammarly is fighting bias, and what are some of those solutions that are currently in place? And maybe thinking about, in the future roadmap?
Bhavana Ramachandra: Yeah. Our AI is one of our biggest tenants, as Heidi said, and we’ve always invested in this area. The couple of things that we have done are actually very, very public. We have blog posts about how we look at pronouns or bias in gender bias in data, and how do we make sure our generative AI suggestions, how do we measure that and how do we prevent that as something that we do? And as Heidi said, this is part of the process.
Bhavana Ramachandra: This is not something that you think about at the end of the day, you plan for this. You plan to have a sensitivity analysis right from the get-go. The other part of this, we’ve published a couple of different papers this year. In fact, in the REI space in, I want to say ACL. Okay, thank you Dana. You’ll actually find a lot of public information. I don’t want to pretend that I know more than I do in this area. I am getting onboarded though. Definitely, we have blogs and papers out there that talk about what are the solutions we have implemented.
Heidi Williams: Maybe just one thing to add to that is part of it is actually cultivating a good data set because you could imagine that you just take, I think we’ve seen this with LLMs as well, you just take the words out there and you might see a bias of a gender bias around when you’re referring to a male, they might be more associated with certain words in the general public than a female. Then you would imagine that percentage wise, it might suggest like, oh, if you’re talking about a man, you must be referring to this, et cetera.
Heidi Williams: We’ve done a good job of cultivating our data sets to help ensure that the data sets themselves are not biased, and that’s a huge aspect of it is just making sure that we’re not having any gender weighting as one example, or it could be racial, whatever it is. There’s just making sure that you have a data set that’s representative and it’s not going to sort of skew things in one direction or another.
Bhavana Ramachandra: Do you want to add to that?
Jennifer van Dam: I wanted to add to that a little bit about how much we care about this investment. We also, besides of course all the deep investments in the modeling, we also have inclusive language suggestions for end users that help basically eliminate gender bias in your language while writing or talking to your coworkers or your team.
Jennifer van Dam:This is an area I also worked on and it’s a really great part of our product. For example, maybe you’re writing, ‘the businessmen are wearing suits’, we’ll underline ‘businessmen’ and we’ll ask if you’re writing to an audience that you want to be inclusive of everyone, maybe replace it with ‘business people’ rather than ‘businessmen’. We also tackled this from the end user standpoint and helping them communicate more inclusively and eliminate bias where they’d like.
Audience Member: <inaudible>
Charlandra Rachal: For those who didn’t hear, the question was, how much do we focus on educating, when people continue to make mistakes in their writing?
Jennifer van Dam: The inclusive language product, I encourage you all to check out because education was a huge part of the product UI and the way we wanted to position it. We always want to be educational rather than forcing you because at the end of the day, the user is in control. You have agency. That’s what we always believe in. We also realized we have to explain why are we saying ‘consider replacing businessmen with business people?’ How do we explain? Because some people don’t realize that it’s so ingrained, you just type it, you don’t really stand still.
Jennifer van Dam: Another example, like whitelist, blacklist, that suggestion, a lot of users didn’t understand why we were suggesting to replace blacklist with blocklist, so we actually focused our UI around education. Why is blacklist/whitelist is perpetuating certain stereotypes, so consider replacing it – and that was a real aha moment because when you say that in a training, it’s different than when in real life you’re writing a text and seeing it and applying it, so it’s actually been really powerful in educating people
Audience Member: To Jennifer and Bhavana. Earlier you mentioned deploying LLM models, initially, were you were skeptical how receptive the users would be and how they would perform.
Audience Member:Can you talk about your A/B testing strategies? Did you roll out to a part of your audience, I mean part of the user base first, and then started gradually increasing, rolling out the new features too, especially the generative AI features? Could you talk about your A/B testing strategy and how did you scale it to the whole user base? And after you employed gen-AI features and these new features that you earlier talked about, how did it impact the revenue subscription revenue and the user base?
Heidi Williams: Start?
Bhavana Ramachandra: I think Jen covered maybe some of the A/B tests. I’ll let her, I can maybe speak to the launch plans, the alpha testing. I can cover that a bit. Especially for projects like Tone where we did iterate over quality quite a bit, we would try to identify one. We had internal annotations. Every time we improve our quality, we do more internal annotations to understand how much of a bump is it? And once we have a fair understanding, we run experiments with Gen AI, we had to take a slightly different process.
Bhavana Ramachandra:As Jen said, we had more alpha testing with users, really deep conversations in terms of understanding what is a useful generative AI LLM feature because we’ve had rewrite features in terms of generative AI for the longest time, but what’s a useful composed feature? What’s a useful quick reply feature? All of that was not really A/B testing. We were building understanding in this case. That was a lot more alpha testing.
Bhavana Ramachandra: Then for the launch plan itself, we have 30 million users, we have five different surfaces, we have extension, we have desktop app, we have an editor, we have our website, and we’re in many different countries, so this to me firstly was the most impressive one, because we had to do a geo launch across many different clients that all have different release cycles, so all of them had to be in sync because we wanted feature parity. We started with certain countries to make sure, one, we can handle the traffic. Two, all the features are looking or performing as they should.
Jennifer van Dam: The difference with how we approach modeling quality, it depends on the maturity of the track. In the zero to one stage we do a lot of offline quality evaluations and make sure it meets our quality bar and the metrics. We don’t necessarily test out multiple models yet, but in the iteration stage we do. One example where we’ve A/B tested our improve it rewrite, which in one click will improve your text, and there was a lot of experimentation we did there with tone behind it and conciseness and what lands the best with improving my text with one click. Typically, we focus a lot on offline quality evaluation of our models, and then in the iteration stages we do a lot of A/B modeling.
Audience Member: After new features, did you see any bump in the overall user base?
Heidi Williams: I can’t talk about specific numbers, but I think obviously there was a lot of excitement and interest in this area. I think we did see that there was new interest, and then also just seeing interest and engagement from our existing users using the product maybe in a different pattern than they had been before as well. It definitely feels like there have been changes, but I can’t speak about specific numbers.
Audience Member: What’s your tech stack typically for the whole GrammarlyGO is hosted on?
Heidi Williams: The tech stack? A lot of different parts of it… it’s hard to answer with a really quick answer. Our particular LLMM provider is Azure OpenAI. And then there’s a variety of tech stacks above that – different things that we’re using for the linguistic side of things, and then there’s Java, there’s Closure, there’s all sorts of different technology stacks and then we run on AWS otherwise.
Charlandra Rachal: Thank you. Then I only have time. Oh, oh, oh. I was coming for you. I know you had your hand out. If you still wanted to answer, we would definitely break the mic over your way. The last I have one time for one more question. Alright, Nancy
Audience Member: Oh hi. Alright. I’ve used Grammarly for a really long time and this may be more of a product manager question because I’m also, I can write circles around everybody, so I don’t really need GrammarlyGO. I’m actually wondering, I’m thinking about, the roadmap further down for advanced writers, people like me who write. What’s coming?
Audience Member: Because I will say now, I use Claude a lot just to be like, Hey Claude, this is what I wrote. What do you think? And then Claude will say, that’s really good or not or whatever. I’m just wondering it’s GrammarlyGO moving in that direction for people who don’t really need help getting stuff on paper or on screen.
Bhavana Ramachandra: These are the comprehension projects that I was talking about. They’re all about trying to understand what the user is reading or what the user has written. For example, tone is something, even if it’s not correction, if it’s not editorial, you still might want to understand how your tone is coming across, especially in cross-cultural communication.
Bhavana Ramachandra: That’s something that’s helpful and in general as well, especially for long writing, we’ve gotten a lot of feedback about, I think this is one area that we were investing in – how we, so we show top three tones and let’s say people use Grammarly to write books or their fictional books, and does it make sense to show top three tones? Then they want a different – so this is the kind of evolution of the features that we see.
Bhavana Ramachandra: Comprehension is one area. In our generative AI in GrammarlyGO, if you actually open it up in a document, we provide a lot of prompts around understanding the gaps in your document, identifying what are your main points. All of these are just comprehension. This is just not how to improve your writing. Rather like this is what’s there in your document. You can review it based on a couple of different dimensions.
Audience Member: I should use Claude and Grammarly. Yes.
Bhavana Ramachandra: That’s the answer.
Charlandra Rachal: Yes. Alright, I wanted to say thank you so much to all of the speakers here and all you wonderful guests. I’m going to give a shameless plug if you didn’t already see, I’m in recruiting and we are hiring! Definitely talk to us, talk to me. I know we’re going to send a link out as well.
Charlandra Rachal: I believe there are more refreshments in the back and everyone is welcome to kind of hang out, chat, network. If you have more questions, I feel like we got through a lot of them without telling all of our secrets, but feel free to pull them aside and ask more questions. I hope you have a great night. Thanks again for coming out.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Table Topics: Engineering Leadership, Negotiation, Transitioning to a Manager Role, 1:1s, Career Management, Career Transitions, Management & Leadership, Promotion, Breadth vs Depth,Networking, Managing One’s Career
Table #2 – Eng Growth As Individual Contributor (IC) – Mentors:
Table Topics: Making Technology Platform Switches, Mobile Platform, Growing On IC Career Track, A/B Testing & Experimentation, Front End Engineering (Angular / React), How To Tech Lead Efficiently, Handling Conflict, How To Work Cross Functionally, Backend, Web Development, Neurodivergence, Self Care
Table Topics: Product Leadership, Transition to Product Management, Time Management, Stakeholder Management, Product Management, Career Strategy, Career Planning, Interview Preparation, Positioning for Success, Executive Presence, Career Transitions, Layoffs
Table Topics: Legal & Tech, Returning To Higher Education, Mental Health, DEI, Ops, Emotional Regulation, Communication Skills, Startups, Product Design, Design Careers With Non-Design Background, Latinas in Tech, Sales, Marketing, Community, Customer
Non-Coding Roles in Tech Mentors: Jenny Jennings (Commercial Counsel, Twilio), Lucy Chen (Head of Education & Fulfillment, Curious Cardinals), Olivia Ouyang (Product Designer, Finix), Paola Johnson (Director, Community & Customer Advocacy, ThoughtSpot)
Table #5 – Career Development / Promotion – Mentors:
Table Topics: Navigating Workplace Environments, Asking For Promotions, Technical Topics, Career Pivots, Negotiation, Building Your Brand, Elevating The Narrative, Storytelling, Career Trajectory, Negotiating Salary / Promotions, Career Progression, Startup Growth / Sales
Table #6 – Leadership / Building Good Networks – Mentors:
Table Topics: Project / Program / Portfolio Management, Building Good Networks, Glass Ceilings / Cliffs, Building Trust With Your Manager, Career Planning, Promotion, Networking, Career Transitions, Leadership, Executive Presence, Office Politics, Continuous Education, Supporting Teammates, Inclusivity, Embracing Technology
Leadership / Building Good Networks Mentors: Gayathri Kamath (Staff Program Manager, Fastly), Karen Ko (Managing Director, WEST Diversity & Inclusion),Madhavi Basin (VP, Diversity, Inclusion & Belonging, Okta), Nahla El Helbawi(Asst. Director Web Academic Content Management, American University in Cairo)
Table #7 – Managing Your Career – Mentors:
Table Topics: Communications, Early Career Exploration, Taking Chances, Decision Making, E-Commerce, Digital Media, Networking / Growing Your Career, Identifying Mentors & Sponsors, Tips & Traits For a Good Leader / Manager, Career Development, Building Skills, Management & Leadership, How To Overcome Blockers.
Table Topics: Career Pivots, Cybersecurity, Career Paths, Management, Owning & Using Your Diversity, How To Toot Your Own Horn, Networking, Landing a Dream Job, Troubleshooting, Startup Life, Stepping Into Leadership Roles
Table #10 – Career Search / Interviewing – Mentors:
Table Topics: Career Development, Technical Interviewing, Resume & Job Application, Web Services & Backend Engineering, Game Development, Individual Contributor vs. Management Track, Finding a Mentor, Career Switching, Career Transitions, Interviewing, Edtech, Tech, UX, UX Research, Career Search Tips (AI Tools)
The virtual Mentorship Lounge will be open for one hour during ELEVATE, 8AM – 9AM PDT, on September 6th, 2023. The 10 tables will each be open to up to 30 participants. Attendees can hop between tables freely throughout the hour, so you’ll have the opportunity to meet as many of our 40 Mentors as you’d like! (Camera on or off, your call!)
Plus network with fellow attendees, meet with recruiters & hiring managers in the virtual recruiting booths, attend over a dozen tech & career-focused lightning talks from girl geeks working at companies like Amazon, Microsoft, Guild, Google, Lighthouse Labs, and more! Get your FREE pass today!
Our goal began with supporting students at Coliseum College Prep Academy in Oakland, California teaching grades 6-12 with a computer science pathway. We providing access to volunteers and role models from the professional community for students in partnership with the nonprofit Oakland Education Fund, which coordinates volunteer activities with public schools in Oakland and clears volunteers for entry into the schools.
Our Girl Geek X Community volunteers helped teachers with classroom projects to prepare their rooms and hallways for students return to campus for the new school year.
Upcoming events are for October & November 2023 – Please sign up to volunteer! Thank you in advance.
VOLUNTEER WITH STUDENTSIN OAKLAND – OCTOBER2
The nonprofit Oakland Education Fund is expanding our access to students in Oakland elementary schools, starting with volunteering with Latine/x Read-in (Monday, October 2, 2023, 1pm – 2:30pm).
Note: you don’t have to speak Spanish to volunteer at the read-in — we will be reading books by Latine/x and Hispanic authors to students.
Volunteers will read books to 2-3 elementary school classes that celebrate Latine/x culture in the 90-minute volunteer shift. Books and sample questions to guide conversations are provided by the Oakland Education Fund.
Volunteers do not need to identify as Latine/x to participate, and those who do identify as such are encouraged to participate and share about their culture with students.
VOLUNTEER WITH STUDENTSIN OAKLAND– NOVEMBER 3, NOVEMBER 17
Volunteers will be matched with public school seniors to provide crucial feedback on grammar, flow, and clarity of writing for college applications. These “College Crunch Days” are dedicated school days for high school seniors to work on their UC admissions applications.
Note: a writing/comms background is NOT required to participate! Any experience writing in an academic/professional setting will be sufficient to participate in this event. Volunteers will be given the writing prompts ahead of time and samples of responses.
Volunteer shifts are 8:30am-12:30pm on Friday, November 3, 2023 and/or Friday, November 17, 2023 at CCPA’s College & Career Day office.