Angie Chang: I’m the founder of Girl Geek X. With us today we have Rachel Rogers, who is the vice president of product marketing at Bentley Systems. She’s been at Bentley Systems for over a decade. Prior to Bentley was a director at Autodesk and Intergraph, where she began her career as a marketing writer. She has her BSBA from the University of Alabama and Huntsville. We have with us today as well, Natalie Plummer, the director of diversity, equity, and inclusion at Bentley Systems, who will be moderating the Q&A. Please pop your questions into the chat, Rachel and Natalie would love to answer them. Welcome, Rachel.
Rachel Rogers: Thank you so much, Angie. Good morning from sunny San Francisco Bay Area. It’s actually going to be a beautiful day today, and I’m excited to be here, and to share some of the best advice I’ve received. As Angie just said, I’ve been in this career for the last 30 years, over three decades, and I would like to share some of the best advice, and some of the things that I’ve learned myself through trial and error over those last three decades.
Rachel Rogers: As Angie just mentioned, to share a little bit about myself, I started my career at Intergraph in Huntsville, Alabama in the nineties as a marketing writer. Then I worked in the AEC division, which supported architecture, engineering, construction. Needless to say, at that time it was definitely a male-dominated team. In the mid two thousands, I moved across country from Huntsville to the Bay Area and worked at another CAD company, Autodesk, but this time on the sales side, to really help balance out my career and learn a different aspect of the business.
Rachel Rogers: This time I was one of a handful of women on the team that supported the infrastructure division and software industry. 11 years ago I joined Bentley, the third leading infrastructure software company in the world. We’re not the third, we’re one of the major ones, but I’ve always been in infrastructure. And I’ve worked from home for the last decade leading a global team, and I’m happy to report that now I work with so many talented, exceptional women, but we’re still in a male-dominated tech field. All of these adventures over the last 30 years have really led me to where I’m today.
Rachel Rogers: I hope that today I can share some of the insight that I’ve learned that’s going to be helpful for you. I’ve narrowed it down because, trust me, I’ve learned a lot in those three decades. I’ve narrowed it down, all that plethora of advice that I’ve received to what I think is the top five. Let’s get started with the countdown.
Rachel Rogers: Number five, or actually this could be number one too, but number five, life is not fair. I grew up in the deep south with a brother that was two years older than me. At that time, boys had the freedom to do almost anything they wanted, whereas girls did not. They could play sports, ride their bikes around the neighborhoods, stay out all day, disappear for hours without anybody know, avoid housework. You name it, they could get involved with it.
Rachel Rogers: I would often complain to my mother, “That is not fair.” And she would always say, “Who said life was fair?” It really pissed me off every single time, but she was right. As we always say, mothers are always right in the long run.
Rachel Rogers: Life is not fair, but even as adults, we can have a hard time not looking at others and not thinking that life isn’t fair. We have a tendency to focus on what’s happening to other people around us instead of focusing on how we can grow ourselves and our own careers.
Rachel Rogers: Even today I often have, and throughout my management, I’ve always had conversations with team members discussing why other colleagues were promoted instead of them, or how was somebody else chosen to do that project instead of me? Why does that person have a higher title than I do when I have more years on the job? Or the title is different.
Rachel Rogers: It’s really easy to get caught up into that. Why not me? What is it about me? We cannot control what happens to other people unless we’re in charge of making those decisions. Then often, even when we are in charge of making them, we’re still doing things on the recommendations of the people above us.
Rachel Rogers: If you spend your time worrying about your colleague’s projects, their opportunities, their career instead of yourself and your own professional growth in career, chances are you’re always going to be frustrated. Now, I’m not saying it’s easy. I’m one of the first ones that have learned this lesson.
Rachel Rogers: A lot of things are out of our control in the corporate world. Things like teams being reorganized around you, other people being promoted, not knowing about new opportunities. It’s happened to me many times, and I know I’ve reacted badly several times myself, but the best advice that I’ve received over the years has been that opportunities may not happen for you at the same pace as others.
Rachel Rogers: We each have our own unique path, and that’s the only thing that we can influence. And once you realize that you can only influence what you can control, you’re in a better place. Like how you react to things going on around you, how you advocate for yourself, and how you focus on your own career to grow your own successes. Life may not be fair, but it can be a lot easier to deal with when we focus on ourselves and not others.
Rachel Rogers: Number four, because we’re going to do kind of a David Letterman count. So number four, find your voice. I just mentioned that you need to advocate for yourself to support your own career and find your own successes. Finding your voice can be really challenging as a woman in a male-dominated industry. Or even as a person that may not like to speak in front of others, or that may not be something they’re comfortable with.
Rachel Rogers: For many years, I was the only woman on all-male teams. Remember all those engineers I just talked about? I was also one of the youngest, most times. So I would get really frustrated over the fact that guys just simply had no problems just talking over me in meetings, not listening to my ideas, just shrugging it off. It truly was like that FedEx commercial from years ago where it’s an office setting.
Rachel Rogers: A woman says, “We can use FedEx for shipping to save money in our budget.” And no one listens to her. And then a man says the exact same thing and everybody says, “Oh my gosh, that’s an incredible idea.” They heard him, they didn’t hear her.
Rachel Rogers: It’s not as funny in real life as it is in that FedEx commercial. Not that it really was funny. But men are comfortable leading conversations and they really usually don’t have a problem interrupting. Whereas women are much less prone to doing so. We’re less prone to leading it, we’re less prone to interrupting conversation. We sit back and wait to be included, oftentimes.
Rachel Rogers: You can’t wait for someone to ask you for your opinion or share your ideas. You have to speak up. You have to find your voice. You have to learn to share your ideas and your opinions, and really establish yourself as a valuable member of the team.
Rachel Rogers: Once you build your own credibility and showcase your skills and knowledge, then people around you are going to start listening more. They’re going to start asking questions of you. And they’re going to want your opinion, and they’re going to see your value, so not only are you building your own confidence in what you have to offer, but you’re also building a really strong brand for yourself.
Rachel Rogers: Make sure that you share your successes, your wins, and not only to your team, but to your manager. It’s the only way to make sure that your contributions are going to be known across the organization. You’ve got to speak for yourself so that you know you’re being seen and heard.
Rachel Rogers: You’ve also got to let people know, you need to make sure that your managers and others know what your wishes are for your career. You’re responsible for your own career growth. Let’s make sure that people know what your wins are, and make sure people know, especially your management, where you want to go with your career. Advocate for yourself, find your own voice.
Rachel Rogers: Number three, building relationships. At the beginning of my career, I used to think that I would simply grow my career based on all my hard work, my successful projects, my proven leadership, my other contributions. I was really naive, because, hopefully this isn’t too old of a phrase, but if it takes a village to raise a child, it really takes a network in support of others to help your career.
Rachel Rogers: You need a team of people inside your own organization helping support you, and helping find opportunities for you to shine, to grow, to build your own career. It’s difficult to do that just by yourself as a solo-person. You need to have that network supporting you, but building a network doesn’t just happen. I mean, you really have to invest the time and effort into connecting with people and nurturing your network. It’s difficult. It takes time.
Rachel Rogers: Your network, it can include a lot of people, like your current colleagues, past colleagues, managers, even managers from other companies you continue to stay connected to, industry groups that we can connect to, and a variety of social platforms now, alumni, other online connections. All these things are really important. These groups of people can really help pull together to help progress your career, let you know about new career opportunities within your company.
Rachel Rogers: They can help be influential in your own promotions or opportunities, new opportunities for you. And really help provide insight and support when you need it. Remember to invest the time in building them. It’s really easy to get caught up in, and I’m guilty of this as well. We get caught up in our daily job, all the urgent tasks, but we need to find the time to build those relationships so we have a support system in place when we need it.
Rachel Rogers: Remember, build your network, but just as another critical, find a mentor or a coach to help guide you. Look around your own organization, reach out to somebody you admire and ask if they have time to mentor you. I found that most people really are willing to help and invest time into giving back. I know that I really enjoy mentoring others and giving back, and for the most part others do as well. Find a mentor, it’s really going to be helpful in your career.
Rachel Rogers: Number two, and one of my personal mantras, is trust your instincts. Listen to your gut or your intuition. It’s telling you the right thing to do. Another lesson I’ve learned over the years is when I ignore that voice inside of me, things do not go well. I have so many examples of ignoring my intuition, and trusting other people’s opinion, and it not working out well for me or for my team.
Rachel Rogers: Perfect example is, years ago, we needed to hire a new product manager for the team. And even though the position was several layers below mine, I always like to interview candidates at the end to make sure that not only do they have the skills, but will they be a good fit for the overall team?
Rachel Rogers: Several colleagues and the manager interviewed the candidate. Everybody thought they had the right skills, they had the knowledge, they had the experience in the industry, they thought they would be really successful in the position. That person bubbled up to the top. I talked to them, but after I interviewed them, I really didn’t think that they were going to be a good fit personality-wise, and I thought that it could have a negative impact over the overall health of the team, but I wanted to support the manager and the team that interviewed them, so I ignored my gut and we made an offer and hired the person.
Rachel Rogers: Well, six months later, it was clear that the new colleague was very disruptive to the team. Production was down, frustration was really high, and then I had to handle the situation. It could have all been avoided if I just trusted my instinct. That little voice inside of you knows what’s best for you, learn to listen to it and to trust yourself. You know what’s best for you.
Rachel Rogers: Number one, and the most important of all, know that you can do anything. After you’ve learned to trust yourself, your intuition, you need to realize that you can do anything. Early in my career when I doubted myself, I would tell myself over and over, you can do anything. It was my mantra. I wrote it down in meetings when I was unsure of myself. I would write it down in the notes I was taking, or doodling.
Rachel Rogers: When I would feel overwhelmed by new projects, or opportunities, or having to build new teams or new companies or all the things that you’re asked to do. When you start feeling intimidated and overwhelmed by new challenges, just tell yourself, I can do anything. Have faith and belief in yourself that that’s the first step to your success, is knowing that you can do it.
Rachel Rogers: It’s really easy to get caught up in that noise, that others are smarter than you, they’re more educated, they’re more whatever, but knowing that you can do anything helps put you at ease. It gives you that confidence. It’s your greatest gift to know that you’re in charge of your life, and you can accomplish anything. Keep telling yourself that over and over until you believe it, because you can. You can do anything.
Rachel Rogers: Don’t forget. Focus on what you can control. Advocate for yourself. Build that network for support. Trust yourself and the intuition, and keep telling yourself you can do anything.
Rachel Rogers: With that, I’d like to welcome Natalie Plummer. She’s our director of diversity, equity, and inclusion to join me. And we’ve got a couple of minutes that we can ask any questions that we may have.
Natalie Plummer: I think I’m just going to go through some of the things, the questions and some of the points you raised. People can continue to put questions in the chat. I’m kind of monitoring the chat to see any questions.
Natalie Plummer: Here’s a question I want to ask you. One of the things that you said is people need to build a network. Here’s my question and it has a bunch of sub-parts because I could pick your brain all day. How do you build that network? Is it a mentor? Is it through an advocate?
Natalie Plummer: There’s a difference, and does it matter whether that advocate or mentor is a man or a woman? And does it matter if they’re even in your field?
Rachel Rogers: Oh, that is a great question. A, I don’t think it does. Many good things. I do not think it does matter that it’s in your field. Honestly, think it’s better to not be in your field because you get a different perspective. Does it matter if it’s a male or a female? Absolutely not. What matters to me, it’s somebody you admire, that you will listen to, that they, you know, will have insight. So they’ve been through things. They can coach you, you can help learn from their wisdom.
Rachel Rogers: I do not think that you have to have a woman to help you. I mean, it’s great because they can understand a lot of things, but I’ve had a lot of great male mentors. It helps me. Of course I to, I didn’t have that many women around the first 15, 20 years, so, I did. I had a lot of great guys that helped advocate for my career and helped do things. I don’t think that that matters. I think it’s whoever you’re comfortable with.
Rachel Rogers: I do want to go back, and I think you said, what’s the difference? Your network, totally different from your coach or your mentor. And by the way, an executive coach, a great investment, one of the best things I’ve ever done. I think a mentor is great. That network is, the people that you’ve worked with, that you build, they’re all over so they can help you. Whether they’re in your current company or in other companies. Because those are the ones you’re going to look to when, maybe it’s time for you to start a new career.
Rachel Rogers: Maybe you’re looking for something else. Maybe you just want a new opportunity within your same company, but you don’t know where to start. You want to broaden, like I did. I went from always being in marketing to being to the sales side. Maybe you want to broaden your experience. Having that network will let you know new opportunities and help advise you and what’s the best direction to go,
Natalie Plummer: Then logistically, how do you go about doing it? Is it something you carve out time for at the end of your day every day? Is it a weekly event? Is it monthly? Is it going to networking events once a week? Is it going on LinkedIn and seeing people you admire, and reaching out from them? Logistically, how do you recommend it when you have a busy day and you have your life afterwards?
Rachel Rogers: I’ll have to say, I’m not the best at it. Because as I said, this was something that I learned. I was very naïve. I thought for years and years that everything’s just going to happen based on my contributions.
Rachel Rogers: I do think, yes, you have to make time for it. You do need to schedule as a part of your day. I mean the LinkedIn, the social, doing those kinds of things. Obviously you can do that after hours and be able to build your own network, but I think building your network within the company, absolutely.
Rachel Rogers: You need to schedule meetings with people. You need to ask if they have the time to chitchat with you, if they need to ask for their advice. People appreciate that too. I think making those connections, and building them and nurturing them, just like friendships, you have to nurture relationships with people.
Rachel Rogers: Having a relationship at work is not unlike having a relationship with friends, that you need to put the time in and nurture that relationship. I’m guilty of not doing that as well. We all get busy in all of our demands, and our tasks, and our deadlines. But we really need to do it. It’s going to make you feel better as well. You need that personal time. You need time to connect with people, and not just heads down doing your job.
Natalie Plummer: It’s almost part of your job, as far as just expanding your network. That’s part of it as well.
Rachel Rogers: It’s definitely part of your own personal growth. As we talked about before, one of the things that you really need to learn, and it’s hard to learn is that you’re responsible for your own career. We think that other people are, oh, I’m going to promote them, or I’m going to do that. I’m just going to sit.
Rachel Rogers: No, you have to advocate for that, and then building that network helps. Because then people know what you have accomplished, and what you’re able to do. You may have opportunities in other areas, that’s certainly happened for me. Someone else advocating for me in a totally different area because we had a relationship, or we’d work together, or they knew … I would’ve never done that if I hadn’t been volunteered for different teams, or done this or that, or reached out across organizations to build networks.
Rachel Rogers: That’s one way that you can help do that, is to definitely reach out in your own organization and talk to others.
Natalie Plummer: All right. Going on to your next slide, you mentioned trust your instincts. Now, if you’re a younger person coming into this field, or even if you’re a person who just doubts themselves a little bit, how do you know what’s your instincts versus what’s your fear? That kind of negative voice in your head?
Natalie Plummer: How do you distinguish between the two when you’re entering a new field and maybe you don’t trust yourself as much as you should?
Rachel Rogers: I think that’s a good question. We’re now at 10:20, so I don’t know if Angie, if we’re out of time or if we can just … You definitely have to, that’s a great question, about learning to listen to yourself. Not listening to the negative thoughts, but listening to the positive thoughts and trusting your gut.
Rachel Rogers: It does take experience and time. I think that we’re out of time and I totally appreciate it. I can answer other things. And Angie, I just want to say thank you for giving us, Natalie and I have the opportunity to talk today.
Angie Chang: Thank you both for joining us today. I know we will see some of your faces at the virtual booth later, so everyone who has questions for them, please hang on to them and connect with them on LinkedIn and add in the virtual booth. See you in the next session. Thank you.
Natalie Plummer: Thank you so much.
Rachel Rogers: Thank you.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Angie Chang: With us today, we have Karen, who is a director of engineering at JLL Technologies, where she leads a team of talented engineers, engineering leaders across eight products with an ability to drive innovation and strategic business goals. Prior to JLLT, she was a software engineer at Intuit and Yahoo. And as a seasoned engineering leader, we’re excited to hear about how her experience in web development, machine learning, and data informed her guidance on influencing others and decoding unspoken rules. Excited for this talk. Welcome, Karen.
Karen Lo: Thank you so much, Angie, for the very warm welcome. Hi, everyone. Thank you for tuning in today. I know you’ve got a lot of things that you could be doing, so I appreciate your time today. I will be speaking about Rewriting the Leadership Manual: A Playbook on Influencing for Non-Influencers.
Karen Lo: For this talk, I was inspired by the other woman on my team and wanted to really write down the different things that I learned over the years and how I’ve applied them through every position I’ve had outside of just my management position that have helped me over the years in providing me with an influencing style, whereas I’m typically somebody who’s not really that open spoken or talking about my own promotions, things like that. This is my approach for how I’ve been influencing, and let me go to the next slide.
Karen Lo: Here’s an overview. It will consist of three parts. Part one, learn the rules to break the rules, part two, de-weaponizing incompetence, and part three, legitimizing invisible work. I’ll dive deeply into each of these parts, and please feel free to type in any questions you have so I can answer them later.
Karen Lo: Part one, learn the rules to break the rules. No one likes to be solutioned at. I know that I don’t like to, especially when sometimes I’m just trying to rant about something, whether it’s about a process that’s really annoying or something that seems to be broken from my point of view. And sometimes I just want somebody to listen to me without providing me with solutions.
Karen Lo: I think that this applies to several people, especially when you join a new team, right? You don’t want to be the person who thinks that they know everything already before they’ve actually understood the nuances of each problem. When you listen and observe, which is step one, you are listening and validating that a problem or challenge exists. This in itself is empowerment. If you validate to somebody that, “Hey, I hear you and I totally understand your pain,” sometimes you may not have to agree with it, but just by saying you hear them, that provides a lot of trust that the person will have in you that, hey, you’re actually trying to listen to the problems that they have.
Karen Lo: A couple of examples of how you might see this conversation play out in real life is sometimes you might say something like, “Hey, I empathize with you feeling frustrated with the engineering team’s technical setbacks, and that is causing you to lose trust in our ability to deliver.” Sometimes you’re talking to a product manager or sometimes even a client, and we have engineering setbacks all the time, but it doesn’t mean that people are happy with it. And so you saying that instead of being very defensive about, oh, we have to do this or we have to do that, just acknowledging that goes really far.
Karen Lo: Another example is, “Oh, I thought the ticketing system was really cumbersome and unnecessary, but I now see it was put in place because people kept pinging your team for help and updates without sufficient details.” We all know that context switching is incredibly expensive. And so if a team has put a ticketing process in place and it is annoying to go through, sometimes you just want to see, hey, how did we get there in the first place?
Karen Lo: This is step one, first, listening and observing. Once you’ve listened and you feel comfortable enough to say something, step two is offering intentional support. This is very different than offering support. How many of you have ever found yourself in a position where you yourself have said or someone has said to you, “Hey, let me know if you need anything.” That kind of puts the receiving end in a position where they’re like, “Well, I have to put in extra work to reach out to this person to tell them what I need.”
Karen Lo: A couple of examples for what you can use instead that’ll be more effective are, “Hey, would you mind if I reached out to my leadership team about this? I think they’ll be able to help, and I’ll CC you in the email.” Or, “How about I take this off your plate?” Or, “I know a person who can help. Let me start a group chat.” Or, “I’ve faced a similar issue. Let me send you the documentation on how to fix it.” Each of these things are very intentional in you actually recommending a path forward and providing them with just a yes/no instead of, “Hey, tell me whatever you want under the sun,” because that is probably not going to happen and they’re probably not going to ask you for help.
Karen Lo: Part two, once you’ve listened and you’ve gained some credibility, the second part is to de-weaponize incompetence. I know this has many charged words, de-weaponizing and incompetence, but how many of you have ever been in a position where you’ve done something because you were either faster at it, you could do it faster, someone else asked you to do it because you were better at it, or you were maybe somebody who did it more thoroughly? Sometimes people are like, “Hey, can you do this? You tend to be a lot better at it than I am.”
Karen Lo: These are signs that I’m just sharing on how to be mindful of when you see this happening. Because while not all requests are inherently bad, sometimes you’re asked to perform a task that you may not really feel like you should be the one doing, but then you find yourself gaslighting yourself by saying, “Oh, well, that engineer or that manager is too busy. I guess I’m the only one who could do it.” Or for example, am I being asked…
Karen Lo: These are the different ways, I guess, to pinpoint if this is happening to you. First one is, am I being asked to do something purely because I’m better or faster at it? Am I being asked to undermine a process for someone else’s convenience? Is someone asking for help but expecting me to do the bulk of the work? Or is someone using urgency or impending deadlines to convince me to do something that could otherwise be done by someone else?
Karen Lo: Step one, identifying when you’re being put in a position where someone may be using incompetence to get you to do something, whether or not it’s intentional or not, right? Some people, that’s just the way that they operate and it’s not really meant to be hurting you or anything, but you need to identify if you think you’re being hurt by it and being put in a position where you’re doing work that you think could be done by other people or that person themselves. So remember, we’re all data-driven people here, so no need to use data to gaslight yourself too.
Karen Lo: Once you’ve identified if you’re in that position, step two is mitigating. Once you have that in mind, you’re like, “Okay, what can I do now?” You control the precedent that you set. You give yourself a choice. In a lot of these positions, when I’ve been part of situations like this, I typically think to myself, “Oh, I don’t have a choice,” or, “Oh, I have to do it.” Which sometimes you accept that you have to do it, but sometimes you don’t.
Karen Lo: A couple of ways that you might be able to mitigate the situation is you can say something like, “Hey, I appreciate that you think I’m someone who’s much better at this, but let’s spend some time for me to watch and provide some feedback as you go through the steps so you also become proficient.” This puts them in the position where they’re actually doing the work and you’re just providing feedback so that they become better at it. And that way in the future, it’s less likely that they will say something like, “Hey, can you just do this? You’re a lot better at it.”
Karen Lo: Second way is, “I understand the urgency of this request. However, I want you to be aware that if we bypass the existing processes to release sooner, we risk breaking core functionality and need to perform an emergency rollback. But I’m happy to oblige if you are willing to accept that risk.” Instead of saying, “Hey, I didn’t have a choice. That director asked me to release this right now and bypass going through QA or going through our regular things.”
Karen Lo: At the end of the day, you’re the one who signed off on doing this thing and you’re the one who actually executed it. Instead of saying that you don’t have a choice, you reflect this back at the requester. And when you reflect the responsibility back at the requester, they need to provide you with an official sign-off on the thing that they’re asking you to do because then you will truly be giving them the option to do it or not and not feeling like you yourself had to break rules in order to accomplish something for someone else.
Karen Lo: One more example is, “Hey, let’s discuss with our manager PM to see what the urgency and priority is. I have work on my plate that will be dropped if I need to jump on the task you’re requesting help with.” This is probably a very common scenario in which you might get pinged by somebody else, or maybe your PM themselves will ask you, “Hey, can you jump on this very urgent bug or deliver on this feature by the end of the week?”
Karen Lo: Well, you can reflect that back to them and say, “All right, I will do it. But just to let you know, I’m working on this thing that we discussed at the beginning of the sprint, and that for sure will slip. Is that the choice you want me to make?” And that way they have to think about what the prioritization is and you do not drop it and then have repercussions at the end when maybe you ended up not delivering on the initial work that you had signed up for at the beginning of the sprint. All right, so that was part two.
Karen Lo: Part three is legitimizing invisible work. What exactly is invisible work? I think a lot of us have done this type of work before. Invisible work is generally very habitual work that we do. Maybe it’s setting up meetings, writing documentation, taking notes to send for a meeting afterwards, remembering people’s birthdays, and celebrating milestones. I don’t know if that sounds familiar to you, but those are a lot of things that I personally have done that felt invisible.
Karen Lo: The first step to legitimize that type of thing is we want to cascade the recognition. Instead of you saying, “Hey, I’m doing all this stuff, guys. I should be recognized for it.” It’s much more about being intentional about who you recognize, how you recognize them, and just being present in being aware that someone did something that maybe went above and beyond, or they did exactly what was within their job description, but they did it really well.
Karen Lo: When you create a feedback loop that takes away pressure to self-promote, you promote other people to start doing the same thing. An example is maybe you’re sending an email to a person that you enjoyed working with, or sorry, maybe you really enjoyed working with somebody. You send an email to their manager, your manager, letting them know about this positive experience you had and how the person assigned helped you accomplish your tasks. Stuff like this that’s day- to-day, they generally are not recognized, but I don’t think there’s any harm in just saying, “Hey, thanks so much for helping me with this request. I know you guys are swamped with a lot of things, but just by the way, I don’t know if you even realize how important it was for me, but this helped me unblock my deployment. And normally, I’m waiting two weeks back and forth.” Something like that, it goes a long way.
Karen Lo: Another way you can do this is privately pinging a person in leadership or maybe even a peer to suggest that they provide recognition for this person because you think that it’s important that they know that this person either helped to go through an architectural review for the first time when the company is trying to make that more standardized. Maybe they did something really well. Maybe they performed their on-call duties really well and the incident management team would love to know that.
Karen Lo: You are just directing the people who might care about it and you’re surfacing what you want to talk about and recognize them for, but you’re not doing it yourself. You’re asking someone else to do it. And generally they’ll probably say, “Oh yeah, I would love to recognize this person.” Or even just give a quick shout-out or send an email just to let broadly more people know about the work that this person did.
Karen Lo: And step two is setting up a framework. Like I mentioned before, invisible work is generally habitual work. And habitual work is the prime candidate for automation and structure because we know that if you’re going to be repeating the same thing over and over, why not make life a little bit easier for not just yourself but everyone else?
Karen Lo: First thing that goes into this is to set up the framework. What does it look like if you were to repeat this task and be able to share it across not just your team, but maybe other teams as well? Because remember, a framework is something that is repeatable structure-wise, but not all the steps in a framework need to be the same. So they can always be adapted to fit any other use case.
Karen Lo: Some prime examples of how you might set up, or what you might set up a framework for are scrum leader responsibilities, maybe meeting scheduling, team engagement activities, documenting, metric reporting. All of these things you can put together, hey, as a scrum leader, every week you have stand-ups that you run, grooming sessions, sprint planning, retros, sprint review. These are all very straightforward things that every time you’re a scrum leader, you’re probably going to be doing these things.
Karen Lo: Something like team engagement activities and planning, why don’t you just have everybody put in their birthdays and then maybe integrate with a Slack bot so that it reminds everyone when it’s their birthdays. Things like that are prime examples of how you can set up a framework. And of course, there’s many other ways you can do it, but these are just some examples for you to look at.
Karen Lo: The last step of this playbook for part three is to democratize that. Now that you have the invisible work lined out step by step, it is something that you don’t have to do by yourself anymore. Once you have that together, you can then set up a meeting and say, “Hey everyone, here’s a framework I put together for what you would do as a scrum leader. It would be really great if we each owned, every one of us owned the scrum process because this is for us and this is for our team. I’ve laid it out really simply so that you just need to follow these things and then this is the way that you lead them. And I can definitely provide some feedback along the way. I’m not going anywhere, but I want this to become something that is a shared responsibility.” And like I mentioned earlier, automated birthday milestone reminders.
Karen Lo: Third one is to have explicit driver expectations outlined on who does what and when documented and frequently referenced. And I say frequently referenced because we all know we write documentation that goes into the abyss and no one ever reads it ever again. The framework type of documentation is what I would generally expect drivers to frequently reference until they forget about what is expected of me and they just know, and then they share it with anyone else who joins the team because then those people will also be taking on the responsibility.
Karen Lo: And finally, we want to incorporate team citizenship as a value and form of recognition. You don’t have to be a manager to do this, but if you are in a management or leadership position, it is critical that you are recognizing citizenship. Because it’s so easy for us to get bogged down by the day-to-day things of we’re coding or we’re going to meetings about things and we’re creating pull requests. And we know what our work is. But the part that’s important is the way that we interact with each other. That doesn’t always get intentionally highlighted.
Karen Lo: Intentionally highlighting, “Hey, thanks so much for replying back to me so quickly.” Or, oh, so-and-so, I was running into an issue and they were like, “Why don’t we hop on a chat and I’ll walk you through this thing?” And it just goes a really long way to providing a very healthy culture in which people start to be a lot more mindful about the things that they recognize each other for. And then they’re more aware of what they’re doing so that they can share the love with everybody else. And eventually you create a culture in which people recognize each other without needing to prompt for, “Hey, you should probably recognize that person,” into your culture.”
Karen Lo: That’s pretty much in a nutshell three parts for how I have learned to influence over the years. I hope that this has provided everyone with at least some nuggets of knowledge. Again, thank you so much for joining me today. I know you guys are busy, so have a great rest of your day.
Angie Chang: Thank you, Karen. I know you wanted to share your slides. If you could do that on LinkedIn or whatever social network that you prefer, we can reshare it and people can save it and look at it for themselves. You can always replay this as well. Thank you so much for joining us, and we’ll be hopping to our next session.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Watch the Bentley Systems intro from Gen Taurand (Product Manager, Bentley), Stephanie Robinson (Director of Services at Cohesive, a Bentley company), and Meghan Goff(Manager of Talent Acquisition at Bentley) on why they enjoy working at Bentley, what they are working on, open roles, and how the hiring / interview process works.
Angie Chang: Welcome to our third ELEVATE Conference! With us today we have our Career Fair Kickoff with Bentley Systems, and we have a bunch of their staff and recruiting here to tell you more about themselves and the roles. You’ll be meeting them later as well in the employer booth from 12 to 1 PM Pacific Time, sorry, we’re on the west coast here and we would love to talk to everybody. I know there’s some introductions, so I’m going to hand it off to Gen.
Gen Taurand: Hi. I’m Gen. I’m based in the Quebec city office. As you can see, I’m in the office right now, but we are a company that some people work from home, some people work from the office, some people are hybrid, some days at the office, some days at home. I’ve been with Bentley for about five years now. I come through an acquisition, so my company was acquired five years ago. It was a company, a startup for an artificial intelligence.
When I joined Bentley, I was in the applied AI team that builds and constructs machine learning models for different products. And as I worked for a couple of years, I really wanted to go into product management and my management team was behind me 100%. They gave me training, they gave me mentoring opportunities, books to read, and then eventually they even found opportunities for me to switch teams and become a product manager.
Even if it means losing a colleague within your team, Bentley is all about growing your career within Bentley systems. That’s really what I love about Bentley. I don’t code anymore, but AI – people will work with PyTorch, Python, And then we also have desktop products where we do C++, Delphi, Java, and we have a big cloud platform where we do C#, Grid, React, Kubernetes, SQL servers and so on. Lots of technologies we work with.
What I love most about Bentley is working with people globally, so we have colleagues at Europe, in India, Pakistan, Australia, and I get to meet them and learn so much about their culture and that’s great, but I can also meet other colleagues outside of work, I guess through different groups. We have my wellbeing group, which is about mental health and physical health. We have our diversity equity inclusion groups. We have different groups of that too. And then we also have groups for empowering sustainable goals, so if you have other interests outside of work and you can meet other people that, and then you can work together towards a common goal helping Bentley. That’s what I love about it. Stephanie, do you want to maybe go ahead and talk about your career path?
Stephanie Robinson: Absolutely. Thank you Gen. Good afternoon everyone. My name is Stephanie Robinson. I’m a director with Cohesive, which is a wholly owned subsidiary of Bentley. Cohesive is a services delivery company that was compiled with eight other companies that used to be competitors, and Bentley brought us together under the same umbrella. We are still in the process, we’re still relatively new. My company was one of the first ones acquired, the one I worked for originally, and we’re about three years into our Bentley relationship having just acquired another company under the cohesive umbrella as recently as this past summer.
In short, we help companies who want to manage their physical assets, so their buildings, their pumps, their power lines, their nuclear reactors, whatever it happens to be, that want to manage their physical assets better. And we do that by compiling small teams of industry professionals as well as technical professionals, and we analyze their business processes, we take a look at how they do things, what they want to do, what their end goal is on what they want to do, how they want to maintain their assets. And then we make recommendations for best practices from the exposure that our professionals have had from the technologies that our specialists know.
We also work with some pretty unique software platforms to help achieve those goals. In asset management, we primarily look at utility industry, but we also have solutions for transportation, aviation, healthcare, and general business.
I’m based in Atlanta, Georgia, but as Gen was talking about, I’ve had conversations with people from South Africa, from Scotland, from England. Today we are a global company and it’s one of the neat things that we have within Cohesive. For example, you may hear about some openings in North America. I don’t care where you live, you can work for me because all we require is that you have a strong internet connection and that you understand the business that we’re in.
By design, we are hybrid. Most of the time our clients, our employees, excuse me, are at the client site working. So regardless of where you live, you may fly in from on Mondays and work at the client site until Thursday afternoon and then you fly back home. You’re working with a team, you’re working with customers, and then you’re working with a company, Cohesive and a bigger company, Bentley.
I’m a member of the veterans who work at Bentley. I’m a member of Black employees that work at Bentley and some of the other affinity groups that Jen had mentioned. You get different levels of membership when you join this company. I’ve been in delivery consulting for seven years. I came to Bentley as a trainer.
I’m now a Director. I was an industry specialist when I joined. As I mentioned, I work very closely with my colleagues in Canada and in Brazil and all across the United States. It’s been a real experience becoming part of Bentley after having been with a small company for so long. It expanded our perspective, it expanded our family and it taught us how to apply some things that we actually usually get paid to go into clients to do, and now we’re doing it for ourselves.
There’s lots of opportunities with this acquisition and I believe that we haven’t even scratched the surface, so it’ll be great to have some of you join us in this journey. Megan.
Meghan Goff: Thanks Stephanie. Hi everyone. My name is Meghan. I’m our manager of talent acquisition for the Americas region, which includes the United States, Canada, Brazil, and Mexico. Personally, I am based out of the greater Philadelphia area in the US, so you can probably see from my background, a hybrid colleague. I do travel into our global headquarters when able, but spend most of my time working remotely since the pandemic.
I’ve been recruiting for 10 years now and having spent the past five specifically here at Bentley Systems. What I love most about working here at Bentley is our culture, and I am fully aware that that is a cliche response, but it’s true, and it’s the people here that I work with, and the support that I receive across leadership, that makes me say that.
I could list several reasons as to why I love our culture, but one that’s close to my heart personally is the fact that I have the confidence to unapologetically say that my family comes first, and to be supported in that, so when the unexpected does come up, which it does because I have a three-year-old, and one on the way, I can be there for my family and then return to work with uninterrupted focus and dedication. That’s what I love about Bentley, one of the many things.
Here with Bentley, I recruit for roles across the company from technical to sales, legal, finance, you name it. I’ve supported it and wanted to highlight two roles that we’re currently recruiting for. I know Angie graciously shared our openings in the chat, so please feel free to take a look at those openings. One of the two that I wanted to highlight is our enterprise user success manager, where we are looking for someone with experience of the engineering or construction industry to be able to act as a trusted advisor to our users and help solve business problems and ensure that they are maximizing their potential of the Bentley software solution. Another role we’re hiring for is a VP of user experience and research where we’re looking for someone who has experienced successfully scaling and reaching a steady state of UX across both desktop and enterprise systems.
The interview process at Bentley would be first to apply formally to the job requisition. From there, if there is initial interest from the hiring manager’s and a talent acquisition representative will schedule a screening call to confirm job requirements as well as answer any initial questions that you may have. The result of a successful phone screen would be to then bring you forward and meet with the hiring manager directly. From there, we would certainly introduce you to other stakeholders and team members throughout the interview process. That is a little bit on Bentley and we’re looking forward to joining everyone in the virtual booth.
Over 100 girl geeks joined networking and lightning talks from women working in engineering, product, and design at the sold-out Grammarly Girl Geek Dinnerat Grammarly’s office in downtown San Francisco, California on August 29, 2023.
Grammarly women shared lightning talks about building GrammarlyGO, Grammarly’s new contextually aware generative AI communication assistant that allows you to instantly compose, rewrite, ideate, and reply. Grammarly is hiring!
Transcript of Grammarly Girl Geek Dinner – Lightning Talks:
Angie Chang: Is [this] your first Girl Geek Dinner? Wow, that’s a lot. How many of you have been to more than five Girl Geek Dinners? Yay! So good to see everyone. My name’s Angie Chang, in case you didn’t know, and you could tell by the t-shirt, I am the Girl Geek X Founder, and started Girl Geek Dinners in the Bay Area 15 years ago, so we’ve been doing events like this at hot tech startups up and down from San Francisco to San Jose, and I’m in the East Bay, so I wish there was more events over there as well. Tell your employers they need to have one of these showing off their amazing women in tech and product.
Girl Geek X founder Angie Chang welcomes the sold-out crowd to Grammarly Girl Geek Dinner on August 29, 2023 in San Francsico! (Watch on YouTube)
Angie Chang: I want to say thank you so much to everyone at Grammarly for helping put this event together. They have been so amazing and supportive and they’re definitely hiring, so please talk to someone here that has Grammarly on their shirt. They’re very friendly, so I’m going to say thank you for coming and hopefully you’ve made a lot of good connections. I know I’ve seen a lot of people talking to each other and I hope you have LinkedIn with each other or Facebook or whatever people are using these days, and continue to stay in touch.
Angie Chang: A lot of us are in this industry working to keep women in tech and I think that involves all of us together, so thank you. Keep coming back to events! Keep giving each other job leads! Keep poking other girl geek to get in the car ride together to get to that event after work when we’re all tired! Thank you for coming! I hope you learn something, make a new friend, and have a good night!
Charlandra Rachal: Thanks, Angie. I’m super excited to kick things off and host this fireside chat with director Heidi Williams, who’s been very involved in building our generative AI features for enterprise. Heidi, welcome!
Heidi Williams: Hi. Thanks for having me! Great to see you all here. It’s awesome. Full crowd!
Charlandra Rachal: Yeah. For those who aren’t super familiar with Grammarly, can you give us a quick overview of our company and our product?
Heidi Williams: Sure. I like to make a joke that either people have never heard of Grammarly or they love it! I know I talked to a few folks already that love it, but for folks who aren’t familiar, we are an AI-enabled writing assistance that helps with your communication wherever you write, and I do mean everywhere. Our mission is to improve lives by improving communication. Earlier this year, we also launched our first generative AI product to help you with even more writing and communication assistance beyond just revision, but also getting into ideation and brainstorming and composition and comprehension. It’s been really fun to see the product evolve in the time that I’ve been here.
Charlandra Rachal: I hear that you just celebrated three years here, so woo woo! Three years! Can you tell us what brought you here and what really keeps you here?
Grammarly Technical Sourcer Charlandra Rachal and Director of Engineering Heidi Williams welcome the audience at Grammarly Girl Geek Dinner. (Watch on YouTube)
Heidi Williams: When I was speaking about the mission, improving lives by improving communication, I do feel like I got to a point in my career, I’m a little farther along maybe than some of you, that I really wanted to work on something impactful and I feel like Grammarly more than any other place, it resonated with me that improving lives by improving communication is so real. It’s not a fake slogan because communication is what makes us uniquely human.
Heidi Williams: I was excited about the idea that we’re not just a platform to help you communicate more effectively, but also to help educate you along the way, especially thinking about things, like insensitive language or bias, there’s an opportunity to help educate people about the possible impact of their words that they may not even know is having a negative impact on someone, and so I got really inspired about the mission.
Heidi Williams: I’m also a word nerd, so that part was really fun as well. I think what keeps me here, is that everyone is so excited about the mission and the people. I think our values are amazing. We really live by our values, we hire and fire by them.
Heidi Williams: The last thing I’ll say is, we’re an amazing size company, where there’s still interesting problems to solve, but we’re small enough that people can really take the initiative if they see a problem that needs to be solved, or they want to advocate for something to change in some way, they’re really empowered to do that. I love being at that size company and our values really help us be successful doing that as well.
Charlandra Rachal: Nice. I like you mentioned initiative and impact. Do you have any sharing stories that you can share that where you seeing either yourself or someone else really make impact?
Heidi Williams: I have three examples if you’ll bear with me for a minute, but I see it all over the place, and it’s not just in the product. It’s about our organization, our culture. There’s an engineer on my team, her name is Lena, and she recognized on the product side that engineers were struggling with a certain pattern of ‘how do I reliably save settings about the individual, about their team and their organization about specific features. Then if I have all of these settings, how do I combine them and know which setting to apply at any time?’
Heidi Williams: She interviewed a bunch of engineers, realized it really was a problem for folks, and then proposed a new project called the Settings Registry, and advocated for it to be on our roadmap. It’s been exciting that she could spot an opportunity and a challenge for our developers and really advocate for that. That’s exciting!
Heidi Williams: The second one, I actually led an initiative where I noticed that I love our hiring process, but I noticed that we had one particular gap, which was that we didn’t necessarily have an interview where we asked people about their experiences. We ask about their knowledge, but we don’t ask, ‘what is the proudest thing that you ever built and tell me how it was designed and what did you learn?’
Heidi Williams: I noticed similarly that we weren’t necessarily getting the accept rates from underrepresented groups that I thought we should be getting, and advocated that this might give people an opportunity to talk about themselves, and for folks who aren’t used to bragging about themselves, that might not come out in a normal interview, but if you give them an opportunity to talk about themselves, then they can actually show off how good they are at stuff, which is exciting.
Heidi Williams: That pilot was successful, showed that we greatly increase the accept rates for folks from underrepresented groups to a really high degree, and now we’ve rolled that out as an interview across the engineering organization, so really proud of that.
Heidi Williams: The last one I’ll mention related to culture, Bhavana, who you’ll hear from later, identified an opportunity that folks were looking for mentorship inside of our women in tech group, and so she started a pilot with a few other folks to introduce an internal mentorship program for women in tech and we’re kicking that off in September.
Charlandra Rachal: I love that. Yes. I feel like the last two really spoke to me, especially being in recruiting so I love that a lot. Now, Grammarly continues to expand in its enterprise space. How do you drive value for Grammarly business with generative AI?
Heidi Williams: It was very exciting to see our generative AI product come out. A little bit of context: the part of the product that I’ve worked on is Grammarly Business, which is our B2B product for teams and organizations.
Heidi Williams: As we all know, communication is not a one person sport. There’s a team dynamic, there are team norms, there’s organizational knowledge that are part of the communication that you have at work. We looked at opportunities for how to incorporate organizational knowledge.
Heidi Williams: We have a feature called Knowledge Share that helps you define terms, definitions related links, key people, and then we can use that as part of the generative AI output to help you have something that knows something about your organization instead of maybe a more generic response.
Heidi Williams: We did things like that and then incorporated some of our Grammarly business features like style guides and brand tones, which help you speak with a consistent voice, and brand tones in particular, you can have a response from our generative AI product, and then choose ‘make it sound on brand to my company’.
Heidi Williams: That was a way that we could really make the information, both the information and the tone be tailored to your organization.
Charlandra Rachal: Nice. Well, I heard that there was some quick turnaround times. Can you tell us more about that?
Heidi Williams: It was definitely felt like this huge opportunity, this huge moment where a lot of folks are talking about generative AI and it’s an area (LLMs) we’ve been investigating for a long time and understanding what their capabilities and limitations were and whatnot, and so I think we really rallied as an engineering organization, and I think the way that we were able to turn things around quickly really came from our leadership approach, which is the idea that we really want to empower teams to make the best possible decisions on the ground.
Heidi Williams: The way to do that is to help with transparency and sharing context around ‘what are the business needs, what are the product needs, what are our customer needs, what problem are we solving for the user?’ Let me give you all of that information, all of that context. At the end of the day, if you need to choose, should this be a radio button or a dropdown or this should work this way or connect with that system, you can make that decision because you have all of that information. Really trying to be transparent and share context so that people are empowered to make decisions on the ground and not feel like they’re stuck with somebody else making decisions and kind of blocking them from things.
Charlandra Rachal: I hear you that you mentioned customer feedback. Do you have any feedback that you’re able to share with us?
Heidi Williams: Sure. You’ll hear more about it in one of the talks today. We did run a survey after launching GrammarlyGO and wanted to know how are people using the product and what’s working and what’s not working. Through that feedback, one of the themes that we heard was that ‘it didn’t sound like me’.
Heidi Williams: We started investigating – ‘how do you tailor the output to sound authentic to you?’ And it sounds, I see a lot of head nods, that resonates and what not. We invested in an area called My Voice and figuring out how to have your own voice profile and use that for all of the responses that are generated, so it’s more likely to sound like you than not and saves you an extra step for trying to even interpret what your own voice is. We can actually help you with that, so you’ll hear more about that when Jen talks about it.
Charlandra Rachal: Great. Well, I know this is one question that I know a lot of people probably want to ask but probably wouldn’t ask, but what would you say really sets us apart from our competitors?
Heidi Williams: Yeah, I was talking to someone ahead of time who asked this question, I’m like, oh, you’ll have to wait. <laughs> Great question. There’s one thing. First of all, I mentioned earlier, we work everywhere and that is one difference from some of the other products that are out there. We work in every writing surface, desktop web, and so we can be right in line for where you’re already doing your thinking, your writing your communication, so that’s certainly one.
Heidi Williams: The two I wanted to really call out, which I think are kind of reinforced by our engineering culture, is our important focus on security, trust and privacy, and also responsible AI. Because at the foundation of everything we do, we really want our customers and users to trust us with their writing and to feel like we can do things to make personalized experiences and what not, and so, what’s interesting to me, I feel like more than any engineering organization I’ve ever been at, because we are so mission-aligned, we recognize we have this huge responsibility to our users to be thoughtful about their data and their privacy, their security.
Heidi Williams: I feel like we care a lot about security maybe earlier than most engineering teams where at the very end before you ship security goes, ‘oh, not yet!’ And you’re like, ‘oh, I can’t’. The whole idea that engineers will advocate for, am I doing this right from the beginning, and wanting to make sure that’s so they’re proactive about asking for feedback about security and privacy. Or even there was a scenario where we had an idea about a feature and people are like, ‘That feels like it might invade privacy. Can we talk about that before we launch it?’
Heidi Williams: I really loved that people could bring that up and that we’re all trying to achieve the same thing, and so it’s a very fair question and let’s make sure we’re holding that to high regard.
Heidi Williams: Then on the responsible AI side, I think we’re so lucky to have an incredible team of linguists who can help us beyond what other competitors can do who don’t have a team of linguists where we can help sort of filter things like the inputs to generative AI to make sure that people are not asking for something harmful, but also that whatever they type in, they’re not getting harmful responses, which are either insensitive or inflammatory or traumatizing in some way.
Heidi Williams: I love the fact that we have the capabilities of being able to create these filters and create a safe environment for people to use these large language models, which have who-knows-what in them. Love that we are actually able to do that. We’ve also been able to build that not just through humans, but figuring out how to build automation and testing and all through the development process help you understand that you’re not going to create a feature that unintentionally create some sort of biased output or something like that, and so just tremendous examples over our long history in this of finding ways to make sure that we are building a product that is responsible and then also keeps everybody safe, secure, and all their information private as well.
Charlandra Rachal: Nice. Well that was fascinating, right, everybody? Alright, so we are going to dive deeper now to exactly how our generative AI features were built. As a heads up, we are going to ask for questions at the end and I’ll bring up all of the speakers including Heidi herself. For now, welcome Jennifer van Dam, who’s a senior product manager here!
Jennifer van Dam: Hey everyone. I’m Jennifer van Dam, product manager here at Grammarly. I’ve been here for three years and I worked on our features like emotional intelligence, tone detection, tone rewrites, inclusive language, and most recently I helped build out our generative AI product, GrammarlyGO, which I’ll be talking about today, so super excited to take you all through the journey.
Grammarly Senior Product Manager Jennifer van Dam talks about building the generative AI product GrammarlyGO from zero to one. (Watch on YouTube)
Jennifer van Dam: First off, I want to give a huge shout out to my fellow girl geek PMs that helped build GrammarlyGO together with me. We were a team of three PMs leading multiple product efforts. Specifically, my product focus was on the UX and also on the zero to one stage, so figuring out the UX framework and the zero to one building process. That’s what I’ll dive in deeper today. First off, I wanted to start with a refresh of Grammarly before GrammarlyGO.
Jennifer van Dam: What Grammarly has been focused on for many, many years is helping make your communication more effective by proof writing and proof reading and editing your writing. Anywhere you write, let’s say you’re writing an email, a message, a Google doc, Grammarly will read the text that you have written already and make sure it’s correct and clear and delivered in a way that you want to come across. But, we have a big mission of improving lives by improving communication, so we fully were aware that this is a small part of communication that we want to help with, and we’ve had many dreams beyond proof writing and editing.
Jennifer van Dam: One big user problem we always heard, for example, was the ‘blank page problem’. For years, we’ve heard that our users really struggle with the inception stage of communication – the writing, getting those initial ideas on paper – and it was a huge productivity blocker. That’s just an example from user problems we’ve been hearing for years, and we always dreamt about solving it, and we were super excited with this recent technological leap that with generative AI – now we have the technology to solve all those user problems we always dreamt about.
Jennifer van Dam: That’s how we built GrammarlyGO. We went from proofreading and editing towards helping solve composition, brainstorming, and all these new use cases, which was really, really complex, because we went from a decade of in-depth expertise of rewriting, towards composition and brainstorming, and we had a pretty aggressive timeline as well. This was super, super challenging.
Jennifer van Dam: What made it really challenging? First of all, it was zero to one. We had no prior experience how this would land with our users and there was no data we could rely on, so we had to make a really risky decisions because we went from a proven product concept with product-market-fit towards a huge uncertainty and risk area, which it was really exciting, but super, super challenging. How can we predict how it will be received with the absence of data?
Jennifer van Dam: Essentially we really had to take on a beginner’s mindset to solve these new use cases and almost operate like a startup again to build this new product from scratch, but we’re also an established company – pretty big – and we have millions of users, 30 million daily active users that have a super high bar of our product. We were building zero to one moving fast, but also had a very high bar we wanted to meet for our users in terms of quality, responsible AI, and security that we wanted to deliver.
Jennifer van Dam: How do you solve such a huge, huge problem? What we did was, let’s just start with the earliest draft possible, and get it out – get it out to users. What we did is we created this really highly-engaged alpha community, and we built very early prototype, and we shifted and we asked for continuous feedback, and it was really, really engaged community that would give us feedback super fast and inform next iterations. We really focused on the core experience before we wanted to invest in any type of polish or any type of design polish, we made a commitment – let’s not focus on that. Let’s figure out the UX framework.
Jennifer van Dam: We had a big challenge. How do we create a UX where someone can brainstorm and compose something from scratch? What is intuitive? What will land with our users? To give you an example of the fidelity of prototypes, what we did is we started in grayscale, because we made a commitment to figure out the framework, before deeply investing into building something out, because we weren’t sure if this is the version to commit to.
Jennifer van Dam: This turned out to be a great idea because we did end up throwing away a couple of prototypes, and the third prototype was the one that we felt landed the most and that we committed to building out and refining, which was of course a huge process as well and took us a lot of time. Sarah will actually be giving a fascinating talk later about all the design and brand work that went into polishing this prototype, so I won’t go too deep into that.
Jennifer van Dam: What was really cool about this prototyping stage is, the user empathy led to innovation. We came up with things that we didn’t necessarily plan from the start. One thing we kept on hearing when we were asking for feedback on the UX and was it intuitive to compose and brainstorm?
Jennifer van Dam: A lot of the feedback we were getting was ‘it just doesn’t really sound like me’. And that made people drop off. They would compose an email or a document, but it didn’t sound like something they would write or want to use, so this was a huge risk of people dropping off and also, it wasn’t of the quality we wanted to meet. This led us to come up with the voice feature that Heidi talked about with before.
Jennifer van Dam: This is a classic example – it wasn’t on our roadmap from the start, but it really, being in tune with the user made us come up with this feature. I remember when we launched our first basic version of it, how excited everyone was, and that made us realize how important voices in generative AI – and it led us to much deeply invest in this area, so we’re keeping investing in this and also it helped actually become an important competitive differentiation.
Jennifer van Dam: To take it even further, we would also hear from users, okay, now it sounds like me, but in this situation it doesn’t sound how I want to sound, which was also a really hard problem. We heard this a lot in the email reply use case, and what we came up with is harmonizing your voice preference with your audience as well. Let’s say, I prefer to sound casual maybe 80% of the time, but I got this super formal email, it would be a little bit awkward if I replied casually there.
Jennifer van Dam: We also created a model that looks at the context of your communication and your audience, and harmonize that with your voice preference so it doesn’t diverge too much, but lands somewhere in the middle. This was an awesome, awesome project and Bhava is going to do a much deeper dive into replies after this.
Jennifer van Dam: Looking back, this was a huge product, and when I reflect back on what were the things that made it successful, I think first of all the team was really, really important when we started this project because building in a zero to one, very high ambiguity, it’s not for everyone. It can be quite chaotic.
Jennifer van Dam: We started with a very small team that was comfortable with iterations and ambiguity and okay throwing away work for the sake of learning. We intentionally kept this team very, very small until we resolved the main ambiguities, we started to scale up the team a little bit or slowly.
Jennifer van Dam: We were very intentional about the initial zero to one stage, and then scaling the team, and we had a lot of high alignment and energy because of this, because the people on the team were excited about these problems.
Jennifer van Dam: We also learned that prototypes are huge to align leadership, because it’s easy to get stuck in discussions, discussing strategy or design flows, but there’s nothing like proving it with real concepts and real user feedback and real prototypes. And then also our transparency principle really helped. We had a ton of cross-functional collaborators and of course it’s inevitable zero to one, there’s going to be all these changes and all these teams are relying on you.
Jennifer van Dam: We were super, super transparent with changes and reasoning and this really helped us creatively problem solve. In the case when there were changes, we would come together and this basically set up a place for innovation with cross-functional collaboration as well.
Jennifer van Dam: What’s next for Grammarly? At Grammarly, we believe that AI is here to augment your intelligence. That is really our product philosophy. We believe that AI is not here to take over your life or dictate you, but it’s here as a superpower, to help you communicate more effectively.
Jennifer van Dam: This is the product philosophy we’ve taken building GrammarlyGO, and this is our philosophy with all our next products and features that we’ll be launching. I can’t share too much about it, but I can share that this is the philosophy we take in building the next features that we’ll be releasing. Thank you. Alright, next up is Bhavana who’s going to be talking about the fascinating project called Quick Replies.
Grammarly Machine Learning Engineer Bhavana Ramachandra talks about engineering the generative AI product GrammarlyGO at Grammarly Girl Geek Dinner. (Watch on YouTube)
Bhavana Ramachandra: Thanks Jen for that awesome overview. Jen spoke about how Grammarly expanded into the user’s writing journey, and we’re going to take a small detour into one of the features that we built, which was Quick Reply, or replying quickly to emails. My name is Bhavana, I’m an ML engineer at Grammarly. I’ve been here for about three years. I was one of the many engineering geeks on this project. There are a couple of folks here in the audience today – Jenny’s here, Yichen is here, yeah, wanted to give a shout out to the team.
Bhavana Ramachandra: Today, I’ll be really talking about foundations in motion and with respect to the quick reply feature. Jen touched upon this previously. We have invested quite a bit in understanding what our users want in terms of their writing, and we were looking at expanding into this user journey, so really we built a lot of fundamental understanding over the years that helped us accelerate into the new product areas that we wanted to go into.
Bhavana Ramachandra: Another shout out is that, the team that worked on Quick Reply, all the point people, all the cross-functional people, were women. We have an analytical linguist, computational linguist, ML engineer, senior PM, and interestingly this has not been my first project when we were all women but yeah. Four foundations deriving from two projects that I have worked on coming into this one. One was Tone. Jen mentioned she has been working on Tone as well.
Bhavana Ramachandra: I’ve been here for three years. She’s been here for three years. Tone was our first project together, so Tone was one of them as well as Recap, which is our investment in 2022 to really go beyond the writing phase and to also to the reading phase to help users read faster so that we can help them write better. With the respect to tone, this was the first version of this. We also have tone rewrites, but this one helps users identify the top three tones in their text so that they can reflect on if that’s exactly how they want to sound.
Bhavana Ramachandra: Zooming into what was the fundamental understanding we built in each of these projects. The first one I’ll cover is Tone, and the three areas we invested is product definition, quality and our AI. For product definition, some of you might be thinking like, ‘Hey, this sounds like sentiment analysis and that is a pretty well solved problem’ but really our product team tries to think about what is the user value of sentiment. If you look at user text, honestly, eight out of 10 times users sound positive. That is not helpful to know.
Bhavana Ramachandra: What the product team did was actually define 50 tones over different aspects of your writing that’s actually helpful for you to know. Do you sound optimistic? Do you sound direct? Do you sound confident? Do you sound worried? Do you sound concerned? The product team really came up with a wide range of tones. In terms of quality, we iterated quite a bit over it and during this phase we actually came up with three levels of hierarchy.
Bhavana Ramachandra: When you have 50 tones, especially if you’re building models for 50 tones, it’s a bit hard – one to get data and to make sure you’re iterating over quality of all of ’em. The way we tackle this is, we define three levels. We have the tone at the really granular level, we have the sentiment at the highest level, but we also came up with tone groups that was maybe around, eight tone groups, and that helped us identify quality at different levels. Then, we really try to nail quality in terms of what is the user values.
Bhavana Ramachandra: Now as an ML engineer, I like to see quality always improving, but is it really worth it to invest in taking one tone from 90 to 92% or is it better for us to improve on a certain tone group that is really valuable to our users?
Bhavana Ramachandra: That’s the kind of trade-off that we had to make and then we really derive over time. I also want to mention our AI is one of our biggest tenants as Heidi mentioned. In this case, this feature was one of the first few pieces to pilot our sensitivity process. The REI manager today was during this process shaping up our formal sensitivity process. We’ve always done it and I think she was making that a very formal process.
Bhavana Ramachandra: Apart from that, we also wanted to make sure that any tone suggestions we make – because we have a varying level of quality, we did want to understand – what are the sensitive cases, and what is our risk of quality with respect to sensitivity. That’s something that we understood during this project as well.
Bhavana Ramachandra: The second one is Recap, which was our comprehension project that we worked on in 2022. Here we were going beyond the writing journey into the reading journey of the user. We invested a lot in understanding the user problem. We had many, many discussions about certain areas that surprised me that I’ll get into. There were also technical challenges because now we needed to again look at the context outside of the text that you’re writing. Where is it? Where are we getting this context from? And then we had a whole new set of ML problems, which is exciting for me.
Bhavana Ramachandra: For the user problem, I wanted to touch on two things. Delight versus value. We wanted to provide summaries and so we identified emails and we wanted to provide summaries as well as to to-do items. But does it really make sense in all use cases? For example, if you have a one line email, it doesn’t make sense to summarize that.
Bhavana Ramachandra: Or, if you have a social promotional email that says, sign up now, that sounds like a task, but all of us know that’s not really a to-do item for any of us. These are the kind of gotchas that we were like, ‘oh, we have a model, but is it actually useful in all cases’ or ‘how long should a summary for a really long email be’ versus ‘a one paragraph email’ be? These are the kind of things that we iterated over quite a bit. And also understanding the context and intent of the user.
Bhavana Ramachandra: Imagine you have an email, an announcement to your entire organization. If you’re a manager versus if you are an engineer versus if you are in design, you might have different takeaways from that email. Trying to understand a bit more of what is that context and what is the intent of the user.
Bhavana Ramachandra: We also solved a lot of technical challenges. Again, shutting out like our AI is one of our biggest pillars. Privacy is also our biggest pillar. We are very, very cautious about what is it that we are asking users to share with us and are we really providing value from it? Before this we didn’t look at the context of the user because we looked at suggestions of what they were writing. Now we wanted to provide value from that, so then we had to update our privacy policy and we also had to update our client side logic to derive this context.
Bhavana Ramachandra: Coming down to the ML problems itself, like I said, there were two things that we were trying to provide – summarization – as well as – task extraction – or to-do list, but because we were talking about delight versus value, context and intent, we also invested in a couple of different areas, including signature detection, intent understanding, and email taxonomy. Trying to understand what is the context of the user – that was more of email taxonomy and intent understanding, and signature detection really helped us. When you look at emails sometimes especially short emails, if the signature is longer than the email, then the model sometimes gets tripped up.
Bhavana Ramachandra: This is true for generative AI as well because yeah, for many different reasons, sometimes models are not perfect, so it helps to help them along the way, and signature detection was one of those areas.
Bhavana Ramachandra: In all of these areas, we spend time annotating our own data sets because email is a space where data sets are not as public, so this was one where we had to understand what data sets existed, what were the things that we were trying to build. As Heidi said, we have a big internal team of analytical linguists, and they help us identify the data, identify what our guidelines are, and go get us annotations that we can build models with, and these were all the areas that we collaborated with them on.
Bhavana Ramachandra: Putting all of that together, from the Tone project, we knew tone was something that our users cared about that we wanted to bring into this feature. You’ll see it says, ‘Jason sounds caring’, but that’s not like models don’t know to think about that. That’s something we have to prompt them to think about.
Bhavana Ramachandra: All of the tone taxonomy that I spoke about, the 50 tones, that made it into the prompt as well. In terms of the recap project, we really built a reply user – like, who are the users? You might get a hundred emails, but you probably reply to 10. What are these reply use cases is something that we had built an understanding that came into this project as well. That really helped us understand quality for launch.
Bhavana Ramachandra: As Jen said, we were not trying to polish, but we were trying to aim for user value. That meant, are we comfortable with the quality for launch? We know that we’re going to iterate over it, but for launch, does this look good? It’s something we try to understand.
Bhavana Ramachandra: Then, on the client side, a lot of the logic, that we built for the earlier project, got repackaged and reused for this one as well. We were using a new protocol, we were using, so it wasn’t just copy paste, but repackage. And then our AI as always, because it’s a generative AI output, we want to be sure that any output that we’re sharing with our users does not have bias, and some high risk scenarios. That’s something as well that we made sure this feature and the output of this feature goes through.
Bhavana Ramachandra: This was one of the few features we built for launch, but it did get a couple of different shoutouts. I know WSJ called out, we had a lot of users sending us, this was awesome. I specifically wanted to, we had a segment on NBC where Courtney Naples, who is our director of language research, spoke about this and the host in fact called out the feature and mentioned how the output of GrammarlyGO sounds like him, versus OpenAI does not. And yeah, that was a really nice moment for us to see. That’s it. Next off, we have Sarah who will be talking to us through all the explorations that the brand design team did for launch as well as our product.
Grammarly Brand Designer Sarah Jacczak talks about designing the generative AI product GrammarlyGO at Grammarly Girl Geek Dinner. (Watch on YouTube)
Sarah Jacczak: Thanks so much, Bhavana. Hi everyone, my name is Sarah and I’m a brand designer at Grammarly. I’ve also worked here for three years and I’m so excited to share the brand design team’s work and show some of the behind the scenes process of the GrammarlyGO launch.
Sarah Jacczak: To start off, I want to intro the go-to-market design team. The team consisted of product brand designers, motion designers, content designers, brand writers, design researchers, and design operations. This was a complex launch and we were designing something completely new, and there were a lot of moving pieces and constant changes. On top of that, we needed to move fast. Having a team with a wide range of expertise, it allowed us to work quickly and collaboratively, and we were able to impact areas across product, brand, and marketing for this launch.
Sarah Jacczak: I want to give a special shout out to the brand and content designers and brand writers. I’ll be sharing some of their incredible work on the GrammarlyGO identity and campaign later on. To give a quick overview of the scope of work, the brand design team worked on in-product systems, a new brand identity that included a new logo and color palette, and a go-to-market campaign toolkit, which included guidance on how to design and write about Grammarly’s generative AI features.
Sarah Jacczak: To do this work, we had to consider how users will interact with this new experience and how we would differentiate Grammarly go from competitors. This required close collaboration with product and engineering teams as well.
Sarah Jacczak: When designing GrammarlyGO, one problem we identified early on was, we needed a way for users to access this new experience. We knew that users were familiar with clicking on the Grammarly icon to open the Assistant Panel and accept writing suggestions, but integrating GrammarlyGO features with this existing UI was not an option for the launch and it was something we would have to address in the future.
Sarah Jacczak: For the launch, we needed to keep these two experiences separate. and we decided to add a second entry point into the Grammarly widget, which would open the GrammarlyGO experience.
Sarah Jacczak: Here are some early explorations of the GrammarlyGO entry point. So on the left we tried two different button designs for the desktop app and browser extension, and we consider it a badge treatment on the desktop app, which has floating widget. The benefit here is that on desktop, the widget wouldn’t be much larger, so it wouldn’t interfere more with text fields.
Sarah Jacczak: However, the visual treatment, it felt kind of like a notification and because of its small size, we were worried it wouldn’t attract much attention. And so we moved on to another exploration. On the right is another exploration where we considered having multiple inline buttons with different icons, so there would be a new unique icon for composed reply and rewrite features, but when prototyping this design, we found that it was a little too cumbersome, and so we decided to simplify it down to one icon for all GrammarlyGO features. And this is what we launched with a single light bulb icon to open the GrammarlyGO assistant window.
Sarah Jacczak: Having one icon as the entry point gave us room to surface prompts that have unique icons. You can see on the example on the right, we have the improve it icon with the pencil, and this prompt appears when a user highlights their text and it gives them a quick and easy way to generate another version of their writing.
Sarah Jacczak: While we were designing how users would access GrammarlyGO, we’re also designing icons. We started exploring icons before we had a name, but we knew it needed to be unique and it would live next to the G icon. We explored a wide variety of approaches. Some were more literal and represented generative AI, like writing and pencils and sparkles and magic, and other explorations were more focused on abstract representations of speed and ideation. But yeah, we could have kept going and going, and this is not even all the explorations, but because of the tight timeline, we had to make a decision.
Sarah Jacczak: We went with the light bulb because we felt it was effective in conveying the new ideation capabilities of GrammarlyGO. We also saw an opportunity to design new product iconography for prompts. These icons would accompany the suggested prompts that appear based on a user’s writing.
Sarah Jacczak: Early testing showed that prompt writing is challenging, and so we prioritize these suggested prompts that are based on a user’s context, and we wanted to make this experience more visual and more delightful. Again, icon explorations range from abstract to literal, but we saw that these icons needed to convey meaning, and also support the prompt compi so we move forward with the literal direction.
Sarah Jacczak: Another discovery was that operating within the new limited color palette was challenging and it didn’t quite feel unified with the existing UI, so we looked to Grammarly’s tone detector iconography, and these emojis, they would appear in the same UI as the prompt icons, so it made sense to create a cohesive experience here.
Sarah Jacczak: We referenced the colors and styling of these emoji to create the foundation for the new prompt icons and here the prompt icons that we designed for launch. You can see they’re literal in that they depict the meaning of the prompt in a simple way, and keeping them simple also ensured that they could scale and be legible at small sizes. We also selected colors and subtle gradients that felt cohesive with the existing emoji icons. This resulted in an icon set that feels warm, friendly, and is hopefully fun to interact with.
Sarah Jacczak: We also needed to consider scalability. There would be hundreds of prompts and we wouldn’t be able to design an icon for each prompt, so we grouped them into categories. Each category has an icon, and within that category are prompts that share that icon. For example, any prompts about writing or composition, we’ll use the pen and paper icon and any prompts about ideation, we’ll use the light bulb and so on. We also identified which prompts we feel would be used frequently and created unique icons for those to add variety and more delight.
Sarah Jacczak: While some of the team was working on iconography and content design in the product, others were working on the identity and go-to-market campaign. Here are some of those explorations – a variety of logos were explored and taglines, as well as graphics for the campaign, and some visual explorations use gradient orbs while others focused on movement and transformation by using over layers of shapes and lines. And for the go-to-market campaign, we created a new tagline as well – ‘go beyond words’. It’s active and it conveys Grammarly’s ability to assist users beyond their writing.
Sarah Jacczak: We also designed a new logo that incorporates a bolder G with a circle forming the O as a nod to the classic Grammarly button. For the GrammarlyGO identity and campaign, the brand design team landed on a concept that uses overlapping shapes to convey transformation and the iterative process where one idea is built on the next. The softness of the gradients also speak to the human qualities, and they’re juxtaposed with hard edges to represent technology, and these overlapping shapes were further brought to life with animation.
Sarah Jacczak: The team also worked on a design toolkit, and this toolkit was shared across the company. The toolkit included logo, color palette, illustrations, photography, motion guidelines, and a library of product examples to be used across the campaign. A style and verbal direction guide was also created to ensure how we speak about GrammarlyGO is consistent. The brand writers provided headline examples based on themes. There was headlines about creativity, such as let your ideas take shape, headlines about productivity, such as ‘discover new ways to get things done’, and headlines about trust like ‘AI innovation with integrity at its center’.
Sarah Jacczak: This campaign was pretty large. We had a lot of requests and a lot of marketing channels to design for, but because the brand and brand writers and brand designers collaborated and built these systems and guidelines, we were able to move quickly and create consistency despite many people working on the campaign production.
Sarah Jacczak: Here are just a few examples of the work created for the campaign. The team created a series of demo videos and animated gifs that show product functionality, and these were used across marketing and PR. The team also worked on onboarding emails, landing pages, in product onboarding, blog posts, ads, and social assets,
Sarah Jacczak: To get a further sense of the scope of work, here’s some numbers from the naming and identity work. Over 500 names were considered, 188 Jira tickets were completed, over 105 taglines were explored, 45 videos explored, 39 product examples designed and animated, and over 230 logos were explored. And so, while these numbers don’t tell the full story, and we had challenges along the way, the team was able to overcome this and collaboratively design a new experience and produce a successful launch in a short amount of time. Thank you.
Charlandra Rachal: Thanks Sarah. To all the speakers that put together this incredible presentation, I learned a lot and I work here, so I hope you guys really enjoyed all of that. Let’s welcome back all of our speakers for Q&A. There is someone who is going to be in the audience with a mic, and I see it first hand already, so we will get a mic right over to you.
Audience Member: As mentioned, it was uncharted territory. I was curious how you went about ideating the first project. Was it based on existing user information you had? Was it academic papers? How’d you go about it?
Jennifer van Dam: That’s referencing my talk, so happy to talk more about it. What I mean with uncharted territory is the solution. We knew it was a problem – we’ve been hearing for years that since we started, we hear from our users, they struggle with these communication problems. What was the uncharted territory is the solution and delivering the product in a way that lends and resonate with our users.
Jennifer van Dam: The approach we decided to take is directly into the prototyping stage because we felt it was really important to connect the text and the product to the user. Let’s say, you want to compose an email, we can design and show you concepts, but we need that moment of you writing your text and seeing the output. That’s why we jumped right into the prototyping stage as our way to research the solutions and the design approach.
Charlandra Rachal: There was another hand right here…
Audience Member: Hello, my name is Kate, and probably question also to Jennifer because it was on one of your slides. When talking about prototyping, you were speaking about more empathy and I took a screenshot. Let me see how it looked like there.
Audience Member: ‘Deeper user empathy.’ Can you please elaborate a little bit more on that, how it worked? How did you do it while you were still prototyping, please?
Jennifer van Dam: Yeah, deep user empathy. What I really meant with that was to understand and dive into the types of things our users are trying to achieve. What are the types of use cases that, here’s a prototype, did you use it for rewrites? Did you use it for emails? What were those things?
Jennifer van Dam: We did so many sessions talking to people and getting their feedback to get empathy and then of course we had questions, but then the feedback we got was, ‘oh, but it didn’t sound like me’. This is what I mean by deep user empathy is really getting into the mindset of empathizing what is working and what is missing. That really helped us inform iterations and changes and new features or scrapping features.
Audience Member: Thank you. I would assume that the launch of ChatGPT definitely affected Grammarly. What would be the key learnings? What were the key learnings for you as product leaders from getting the LLMS viral and thanks for the presentations. My name is Maria.
Heidi Williams: One of the things which I think was interesting – we’ve been doing AI for a long time, we’ve used a lot of different technologies, whether it’s rule-based or machine learning, or all sorts of different technologies, exploring LLMs on our own.
Heidi Williams: The biggest thing about ChatGPT that ended up, was actually the discussion in the world about how to use AI as an augmentation tool. Before, you would have to convince people AI was okay and trusted and then all of a sudden overnight everyone’s like, ‘of course you trust it. Look at this’.
Heidi Williams: Now all of a sudden we don’t have to waste time talking about should you use AI? Now it’s about how can we be a trusted partner on how best to use AI and help you be more effective and help you succeed in your job or in your life. It has changed the conversation of ‘should I’ to ‘how should I’, and that’s been interesting and amazing that we can now focus on just solving real problems as opposed to convincing people they have a problem that AI can help with.
Audience Member: Thank you so much. First of all, want to say huge shout out for Girls Geek and Grammarly for putting together. Thank you. That’s a great event. I’m a huge fan of Grammarly. Go now. It finally sounds like Shakespeare in my emails and not like a broken machine. I want to ask you this question. I think Heidi and Bhavana, that’s questions for you. You said that one of the feedback was, ‘it doesn’t sound like me’.
Audience Member: Are you using LLMs to train data which your users are input? And if so, how do you also prevent some data security in terms of, for example, I’m putting something in my email as a product manager about revenue or about some specific of the product, which haven’t been on the market, and I’m always a little bit worried where this data is coming from. Yeah, I think it’s a good question. One is LLMS for make me sounds like it’s me. And the second one is the data privacy. Thank you.
Heidi Williams: I can talk about at least part of it and then if you have things to add as well. Because Grammarly has been around for so long, and that we are a trusted source, we were able to negotiate a really amazing contract with our LLM provider, which means that they don’t train on any data that we send to them. Not everybody could negotiate that, but because we’re Grammarly and we’ve been around so long and have such a big user base, we were able to do that.
Heidi Williams: I feel like that was a huge thing that is very differentiated from just using whatever’s on the market is that they’re not training on any of the data that we send. From that perspective of the data privacy, but if you want to talk more about the my voice and about how we do that and how do we then, if either of you want to add…
Bhavana Ramachandra: I’m going to pass to Jen.
Jennifer van Dam: Could you repeat the question?
Audience Member: Sure. How you make, if we’re not sending data to any LLMs, how you make it sound more like me, for example, GrammarlyGO, always suggesting me to be more assertive, which I think I’m already too much and then I’m like, no, no, no, let’s make it more positive. Yeah, how this happened, how it sounds more like me is like data being trained.
Jennifer van Dam: We look at context and communication patterns, so it doesn’t necessarily train on your data per se, but on the patterns of your communication and the context. That’s how we understand your voice profile across.
Bhavana Ramachandra: To add to that – we understand what your tones you prefer or you use are, but we don’t actually pass that on to LLMs. We had our tone detectors since 2019. We’ve been telling users how they sound for a bit now. We use that information to really update the writing rather than train the LLMs itself with your data.
Charlandra Rachal: I feel like I’ve been neglecting this side over here, so right here in the front.
Audience Member: More of a quick technical question. Do you list all of the tones that you detect for and measure somewhere publicly, or is that behind closed doors?
Jennifer van Dam: We have a homepage that lists a lot, but not all, of our tones. We feel it’s too competitive to reveal 50 plus. But yeah, you definitely can find information about a lot of our tones that we support with tone detection.
Bhavana Ramachandra: Maybe this is a challenge. Can you write enough with Grammarly to find all of them?
Audience Member: The tables have turned. I wanted to direct a question to the second speaker after Jennifer van Dam. Yes, you, um, why does my voice sound like this? Ah ha ha I see what you did there. The question I have more specifically was, when tone is being cited as a suggestion, when you write a sentence and it connotes that, ‘oh, your tone is serious and neutral’, and when you add a word or two and it changes the tone entirely, I’m curious, what quantitative scales do you use behind the scenes to make those on the spot judgements? You mentioned your team had a lot of linguists on it, and I was hoping you could expand on that because that has been an object of curiosity of mine for a while, possibly. Thank you.
Bhavana Ramachandra: Yeah, I’m just going to Jen about… Yeah, I think for, in terms of how do we decide, in fact, when we started looking at rewriting for tone, I think our initial exploration had just neutral and new tone versus in certain cases we were actually able to provide three levels, friend, friendlier, friendliest, but really depended on how much data we’d have. If you can actually, you can be neutral, but you can’t be more neutral. It really depended on the tone and how much data we have.
Bhavana Ramachandra: I think this is a part where our linguist really helped us really dive deep into this and look into each of the tone that we have, get data for each one of these tones. I think for our tone retype explorations, we started with the tones where we had most understanding and first started with two levels and then moved on to three.
Audience Member: Thank you. Hi, my name is Leanne and big fan of Grammarly. My question for whoever thinks that they’re best equipped to answer this is, could you tell us a bit more about how Grammarly is fighting bias, and what are some of those solutions that are currently in place? And maybe thinking about, in the future roadmap?
Bhavana Ramachandra: Yeah. Our AI is one of our biggest tenants, as Heidi said, and we’ve always invested in this area. The couple of things that we have done are actually very, very public. We have blog posts about how we look at pronouns or bias in gender bias in data, and how do we make sure our generative AI suggestions, how do we measure that and how do we prevent that as something that we do? And as Heidi said, this is part of the process.
Bhavana Ramachandra: This is not something that you think about at the end of the day, you plan for this. You plan to have a sensitivity analysis right from the get-go. The other part of this, we’ve published a couple of different papers this year. In fact, in the REI space in, I want to say ACL. Okay, thank you Dana. You’ll actually find a lot of public information. I don’t want to pretend that I know more than I do in this area. I am getting onboarded though. Definitely, we have blogs and papers out there that talk about what are the solutions we have implemented.
Heidi Williams: Maybe just one thing to add to that is part of it is actually cultivating a good data set because you could imagine that you just take, I think we’ve seen this with LLMs as well, you just take the words out there and you might see a bias of a gender bias around when you’re referring to a male, they might be more associated with certain words in the general public than a female. Then you would imagine that percentage wise, it might suggest like, oh, if you’re talking about a man, you must be referring to this, et cetera.
Heidi Williams: We’ve done a good job of cultivating our data sets to help ensure that the data sets themselves are not biased, and that’s a huge aspect of it is just making sure that we’re not having any gender weighting as one example, or it could be racial, whatever it is. There’s just making sure that you have a data set that’s representative and it’s not going to sort of skew things in one direction or another.
Bhavana Ramachandra: Do you want to add to that?
Jennifer van Dam: I wanted to add to that a little bit about how much we care about this investment. We also, besides of course all the deep investments in the modeling, we also have inclusive language suggestions for end users that help basically eliminate gender bias in your language while writing or talking to your coworkers or your team.
Jennifer van Dam:This is an area I also worked on and it’s a really great part of our product. For example, maybe you’re writing, ‘the businessmen are wearing suits’, we’ll underline ‘businessmen’ and we’ll ask if you’re writing to an audience that you want to be inclusive of everyone, maybe replace it with ‘business people’ rather than ‘businessmen’. We also tackled this from the end user standpoint and helping them communicate more inclusively and eliminate bias where they’d like.
Audience Member: <inaudible>
Charlandra Rachal: For those who didn’t hear, the question was, how much do we focus on educating, when people continue to make mistakes in their writing?
Jennifer van Dam: The inclusive language product, I encourage you all to check out because education was a huge part of the product UI and the way we wanted to position it. We always want to be educational rather than forcing you because at the end of the day, the user is in control. You have agency. That’s what we always believe in. We also realized we have to explain why are we saying ‘consider replacing businessmen with business people?’ How do we explain? Because some people don’t realize that it’s so ingrained, you just type it, you don’t really stand still.
Jennifer van Dam: Another example, like whitelist, blacklist, that suggestion, a lot of users didn’t understand why we were suggesting to replace blacklist with blocklist, so we actually focused our UI around education. Why is blacklist/whitelist is perpetuating certain stereotypes, so consider replacing it – and that was a real aha moment because when you say that in a training, it’s different than when in real life you’re writing a text and seeing it and applying it, so it’s actually been really powerful in educating people
Audience Member: To Jennifer and Bhavana. Earlier you mentioned deploying LLM models, initially, were you were skeptical how receptive the users would be and how they would perform.
Audience Member:Can you talk about your A/B testing strategies? Did you roll out to a part of your audience, I mean part of the user base first, and then started gradually increasing, rolling out the new features too, especially the generative AI features? Could you talk about your A/B testing strategy and how did you scale it to the whole user base? And after you employed gen-AI features and these new features that you earlier talked about, how did it impact the revenue subscription revenue and the user base?
Heidi Williams: Start?
Bhavana Ramachandra: I think Jen covered maybe some of the A/B tests. I’ll let her, I can maybe speak to the launch plans, the alpha testing. I can cover that a bit. Especially for projects like Tone where we did iterate over quality quite a bit, we would try to identify one. We had internal annotations. Every time we improve our quality, we do more internal annotations to understand how much of a bump is it? And once we have a fair understanding, we run experiments with Gen AI, we had to take a slightly different process.
Bhavana Ramachandra:As Jen said, we had more alpha testing with users, really deep conversations in terms of understanding what is a useful generative AI LLM feature because we’ve had rewrite features in terms of generative AI for the longest time, but what’s a useful composed feature? What’s a useful quick reply feature? All of that was not really A/B testing. We were building understanding in this case. That was a lot more alpha testing.
Bhavana Ramachandra: Then for the launch plan itself, we have 30 million users, we have five different surfaces, we have extension, we have desktop app, we have an editor, we have our website, and we’re in many different countries, so this to me firstly was the most impressive one, because we had to do a geo launch across many different clients that all have different release cycles, so all of them had to be in sync because we wanted feature parity. We started with certain countries to make sure, one, we can handle the traffic. Two, all the features are looking or performing as they should.
Jennifer van Dam: The difference with how we approach modeling quality, it depends on the maturity of the track. In the zero to one stage we do a lot of offline quality evaluations and make sure it meets our quality bar and the metrics. We don’t necessarily test out multiple models yet, but in the iteration stage we do. One example where we’ve A/B tested our improve it rewrite, which in one click will improve your text, and there was a lot of experimentation we did there with tone behind it and conciseness and what lands the best with improving my text with one click. Typically, we focus a lot on offline quality evaluation of our models, and then in the iteration stages we do a lot of A/B modeling.
Audience Member: After new features, did you see any bump in the overall user base?
Heidi Williams: I can’t talk about specific numbers, but I think obviously there was a lot of excitement and interest in this area. I think we did see that there was new interest, and then also just seeing interest and engagement from our existing users using the product maybe in a different pattern than they had been before as well. It definitely feels like there have been changes, but I can’t speak about specific numbers.
Audience Member: What’s your tech stack typically for the whole GrammarlyGO is hosted on?
Heidi Williams: The tech stack? A lot of different parts of it… it’s hard to answer with a really quick answer. Our particular LLMM provider is Azure OpenAI. And then there’s a variety of tech stacks above that – different things that we’re using for the linguistic side of things, and then there’s Java, there’s Closure, there’s all sorts of different technology stacks and then we run on AWS otherwise.
Charlandra Rachal: Thank you. Then I only have time. Oh, oh, oh. I was coming for you. I know you had your hand out. If you still wanted to answer, we would definitely break the mic over your way. The last I have one time for one more question. Alright, Nancy
Audience Member: Oh hi. Alright. I’ve used Grammarly for a really long time and this may be more of a product manager question because I’m also, I can write circles around everybody, so I don’t really need GrammarlyGO. I’m actually wondering, I’m thinking about, the roadmap further down for advanced writers, people like me who write. What’s coming?
Audience Member: Because I will say now, I use Claude a lot just to be like, Hey Claude, this is what I wrote. What do you think? And then Claude will say, that’s really good or not or whatever. I’m just wondering it’s GrammarlyGO moving in that direction for people who don’t really need help getting stuff on paper or on screen.
Bhavana Ramachandra: These are the comprehension projects that I was talking about. They’re all about trying to understand what the user is reading or what the user has written. For example, tone is something, even if it’s not correction, if it’s not editorial, you still might want to understand how your tone is coming across, especially in cross-cultural communication.
Bhavana Ramachandra: That’s something that’s helpful and in general as well, especially for long writing, we’ve gotten a lot of feedback about, I think this is one area that we were investing in – how we, so we show top three tones and let’s say people use Grammarly to write books or their fictional books, and does it make sense to show top three tones? Then they want a different – so this is the kind of evolution of the features that we see.
Bhavana Ramachandra: Comprehension is one area. In our generative AI in GrammarlyGO, if you actually open it up in a document, we provide a lot of prompts around understanding the gaps in your document, identifying what are your main points. All of these are just comprehension. This is just not how to improve your writing. Rather like this is what’s there in your document. You can review it based on a couple of different dimensions.
Audience Member: I should use Claude and Grammarly. Yes.
Bhavana Ramachandra: That’s the answer.
Charlandra Rachal: Yes. Alright, I wanted to say thank you so much to all of the speakers here and all you wonderful guests. I’m going to give a shameless plug if you didn’t already see, I’m in recruiting and we are hiring! Definitely talk to us, talk to me. I know we’re going to send a link out as well.
Charlandra Rachal: I believe there are more refreshments in the back and everyone is welcome to kind of hang out, chat, network. If you have more questions, I feel like we got through a lot of them without telling all of our secrets, but feel free to pull them aside and ask more questions. I hope you have a great night. Thanks again for coming out.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Breanna Carodine (People Operations Specialist at Vannevar Labs), Caitlin Stangland (Senior Talent Acquisition Specialist at Vannevar Labs) and Ann Zeng (Software Engineer at Vannevar Labs) talk about the company’s culture, values, and mission, as well as the opportunities for growth and development within the organization.
VANNEVAR LABS combines top software engineering talent with decades of mission experience to get state of the art technology to the people that keep us safe. Vannevar Decrypt is a foreign text workflow platform built for national security.
Angie Chang: We are here today to kick off our career fair session with Vannevar Labs, and I want to introduce Breanna.
Breanna Carodine: Hi everybody, super happy to be here and to talk to you guys all. My name is Breanna Carodine. I work for Vannevar Labs as the people operations specialist, and that just means I basically work with a lot of onboarding stuff and employee experience and a couple of other things like that. I’ve worked here since December of last year and it’s been such a great company so far.
Breanna Carodine: We’re growing so fast and we’re doing a lot of really great things. Some of my favorite parts about the company is the way that we value certain things. Transparency is really important to us. We want to make sure that everyone is aware of everything that’s going on behind the scenes. We’re not afraid to share some of our trials and tribulations with each other but also celebrate our successes.
Breanna Carodine: One of the things that we really love is that we are a team of Jedi, meaning that all of the people who work here are really good at what they do, and that means that we can accomplish so much more. And everyone has very similar mindsets around being user and mission focused on our work.
Breanna Carodine: Everyone comes together with the same mission, being really great at what they do so we accomplish so much. Also, a value that I think is really important as that we put your wellbeing first. Something I tell every new hire who meets me and everyone within the company, “You cannot do your best work if you’re not your best selves.” And we truly, truly believe that here.
Breanna Carodine: We hope that everybody who comes into this company understands that we have time that you can take off. We have mental health benefits so that you can always be working at your best selves because we truly, truly do care about that.
Breanna Carodine: We are also a remote-first environment so that you can create the environment around you the way that it’s going to be best for you to do your work. I know that’s really great for me because sometimes I don’t need a lot of people around to get a lot of work done. Sometimes, I remove myself from this space and go to a coffee shop and work in a space that I can have all that energy, so that’s really awesome.
Breanna Carodine: Just a little bit about Vannevar and our mission, our basic spiel is that we bring together multidisciplinary groups of people with a wide range of experience and over 40 years of military experiences, engineers from some of the top tech companies and startups, and a passion for delivering mission-critical tools to support public servants on the front lines of the country’s most important national security problem.
Breanna Carodine: A lot of the work that we do, we work right alongside with the Department of Defense and we do some really critical and important work for the country. So to get into a little bit more of those positions and even more details, I’m going to pass it over to Caitlin.
Caitlin Stangland: All right. Hi everyone, so excited to be here today. My name’s Caitlin, I’m based in Northern California, I am a senior recruiter here working on technical and non-technical positions. I’ve been with Vannevar now since January. It’s gone really quick, it goes fast here. And some of the open roles that we currently have, I’ll just give high level overview.
Caitlin Stangland: Currently, we have our mission team and mission roles. Those are broken up into groups, mission success and mission development. Mission success, think of it as customer success within an organization. So most profiles we’re looking for are people with military backgrounds and then have worked within a customer success team at a tech company. So they’re used to working and communicating with clients.
Caitlin Stangland: These positions, our employees on the mission success team are communicating with our customers within the Department of Defense. And there is frequent travel involved with these roles. Mission development, think of it as growth and sales. Usually, we’re looking for someone with military background, and active clearance, and have that sales background and mentality. We also have various engineering roles open.
Caitlin Stangland: We offer internships. Currently, we have deployment lead internship openings for this year. We’re actively recruiting on those roles. We also currently have an opening for a data researcher position. This role’s actually very highly confidential so I’m not able to say too much about what this position does. Then we also are currently looking to build out our finance department. We have a controller opening right now who would own all of the accounting function at Vannevar and would build out that team.
Caitlin Stangland: Those are currently high level overview of our current openings. You can see them all posted on our site and on LinkedIn. Usually at Vannevar, our interview process, we like to really keep it to the same process across all positions at the company. Always starts with a recruiter screen, then a hiring manager interview dependent on the role.
Caitlin Stangland: For mission roles, we usually like to include a decom, think of it as like a conversation prep document that we would send you prior to meeting with the hiring manager so you have a focus of that interview. From there, there’s usually homework or a technical assessment. And then the final part is a top grade interview that’s usually one hour long where someone at the company just gets really, it’s just a lot of questions about your background and goes kind of role by role. So we like to just streamline it and keep it as quick as possible.
Caitlin Stangland: And overall, like Breanna was mentioning, we have awesome benefits. The company culture I love, the startup environment, Vannevar, we’re fast-paced, it’s very collaborative, we’re growing like crazy. But first and foremost, I feel like everyone just genuinely really likes each other.
Caitlin Stangland: We do a really good job of hiring nice people. Every team, no matter who you’re working with, they just do a great job of collaborating with one another. When I joined Vannevar, I did not have a defense tech background, so this was new to me and there was a learning curve there. And I never felt like no question was off limits, no question’s a bad question here.
Caitlin Stangland: Everyone’s open to educating and just giving all the information that you need to start your position at Vannevar. And we have great benefits, competitive pay, equity, the flexible working environment, remote working environment. We do have the flexible PTO policy.
Caitlin Stangland: Mental health like you were mentioning earlier, we definitely prioritize that so we do have a mental health benefit. And overall, I just feel like we’ve done such a good job of creating such an amazing culture. And I’m excited to talk to more of you later. All right, and I’ll kick it off to Anne.
Ann Zeng: Thanks, Caitlin. Hey everyone, my name’s Ann, my pronouns are she, they, I’m a software engineer here at Vannevar Labs and I’ve been here for a little over a year now. Also, my internet has been going on and off so I hope I don’t cut out. Well, I’ll do my best.
Ann Zeng: I currently work on the team that focuses on collection and ingestion of the foreign media that powers the Decrypt platform. Like Breanna said, we’re focusing on solutions in the national security space. And one of our biggest products is called Decrypt.
Ann Zeng: Let me talk everyone a little bit about day-to-day life as an engineer. We’re at about 30 to 35 people on the engineering department among a couple of different teams. I wouldn’t say that the team boundaries are super solid. Somebody described it as a semi-permeable membrane. There’s lots of projects that are across teams. We really welcome people to advocate for themselves in terms of what they want to work on, what they want to learn, and to the extent that there’s room to have people work on things slightly outside of their job description. That’s definitely still there. I mean, we are a startup. No one’s going to say, “Hey, we don’t want you working on more stuff. Why wouldn’t we want people to work on more stuff?”
Ann Zeng: In terms of the tech that we use, it kind of depends on the team, but I would say broadly Python, Node, React, TypeScript, Postgres, OpenSearch. Our stuff is deployed in AWS so yeah, if you have any experience in any of those or are you interested in any of them, feel free to come by and say hi.
Ann Zeng: Because we are remote first. We’re pretty understanding about people’s time zones. There’s no hard start or stop times for working hours, like if someone messages me but it’s like 7:00 AM my time, nobody’s like, “Why aren’t you answering my messages?” For required meetings, at least on the engineering side, we’re pretty light on those, sprint rituals, standup retro planning refinement, one-on-ones with your manager. Those are usually the required ones.
Ann Zeng: We really encourage people to sit in on other meetings. So there’s an engineering department wide round table, there’s a company-wide demo space for engineers to show off like, “These are the things that we’re working on.” I love to lurk in different Slack channels just to see what’s going on. We really encourage people to get involved or to just spread transparency about the progress of work, that kind of stuff. Like Caitlin and Breanna said, we’re trying to solve problems in the national defense space and we want to do it with some full people. I’m really excited to meet everyone. Please feel free to come by to the booths. Thanks.
Angie Chang: Thank you so much for these great introduction to you all and the roles and the teams. We’re going to meet you in your booth in about 10 minutes. And now, we’re going to move on to our next session. Thank you so much, Vannevar ladies!
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Sukrutha Bhadouria: Sanchika Gupta and Tutu. The talk’s title is “Beyond the Algorithm: the Human Element in Developing Trustworthy AI”. Welcome, ladies.
Yunwen Tu: Thank you. Thanks everyone for joining the session today. Today we will share our thoughts and learning about building trustworthy AI. First of all, we will share a little bit more about ourselves, ourselves. My name is Yunwen, go by Tutu. I’m a user experience designer. I enjoy using design as an approach to learn and solve problems for people in both digital and physical way.
Sanchika Gupta: Hey, I’m Sanchika. I’m a data scientist with experience in the field of technology, cybersecurity, and human-centered AI. In a former life, I was a computer science professor, though while we are building the human-centered trustworthy AI and have been working with each other. I’m generally curious about this thought of what is the human involvement in building the trustworthy AI.
Sanchika Gupta: There are the three topics that we are going to discuss in this talk today. The uncertainty and concerns of AI development. How do we build and maintain trust relationship in AI and the unique value of human expertise in the age of AI.
Yunwen Tu: What’s your reason, conversation, or search about AI? For me, I chatted with my designer friends, and we talk about how AI can streamline some of our work, and is there anything can be replaced by it so that we can do things faster, such as making icons or some simple websites.
Yunwen Tu: We also talk about the new design trends that brought by the new development of AI. Since the breakthrough of the large development in large language model, more people have exposed to this technology and learn about the possibility how AI can be applied to their work. Lately, I have even heard users, they share that they have the desire to add this and that function using AI in their product after their self-education.
Yunwen Tu: In general, I got a sense that many of us are concerned and uncertain about the impact that AI can bring to us or our society. We start wondering, has the time finally come, is finally replacing us now? As we already see AI in is applied in many fields such as healthcare, automobile and et cetera. I ask my co-speaker Sanchika, where do you feel AI has already made a big impact and what kind of role is AI playing?
Sanchika Gupta: Automation of AI may lead to replacement of certain human roles. However, AI also presents us with newer opportunities and creates our jobs easier. Let me talk about certain examples where I feel AI has been present around for so, so many decades now, and it already feels as a partner. First example that I would like to talk about is natural language processing.
Sanchika Gupta: AI has significantly improved natural language processing capabilities. There are virtual assistants like Google Assistant, Amazon Alexa, which utilize the AI algorithm to understand and respond to voice commands. Now, it may seem a little trivial to talk about these because they have become a part of our daily lives, and I personally use them on a daily basis to like set up reminders, ask about the weather, ask it to provide directions for a destination while in the car, making our daily lives more convenient and efficient.
Sanchika Gupta: Another example I would like to give is of natural language translation. There are platforms like Google Translate, which leverage AI algorithms to provide real time translations between different languages. If you go to a country where they don’t speak your language, you can still communicate effectively. I have done it myself so many times. With this, I want to say that AI may automate tasks that require basic skills while human can focus on higher level responsibilities, harnessing creativity and imagination.
Sanchika Gupta: The next question is how can we have more access to the system so that we can use AI as a partner? Generally speaking, education and awareness are crucial in fostering trust in ai. Trust is very essential for building a reliable, transparent, and available use of system. AI has been present in various forms for many decades now., even during my university studies, I delved into the neural networks topic and AI’s potential to replace job was already circulating at that time.
Sanchika Gupta: However, the conversation around trustworthy AI only gained prominence with the emergence of large language models, instances of AI generated hallucinations where the system just make stuff up, started gaining attention while getting a recommendation for an unwanted TV show may have minimal impact. A recent incident in which an AI system made a judgment on a legal case without the attorney’s verification highlighted the potential consequence. These recent repercussions have brought the issue of trustworthiness to the forefront, causing it to enter the collective consciousness of all of us.
Sanchika Gupta: Let me throw another example on you and you tell me which would you prefer. If we were to compare a trust in an AI driven car accident versus a human driven car accident, what would you choose the second time? My opinion as humans, we tend to trust other humans more than AI. How can we bridge the gap?
Sanchika Gupta: I believe that by focusing on AI literacy, upskilling collaboration, and ethical considerations, us individuals can also be empowered to embrace AI as a tool to enhance our skills, productivity and relevance in the job market. Now, I would like to ask Tutu, how do you as a designer build and maintain trust relationship in ai?
Yunwen Tu: As a designer, I started this journey by understanding the technology, especially why AI failed and why people don’t trust AI. The major distrust I learned in AI is the lack of transparency. We haven’t considered enough that trustworthiness is a high priority in building that.
Yunwen Tu: Many AI models feel like a giant black box sitting between the input and output. We don’t, we don’t know how it works and it just did it work. When handling very mundane tasks such as grammar correction, language translation, it’s great when they’re magically done by machines. But when it comes to like riskier cases with bigger impacts such as loan approval, it’s impossible to rely on a black box like this, which you don’t even know if can understand 10% of your problem. There are also potential ethical biases in a model that needs to be monitored closely.
Yunwen Tu: We help user increase the transparency, the observability and the visibility to make the process and the model more interpretable and explainable in their context work. That’s also baked in our design principles.
Yunwen Tu: And as part of our design principles, we also use design thinking method that to work with our users to understand what does trust mean to them, and discover how AI can solve their business problems in a trustworthy way. For here, I would like to give you two examples that we use the user interviews and other thinking methods to solve problems for our users.
Yunwen Tu: The first example that we are working with an insurance company to reduce their business loss. Through many rounds of discovery interviews with underwriters and their managers, we actually found their primary challenge is not about finding the best algorithm to analyze their internal data.
Yunwen Tu : Instead, they want to understand better how the events happened in the past 10 decades that had impact their current business performance. In the meanwhile, we feel everything moves way faster now. For our underwriters, they need to quickly catch up with all the new updates such as the regulation change, the new settlements of lawsuits in their professional area. In the end, we build a tool that use natural language processing to help our user to connect the dots and find the needle in the ocean of internet data, which is the result we would not expect if we didn’t spend that much time to talk with our users.
Yunwen Tu: The second example I would like to share is related to our ops platform. And as part of my UX research, I regularly chat with different users such as data scientists, business ops, and et cetera.
Yunwen Tu: I found the expectations from our data scientists on monitoring AI models are very different from other general business users. They’re not looking for a no code or a fully automated experience. Instead, their philosophy is not to trust the data, not to trust the model until they have seen enough evidence to take action. It’s very crucial for us to deliver those insights clearly and efficiently.
Yunwen Tu: From our users, I actually learned trust is not purely top performance, not the best performing model. Trust means making informative decisions after peeling off the complexity and the root causes. Now Sanchika, what does trust mean to you as a data scientist and how have you built trust in your practice?
Sanchika Gupta: Demystifying the AI systems and ensuring reliability helps human use them with confidence. There are certain visible limitations of AI, for example, drift observability, root cause analysis, bias and ethical use are important to establish trust. So let me explain with an example how I establish trust in AI.
Sanchika Gupta: We as data scientists do not tend to trust our model or data. Instead, try to gather enough evidence around it and then be able to trust it. Let me talk you through that process.
Sanchika Gupta: Let’s consider a case of an AI driven customer service chat bot used by an e-commerce company. The AI chat bot is deployed to handle customer inquiries and support requests. Over time, they noticed that there is a decrease in the customer satisfaction scores and there is an increase in the unresolved issues.
Sanchika Gupta: The first step at this point, which I as a data scientist would like to do is to check on the model performance, let’s say, model performance evaluation, reveal that there is a decline in the chatbot’s accuracy and performance compared to previous months indicating a potential issue. Then we can begin closely monitoring the chatbot’s interactions and collect data on input queries, chatbot responses and user feedback. Now by looking at all of this data, closely plotting it, we might be able to identify certain patterns or anomalies in the chat bot’s behavior. During all of this observability and analysis, let’s say that we were able to identify a set of queries that consistently receive incorrect or nonsensical response from the chat bot. Now these queries stand out as potential outliers as they significantly deviate from the expected behavior. Then the next step could be to, let’s say, look at drift analysis on the chat bots performance.
Sanchika Gupta: We can compare key metricses like customer satisfaction scores, response accuracy, and resolution rates over different time periods. During this analysis, we notice there is a significant decline in performance starting around the same time as there was an update in chatbot’s knowledge base. Based on all these findings, we start the root cause analysis and discover that the update to the chatbot’s knowledge base introduced some incorrect or incomplete information resulting in chatbots diminished performance.
Sanchika Gupta: Through this example, we observe how model performance analysis, observability, outlier detection and drift analysis collectively contribute to the identification of the root cause leading to targeted corrective actions for enhancing the chatbots performance. This provides a glimpse into the case that I mentioned above showcasing the methods employed to established trust in an AI system. This also demonstrates the importance of human involvement in analyzing and improving the AI system, reinforcing the notion that despite all of its capabilities, AI cannot fully replace human judgment and decision making.
Sanchika Gupta: Now this leads to my third question that we would like to discuss here. What is the unique value of human expertise in this age of AI? Now again, creativity, imagination, and diverse opinions are very unique to humans. If let’s say there were to be a discussion, we humans can participate in a discussion and arrive at different conclusions in the same situation
Sanchika Gupta: At the same time, AI lacks the ability to participate in a discussion as an equal lacking both opinions or any standing in human conversations. Now let me quote another example here. New neural networks founder Professor Jeffrey Hinton, six years back in 2016 said, we won’t need radiologists to analyze scans and image perceptional hams can do all the scanning and diagnosis by themselves. Six years have gone by and we are nowhere nearby. It is not because of compute power or resources because I feel that compute power and resources have only been growing in the last couple of years.
Sanchika Gupta: What I believe is that AI can only solve very well-defined problems. What happens when it is posed with ill-defined problems? That is where human ingenuity comes in. All AI attempts to do is to recreate memory and computation capability of human brain. But what makes human a human is not just being able to solve the task, but be able to synthesize the complexity of this world and make decisions on the basis of that. Now, at this point, I would like to ask Tutu to give her thought process around this topic.
Yunwen Tu: Thank you, Sanchika. Those are great takeaways and what you just shared also, remind me again the user interview process in our design method. We do that to understand user’s journey and the pain points, and then present a personal story that summarize our learnings and the synthesis. Sometimes I feel AI is like an abstract persona that is summarized in a way with a well-designed cover and well-defined title. However, when we are doing the interview, the persona story, it’s not about creating an abstract figure, but to emphasize with our users’ needs, their feelings and the the reason why they’re making those decisions. There are all, and also this process are all done through our communication and synthesis in person. But AI does not learn new insights as we do in those contexts. They also don’t understand the complexity of the world like us.
Yunwen Tu: For example, when I read the news, the debates on the news, when I also work with different people, design for different users, I always feel so, and this comes from our unique experience, our desire and the belief that makes us like diverse, unexpected. And sometimes we argue we also have conflicting ideas, but this also made the world and humans like very unique that AI cannot replace with. That covers all we want to share today. Thanks for staying with us. And if we have a little bit more time, please feel free to ask us question now or reach out to us afterward. Thank you all.
Sukrutha Bhadouria: Thank you. Thank you ladies. Yes I would definitely encourage everyone to reach out to you both on and ask their questions on LinkedIn. Let’s keep the conversations going and I really encourage everybody to rewatch this content, share this content as much as possible with everybody. We really appreciate the time that you’ve taken out Sanchika and Tutu. Thank you everyone for attending. Bye everyone.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Angie Chang: With us today, we have Rahmira Rufus. She is a next-generation enterprise security architect, a university professor, academic and industry scientist. Welcome, Rahmira.
Rahmira Rufus: Thank you. Thank you very much, Angie. Hello everyone. Very honored to be here to share with all of the insights, all of the wonderful things that’s happening at ELEVATE this session. Like Angie said, my name is Dr. Rahmira Rufus. I’ve run a gamut of things that I’ve done in the cyberspace – lot of technical work with the elements that she spoke about, contracting areas, purely and academic. This element focuses more on a lot of the outreach work that I like to do that I’m not really able to dive into within, you know, places of employment and things of that nature. That’s what I’m gonna talk to you about today. I thought this would be a really good topic to bring up at a Career Fair, as folks are trying to find their way and navigate through the employment process, excuse me, workforce and academia and that whole gambit, that pipeline.
Rahmira Rufus: I’m gonna dive right in. Like I said, the title is Exposing Gaps in Cybersecurity Workforce and Education. I like to say that this particular work or slide deck is stemming from some work that had a different direction. This is more of a tangent of that work. I’ll start by addressing why are we even doing this analysis? Originally we wanted to find out what was happening within the cybersecurity paradigm as far as that pipeline for supply and demand. The workforce for the talent and things of that nature. What we discovered is that this is not a unique problem. This actually occurs in other industries. One in particular biomedical things of that nature, where you have very specialized kind of criteria for resourcing talent and staffing needs.
Rahmira Rufus: What we discovered here that’s the reason why we’re doing a gap analysis. A gap analysis is a very simplistic term. We’re trying to see what are those kind of hindrances, are those kind of blockers for where a particular environment or situation is at an actual level of performance? How do you get to a desired level? How that relates here through this series that I use when I work with my clients, how that relates here is that we’re trying to see what is this huge disparity against folks trying to either jump into the cyber field, maneuver within their career path, or transition to higher levels, or, even in some ways, try to make an entire pivot in another direction. What we discovered in this work was that there are some elements that are not being taken into consideration that makes the process a little more convoluted than it needs to be. That’s pretty much what we’re addressed. If there’s anything someone needs me to go back, just let me know. but other than that, I’m gonna dive right into it.
Rahmira Rufus: What we started was, we looked at a generalized problem,. What’s happening within the cybersecurity workforce and basically feeling that void of the workforce, of the talent that’s out there and the talent that’s more importantly needed. We started at a global scale, and the diagram to the left, the far left, is what we collected as data, globally. Every single place on the planet is not represented, but we try to get a proper representation of the skillset and those particular numbers that we would thought were relevant for the sample population that we were looking at.
Rahmira Rufus: We saw a disparity when in the US and wanted to focus there. Now, I do want to say the data that was collected for this very huge gap that shows for China, <laugh> and India, there is some emphasis there. However, that’s not a part of the scope of this talk, but that is something that as time progresses, we would like to see if we could journey into that gap.
Rahmira Rufus: We have some assumptions, but, you know, since we’re scientists, we’re not allowed to do that. We actually have to follow a process. Right now we’re in pretty early qualitative work with that study. If I dive over to the area that we’re looking at, we then wanted to focus in the United States. In my next slide, I want to show where we started seeing some disparities. I’m taking those two slides over the far right and they’re on the far right on this slide. We had looked at basically a percentage in a kind of you empirical actual value for each one of the data points for the job openings up against, compared against the employed workforce, right? The diagram on the right looks like it kind of flows in a normal fashion.
Rahmira Rufus: The problem is that the diagram in the middle, we saw little disparity around 2021, moving into this year. It didn’t quite match, it wasn’t like a huge disparity, but that triangle with that slant at the bottom, that that variation was really, really off. We said, okay, look, let’s dive into more granular elements. And we started looking at for state, and once again, every state ends represented here, but this is a sample population that we think would be a best representation. We noticed was by state that matched or was similar, kind of congruent to the, the diagram on the far right, but we were still concerned about that percentage. Why did we get a percentage that did not look like the two diagrams that basically are on the far left and the far li right of the middle one?
Rahmira Rufus: That’s where we started diving into what we considered to be this problem. And the reason why this is important for this particular forum is because in the next few slides, you’re gonna start to see some of the reasons why when folks are looking to streamline and understand exactly what they want to do in cyber, it’s a little more complex than than you actually thought from us just taking that tally that you saw on the previous slide, a very general approach to trying to solve or answer this question.
Rahmira Rufus: What we discovered was that approach was a little off because we didn’t look at the true nature or components that were involved with the supply and this demand. Now if we jump to this slide, we’re talking about a simple supply and demand problem, right? We then associated demand with the knowledge, the education that’s requested, as you see in the chart in the top, and then we looked at entry-level jobs and then against paired up against their requested education. If you notice in each one of these charts, it seems like folks in the bachelor’s degree area, they’re winning, right? For these particular cyber skills. Then, you have some luck in the kind of the sub, those are pretty much a lot of the associate degrees, junior college, but also a lot of the certifications and licensing that people engage in when they either have done a lot of higher education / learning, or they could be adult learners, it could be all kinds of different reasons, but they don’t necessarily go through a traditional degree program.
Rahmira Rufus: Then you see the graduate level of folks having that kind of smaller percentage of kind of matching a little bit of the sub bas. Now, what is funny, thank you. What is kind of funny here is that in actuality, because you know, I’m a teacher / professor as well in my background, I notice that’s not true to what’s happening here. Students, uh, adult learners, uh, folks in the field trying to figure out where they are, where their next step is gonna be, what’s their next direction, they’re not having the success that’s being represented in these charts. What we had then decided, okay, since we’re talking about supply and demand, well now let’s look at the supply side. Now let’s look at the workforce. What we had discovered was that this was a much more complicated problem than we thought, than someone just saying, Hey, I would like to get into the cyber field and I’m just gonna protect, I’m just gonna pick this particular area and I should be fine.
Rahmira Rufus: I’m just gonna grow here in years. Now an example that I will give, and not to shed any kind of negative light or anything on this particular field, but if we look at fields like psychology, most folks know this is something that’s been told by everybody in that field. Psychology, psychiatry. If you are not pursuing the master’s or PhD level, you’re gonna have a difficult time finding the type of placement that you thought you would being that breakthrough psychologist or psychiatrist. That’s something that’s a normal situation in that field. From what I’ve told, it’s been worked on over the years, but that’s usually something that’s been accepted in this space. That’s not necessarily the case. There are absolutely so many options and so many variations of what you can do, not only in cyber, but if you go down into its larger focus, like a computer science or pure network admin, or you name it, this becomes a little convoluted.
Rahmira Rufus: Now, I’m being very, very nice with this slide. Thank you. I’m being very, very nice with this slide because the next slide I’ll show is actually a very large data set that we had to work through so that I could bring you such a pretty visual right here as far as just some of the things that you have to take into consideration when you’re trying to think about what’s going to be your cybersecurity career path journey.
Rahmira Rufus: One thing really quick, I want to note if folks are paying attention right here to this graph over here, notice how low security intelligence is, I can give you a little hint of that. Remember, this is a trend, a growing field. Right now, the reason why this number is so small, that is actually being developed as a track.
Rahmira Rufus: All of the data analysis, all of the business intelligence, all of the different elements out there are now being able to be fueled and tunneled into a path that’s going to be security intelligence. Just letting you all know, look for that on horizon. However, these other tracks, they had experienced the same thing previously. That is another added complexity to this field, as things change, as things morph, as we’re talking about the next generation of computing, how do you know where to go, where you’re at, what you need to be? was able to break down some of this in some of these charts, but, I think everyone can kinda understand what’s going on here. Right here, you’re talking about, we’re, we’re usually using like as the little bit of a control, right?
Rahmira Rufus: The job opening, say, nationwide. Then you have to look at the entry-level jobs versus the knowledge that you would need. Then also you’d have to look at the skills and knowledge that would be required for particular sets. And then on top of that, what type of certifications and licensing would I need to perfectly be where I wanna be or where it is I’m trying to go see just that quick. On the next slide, as I said, this is just to give you an example. We have so many repositories that we pulled this information from, and I was only able to get 25 in these tiny little visuals on this screen. That is not something we want you to do. What does this mean to all of you? All right, here’s a little in the second slide you saw that I was gonna talk about dilemma, and then I’m gonna jump into the, what I think is a quick solution.
Rahmira Rufus: Here’s a method that you could help you without having to go through all of that granularity that we discovered in this process. One, be proactive, right here is a diagram of a chart, right? That’s all of the data that you see before, and I’m going to have this recorded, basically provided for you in the next two slides is from platforms like cyber seek.org, ISC squared, the, their platform, their education focus of the nice framework, things of that nature. You can go and get all of this data, get all of this information, and you can get metrics about these particular fields, right? Right here is a combined chart, a visual where you should go out there instead of waiting to go to an employer or waiting to figure out what’s out there or talk to your professor or whatever, kind of overindulgent process that you normally do.
Rahmira Rufus: You go out and be proactive and find out what these, what these particular roles and these skillsets are, and start diagramming them and them in a skillset similar to this slide. Now, to break it down even more, and I said to DIY – do it yourself – break them down into different skill sets that you find to be requested skills. And like I said, and these particular prac platforms, you can go to the go there to find this information. Right here, this is basically a broken down version of the slide you just saw. And you can pick particular skills and find out where you measure right now, right? And what is it that you’re gonna need to either increase that proficiency or, you know, whatever it is that you’re trying to kind of get at, right?
Rahmira Rufus: Be realistic with yourself, okay? Because this is about improving you and making yourself a much more viable product as you move forward. Use elements like that to set yourself up for success. Some real quick wins, as I said – I can tell you about – is do your own analysis. Like I said throughout this slide, you don’t have to dive deep like we did with the tangent that we found as a new area of research for the disparity in the path between the cybersecurity workforce and its talent and resourcing, but at least try to leverage different areas, like I said, cyberseek.org, these different platforms.
Rahmira Rufus: On top of that, when you actually capture all of these roles, develop proficiencies with analytic service or, data, data analysis… You can leverage different visualizations, power bi, you know, build your chops in those areas or create diagrams or metrics like these. I will let us say, since this is a Career Fair, these types of visualizations go very, very well in front of potential employers. Department heads, You know, folks that are trying to move up within an organization and even for folks that have their own endeavors, you know, clients like to be able to see the skillset and, proficiency of your talent, of your personnel, and being able to provide these types of metrics really, really set the right type of parameters for what you want to show as one, being proactive and kind of owning your area.
Rahmira Rufus: Some of the other things do self-assessments and skills inventory, slightly like the one that I just showed you in the previous two slides. Try to keep a skills inventory. If these things ever come up, you can tell people, “Hey, I’m at a level three or six at this particular coding element” (or this particular project management software, whatever it is). Gauge where you are and know where you’re headed, right? Or where you’re trying to go. Put your skills to the test battle, test your elements, okay. And things of that nature. I’m sorry, Angie, are you gonna say something?
Angie Chang: I was gonna say thank you so much for this talk. We’re at time, but we will say that you are on LinkedIn and people can connect with you there. Thank you so much, Rahmira.
Rahmira Rufus: Thank you. I hope everyone enjoy my talk and enjoy the career fair. Best of luck and be the best You.
Angie Chang: Thank you.
Rahmira Rufus: Thank you. Bye-bye.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Sukrutha Bhadouria: Soundarya is a seasoned senior product manager currently leading growth for small and medium businesses on Instagram. Before Meta, she was a lead product manager at Yelp, where she’s spearheaded a team managing over 10,000 US restaurants. She launched Yelp waitlist kiosk, driving growth and innovation in the restaurant technology industry. Welcome, Soundarya.
Soundarya Chandar: Thanks. Excited to be here and really distill and share what I’ve learned over the years. I’ll kick off this talk on acing product management interviews with a little bit about me. This year is special because I’m celebrating a decade of being a PM in the Bay Area, which has meant that I’ve worked at different companies and I’ve been an interviewee over 200 times, and I’ve been on the hiring panel maybe about half of that time. As I was prepping for this talk, the stat actually surprised me too, and I was like, whoa, that is a lot of information and insights that I’ve gained over the years.
Soundarya Chandar: The goals for today’s talk are twofold. After being in over 200 interviews, I realized I did half of them quite mindlessly. I followed all the prep tips that you felt that you read on the internet, and I will be honest interviewing did feel like a second job. Sometimes the search took months and there would be times where I get through the onsite rounds and things don’t work out, and it really sucked because I was back to square one trying to look for the for the next gig. However, today I’m at a point where I actually love the interview process and the task, both being an interviewee, but also being on the side of an interview panel.
Soundarya Chandar: Today I wanna walk through a little bit of how that mindset shift happened for me and how that my day-to-day job played a big role in it. And the second goal I want to highlight for today is how developing this continuous learning mindset is so important, and it took me a long time to get there. I’d really want to help you all see the value of building your skillset, especially – this is so tough because as a PM our job day-to-day jobs are already so difficult. There’s so much context switching, there’s just so much your task to-do list is never ending, and there’s a life outside of work that we want to participate and be present in. There’s a way to actually get around it, especially as we think about time and balancing work and time and investing in our skill sets.
Soundarya Chandar: Before I jump in, I wanted to take a step back and talk about perspective – on things that we absolutely cannot control. Especially now, the market conditions are so tough. And, and all of this, just as we were starting to feel like we were recovering from a global pandemic crisis. There’s the economy, there’s the market conditions, there’s global factors, politics, policies that are changing. Every company seems to have a different hiring process. And of course, we are at the age of AI, and there’s technology advancements are happening so fast that it’s, it sometimes feels hard to really keep up with it. As PMs, we like to be in control a lot – so when all of these things are happening, it can feel pretty depressing and deplete our energy resources. Which brings me to my next point on what can be actually controlled here.
Soundarya Chandar: In my opinion, the only thing that you can control is in investing in your skillset. This is something that you keep building growing and it sort of never leaves you. The closest analogy is, of course, if you regularly go to the gym, your physical fitness is going to get better. The more experience you gain, you in theory will get better at decision making. We often forget that how important and critical it is to keep developing our skillset. I really believe that by focusing on building our skillset, we can actually showcase all of that very easily in an interview setting.
Soundarya Chandar: Let’s talk a little bit about what these skill sets are. The first time I looked at this map, I felt pretty overwhelmed. This is a resource that I picked from this really amazing PM coach I follow on Twitter – Shreyas Doshi – who has been a PM leader at many different companies in the Bay Area. This skillset map looks so daunting – like where do you even start?
Soundarya Chandar: The good news is – if you’ve been a PM for some time, you’re already flexing a lot of these skills in many different ways. But, we don’t often take a step back to think about how we are calibrating. Are we actually building and sharpening these specific skill sets? Early on in my career when I was interviewing, I used my intuition a lot. That is just something that I’m naturally drawn to. I would research the companies, I would research the products, I would have all these ideas of how they can improve it. In a way, I was really getting good at building my product sense, and I would do really well in those interviews.
Soundarya Chandar: But when it came to some other aspects of interviewing, like analytics, storytelling, communication, something or the other would just wouldn’t work out. When I started later on interviewing at FANG companies where the interview structure is very closely aligned to how you measure your skillset, I worked with a program online, which I can share later that, that modeled the interview set up along the lines of the skillset. I was doing more mock [interview]s and getting more into the program, I realized the side effect, which was that I’m getting better at my day job.
Soundarya Chandar: I realized that I’m avoiding doing certain things at work because it was just not a skill that I felt really confident in. The a-ha moment for me was, oh, I’m doing these mocks. I’m improving the skillset. How do I do that in my daily job? A typical product sense question would look something like this. You are the PM for an amusement park. How do you improve the visitor experience? The old me would jump at this question. I would have all these ideas buzzing in my head, but I needed to learn to pause, breathe, remember, what am I really trying to answer here? Remember the user, remember the pain points? Remember what business outcome I’m trying to drive? Because that is the structure ideally that you follow in an interview, but you of course adapt it to your style.
Soundarya Chandar: If you actually think about it in your day job, you, you are constantly exercising and flexing these skills when you write a PRD or when you’re doing a product review or a strategy session where you are honing in on who the user is, who’s your audience, and why should that audience care? When you’re working with your designer, either whiteboarding sessions, you are mapping out the various happy parts and unhappy parts, or when you’re talking to your partner teams, if you wanna get some collaboration opportunities identified. At all times you are actually flexing your products and skills.
Soundarya Chandar: Similarly, a common question would be, you are the PM for talk’s feed. How would you set goals for your team? You may not. It’s a hypothetical scenario, of course. You start to think about, oh, what’s talk’s mission? Who are its users? What are their pain points? How do I measure the impact? Again? But again, you’re doing all of these things when you write your PRD when you’re setting up OKRs, maybe you’re debugging an issue, you noticed a dip in your metrics and you’re trying to figure out what’s going on. Or if you’re talking to XFN folks on tradeoffs, like, which features should we build? What has a better opportunity size? What will give us a more higher ROI?
Soundarya Chandar: To me, the shift really happened when I started to see the value of doing interview prep, the skills that it tests for, and how often we actually get to apply it in our day-to-day job. If an interview setting is so similar to a job setting, why aren’t we all acing our product management interviews? Well, obviously because there are some key differences, right? There’s time constraints. In an interview setting, you are you know, you’re given like 30 minutes, 40 minutes to solve that question. There is a lot at stake because you’re trying to prove who you are. There’s a lot about how you come across and how you are a fit for the role.
Soundarya Chandar: You’re interacting for the very first time with hiring panels, so you’re trying to make a great first impression and you’re evaluated on your ability to communicate clearly at all times. And this process can go on for many hours in a single day. But in your daily job, this is like a continuous process and you’re evaluated based on the success of the product. You’re interacting with customers, you’re interacting with your team members who you feel more comfortable with. There’s obviously some key differences, but I do believe that if we start to look at them slightly differently, start to look at how our daily job is preparing us for the next interview and how the interview process of prep preparation makes us better at our daily job. It can actually make the whole process much better give us more fulfillment from a career standpoint. And it does certainly work like that for me.
Soundarya Chandar: I think there’s two very important things of a mindset shift that needed to change. One is, how do I think about time and how intentional am I in taking the time to step back and think about how I did in my day job? It’s very rare that I’m actually questioning was my, this was my PRD number two better than PRD number one was, what was my opportunity sizing or my intuition on how a feature would perform? Is it improving over time?
Soundarya Chandar: A lot of times we do retros for the team to improve team processes, team culture, but very rarely do we apply that retro to our personal growth. The difference is, we certainly do it when we are prepping for an interview, but I really believe that not doing that on a regular basis, not allocating the time to check in and see how we are improving our own skillset, we’re missing out on the compounding effect it could have when it comes time to looking for the next gig.
Soundarya Chandar: This is a cool image I like to look at. This is a grid, from this blog “Wait But Why” – If you haven’t checked it out, I highly recommend that you do. This entire grid, both on the left and right is about 1780 hours in a day that is divided into 10 minute little squares. Obviously a big part of our day is occupied by work. All I’m saying is that we need to set aside three to four little blocks. It doesn’t have to be contiguous, but for this intentional practice of investing in ourselves and building our skillset and using that eight R chunk to actually keep flexing and checking in on how we’re building those skillset.
Soundarya Chandar: One of my mentors, she has a CPO role at a high growth company, two kids, does a lot to contribute and give back to the community, product community broadly. She was telling me how she finds time to journal every day. If you look at all the big leaders, they always talk about all these books and podcasts that they’re reading. It’s because I think somewhere along the lines, people have started to internalize how important it is to build this intentional practice of reading and really sharpening our skillset.
Soundarya Chandar: When it comes time to interview, when you’re actually ready to look for that next role, that intentional practice time block is probably occupied by mock interviews, which in my opinion is the most effective way to get conf to gain confidence while you’re interviewing. All the skills that you’ve been practicing in your work by recalling and being intentional about every time you have a conversation on who the user is, what their pain points are, by reflecting on it every single day or every now and then, that’s habit sort of naturally flows in to a mock interview time or in interview time when, when you’re just sort of then focused on keeping to the structure, keeping to the time constraint, but you’re more creative, you’re more insightful, you bring about net new ideas because you’ve been already practicing that previously.
Soundarya Chandar: Here are some ideas that has really worked for me about how to build that intentional practice, this image on the right, something I absolutely love. It used to be my screensaver for the longest time because it took me a long time to internalize this, the power of being consistent. This image on the right is about how the more consistent you are, the further you can get, but the more inconsistent you are, the harder it is to get to that next level. Certainly for me, when I started interview in my first early days of interviewing, I was pretty inconsistent. I would only care to reflect about what I learned, how the features, like how do I frame the success of the feature? How would I do it differently when it came time to interviewing? But now I’ve built a more consistent habit of writing, of occasionally sharing on the internet of what I think.
Soundarya Chandar: That has really helped change how I do my day-to-day job and also practice for interviews, pausing and critiquing products you see every day. This is an idea that I stole from Julie Zhuo, who’s a XPM design lead at Meta. She’s written a book on making the manager. She’s an amazing writer, highly recommend if you don’t already follow her to check her newsletter out. She talks about how there’s products all around us. All we need is to set that intentional time aside to think about, oh, what is it about this product that’s different?
Soundarya Chandar: The new Apple Vision Pro headset just launched yesterday. Did we take the time to think about, oh, it’s $3,500. Who would pay for it? What other use cases exist? Even if that’s just on a own, on your own personal notes, that is building intentional practice that really helps build a muscle. And the last thing about around numbers and execution. When I was prepping for execution rounds, I would study like what the population of the of United States is. What is that of California? And now when I look my look at my MAU, DAU numbers, or if I look at the stats of OpenAI and ChatGPT, I’m trying to actively put that in perspective.
Soundarya Chandar: Some closing thoughts on this. A mentor once told me that your next job is really a stepping stone for the job after that. That’s how you sort of build this practice of thinking about your career and you’re being more intentional about it, and you’re building the skillset that will take you to that next level. I’ve already talked a lot about making the time to invest in sharpening your skillset, but I also wanted to highlight the importance of, you know, occasionally paying for it.
Soundarya Chandar: Do courses, attend conferences to step back and gain some perspective. And then lastly, consider interviewing even if you’re not actively looking. One, you never know what job opportunity you may be passing up on. Two, it’ll quickly tell you where you stand and what you need to pay attention to in your daily job so that you can keep flexing that skillset muscle.
Soundarya Chandar: Lastly, I wanted to share this really cool visual on product manager competencies. It’s from Ravi Mehta’s blog. He’s also a very famous PM leader. If you notice that the top half of the section, as you start off early on in your PM career, your focus is on the top half of the circle, which is really about feature specification, data fluency, voice of the customer. As you grow, the competencies that you need to actually develop are completely different. By the time you get to a VP or CPO level, it flips to the other half of the circle.
Soundarya Chandar: This was pretty interesting for me to look at because it’s very easy for us to forget that when we want to get to that next level, we need to be intentionally building the skillset that’s going to be exercised at that next level. In closing, really making the time, being intentional using our day job to sharpen our skillset, which, trust me, will help you in your interview. I will set you up for success. That’s it. Good luck with your journey today.
Sukrutha Bhadouria: Thank you so much, Soundarya. This was wonderful. Thanks everyone who attended.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Angie Chang: With us today, we have Cindy Deng who is a leadership coach working with leaders to in tech to succeed in their careers while enjoying life. She has been working for over 20 years in the tech industry and her experience includes over five years at Google. She has worked as a business analyst, program manager and a product manager. Welcome, Cindy.
Cindy Deng: Thanks Angie. It’s great to be here today. Hi everyone. I’m here in the San Francisco Bay Area. Hello to those of you also here, and everyone else in the world. All right, we’re gonna start with a little brief story.
Cindy Deng: It was really early in my career when I hit this one critical point. I remember doing this one offsite at this beach to staring at the ocean, feeling really overwhelmed. You see, before that point in my early career, I had amazing managers for a good stretch of time. I worked hard deliver results and there was felt like it was a clear path. My scope increased over time and I became a manager, and honestly I didn’t think much about career development at all that much at the time. Or if somebody were to ask me, I probably wouldn’t even really know what that meant. <laugh>
Cindy Deng: Hardworking seems to simply led to good outcomes at the time. It felt straightforward, and then one day things changed. We had new leadership in place and everything felt different. I was no longer working with managers that knew my history and my strength and many of my new colleagues had really opposite style from mine and they were excellent at advocating for their ideas, it’s just amazing. But me on the other hand, I prefer deep analysis and I was a bit shy at speaking up in meetings, especially because English was not my native language.
Cindy Deng: Even though I liked the people I was working with, I didn’t feel equipped to navigate the changes and I felt lost. That day, I was sitting on the beach, I felt like I was in the middle of that ocean and just like waiting for the waves to come crashing down on me, but all of a sudden down on me that I could sit there and felt sorry for myself or I could get up and learn how to surf that wave.
Cindy Deng: I didn’t actually learn how to surf, but I could learn how to navigate the unpredictable changes and ultimately my career was up to me. A lot has happened since then. After about 20 years of working with engineering teams, I realized, while I really like working on products, I actually enjoy working with people even more. I now primarily work with female leaders in tech to navigate career challenges and elevate their leadership skills. What I’ve noticed in the last few years is that while career development is on most people’s mind these days, unlike that younger version of it’s not always obvious what to do, which is what brought me here today.
Cindy Deng: Today we’ll talk about how we can apply product management thinking into managing our careers. It’s just one of the many possible approaches to tackle this, but it’s certainly not the only way. Hopefully this will become another tool for you on your career journey.
Cindy Deng: Where are you on your career journey? Which one of these fields that resonate more with where you’re at? Are you looking at taking the next leap to reach for the next big thing? Whether that thing is a big project or promotion or something else that you’re excited about or are you more on the crossroad trying to figure out where to go next? It could be a pretty significant shift of your career direction changing type of roles that you have been working as or you’re moving from IC to a manager or potentially vice versa, whatever it is.
Cindy Deng: You might be debating options and considering how to navigate from where you are to the future direction or perhaps you’ve accomplished a lot in your career so far and you’re looking at how do you carefully balance all things that you’ve accomplished and what’s important to you and what do you do to put on top of that stack of stones while maintaining that balance – whether it’s health, fun, family, relationships or something else.
Cindy Deng: Throughout the talk today, we’ll be kind of going through a number of questions, kind of like this, really for you to think about where you are at, where you wanna go, and if it serves you, feel free to put in your thoughts in the chat or grab a notepad and then jot things down along the way. Yeah, I see some people already starting doing that. That’s great. And as you continue, what we’re talking about today really is a general framework. You can apply the formula and adapt it yourself and regardless where you are at with your journey. Throughout my career I worked on products most of my time. One way or another. I enjoy working with customers to understand their needs, figure out solutions with designers and engineers and delivering products ultimately make users’ lives better.
Cindy Deng: Over the last few years, as I consider how people navigate their career journeys and reflect on my own, it hit me one day, what if we were to look at ourselves as products? If you think about a product leader that you admire, whether it’s somebody you work with or somebody you observe from far, far away that on the stage, how would they approach a product that they’re passionate about? What would be their vision for it and what would be the strategy and how they approach it for the next few years and what would be needed during this execution so that it can launch successfully?
Cindy Deng: How would they advocate for the product that they’re passionate about to the world? Sometimes especially when we might feel stuck kind of in our journey or sometimes have a hard time talking about ourselves and thinking about your career this way gives you a little bit distance. Just backing off a little bit and thinking about what if for you with the product can potentially make it easier to think about it a little bit more objectively. What would you do? Think about this product is truly one of the kind. Anybody would be lucky to have it and one that you care deeply about. I know that because otherwise you wouldn’t be here today.
Cindy Deng: What’s the vision? For those that are ready to engage in the chat, if you just think about what’s your dream vacation look like and just pop in the chat like this one place you would go if you don’t have to worry about time or money or anything else, what would be your dream vacation look like? Just like the images here on the slide. Some people would want to go to the top of mountain and some people would want to get on a hot balloon. Some people say beaches are my way of doing lots of different ways. Everyone is different. Yeah. Like some of the answers coming in, it’s a lot of fun and one person’s dream job, it could be nightmare for someone else. Yeah. But so often we compare ourselves with others. If you don’t worry about anybody else’s opinion, what would be your vision for a successful, successful career?
Cindy Deng: Think about at the peak of your professional career, where would you wanna be? What would you wanna be doing? What role would that be and what size of the company would that be? Do you wanna be in a large size international scale? Do you wanna mid-size or small startup? What industry would that be? Do you wanna be an entrepreneur? Everyone’s answer could be totally different and every single one of those answer is perfect cuz it’s for you.
Cindy Deng: Sometimes these answers could be vague. It’s not always clear for everyone. Some people have really clear answers to say, I want to be this CXO of this one company, but for most people it’s not that clear. And it’s okay to be vague., because often when we’re trying to, even when we’re trying to go onto the highest peak, you can’t always see the highest peak sometimes, until you get past some hills before that.
Cindy Deng: It’s okay to be vague, and if you have absolutely no idea like how I was when I was growing up, I really did not like the question of what do you wanna be when you grow up? I would just make it up if I had to make an essay for it. Some of you might take jobs in the future that don’t even exist today. Who knows, right? If you don’t yet know, it’s perfectly fine.
Cindy Deng: Consider some of these questions. Reflect back on your journey. What brought you joy and what drains your energy, and what have you tried that you did not like? Consider time when things feel really good and you were in the flow, what was really good about it. These kind of questions can help you surface what’s really important to you, so you can figure out like just general direction you wanna go and if it’s kinda hard to figure it out and it’s okay to say I don’t have my vision for the next 20 years, cuz most people don’t.
Cindy Deng: What would be the direction you’d like to grow in the next few years? And just focus on that. And ideally it’s something that’s both a little bit exciting and scary cuz that would indicate things that are drawing you and also would help you grow next with a little direction or maybe a good clear vision of where you wanna go. Now it’s time to think about how do you get there. Just like planning on the road trip, you know, we have general direction where we wanna go, but there are some questions that you wanna kinda understand and the principles along the way. What are some key sights you might want to see? For example, if we were a road trip and we’re all going to Las Vegas, some people will wanna spend time in casinos and some will like make sure they pick out the best buffets to go to, and I will not want to miss any concerts or circus shows.
Cindy Deng: What kind of experience and learning do you want to gain in the next few years? When you think about the strategy, think a little bit distant, maybe the next few years or so. Do you wanna go deeper in areas that you do already enjoy, or do you wanna get a like a more breadth in areas that you may not know yet? Get a better kind of broader perspective around things? And how fast do you want to grow? Do you want to really rapid growth, or you prefer it to be a little slow and steady considering other aspects of your lives to for a better balance? And either way, it’s possible to have a rough itinerary but you want to really detail one because things are changing so rapidly these days. You can’t really just control a lot of things and think about what options might be in front of you.
Cindy Deng: There’s almost always multiple ways to get to a place and some of the options might be obvious, some may not. Sometimes you have to talk to others and discover those doors that might be hidden behind another wall that you don’t know about. And when you’re comparing options, think about what excites you. Often I talked about question yourself to say if you are looking to move to somewhere, are you running towards that option? Or you are running away from your current option if you have the time and the luxury to choose. I know sometimes people don’t, but which way do you wanna go and how do you wanna think about this?
Cindy Deng: Ideally there are little sparks that are drawing you and don’t get stuck in analysis paralysis. Sometimes people ask, do I wanna do this? Do I wanna do that? And it feels like I have to have a perfect option and perfect choice to optimize it all. It’s not always possible. Trust yourself that you can walk through a door and you decide from there. You can build build and you design your f way forward as always. Think about along this way also, what are some of the missing pieces that you might need? Are there new skills that you need to learn to get to the next phase?
Cindy Deng: What type of experience or opportunities that you might need in order to unlock the next opportunity? Think about what type of relationship do you need to build, whether that’s potentially mentors or sponsorships or maybe even peer support, like all the wonderful people here today in our conference. And think about the summer learning, circles potentially that will help you support you along the way. And sometimes it could be something a little less tangible like a mindset, especially if you recently transitioned to a new level or a new role. Maybe as a new manager or manager or managers.
Cindy Deng: What is something that you need to kind of change the way you think in order to lead at this level? Sometimes it’s like, how do I delegate better? How do I be more comfortable of I don’t need to be the one doing everything? Where are you spending your time thinking about the most strategic areas? Remember, what got us here sometimes doesn’t get us there. Yeah. The mindsets are important as well on your journey.
Cindy Deng: And last but not least, thinking about where you want to go and planning how to get there. Make sure you embrace your unique values. I sometimes hear people say, oh, well I don’t have a traditional background of this role. How do I go out and talk about myself? I said, that’s great. What is the unique thing that you are bringing that other people don’t have? How do you own your story and be really clear about it and be proud about that too. We can change the past, but you can think about how do you position that and then use that forward.
All right, so execution time. Not a lot different than how you would approach a work project. First, think about kind of how we prioritize things. Breaking down large large projects into smaller milestones and smaller goals. Where would you want to focus maybe for the next year or the next month? Sometimes we have too many top priorities. If the list of things that you wanna do and things you need to do are fairly long, how do you identify the one or two things to to really just pursue first? Because if you have 10 top priorities for a single team, you don’t really have a top priority. That will be the the top couple things to pursue.
Cindy Deng: Think about what kind of support do you need around you. It can take a village to have a successful product. For you to succeed, who do you need around you? For example, do you have a solid relationship with your manager? If you’re currently in a job, do they know about your career aspirations and where do you want to go? If your manager has idea of those, then they could potentially create opportunities for you along the way and allow you to practice the skills or learn things that you want to learn. Most managers are doing that these days, but if your manager has not had career development conversations with you the last six month or a year, try to start that conversation. Use the one-on-one time and just bring it up. It is your time and to look for manager support around things like this. You can share what you’re interested in learning and see what they can do about it to align.
Cindy Deng: Thinking beyond your manager, what about your skip level or other senior stakeholders that you may not interact with on the frequent basis, but they may be involved in the interest in the projects that you are working on? How do you make your work a little bit more visible beyond the people that you are enacting on a daily basis? And if you’re thinking about kind of exploring what’s next, actively on the job search, think about what kind of roles are you most interested in and really have lots of conversations with people. Networking can feel kind of scary at times, but really is the best way for people to discover opportunities and to find those kind of hidden roles that may not even be published on the job boards yet
Cindy Deng: In this phase as we’re trying things out and learning new things all the time, it’s potential to make mistakes. It’s like likely for us to make mistakes when we’re learning something new. Experiment and try things when you can kind of control in a small kind of setting. It’s less risky, but don’t be afraid to try and don’t be afraid to kind of get out of that comfort zone. And really, successes come from failures. We all have that tendency. I don’t wanna fail. What if I fail? It feels scary, but that’s the best way to learn at times.
Cindy Deng: Let’s say we finish all execution steps and we are ready for lunch. Who would be your number one fan? Any guesses here? Thoughts during the chat? For some time I would say, my number one fan might be my mom in my life, but on the career sense, As I mentioned, I have really great managers in the past. And as we think forward forward, what we really want to be our number one fan is ourselves. This could be really difficult for some people. I found it difficult, especially, for a good amount of time in my career. I grew up learning that it’s good to be humble. When people compliment us, my parents would say, “oh no, no, no, you’re too kind” and brush it off. Talking about ourselves is bragging. I took me a long time to understand that I needed to shift my thinking here, especially working in the US and corporate America. If we’ve done it, it’s not bragging. If we could talk about it in a humble way, it is important to really acknowledge what we’ve accomplished and our strength and our ability.
Cindy Deng: The hard truth is, as much as we like it, we don’t have people constantly advocating for us all the time. Even if our managers try to do their best jobs, they often have multiple people on the team. They can’t do that for just us. We have to do our part and advocate for ourselves whenever we can. Doing excellent work is great, but it’s also important to make sure that’s visible and known to other people. All right, I’m gonna fly through the next couple slides a little faster cuz we’re short on time.
Cindy Deng: After launch, how do we continue to improve ourselves? Getting feedback, making, making sure you remember to keep taking small steps. Don’t forget to take care of yourself. Physical mental health are absolutely important, so we don’t build up a tech debt over time. The key takeaways… today’s talk is all about you. How do you invest in yourself, the next step, and how do you keep trying? Love to connect with everyone here. Feel free to reach out on LinkedIn and I have set aside time this year to help with folks on their career journey. If you like some additional one-on-one support, feel free to reach out.
Angie Chang: Thank you, Cindy. That was an excellent talk. Thank you so much for submitting this talk in the speaker call for proposals. Many of our speakers today came through that process. I encourage everyone in the audience to think about what is your TED talk and what can you speak about at the next girl Geek X ELEVATE. I did happily book a time with Cindy to talk and I’m so excited for that time. Thank you for being a leadership coach and helping women in tech. All right, we’re gonna close the session and hop to the next one, but thank you again, Cindy, and there’s a lot of chatter. I encourage you to look at the chat and we’ll see you on the other side. Bye.
Cindy Deng: Thank you, Angie. Bye everyone.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
Angie Chang: My name’s Angie Chang, founder of Girl Geek X and I’m really excited to introduce our speaker. San Robinson is a mobile UI engineer at CrowdStrike, where she’s responsible for develop, developing user-friendly, intuitive, and responsible applications using number, she uses technology to advance education, environmental sustainability, and social good through unique and complex approaches. We’re really excited to hear her talk on AR, VR and her passion project. Welcome, San.
San Robinson: Thank you Angie for the intro. Hello everyone, my name is San. I’m a software engineer and a freelance technology consultant. I have a deep interest in the confluence of education and travel with this vision over the past six years – I’ve researched and begin developing AR/VR technology solutions like The Natives POV, where my mission is to cultivate language learning, cultural understanding, and global citizenship citizenship using immersive technologies and gamification.
San Robinson: Today I’m dedicated to revolutionizing language learning, utilizing AR.VR to create immersive experiences for people around the globe and to inspire and educate people like you so that they too can create world-changing AR/VR apps.
San Robinson: Have you ever been lost in an algorithm like Alice tumbling down a virtual rabbit hole? One minute you’re scrolling through cat photos and next thing you know, you’ve turned into an overnight expert on diabetic cat care. And yes, that’s the thing. From there, you zapped off into a whirlwind tour of at building and before you can say “Hello world”, you’ve teleported into a bustling Korean marketplace, all of this without moving an inch from your couch. Information overload, but without getting into the negatives that rhymed with my phone, let’s talk about AR vr for immersive learning, A new level unlocks, we’re stepping into the realm of augmented real, augmented and virtual reality for learning. It is like switching from a black and white TV to color only this time. You don’t watch it, you live it. A world where anything is possible, that is the power of AR/VR.
San Robinson: Now that we’ve opened Pandora’s box of immersive tech, AR, VR, XR, what is the difference between these things? Let’s start with VR. With VR, we step into a completely immersive and digital environment. Have you ever wondered what it would be like to live inside of a volcano or what it would be like to hang out with your favorite music artist? When you wear a VR headset, you’re transported into a different world. A user could be in a conference room in New York and instantaneously get transported into a virtual beach in Hawaii. Or imagine you’re a language learner looking to build your vocabulary, but everything around you is in the language that you know.
San Robinson: What if you can alter and enhance your world using just your phone? This is what augmented reality is, a place where our existing world and our virtual world merges. Think of Pokemon Go. Now, let’s say you have an important event that you’re trying to figure out what to wear. You may just have to go to the store and try on new outfits, which which can sometimes be a drag, especially when time doesn’t permit. Your other option is online shopping and waiting a few days with the possibility of you not liking it with mixed reality. You can try a new thing before you get it, and if you don’t like the look, you can just change it with the click of a button.
San Robinson: AR, VR, XR, but what is XR? It’s extended reality. It’s the umbrella term for AR and VR is considered immersive technologies. You may be wondering, with all this talk of AR and VR and tech advancements, how does this fit into your personal life? It could seem overwhelming or perhaps even disconnecting you from the real world, but that’s where immersive technologies like AR and VR come into play.
San Robinson: It’s the power to travel without moving to immerse, without diving, and to learn in ways that we never thought were possible. I’d like to take you on a personal journey of mine, of immersive experiences. In 2017, I spent a year in Gudo China. I experienced the intricate dance of language learning, the nuances of culture and the social art of living like local. In just one year I acquired more Mandarin skills than I’ve acquired in four years of high school Spanish.
San Robinson: Let’s be realistic. It’s not feasible for everyone to pack their bags and immerse the immerse themselves in a foreign country to learn something new. And that’s where technology steps in. Using immersive technologies, we can replicate essence of being in a completely different environment and foster learning through virtual immersion. Picture this, you’re living in China and you’re learning Mandarin from the comfort of your home, but it feels like you’re right in the heart of Beijing, surrounded by native speakers and vibrant local life, or, let’s say you’ve been intrigued by Chinese architecture.
San Robinson: With VR, you could transfer into a different world. You can be in the forbidden city experiencing the majestic history as if you were physically there. And that’s where The Natives POV comes in learning through native perspective from anywhere. While I too am a fan of the real world, I cannot overlook the immense potential of these technologies and bringing the world closer and truly making learning limitless.
San Robinson: While this might all seem groundbreaking, breaking, astonishing, even it’s relevant to each one of us as we strive to learn and grow in this interconnected world. Through my research, I learned something really interesting. The human brain processes visuals 60,000 times faster than text, and 90% of the information transferred to the brain happens to be visual. What does this mean for learning?
San Robinson: Let’s take a moment to consider the learning pyramid a visual representation of how different learning methods impact our knowledge retention. At the apex, we have passive learning techniques like lectures and reading, which results in just five to 10% retention. But as we move downwards, we find more interactive methods such as this group discussions, practicing, yielding higher retention rates, and about 50 to 75%. And at the bottom of the P of the pyramid, we find the most effective learning method, teaching others or immediate use of learning, which has an impressive 90% retention rate. It’s clear to see that as we descend the pyramid learning becomes more interactive and more engaging in thus more effective.
San Robinson: This is exactly where immersive technologies like AR and VR come into play, taking us straight to the bottom of the pyramid, enhancing retention and making learning, deeply engaging.
San Robinson: Shifting our focus to communication, an essential part of learning and sharing knowledge. The famed research by Albert Mahar breaks it down into three components, 55% non-verbal like facial expressions and body language. 38% vocal the tone and pitch our speech and only mirrors 7% of actual words used, helps us communicate each day. Imagine then the power of the immersive technologies that allows us to practice and learn environments where all these aspects can be incorporated in a virtual world. A learner isn’t just reading words or hearing a lecture. They are in the midst of action, practicing and learning non-verbal cues, picking up tones and picture the pictures of the language and actively using their learning. This depiction of learning pyramid and our understanding of communication is what makes AR and VR the next frontier of education and training.
San Robinson: Now that we see that leveraging immersive technologies can help with both communication and learning, what does that mean for cross culture communication? That means consider, consider a scenario when you’re American executive preparing for a crucial business meeting in Japan. With the aid of AR and VR, you can find yourself in a virtual environment mimicking a Japanese office, helping you experience the perfect bow or learning the proper way of exchanging business cards. These immersive technologies by creating realistic simulations of different culture, equips us with the knowledge and exposure that might otherwise require physical travel and a significant time investment. In essence, this allows us to explore and understand the riches and diversity of our global community, fostering empathy and refining our communication skills.
San Robinson: In 2019, the University of California began studying ultimate augmented reality. They used it to visualize scientific data via 3D models and videos to provide insight into human biology. In an app called Scholar, the the results are amazing. 90% of students using ar vr felt more engaged. 85% of them understood the subject more, and there was a 10% average in grade grade increase. Students really enjoyed the app and they said that it made learning more fun and much more easier to understand.
San Robinson: Today, technology changed a lot and these AR/VR technologies are becoming increasingly sophisticated and are being used in various ways from gaming and entertainment to education and training. One example of AR technology that we may all know of is Pokemon Go. This game allowed players to use their smartphones to see and interact with Pokemon in the real world. Another example is Snapchat’s AR lenses, and these lenses allow users to gain a digital filter and effects on their photos while VR technologies are becoming increasingly popular.
San Robinson: One example of VR technology is the Oculus Riff. This headset allows users to immerse themselves in virtual worlds. And another example is the HTC Vive. This headset allows users to interact with virtual objects using just their hands. One of my favorites is Google Expedition, which allows users to actually learn about different things in different countries and different places using just AR and VR.
San Robinson: But we should take a moment to really consider the backstage heroes and the cutting of the cutting edge immersive technologies. The people who are powering the development of AR and VR, places like the various software platforms, such as Unity and Unreal Engine, which serves as canvases where creators can design immersive experiences. Unity has a user-friendly interface that is widely recognized for their AR and VR developments helping creators animate and stimulate lifelike environments. On the other hand, Unreal, particularly with its MetaHumans app, is known for its robust graphics and cutting edge technology.
San Robinson: The MetaHuman app, which happens to be my personal favorite, brings photorealistic visuals to the VR world, enabling the creation of a astonishingly realistic human characters. It truly showcases the power of potential, potential of Unreal Engine and shaping the future of digital experiences, especially when it comes to acquiring a new language.
San Robinson: Then we have the hardware, like the magic wands that bring our interactions to life like Oculus Rift, PlayStation VR and new Apple Vision Pro glasses that are bringing digital information in our physical world. But it’s not just about the hardware and software. AR and VR rely heavily on advancements in computer vision and AI. These technologies allow systems to understand and respond to what they’re seeing, making our interactions in a virtual world more intuitive and seamless.
San Robinson: The confluence of these technologies is accelerating growth and adoption of immersive technologies, opening up limit limitless possibilities in our journey towards a more immersive and interactive future. As we step further into this exciting and transformative era, it becomes increasingly important for us to understand the specific advantages and obstacles associated with these innovative developments. Let’s turn our attention now to the benefits and challenges that AR and VR, a key player in immersive technology landscape, and imagine how it reshaping the future using virtual reality while simultaneously presenting new hurdles and that we must overcome as time goes on.
San Robinson: Some of the benefits in the realm of AR/VR, what striking to me was the increase in motivation and engagement, the immersive nature of these technologies, drugs, users, and keeping them engaged for longer and driving a curiosity to explore and learn. It turns learning from, from a task to an adventure.
San Robinson: These technologies also shine in simplifying complex concepts. Visualize visualizing ideas and intricate processes, which is, which are often a challenge for many people. AR/VR brings these concepts to life by making them more comprehensible and relatable. For instance, learning about the structure of atoms comes exponentially easier when you can explore it in a digital realm. AR/VR impact on retention and recall is also profound. The interactive experimental learning that these technologies provides. AIDS and memory retention, it’s one thing to read about the pyramids of Giza and there’s another one to actually virtually explore them.
San Robinson: By placing these learners in the real world of fantastical scenarios, AR/VR fosters problem solving and critical thinking skills Users can learn and navigate and adapt to and make a real decision based off of these environments, which actually helps them develop a more cognitive experience. And lastly, these technologies have the ability to ignite and spark learning. The assignment of exploring a virtual role or interacting with AR applications can turn the most reluctant learners into eager explorers. But in a nutshell, AR/VR not only enhances the way we learn, but it also instills a love for learning, which to me is arguably the greatest gift and benefit. Now let’s move into the challenges. Well, some insights AR/VR has actually shown 20% of increase in learning outcomes. Another thing is that 50% of education and institution plan to implement AR/VR in the next five years.
San Robinson: The insights are there and we can see that AR/VR has many benefits, but we should also focus on the challenges because with every silver lining there is some clouds, AR/VR is no exception. The first challenge we encounter is training. And the second problem we encounter is limited availability. But the really big challenge that I foresee in in the future is the high cost of this hardware, which can be prohibited to widespread adoption efforts to create ongoing affordable devices are u are being completed to bridge this gap. Now there’s also the user experience, which strides have been made to creating intuitive and user-friendly interfaces. It is also a significant challenge. Navigating the virtual roles should be as easy as it is to navigate the physical one. There’s also a question about the accessibility of and the digital divide that AR/VR might cause.
San Robinson: As we embrace these technologies, we have to ensure that they’re accessible to all regardless of socioeconomic status or geographical location. And lastly, there are ethical and safety considerations. Balancing immersive experiences with user safety and privacy is paramount. As we venture deeper into these virtual worlds, defining rules for data protection, user interactions and content moderation will be crucial. The journey towards perfect AR/VR is always challenging, but as we navigate these obstacles, the horizon poles of promising and immersive future future in the realm of education and learning, these technologies are revolutionizing the way we absorb and retain knowledge. They’ll create a more immersive experimental learning environments where students can explore the world, conduct complex lab experiments, or engage with Shakespeare’s plays in ways that we’ve never seen possible all this in the comfort of their classroom or home, what this potential extends far beyond just education.
San Robinson: For example, in healthcare, with immersive technologies, a practitioner can stimulate experiences of patients from different racial, ethnic, gender and socioeconomic backgrounds, or those with specific conditions such as autism, PTSD, and also physical disabilities. This immersive experience can deepen the understanding of how these patients perceive and interact with the world leading to more empathetic and personalized care. In the business world, AR can exist, assist in product design or provide employees with hands-on training. And let’s not forget about the entertainment industry where technologies will continue to redefine our gaming movie and live events experiences making more interactive and more, and making them more interactive and more immersive. But as we stand at the this technological crossroad, the possibility seems endless.
San Robinson: Challenges remains such reducing costs and refining user interfaces. But the trajectory of AR is really clear. AR is set to become an integral part of our life, part of tools that will transform how we work, play, and learn. Now that we have explored the potential applications of AR and VR across various domains, I hope you gain a better understanding of how these technologies can make a significant impact. We are truly standing on the threshold of a new era. I would like to thank you all for your attention and participation at this time. I’m happy to answer any questions that you might have and leave the floor open for discussions. Thank you. Thank you.
Angie Chang: Thank you son. That was an excellent talk on AR/VR. We were really excited to have you sharing your knowledge. I’m going to have to wrap up the session and talk to the next one, but thank you for joining us and there’s a lot of chatter in the chat, so I encourage you to check it out and connect with people. You can message them on this platform if you like. Thank you.
San Robinson: Okay, thank you Angie.
Like what you see here? Our mission-aligned Girl Geek X partners are hiring!
