Girl Geek X Planet Lightning Talks! (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

  • See open jobs at Planet and check out open jobs at our trusted partner companies.
  • Does your company want to sponsor a Girl Geek X webinar in 2021? Talk to us!
  • Transcript of Planet Girl Geek Dinner – Lightning Talks:

    Angie Chang: It’s six o’clock and that means it’s time for another Girl Geek Dinner, and this time, however, we are coming to you virtually for the first time!

    Sukrutha Bhadouria: Just going virtual opens up our access to you and to you to each other, few people in various time zones, some people who say they’re in London at 2 A.M.

    Angie Chang: I’m just super excited to be able to partner with Planet and bring this evening of talks to hundreds of girl geeks.

    Adria Giattino-Johnson: So today I’m going to talk about diversity and belonging and the climate that we’re at right now and how it’s not business as usual, and rethinking what diversity is going to looks like in 2020.

    Lisa Huang-North: And when you do make that leap into your new role, how long do you want to be there? Is there a stepping stone to another bigger career pivot? For example, if you’re moving to a new industry or is it a way for you to grow and really deepen your expertise, for example, within the industry or within the field?

    Sara Safavi: Along the way I’ve had to pick up some new habits, some new practices and ways of working in order to make my staye in remotesville as a remote employee sustainable.

    Barbara Vazquez: What I’m going to talk about today about agile development and estimation, because I’m a software engineer and we do agile development at Planet. These are some tips that might be useful on a day to day.

    Kelsey Doerksen: Today, I’m going to be talking a little bit about how to handle big data in space and the different machine learning projects I’ve been a part of over the past few years.

    Deanna Farago: My name is Deanna Farago and my team and I operate a fleet of satellites that are currently imaging the entire planet every day.

    Elena Rodriguez: I chose a topic because this is something that I’m always thinking about it, and now I have the opportunity to talk about it and I’m going to take advantage of this – this is how I ended up here, so I’m going to show you my story.

    Sarah Preston: Stories are passed to community and understanding. So think about all the stories that you loved growing up. There were some kind of connection that you made, either to a character, to the author or to the setting that drew you in and made it really memorable.

    Brittany Zajic: I’m on the business development team here at Planet. Business development means something different at every company. Here we focus strategic partnerships and the commercialization of new markets.

    Nikki Hampton: At Planet we have always been committed to diversity, but we are doubling down on our commitment and particularly so looking with respect to attracting and retaining communities of color. For all of you online, we are looking forward to and eager to work with you to tap into a broader network of talented folks that you might want to consider referring to us or applying and sharing with a who you know. But we’re super excited to have been part of this and are grateful that you all attended!

    Angie Chang: It’s six o’clock. And that means it’s time for another Girl Geek Dinner… This time, however, we are coming to you virtually for the first time from our homes in Berkeley, California here. Sukrutha, where are you?

    Sukrutha Bhadouria: I’m in San Francisco, California.

    Angie Chang: And behind the wings we have Amy, who is coming from … Amy, where are you coming from?

    Amy Weicker: Pennsylvania.

    Angie Chang: Pennsylvania. Awesome. We have a bunch of people coming in. Can you use the chat below and tell us where you’re coming in from? While everyone does that, Oh my God.

    Sukrutha Bhadouria: Wow. Orange County, San Jose. [inaudible] India, my hometown. What were you saying, Angie?

    Angie Chang: I’m like, normally we get to see you in a beautiful office space. It’s always great to just go to these different companies and go there and meet the people, eat their food, drink some wine — and then hear from their women at the company speaking about what they’re doing at the company. From roles in engineering and product to sales … we’re going to hear from a few sales people tonight .. It’s really great and exciting to hear from many of the women working at the company on what they love to do.

    Angie Chang: We learn a bit about the company. I’m just super excited to be able to partner with Planet and bring this evening of talks to hundreds of girl geeks. These videos will be available on YouTube for free later so if you can’t come because you actually had to cook dinner and eat it with your family, you can still watch it later.

    Sukrutha Bhadouria: I want to just call out a few people in various time zones. Some people who say they’re in London at 2:00 AM, that’s awesome. India, 6:30 AM. That’s amazing, where in a funny way just going virtual opens up our access to you, and to you to each other 100% across time zones and across a variety of fronts. So that’s awesome.

    Angie Chang: Cool. I guess it’s time for introductions. My name’s Angie Chang. I’m the founder of Girl Geek X. I’ve been organizing these Bay Area Girl Geek dinners, as we called them for the first 10 years. Then now we’ve been doing Girl Geek X events. We’ve done over 200 events at companies big and small, at companies you’ve heard of and companies you haven’t. I think it’s really fun to keep doing it all these years because of that. You get to learn about so many companies that you never thought of. You go in there and you hear about all the ways that the company has people working in these different departments that you never knew existed. Suddenly you’re like, “Oh my God, I guess this sounds really cool.” By the end, when they’re like, “And we are hiring,” you’re like, “Yes, I know what you do. I know what team I can join. I heard from people at that company, I know their names. I can now find them on LinkedIn and poke them and send them my resume.” Please do that. They are hiring. Sukrutha?

    Sukrutha Bhadouria: Yeah. Hi, I’m Sukrutha. I’m the CTO of Girl Geek X. Angie and I met several years ago when I had just moved to the Bay Area looking for other like-minded women like yourself to connect with. I found out that there was an upcoming event with Girl Geek Dinner and I saw Angie’s name there. I was like, that’s awesome. I should try to go. For whatever reason, I wasn’t able to go that evening, and I instead managed to get the company I was working at to sponsor. Angie and I played phone tag for a little bit, but we ended up meeting and I was like, this is so exciting because that particular event had over 200 women AND men show up — 200 people show up, basically. It was such a great energy in the room. I just couldn’t get enough of it. I wanted to come back.

    Sukrutha Bhadouria: That’s where our journey together started. That was dinner number 11. We’ve since had over 200 dinners. I’ve actually lost count. At that point it was one every few months. We ended up having the frequency just go up. We then launched into podcasts. We launched into virtual conferences. So you can see all of that content on our website ( Just to catch up if you’re new to this, usually what we do in this situation is we survey the room and we ask how many of you are attending this event for the first time. I don’t know how we would do that now, but I’d be really curious to learn from virtually raising your hands. How many of you are attending for the first time? Wow. I can see the numbers, counting now over 40 people are raising their hands as the first time.

    Sukrutha Bhadouria: Wow. That number’s climbing, Angie. That’s amazing. I’m so happy to see so many first time attendees. Generally, like for us, it has been amazing because we would get so much out of these dinners, the podcast that we do, as well as the conferences, because the energy from just meeting other people specifically like you, you may not have that access in your company. We were getting so much out of it. We would hear from the sponsoring company, how they were getting access to really motivated, smart individuals like yourself, where they ordinarily wouldn’t have the access to. Likewise, the attendees would come to these events and they’d be like, “Oh my gosh, I didn’t realize that were these many people who are just like me.” And then they started to make friendships. Often Angie and I would talk about how important it is to network before you actually need it.

    Sukrutha Bhadouria: I myself was super shy and awkward. And honestly, I still am. Who knows with the pandemic and sitting at home how awkward I’m going to be in real life when all of this lifts, but I do force myself. I learned from Angie, actually, how best to get involved in a conversation and approach people that I know I can benefit from that connection and they can benefit from it, as well. We started to build our circle. From that, I learned concepts like build your own personal board of directors, people who advise you in your career and your work life balance and topics like that. Then people who give you honest feedback on how you can improve yourself. So many things like mentorship and sponsorship and how to go about seeking that for yourself and how not to directly just go up to someone and be like, “Just be my mentor,” but then not give them enough context. So how to go about it the right way. There’s usually tips and tricks like that, that we will benefit most from asking other people who’ve had shared experiences like ourselves. What do you think, Angie? What do you think people get out of this?

    Angie Chang: I really appreciate going to Girl Geek Dinners and then Girl Geek events, because we reach a wide range of women who are working in tech and engineering and product. Also a lot of startup entrepreneurs and operations and marketing people. And they all intersect. I think in our careers, which are going to stand for decades, we are definitely going to be changing our jobs, and our roles will be different. I remember when I first met Sukrutha, she was a software engineer in test, and now she’s a senior engineering manager and it’s been years and it’s been great watching her change her career and grow and continue to look for … I think people look for people like them.

    Angie Chang: If I were an engineer, which I was 15 years ago, I would go to a Girl Geek Dinner and I’d be like, “I want to meet other engineers,” but then you wouldn’t have that happy chance of meeting other people, women who are working in other roles, but then you’d be like, “Oh my God, this is actually really cool.” These weak ties and these relationships are actually really beneficial in the long run. I don’t think I would have asked for it when I was younger, to meet all these different types of people, but now I really see it’s fortuitous and it pays to be a little broader. I like the Girl Geek X umbrella, instead of saying I’m only in product, which I was for a few years, or I’m only an entrepreneur, which I was for a few years.

    Angie Chang: Now, it’s just a great place to meet a lot of people. They keep coming back. We actually keep seeing a lot of faces. There’s always a lot of new people and a lot of people that come back time and again, based on who is hosting. We’ll be having different companies host virtual events moving forward monthly. You can look forward to different companies. But tonight we’re really excited to bring you the Girl Geeks of Planet Labs. I am going to be introducing our first speaker from Planet Labs, Adria.

    Angie Chang: Here’s a quick bit about her. She joined Planet’s federal division in Washington, DC as a people partner, where she was able to continue her passion for innovation and data with strategic human capital. She earned her master’s degree at Georgetown university with a research focus on diversity, equity and inclusion in tech. She is co-lead to Planet’s belonging taskforce. Welcome, Adria.

    Adria Giattino-Johnson: Thank you so much. I’m so excited to be here. This is such a great event, and it’s my first time. Obviously my first time as a panelist, but my first time attending the event. I’m just so excited to have so many people here listening to our talks and just connecting with women in different industries. I’m excited to just attend future events later on. Thanks so much for the introduction.

    Adria Giattino-Johnson: Let’s jump into a little bit about Planet. I’m going to share my-

    Sukrutha Bhadouria: Adria, would you like to turn on your video so people can see you?

    Adria Giattino-Johnson: Oh, I’m so sorry.

    Sukrutha Bhadouria: No worries.

    Adria Giattino-Johnson: I think we can all relate. I think this has happened to probably all of us. We’re all in a remote workforce right now. Maybe everyone can raise their hand if they’ve forgotten their video once or twice. Thank you. That made me feel a little bit better. Let me share my screen really quickly with everyone. We will jump into a little bit about Planet and then … oops, sorry … I will jump into my presentation.

    Adria Giattino-Johnson: About Planet, aerospace know how meet Silicon Valley ingenuity. From our spacecraft to our APIs, we engineer our hardware and software to service the largest fleet of earth imaging satellites in orbit and scale our seven plus petabyte imagery archive, growing daily. Planet designs, builds, and launches satellites faster than any company or government in history by using lean, low cost electronics and design iteration. Our Doves, which make up the world’s largest constellation of earth imaging satellites, line scan the planet to image the entire earth daily, which is really cool. We launch new satellites into orbit every three or four months. Most earth imaging companies don’t build their own satellites, but we’re not like most earth imaging companies. Planet designs and builds its satellites in house, allowing us to iterate often and pack the latest technology into our small satellites.

    Adria Giattino-Johnson: Complete vertical integration enables us to respond quickly to customer needs and perpetually evolve our technology. Operating one satellite is a challenge, but operating 200 is completely unprecedented. If you haven’t checked out our Ted Talk on YouTube, I highly, highly suggest you do. Planet’s submission is really cool. I’ll dive into a little bit about why I love working at Planet in a little bit, but it really is unprecedented. Our mission control team uses patented automation software to manage our fleet of satellites, allowing just a handful of people to schedule imaging windows, push software into orbit and download images to 45 ground stations throughout the world. Planet processes and delivers imagery quickly and efficiently. We use the Google Cloud platform and enable custom processing so that customers can tap directly into our data the same way we do. Our data pipeline ensures easy web and API access to Planet’s imagery and archive. We make every scene available as a tile service, composite scenes into mosaics, and build time slice mosaics so you can see change over time. That’s a little bit about us.

    Adria Giattino-Johnson: I am the first speaker, so I’m just going to dive into my talk. I hope that was a high level overview of Planet. Every person that works at Planet is super passionate about our mission, what we do. I really can say that every time I’m out on the street and I do tell people that I work for Planet, our mission is just so cool, that we build our own satellites and we have daily earth imaging. It really is unprecedented. It’s a really cool place to work.

    Adria Giattino-Johnson: On to my talk. I’m the people partner for Planet Federal. I work out of Washington, DC. Planet Federal, it’s the government arm of Planet. We partner with the government. I function as the people partner, which is basically HR. The people partner does function kind of as an HR business partner. Today I’m going to talk about diversity and belonging and the climate that we’re at right now, and how it’s not business as usual. We’re rethinking what diversity and belonging looks like in 2020.

    Adria Giattino-Johnson: A little bit about me. I like to use the group identity wheel anytime I do any type of speaking related to diversity and belonging, because I think this is a really good representation, at least for me, the way I like to represent myself and my different group identities. I am a cis gendered woman. My pronouns are she/her. I’m a US national, identify as agnostic. I am a Black, queer lesbian living with disability. I’m a millennial, upper middle class, and I do hold an advanced degree. This framework is really good for me. I think it’s really good for others, just to kind of show places where I’m marginalized and places where different group identities that I am also dominant.

    Adria Giattino-Johnson: Let’s jump in. So why I joined Planet. It was an industry jump for me. I had about seven years in human resources. I started as a generalist. I grew into leadership and then I later expanded into consultancy. I’m really passionate about strategic HR and diversity, equity, and inclusion. I began looking for something in the tech industry. I wanted to feel really connected to the mission of the next place that I landed. I was instantly intrigued by Planet and their core values. Why I love working at Planet, and this is what keeps me passionate, keeps me engaged, it’s why I show up to work every day. I love my team. They’re brilliant. I can actually say this globally, across Planet. We just have a really talented group of individuals that work for our company. If we’re at coffee chats or happy hours or whatever you can just listen to people for hours.

    Adria Giattino-Johnson: Everyone is just brilliant at what they do, and everyone is so passionate about how they contribute to Planet’s mission. The work that I do is really great for me. It is what I’m passionate about. I get to do that every day. Planet is dedicated to agility and learning, which is something that’s really important to me, especially being in the people department. I love working on the people team because I really enjoy fostering connection and collaboration between teams.

    Adria Giattino-Johnson: Let’s dive into the topic today of what I wanted to talk about for this lightning talk, which is diversity and belonging. This year has been a tough year, and I think we’re all in agreement. We face a global pandemic. We’re facing systematic racism and police brutality, political unrest, and let us not forget the murder hornet scare in May. Just in case you did forget, I put a little slide here. It did terrify me, I think, as well as some others. Wanted to add a little bit of levity there. This was an addition to our plates, I think, that we did not need in May. But so let’s dive into the topic for today. We are a nation that’s currently experiencing trauma. Filmed police brutality and racist interactions have flooded our broadcasts as well as social media. It’s something that we’re seeing every day. Many, from all backgrounds and racial identities, have filled the streets in protest to support Black Lives Matter. In response to this, a number of companies have put out statements in solidarity, and it’s forcing many companies, including Planet, to grapple with internal diversity statistics and consequently rethink diversity, equity, and inclusion programs.

    Adria Giattino-Johnson: Let’s talk a little bit about statistics. Statistics show that Black employees are left behind. In 2014, Google released their diversity statistics, which many tech companies followed suit after that. But before that it wasn’t something that companies widely released. Statistics over the past six years have shown that despite diversity efforts by most organizations, Black representation remains extremely low with a net change that is almost nonexistent. Statistics do show a slight increase for women in tech, which shows that some diversity efforts are working, but some marginalized groups are still being left behind, which is super important to look at. Let’s look a little bit at the delta for Black employees and tech. So this is a really good representation to just show you over the past five to six years there really hasn’t been a change, despite companies having large funding towards diversity, having diversity programs in place.

    Adria Giattino-Johnson: The numbers still remain extremely low. There has been, as I said, an increase for women in tech. It’s been a small increase. There’s still so much room to go, but there has been some strides made there. So just wanted to show a little bit of visual representation of that data. Let’s talk about why diversity efforts are failing. This is what I mean when I’m talking about diversity, quote, unquote business as usual. This is what companies have been doing for decades. Despite a few new bells and whistles that came about in the ’90s, companies have been essentially doubling down on the same approaches that they’ve been doing since the ’60s, which is diversity training to reduce bias. I think many of us have held trainings like that if you’re in people operations, like I am, or maybe you’ve attended a training like that. Hiring tests and performance ratings that limit bias, and putting grievance systems in place for employees to challenge managers.

    Adria Giattino-Johnson: These tools are really designed to preempt lawsuits. I think that framework is even in the wording. When we do attend these trainings, it’s very fear-based, I would say. They don’t dive further than that. They don’t dive further to promote equity and inclusion. Now we’re seeing a shift. Employees are demanding change. Companies can no longer operate business as usual in diversity, equity, inclusion, and belonging. Employees don’t want a PR statement from the organization, but rather they want to see a clear action plan related to inclusion and anti racist efforts. This really falls in the wheelhouse of the people team.

    Adria Giattino-Johnson: It is an organizational wide effort, but it’s something that I’m proud to be involved in. I wanted to talk a little bit about that today. Moving toward belonging and the new landscape for diversity, equity, inclusion, and belonging. I really, really love this framework and I wanted to make sure I included in this talk. Diversity has no meaning without inclusion and belonging. Diversity is like being invited to the party. Inclusion is being asked to dance and belonging is dancing like no one is watching. Belonging is really being able to show up at work as your true self, and being able to be your authentic self in the workplace. We spend so much time at work that really having this piece where you’re being invited to the party without having these other pieces, it doesn’t mean anything. This is exactly why these diversity efforts are failing.

    Adria Giattino-Johnson: I’m not going to dive super into the inclusion framework here, but I did want to include a visual of the sweet spot for inclusion, which is a high level of belongingness and a high value in uniqueness. What that results in is an individual being treated as an insider, and also allowed and encouraged to retain uniqueness within their work group. Let’s talk a little bit about definitions, because a lot of times, I think you can get these trendy words that are happening within diversity or even happening within HR, within people. Belonging can be pegged as a trendy word and it’s really not. I wanted to be explicit about the definitions. Belongingness has to do with whether or not a person is and feels treated as an organizational insider. Uniqueness is measured by the degree to which an individual feels he or she can bring his or her full self to the work without needing to assimilate to cultural norm.

    Adria Giattino-Johnson: The degree to which an employee can fully engage, feel safe, and feel connected in the workplace greatly depends on these two categories. And like I said, these can often be left out of diversity programs. So let’s dive a little further into diversity without belonging. Like I said, diversity without belonging inclusion allows marginalized groups into the organization, but then it forces them to fit in to the existing dominant culture. Many Black employees, for example, experience a pass on promotion, noting that they should get to know other managers more, or network more, or connect more. There’s really not explicit definitions in terms of what that really means. For many marginalized groups, Black employees specifically, they report not feeling safe to connect at work and be their authentic self due to cultural difference and fear of bias or repercussions. There’s a real barrier there. Statistics show that attrition rates among Black employees and those of other marginalized groups are much higher. A 2017 report surveyed over 2000 tech employees who left their jobs. It found that many people of color felt that they had unfairly been passed over for promotion, faced stereotyping or bias related to quote unquote fitting in or connecting with others.

    Adria Giattino-Johnson: Let’s talk about getting it right. I mean, that’s what I really want to talk about in this talk. When belonging and inclusion are embedded in company culture, it no longer forces employees to fit into the dominant culture, but rather it builds a culture around everyone’s unique identities. Rethinking strategy. Belonging becomes the heartbeat behind an organization’s culture and core values. I’m proud to say that that’s something Planet is working towards and I think that they value. I am the co-lead on the belonging task force. I can really say that that is embedded in Planet’s core values. Without inclusion and belonging, employees do not feel as though they can show up as their authentic self at work, like I said before. This inhibits recruitment, retention, and promotion of marginalized groups, and it also inhibits diverse voices from speaking up and being heard. Let’s talk about creating sustainable change. An internal and external audit is something that must be done.

    Adria Giattino-Johnson: Companies, including Planet, must take a long, hard look in the mirror and they must sit with what they see. What are the diversity statistics amongst marginalized groups, specifically Black employees in this climate? What are the attrition rates amongst these groups? How do these systems that organizations have in place contribute to oppression of these groups? Creating a safe space for employees and fostering belonging is also really important. I’m sure a lot of you have heard about employee resource groups, or maybe you’re a member of one.

    Adria Giattino-Johnson: They’re a great place to create a safe space for employees to connect. They’ve actually been in effect since 1964, and they were established as a response to anti-black prejudice following the 1964 riots in New York. They’ve continued to be a huge part of the tech community, but companies must really be careful to utilize these groups as a safe space, rather than placing extra burden on them by forcing them to do organizational diversity work and education on top of their jobs. Especially with us being women in tech, sometimes the burden can fall on the marginalized group to do the education, to do the work on top of their jobs. That’s not really the purpose of an employee resource group. It’s to create that safe space, to create belonging, and to create connection. Employers should really watch that and be careful of putting that burden on the employees.

    Adria Giattino-Johnson: Looking at the internal and external pipeline of candidates is also really important. Talent and recruitment reform, I think is the biggest part of this. You want to audit your hiring practices, and broadening the schools that you recruit from is really important and including HBCUs, it’s also really important. Recognizing bias against HBCUs and other university programs as being seen as a lower bar is the first step in that. I think that’s something that a lot of tech companies are looking at right now. Also auditing referral programs. So I think referral programs sometimes can fall by the wayside, especially in tech. If a workforce is already homogenous, referrals can further contribute to this as referrals from employees tend to be within their own identity groups.

    Adria Giattino-Johnson: I challenge everyone on this video to think about when you’re referring people into your organizations, are you amplifying diverse voices? Who are you referring, or is it homogenous? This is something that even as employees, we can be thinking about when we’re bringing people into our organization.

    Adria Giattino-Johnson: Addition of external efforts, and this is something I’m really proud to partner or be involved with Planet. Recognizing the disparity of marginalized groups in tech and committing to investment in community partnerships and education is also huge in creating sustainable change. An example of this is investing money to give black and LatinX students exposure to geospatial and STEM studies and potentially creating an internship pipeline based on such programs.

    Adria Giattino-Johnson: The last portion I want to talk about is mentorship programs. I think Angie highlighted, it was either Angie or Amy, highlighted mentorship in the beginning of this. People in senior roles tend to want to mentor and groom people who look like them or remind them of themselves. This is implicit bias. It’s unconscious bias. It’s not on purpose. But this means that people in marginalized groups often do not have someone to advocate for them. Organizations and managers within these organizations, if you’re a people leader on your team, you should be intentional about diversity in mentorship programs rather than leaving it up to senior management.

    Adria Giattino-Johnson: The last portion is stamina. This isn’t a checklist. This isn’t a quick fix. This isn’t a measurable ROI. ROI is like always what executives want to hear is if you’re on the operations team or maybe you’re a people leader on your team I’m sure you talk a lot about ROI, building business cases for everything that you want to pass through. But that’s not the case here. This is systemic change that we’re trying to create at the organizational level, which is sustained over years of hard work to see measurable results. Companies must commit to sustainable change over time at every level of the company to value and prioritize diverse and inclusive workforces.

    Adria Giattino-Johnson: I’ll end this just by saying, I am so excited to be a part of these efforts at Planet. I look forward so much forward to seeing sustainable change within our company, and I hope that your companies are also working to create sustainable change. I hope that your voices are being heard. This is a really important time for all of our companies, especially within the tech community. I’ll be excited to see what type of change happens within the tech community in years to come. So thank you so much.

    Sukrutha Bhadouria: Hi. Thank you so much, Adria. That was wonderful. It was really inspiring for sure for me. We’re going to switch over to our next amazing panelist, Lisa Huang-North. I’m going to do a quick introduction and then we can jump into Lisa. Wow, great background, Lisa! Lisa is a product and program lead at Planet. The team is responsible for delivering product solutions that help customers scale their business. Before joining Planet, Lisa worked for over a decade in strategic consulting, finance, digital marketing, and full stack software engineering. In her free time, you can find Lisa building Lego Technic sets, coaxing her sourdough starter, and dreaming of the day when we can all travel to see friends and family again. Oh my gosh, don’t we all? Welcome, Lisa.

    Lisa Huang-North: Thank you very much, and thank you for the intro. Let me share my screen. Hopefully, everyone had a great time listening to Adria’s talk. I’m really excited to be following such a fantastic speaker. Can you all see my screen?

    Sukrutha Bhadouria: Mm-hmm (affirmative).

    Lisa Huang-North: Hopefully, yes. Okay, wonderful. Yeah. Really today I’m hoping to speak with you around pivoting, and I think especially with 2020, it’s really thrown the spinner. I think a lot of people’s plan, whether that be life plans or career plans and career pivots, there’s never really a good time for it, but it’s even more stressful when there’re uncertainties around that. I’m hoping today I can share three lessons from our satellite operation team and really get you to think around how you can plan for your career pivot.

    Lisa Huang-North: To start, let’s see. Here we go. All right. Firstly, about me, I’m currently a product and program lead here at Planet, and I’m also a part of our wonderWomen ERG group that Adria mentioned earlier, [inaudible] taskforce. I call myself a Pivoteur with five career pivots. Prior to COVID shutdown, I loved to travel. Hopefully that’s something that resonate with everyone. And here, I just included a short quote because that was part of what inspired my brief or the talk, was Robert Frost’s poem around traveling or taking the road less traveled.

    Lisa Huang-North: The first lesson, what are your areas of interest? A lot of the time for our satellite operation team, the first thing they need to know about tasking on satellite is, where do you want to look, and what do you care about? I will use two use case to try to explain. The first one, perhaps you’re in agriculture. Perhaps you are a farmer, in which case, the area that interests you could be roads. You’re trying to find the roads that will help you travel to your farms versus if you’re a civil government, for example, someone in San Francisco who is doing city planning, the things you care about will probably be buildings or infrastructure, and not so much about the road itself to a farm land area.

    Lisa Huang-North: Using these sample lessons similarly for you, when you’re planning your career pivots or career changes, that will be my question to you, what are your areas of interest? That can be an industry, a vertical, perhaps you really tech or you want to try out finance or non-profit. Maybe it’s a skillset that you want to gain along the way, or perhaps it’s really about a national or geographic location, you want to move to the city or you want to be closer to family. So those are interesting points to consider around your area of interest.

    Lisa Huang-North: In my case, it was a combination of all of those when I did my first two career pivots, I will say. I started off in Chicago, my career as a mutual fund data analyst. So, that was at Morningstar. And one of the things that I personally felt was really important was a chance to work abroad because I think it’s important to learn about different culture and get a chance to work and live in those places [inaudible 00:39:30] traveler.

    Lisa Huang-North: And that’s what brought me to my first opportunity where the company went through a merger and acquisition and I volunteered, interviewed, and ended up moving to Cape Town, South Africa, where I headed up the data operations for our Sub-Saharan African office. And that’s the picture on the left. And after doing that for a couple years, I realized, hey, data analyst is great. I get to learn a lot about data operations and logistics and business analytics, but I really want to do something more creative now. And I love something that’s more customer facing and somewhere where I can work on my marketing or communication skills. So that was my second pivot where I moved and became a food writer. I know, I know a little off course, but it was something fun. I was in my early twenties and for me, it was about the skillset that I wanted to gain and in the immediate format.

    Lisa Huang-North: All right, lesson number two, what are your time of interest? A lot of the time for our satellite operation team, they need to know what the targeted time period for our customers, our users will want to see imagery of. Again, going back to the earlier examples, if you’re in agriculture, for example, a farmer. Your time of interest is probably quite seasonal. For example, with this picture, you actually see a lot of the circular fields. That’s what you’ll spot throughout the U.S. And in their case, their time of interest would probably be spring because they’re planning for the growing season and they really need to know what the health of their fields are. However, going back to civil government, if you’re looking at zoning or city planning, or even thinking about where do I want to develop the city, building more infrastructure, building new highways, some of those time of interest could be longer term instead of a season. You’re looking at your own year or even multi-year horizons.

    Lisa Huang-North: So think about that when you’re going through a career change or planning for it, what is your time of interest? Are you looking at something that will happen within the next 12 months, two years? And when you do make that leap into your new role, how long do you want to be there? Is there a stepping stone to another bigger career pivot, for example, if you’re moving to a new industry or is it a way for you to grow and really deepen your expertise, for example, within the industry or within the field. And feel free to put your thoughts in the Q and A as well, it’s always fun to make it interactive as you are pondering through these lessons.

    Lisa Huang-North: So in my case, I would say while I was becoming a food writer, I fell into digital marketing because a lot of writing and communication are augmented by social media. And from there I discovered one of my passions, which is in public speaking. So for me, my time of interest at the time was really to hone my public speaking skills and communication skills. And one of my capstone projects or goal I set for myself was to speak at the TEDx event. And at the time Cape Town held or organized various TEDx events. There’s ones organized by the university and there’s ones organized by the city itself. And I was able to, again, submit the talk proposal and be selected and really presented. And that was where I had the unique opportunity to meet Archbishop Desmond Tutu, as well. Still one of the highlights in that time of my life.

    Lisa Huang-North: And carrying that forward, now my next time of interest was looking at two to three year horizon where I said, “I have my data analytic skills down. I have my creative marketing skills down. What do I want to learn next?” And I really wanted to be able to build a product so that I’m not just talking about it or selling it or analyzing it if I can build the end to end user experience. And that’s where it brought me to my next pivot into a full stack software engineer role. And I went through a coding boot camp where I really learned the full stack where on the backend learning Ruby and on the front end learning JavaScript, using frameworks such as Ember.js and React.js. And that’s the photo you see on the top right. Again, I like to have milestones or capstone project for myself, and for that one, I really wanted to present some fine learnings in the form of a conference talk. And I was able to present at GDG in Madrid, that’s Google Developer Groups, during my travels when I was in Madrid. Think about the time of interest as you pursue your next career change.

    Lisa Huang-North: All right, lesson number three, and I think this one is actually one of the most important one. And it’s a reasonable or logical extension coming from area of interest, time of interest, and now what are your success criteria? Using the earlier examples, if we are looking at those as an agricultural farmer. This image on the screen, it’s probably not very successful because I don’t see a lot of farming or agricultural land near San Francisco downtown. Whereas if the photo was of [inaudible] with garlic farming or even of Napa Valley with the wine industry there, that probably makes a lot more sense and that image will be successful, right?

    Lisa Huang-North: But again, going back to city, if you are San Francisco government and you’re doing city zoning and infrastructure development, this image is probably perfect for your use case. You’re able to see downtown, you’re able to see Embarcadero. And in fact, you can even see Presidio on the top and the bridge, The Golden Gate Bridge. And even with Karl the Fog, the clouds, we’re always looking up for cloud covers at Planet, even though the cloud obfuscate the left side of the city, you really get to see 90% of the city.

    Lisa Huang-North: So this image for civil government will be successful. So link in to that, what are the factors for your success criteria? Is it about the job, the scope of the role, maybe it’s about salary because you’re at the time of your life where you need to provide for your family and financial stability is key. Or perhaps if you’re younger and earlier in your career journey and for you, personal growth and learning is the key factor for your success criteria. So think about that as you’re planning your career change and planning for the next pivot.

    Lisa Huang-North: In my case, I would say that through those different career changes, initially the success criterias were pretty immediate. Which are, what skills can I learn? And am I having fun with it? Am I having fun while I’m changing these different jobs or learning new things? And I would say on the top left, this was at a friend’s wedding in Durban, South Africa. And for me at the time, the social aspect was a huge thing, too. I really wanted to meet people. I wanted to experience different cultures and those, my lifestyle choices, were integral pieces to my success criteria beyond professional growth.

    Lisa Huang-North: And slowly as I moved back to the U.S., I would say that my success criteria has changed over time. And now, instead of just focusing on perhaps immediate and personal gains, I’m really looking at how I can integrate or how I can be closer to families and what that means for my lifestyle and what I want in the longterm, starting a family, for example, mentoring other women in tech. And that’s how I’ve been involved in Women in Product and Tech Ladies. And in some ways, still trying to get connected with my roots from when I ran the startup by attending startup conferences and just keeping fingers on the pulse about what’s happening in the startup space. So that was really key shift from personal growth lifestyle to professional, family, as well as any mentorship impact.

    Lisa Huang-North: And that ultimately was what brought me to Planet. I think, as Adria mentioned, a lot of us here at Planet, we are fully aligned with Planet’s mission. And one of the success criteria for me when I went through the latest round of job search was around impact. I really wanted to join a company where I myself can be contributing to something that is impactful at the global scale. And really, Planet way surpassed that and some more because I would say beyond global, this is really a planetary and specie level. And I think hopefully with the use case I have shared, you can see how it impacts industries at the time. And I’m sure some of the speakers later will share even more interesting story such as forestry or crisis management. And you’ll get to hear a lot more. So take this time in the question Q and A area, if you can think about what your success criteria are, start sharing that with us.

    Lisa Huang-North: So finally, savor the journey. I think bringing back the three lessons about area of interest, time of interest, and your success criteria, another thing to remember is that while we are in the midst of career change or any pivot, the uncertainties are probably quite stressful. And you may feel like you don’t really know where you’re going, or if you are going to be able to attain the goals that you have set out for yourself. But as a famous saying go, hindsight is always 20/20. And while you’re in it, you may feel like you’re going through a rough divergence, snaking around from place to place. And it doesn’t feel like a linear path, but looking back, or if you zoom out and take a bird’s eye view, you’ll probably realize that you’ve made something beautiful and you have created this fantastic journey for yourself, where all those different skills and experience you pick up along the way were pieces of the puzzle. And ultimately when you piece all of them together, they look really stunning.

    Lisa Huang-North: So I hope that will help to lessen some of the stress, anxiety you’re feeling as you put it through these uncertain times. And to close, obviously, if you have any questions, feel free to reach out and let’s chat. You can connect with me on Twitter, on LinkedIn. I will be here for the networking event later on as well. So definitely reach out and we are hiring. So always happy to chat about Planet. Thank you.

    Angie Chang: Thank you, Lisa. We are running a little behind, so we’re going to skip the Q&A but feel free to ask the questions and we will ask Lisa and we will share them later in a blog post with everyone. But right now our next speaker is Sara. And we’ll bring her right up. Hey, Sara.

    Sara Safavi: Hey, how’s it going?

    Angie Chang: Good. How are you?

    Sara Safavi: All right.

    Angie Chang: So… you can get your slides…

    Sara Safavi: Mm-hmm (affirmative).

    Angie Chang: Perfect. So Sara, by means of intro and [inaudible]. She leads the developer relations team at Planet Labs. Welcome, Sara.

    Sara Safavi: Thank you. All right. So yes, I will get started. Like Angie said, I lead the DevRel team here at Planet Labs. And what I want to talk to you all about today is my experience working remote. I’ve been working remotely, both here at Planet and prior to Planet for about five or six years. So about three years here at Planet and then a couple different companies before. Along the way, I’ve had to pick up some new habits, some new practices and ways of working in order to make my stay in Remotesville as a remote employee sustainable.

    Sara Safavi: Tonight, I just wanted to share some of those tips with you and go through them really quick. I want to give you a starting point, not so much teach you everything, but a starting point you can reference if you’re also somewhere at the beginning of this journey. I know a lot of us are, especially in the last couple of months, so it’s a topic that we’ve all been talking about. And this, if you ask somebody for their one tip for working remotely, this one is probably what you’ll hear most of, establish a routine, make sure you have a routine.

    Sara Safavi: I’m putting this first because it is so common that you’ll hear it. I have a couple of things I’ll mention after this less common, but I do think that this is important. But something important to notice here is that we’re new because I’m talking about establishing a new routine. You need to develop some new routine that works for you because this isn’t the same as your pre-Remotesville routine. Your life is no longer in the same patterns. You’re not going to get up in the morning and pack a lunch, probably. You’re not going to get into your car, stop at the gas station on the way. You probably not even going to put your shoes on in the morning.

    Sara Safavi: So it’s completely different scenario, which means it’s going to take a different routine. But routines are still important because our brains can be stupid. And we want to trick them. A routine helps you trick your brain into understanding that we’re getting ready for work, we’re going to work, we’re no longer sitting at home in bed, it’s not the weekend, it’s still a weekday. So taking that time to get dressed in the morning, do your hair, put on something that makes you feel powerful and professional. It really helps separate that situation in your head between home and work.

    Sara Safavi: So build a morning routine that takes care of you. Maybe do some yoga, meditate, go for a run, whatever it takes to establish that new routine. But some other things that people don’t necessarily talk about, a friend of mine shared this concept with me a couple of months ago, and I really love it. So I had to stick it in here. Teach yourself and give yourself permission to put your body first. What I really mean by this is a lot of times when we’re working solo at home, it can become really easy to just stop listening to our body’s needs. If we’re not changing what we’re doing or interacting with other people, if we’re just sitting at our desks for eight hours a day with a cat or a dog sitting under the desk, then you can really start ignoring your own body’s needs.

    Sara Safavi: So if you catch yourself feeling out of sorts or not able to get into that workflow like you usually do, or just feeling like something’s wrong, or you keep beating your head against the same bug for 10 minutes, take a minute and check in with yourself. See if there’s some body’s needs that you’ve been ignoring. Did you skip lunch? Have you not stood up from your desk for four hours? Since you don’t have like a water cooler to walk towards, maybe you forgot to get a drink of water, hydration is important. But just take a moment, check in with yourself because a lot of times, the ways that we’re feeling are actually directly related to ignoring what our body’s asking for.

    Sara Safavi: And similarly, talking about stepping away from your desk, when you’re working remotely, you really have to make space for scene changes. If you’re in an office, many times a day, you’re going to get up, you’re going to go to a conference room, you’re going to go visit your coworker’s desk, you’re going to go to somebody else’s desk and ask to see what they’re working on. You’ve got all these opportunities to change your scene, but when you’re working at home, you don’t have those opportunities anymore. So you have to deliberately make space for them. Schedule them into your daily routine. Maybe you’re going to take your dog for a walk for a half hour every afternoon. Put that on your work calendar. Or maybe every Monday morning, you water all your plants, put that on your calendar. Put dancing breaks on your calendar, I have friends that do that and I love it. You’re working remotely though, your schedule can be flexible, maybe you can do a yoga class at 1:00 PM. Maybe you have the freedom to do that, but you have to deliberately seek out those opportunities to change your scene.

    Sara Safavi: Similarly, you have to seek out connection. You really have to rethink what it means to make connection. If you’re working remotely, like I said, you don’t have those coworkers desks to walk to. You don’t have a water cooler. You don’t have a break room to go make a cup of coffee or grab your lunch and heat it up. You don’t have those natural opportunities for connection. So as a Remotesville citizen, you need to be deliberate and intentional about this. Instead of just telling a coworker on Slack, “Hey, we should get coffee sometime,” you should send them a calendar invite for 2:00 PM on Wednesday and say, “Hey, I’m going to be on Zoom, having coffee. Let’s chat.” Make it an intentional and easy way for them to accept and say, “Yeah, let’s connect.”

    Sara Safavi: Find opportunities to network. Find a network of other people working remotely, whether it’s at your current company or friends that you know who are in different companies. And if you don’t have a network already and you can’t find one, maybe that’s a perfect time for you to make your own. Something that’s really great that we overlook in remote work is coworking. It can be really great to just cowork with somebody. And I don’t mean an active Zoom chat, like a coffee break, where you’re talking back and forth, but maybe you just open a video call with a coworker and you guys just sit there in silence doing your own work together. It’s really companionable.

    Sara Safavi: So rethinking what we mean when we’re thinking about human connection and then being deliberate and intentional about it, is what’s going to make that remote work environment more sustainable. Something to watch for is to be aware about the creeping attraction of home comforts. So if you’re working in Remotesville, you’ve got a comfy couch, you’ve got a comfy bed, you’ve got all of the comforts of home, but I strongly recommend that you don’t work from your bed.

    Sara Safavi: So I know Deanna is going to talk to us later about satellite operations from bed, and I totally fully endorse it. I think that’s awesome. But what I mean when I say don’t work from bed is, don’t make this your normal Monday to Friday, nine to five office space. Like I said, brains are stupid. You need to trick your brain into understanding home versus workspace. You have to use sensory cues to signal that difference. You have to let yourself close an office door at the end of the day. So maybe you don’t actually have an office at your house, but maybe you have to mentally be able to close that door.

    Sara Safavi: If you’re working from your bed all day, it’s super comfortable. It’s awesome. Maybe you’re even really productive, but then the problem comes when it’s time to go to bed and you want to sleep, but your brain is like, “Oh, this is where I’ve been working all day.” So you start thinking about work again, and your brain starts turning the last problem you’re working on over in your head. And it’s really difficult to have that isolation. So maybe at home, you don’t have a lot of space, maybe you’re working from your dining table. That was me for the first two years of my remote career. But something you could do is put a lamp on that table and turn that lamp on only when you’re working. And when you’re done working, the lamp’s off. Little stuff like that, those sensory cues can really make a difference in being able to mentally close that office door.

    Sara Safavi: I’ve given you a lot of advice and I do want you to remember, these are interesting times where we’re living through right now. This isn’t the normal time that you would be switching to working remote in tech. So give yourself permission to practice a little self compassion and be kind to yourself, but also be honest because compassion doesn’t mean lying to yourself. So if you forget to step away from your desk for eight hours, or maybe you fail to put anything besides coffee and LaCroix in your body since 8:00 AM today, it’s okay. But it’s important to be honest and name that and understand that it happened and then just try again tomorrow. You understand that it’s important to listen to your body, to stay hydrated, to take those opportunities for scene change, and just try again tomorrow.

    Sara Safavi: So try to create a routine that works for you. A new routine. You’re not going to make your old routine work here. Take breaks. Remember to move around. Listen to your body and brain’s needs. Intentionally seek out human connection and make invitations to people that are easy to act upon that are not passive. And don’t let comfort creep overtake you. Try not to work from bed all day every day. Don’t ignore your body and your brain’s needs. Don’t skip meals. It’s okay to take a break and step away from your desk, but above all, don’t be too hard on yourself.

    Sara Safavi: So I don’t know if we have time for Q?A. I would love to take questions if I can, but otherwise that’s my contact info. I would love to hear from any and all of you.

    Sukrutha Bhadouria: That was great. Thank you so much. We’re definitely going to take questions later, like Angie mentioned, but thank you so much. All right, next up… Barb is a software engineering manager and developer on the applications team at Planet. Take it away, Barb. Welcome.

    Barbara Vazquez: Thank you. Hey, everybody. My name is Barbara Vasquez. I go by Barb and I’m a software engineering manager and developer, as well, at Planet. A little bit about myself, I was born and raised in Puerto Rico. I have been working in the geospatial industry as a software engineer since 2008, when I moved to the DC area. And I have been living right now, I’m in Maryland, but I’ve been in the DC area since then. I joined planet about three years ago in 2017. And I’m part of the web applications team. We build some of the tools that help people have easier access toward data.

    Barbara Vazquez: The main thing that, if you’re familiar with Planet, is an application called Planet Explorer. If not, go check it out, Explorer. Now what I’m going to talk about today, it’s about Agile Development and estimation. It’s mostly focused because I’m a software engineer and we do Agile Development at Planet. And these are some tips and things that might be useful for people doing Agile. Even if you’re not doing Agile, thinking about estimation and how much something will take you to do is useful on a day to day. But with further ado, if you’ve done Agile Development and you do the daily scrums or the daily meetings, you’ve had these thoughts, what are points?

    Barbara Vazquez: Why are people asking me so many questions so many times, when will it be done? Why do I have to give status every day? And it can get tiresome. And you might just want to flip the table and say, this is not what I signed up for. This is not why I want to do software engineering. But through the years, I’ve learned that it can work in your favor. It can actually help you be more organized and communicate better, to have less stress.

    Barbara Vazquez: So estimating with points, if you’re not familiar with Agile or Points. Points is a system that tells people, mostly managers, how difficult do you think a thing is and how long it will take you. But in my perspective, yes, that’s one benefit, to tell your manager when things will get done, but it will help you be honest with yourself.

    Barbara Vazquez: Can I really do this? Is two weeks enough? Or however long you have to develop something. That doing the mental exercise will get you in a better spot where you might not need to pull all nighters. If you have to work weekends to meet your deliverables, you’re probably signing up for too much. Or you might be underestimating what is being asked from you.

    Barbara Vazquez: In Agile, the way it works, you sign up for work and you have X weeks to do something. I’ll use our example. We do two weeks of development. If after those two weeks, every time you’re rolling over things, rolling over means that you did not complete it. That means something is wrong in the process. It’s not necessarily you. It’s a team thing. It’s being underestimated.

    Barbara Vazquez: Scope creep happens. You’re midway. You’re almost done. And then somebody is like, did you think about this? What about you do that? And you go on a tangent and you forget about your original goalpost, or the biggest one that nobody wants to admit is you probably don’t have enough information, but how do you tell your manager that you don’t have enough information?

    Barbara Vazquez: Shouldn’t you be able to do it on your own? Not really. That’s what the whole point of Agile and team development should be. And points are there to help you communicate that.

    Barbara Vazquez: How to start doing better estimates. One thing I do with my team is ignore numbers. Just give me T-shirt sizes, small, medium, large, or extra large. Extra large, can I do this in two weeks? If it’s an extra large, no. It probably needs to be broken apart. You probably need to talk more about it. A large size, will probably take me the two weeks. I’m threading there on borderline not completing it, but let’s give it a shot and let’s see how it goes. Medium, I can get this done. I don’t know how long it will take me. It’s definitely going to be more than a day but I can get it done. And small is I can do this with my eyes closed. It doesn’t matter.

    Barbara Vazquez: That’s my rule of thumb. When I go to do estimates, it’s give me a sense, how do you feel this is so that we can have that conversation of how long it will take. As soon as you do this mental exercise, you’ll get in a better habit and you’ll start recognizing better. I don’t have enough information or this is super easy. Why am I even thinking about it? Let’s get it done.

    Barbara Vazquez: So once you get the T-shirt sizes down, you can map this to whatever point system your team uses if that’s the preferred methodology. A lot of people use the Fibonacci sequence where it’s one, two, three, five, up to 13, where a 13 is the extra large equivalent.

    Barbara Vazquez: So this once you get used to, and you’re like being able to do t-shirt sizes, you can move up to doing the point systems. In any case, even if you don’t do Agile, thinking about your tasks in t-shirt sizes can help you think about difficulty, can help you keep yourself organized and just do that mental exercise of what do you need to get done that week?

    Barbara Vazquez: The other point, two points, no pun intended, is keeping your other responsibilities. Add some buffer. You might be able to sign up, just keeping with the example, two medium things, because life happens. Add some buffer, COVID has taught us that life is unpredictable and your normal cadence is not the same anymore. Distractions happen, you might have family at home. Take that into consideration as well when you’re doing these estimates.

    Barbara Vazquez: And the other point, the other thing to think about with points is it helps you negotiate. It helps you make sure priorities are clear of what needs to be done first versus what needs to be done later. If your plate is full, whether it’s with actual tasking, if it’s with life, use the points to help you drive conversations. I can only do so many mediums stories. If I sign up for one more, I will definitely roll it over because that’s what I’ve learned.

    Barbara Vazquez: And in the end, having slightly more predictable cadence is valuable for everybody. And again, I say slightly because life happens and we cannot be 100% predictable, but we can get there. And that’s all I have. Thank you everybody. I know we don’t have time for Q and A, but that’s my email, If you want to reach out or we can talk later.

    Angie Chang: Awesome. Thank you, Barb. That was really great. I’m going to find Kelsey. Video, it’s perfect. Great. We can see you. So Kelsey is a space systems engineer at Planet. Welcome, Kelsey.

    Kelsey Doerksen: Thank you. Perfect. So good evening, everyone. My name is Kelsey Doerksen and I am a space systems engineer at Planet. I started about four weeks before work from home was an order for the San Francisco office. So I got only a little taste of what it was like to work in the physical San Francisco office, but I’m really happy with my past five months being a part of the team.

    Kelsey Doerksen: And today I’m going to be talking a little bit about how to handle big data in space and the different machine learning projects I’ve been a part of over the past few years. And so I’m just going to jump right into it. So first I wanted to start off with what is machine learning and what do I really mean by big data?

    Kelsey Doerksen: So big data is really just that, it’s a large volume of data or a lot of data. And we use machine learning with this big data to seek statistical patterns, to enable computers and algorithms to make either a classification, such as differing between pictures of dogs and cats, or prediction about the data.

    Kelsey Doerksen: I really like this three step image here that basically breaks down what machine learning is really at a high level, where you start with this big conglomerate of data, you can’t really make sense of it or extract any meaningful information from it. You apply analytics to it. And in this case it would be a machine learning algorithm. And from those analytics, you’re able to make informed decisions about the data in question.

    Kelsey Doerksen: I’m going to be talking about three different projects I’ve worked on at a very high level. Don’t be worried if you don’t know anything about machine learning. And I’m going to start off with my first project I worked on, which has to do with machine learning on Mars.

    Kelsey Doerksen: For those of you who are unfamiliar with the Mars exploration Rover mission, this was a NASA mission that launched in 2003, and it sent two twin Mars rovers, Spirit and Opportunity, to the surface of Mars. Unfortunately for the Spirit Rover, its wheel actually got stuck in the Martian soil. You can see in that black and white gif image there that is taken from the Spirit Rover itself. And unfortunately the mission was lost in 2010 for the Spirit Rover because its wheel was stuck in the sand and they weren’t able to get it free.

    Kelsey Doerksen: How could we have used machine learning in order to prevent this from happening for future Mars Rover missions? As we know, Perseverance is launching, hopefully soon, barring any delays. This is a project I worked on at the NASA jet propulsion lab called the Barefoot Rover project. Essentially what the Barefoot Rover project purpose was, was to use what is physically felt by the Mars Rover wheels, to be able to detect different things about the surface it was rolling across of.

    Kelsey Doerksen: My work was specific to making sure the wheels were not slipping or sinking into the different types of sand material we had at the JPL campus. And it was also, I worked on the terrain classification and detecting if there’s any subsurface rocks that could possibly penetrate the wheel and cause damage to the wheels.

    Kelsey Doerksen: How this worked from a machine learning perspective at a very high level, essentially what we had was a yellow pressure pad wrapped around the outside of the Mars Rover wheel. And we took those pressure pad readings and trained that in a classifier to be able to detect these things that are on the bottom of the slide there. So we were able to tell the hydration content of the soil, anomaly detection, safety, and stability of the Rover, slip and sinkage, which is what I worked on, terrain classification, rock detection, and other different tear mechanical properties.

    Kelsey Doerksen: This is a really cool project I worked on and it’s going to be implemented on future Mars Rover missions. The second project I’ll talk about is machine learning for the sun and for our Earth atmosphere. So this very terrifying image you see on the slide here is a picture of a Coronal Mass Ejection event. What a Coronal Mass Ejection event is, is a huge explosion on the surface of the sun.

    Kelsey Doerksen: And essentially what happens is these huge explosions send out high energy particles into space. You can see there, Earth is to scale in terms of the size of a Coronal Mass Ejection and the sun as compared to the size of our Earth. The distance is not to scale, but the size of the two planetary bodies is. So why this is of concern other than the fear that it strikes of course from this image, don’t worry. It’s not going to cause any … The flames will not reach our surface. But what they do do is send these high energy particles to our Earth’s atmosphere that essentially push our satellites around. So from a satellite operator perspective, the satellites can actually be moved off of their orbit path and collide with other objects in space, which is obviously really detrimental to the satellite operators.

    Kelsey Doerksen: How can we use machine learning to tackle this sort of problem? Well, we can’t stop these Coronal Mass Ejection events from happening, pictured there is a gif image from the Soho telescope that is showing what a Coronal Mass Ejection looks like. So we can’t stop these huge events from happening, but we can at least try to learn as much as possible about them and how they are affecting our satellite. And this was my master’s thesis work using the satellite accelerometer data to detect these solar storms. So I mentioned before that these solar storms send out huge amounts of high energy particles and they reach our Earth’s atmosphere. The way you can think about this is if you’re walking outside and it’s very, very windy and you’re getting blown back by the wind, that’s kind of is what’s happening to our satellites when these particles reach our atmosphere.

    Kelsey Doerksen: And that can be captured in the satellite acceleration data. The two graphs I have pictured on the slide here, the top graph, it shows the acceleration of the satellite when there’s solar storm happening. So you can see the signal is quite erratic and it’s actually doubles and above in the linear acceleration of the satellite itself. Whereas during a period, when there is no sort of solar storm, the satellite is very periodic and the signal isn’t fluctuating at any alarming rates.

    Kelsey Doerksen: The last project I worked on and want to introduce is, of course, using Planet data, and this is machine learning for our Earth. So I’m really happy to be a part of the new partnership with the Frontier Development Lab and Planet, which is an eight week research sprint with the NASA and SETI Institute, and Planet is working with the Waters of the United States team, which is using Planet’s daily imagery with machine learning, to assist with drought detection and prediction in small streams in the continental United States.

    Kelsey Doerksen: Pictured here is the Seminole reservoir in Wyoming, United States. And the first signs of droughts can be identified in the small streams that branch off of large bodies of water like these. So by comparing pixel values in these streams using Planet’s daily imagery of sites, similar to this, the team of researchers will be able to detect and predict future droughts across America with the aim to scale this work to other areas across the globe.

    Kelsey Doerksen: I can’t get to my … There we go. I really hope you were interested and able to follow along with those three different projects I worked on. I think machine learning, it’s such a new and growing field and space is the perfect application for machine learning because we have so much data. And if you have any questions, you can feel free to reach out to me, and thanks very much for your time.

    Sukrutha Bhadouria: That was excellent. Kelsey, are you seeing the comments? Awesome, Kelsey [crosstalk].

    Kelsey Doerksen: I can’t see them, but thanks a lot.

    Sukrutha Bhadouria: Someone said I want to be all the speakers. That was just amazing. I learned so much. So moving on to our next speaker, Deanna. Deanna leads the team at Planet responsible for operating and maintaining the over a hundred imaging satellites, or Doves, currently on orbit. Welcome, Deanna.

    Deanna Farago: Thank you. I’m so happy to be here. This is my first Girl Geek event. I’m excited also just to hear from other Planeteers because, sadly, it’s a large enough company that you don’t automatically know everyone. I love hearing everyone else’s stories, as well. All right, so I will present. Hopefully everyone can see that okay.

    Deanna Farago: All right, as I mentioned, my name is Deanna Farago and my team and I operate a fleet of satellites that are currently imaging the entire planet every day. And, traditionally, satellite operations can be very time and resource intensive. For example, in order to operate one spacecraft, you could have a room full of engineers around the clock, 24/7 monitoring, telemetry and contacts, and just system performance.

    Deanna Farago: And our satellites operate in a different paradigm and risk posture. This has allowed us to be able to automate a lot of the operations. Even before COVID, we could operate essentially anywhere as long as we had a good internet connection and our laptop. Before I describe what that looks like, it’s important to understand what the mission is and the scale of our operations.

    Deanna Farago: Our company’s mission one is to image the entire planet every day. And you need a lot of satellites in order to do that. And we actually, in addition to operating satellites, we design, build, and test all of our satellites in house. And this is a big advantage for us as operators, because if and when we run into issues on orbit, we can work directly with the engineers that designed the satellite in order to troubleshoot the problems and help come up with on orbit mitigations, as well as design out these bugs/features in the next spacecraft iteration.

    Deanna Farago: And then once in space, we use just a little bit of atmosphere that we have to use something called differential drag to space out the satellites over time. And as one satellite images over a strip of land, the one right after it should image this strip of land, just adjacent to it. And this essentially creates alliance scanner. What you’re seeing here is a 24 hour snapshot of what the imaging strips could look like that the satellites are capturing. And we have a distributed team operating our satellites. We have four people in San Francisco, one person in Toronto, and a team of four in Berlin. And we send tasks to the ground stations, which then send the schedules up to the satellites. And just a fun fact for this group that at Planet, we have three satellite operations teams and they’re all managed by women.

    Deanna Farago: The concept of operations is actually quite simple for these Doves. We don’t image over the ocean. We only image over the land, but basically anytime they’re overland, they just point down, take pictures. If they’re over ground stations, we downlink those pictures in logs and we communicate with them. And then in the background we’ll just run maintenance activities, essentially thinking of them as like tuneups and checking in on like subsystems and keeping an eye on any degradation that might be happening or running experiments. And, in theory, if the satellites are performing well, they should just be as easy as this man’s rotisserie grill, where we just set it and forget it. We can even run it custom experiments, and we set up the tasks and not have to worry about it.

    Deanna Farago: However, things don’t always go smooth. There’s a lot of fires that can happen. And that’s kind of how we know we’ll never really be able to automate ourselves out of a job. These are just some examples of issues that we’ve seen on our satellites. So a satellite suddenly starts spinning up, and we have to figure out why is it spinning up? And we need to de tumble it. We noticed that the satellites have low battery, that’s voltage, and we need to take action before they start browning out and rebooting rapidly. We see that telemetry sensors are reading zero value. Is this a real thing? Or is the sensor it just being faulty? And we have to reset it. Or sometimes satellites just are unresponsive out of the blue and we have to spend time to figure out, did something change, did something break on the satellite?

    Deanna Farago: Or can we just set up some automation to keep an eye on it? And all of these actions started out as manual. We would detect these problems and then operators would spend time triaging it and then eventually taking action. And now our teams have automated responses to all of these so that they trigger off of just telemetry on the satellite. As soon our automation sees like the driver readings are reading up. Then we know the … Sorry, the robot just basically sends a task to respond to this, so an operator doesn’t actually have to. And this decreases latency in the system and gets the satellite back into production as quickly as possible. And there’s always going to be unknown unknowns, and we’re constantly trying to find these new problems and automating responses to it.

    Deanna Farago: What does a day in the life of an operator? Well, we work nine to five and we have a checklist that we rotate among the team members. This enables our team to be able to have weekend or holiday coverage. Even though we’re working normal office hours, we want to make sure that there’s always going to be satellite operators, eyes on the system every day. And for this number of satellites, we have to aggregate our data. Aggregating our data is key. What that means is we build lots of dashboards based off of our telemetry, and off of our logs from the satellite. And it allows us to be able to easily see if there’s any satellites that are responding and acting out of family. And that will then trigger an operator to say this one’s not behaving the same as its fellow satellites. I’m going to dig in further and try to triage it.

    Deanna Farago: We have weekday team standups and we’re supported by amazing other teams in mission control. And those teams also have their own on-call. And so if something does break in the middle of the night, that affects the whole fleet. Those teams help support us. I wanted to show this because it’s one of my favorite things that we’ve taken a picture of at Planet. And it’s actually a series of pictures that we stitched together into a video. And just before a rocket launch, we’re able to opportunistically schedule a Dove to take a series of images of a rocket delivering more Doves to space. Just a real quick cool shot. And that’s shot by one of our satellites. So very cool. And then sadly, we won’t be doing any missed high fives and hugs and mission control in person anytime soon, like our former coworker here Rob Zimmerman. But we can still enjoy having first contacts and commissioning with one another virtually. And this is our, I guess, equivalent version of that from a few years ago when we were able to successfully make contact with 88 satellites right after launch. And with that, that’s all I really wanted to share. I couldn’t go into too much detail, but I’m happy to answer questions. If you’d like to email me. I am at Thank you for having me.

    Angie Chang: Thank you, Deanna. That’s really awesome. And you … Let’s see. And now we are going to bring up Elena, who has over two decades of experience in sales and she’ll be telling us her journey.

    Elena Rodriguez: Excellent. Good evening, everyone. I’m so happy for this invitation. I just joined Planet three months ago and I really wanted to talk about … sorry, this is my first time, I wanted to talk about the adventure of making a decision, how important it is for our career. But first, let me introduce what I do here at Planet.

    Elena Rodriguez: As I said, I joined the company three months ago, I’m this salesperson for Mexico, Central America, Ecuador, and the Caribbean. I have been in the business for more than 20 years, and I am so, so honored to be part of the Planet team. I’m so happy and so proud of working for the company that is offering solutions that are critical to mitigate some of the main challenges that we are facing right now, like climate change, food crisis, fighting poverty, so many applications, and I feel so proud to talk about our business when I go out there and meet my clients and listeners. So I chose a topic because this is something that I’ve been always thinking about it. And now I have the opportunity to talk about it. And I’m going to take advantage of this — is how I ended up here. I want to show you my story.

    Elena Rodriguez: Ever since I started back in the 80s, I have all the dreams like I wanted to be a fashion designer, because that’s something that I really enjoy since I was a little girl. And I took … but it was difficult for me because fashion was a very expensive career in Venezuela, and I had a scholarship, and I moved to Seattle from Venezuela to study sales and advertising. I have no choice. So let me tell you that, that was the first time I didn’t make any decisions.

    Elena Rodriguez: I had to choose what I thought was available for me that time. So I remember my sales teacher, Mr. Fine, it’s impossible to forget him. That he was always saying that a good sales person is capable of selling anything anything. Selling water to a fish. I wasn’t growing that idea of on my mind, but I was thinking, I don’t know if I’m really right for this career, sales is like — I don’t know — However, I was already thinking like when I was a little girl, I was drawing paper dolls and I was selling those to my friends at school. I was making bracelets with the colorful telephone wires, and I was selling those. I was a sales person already!

    Elena Rodriguez: I went back to Venezuela and I graduated, but I was still thinking, I don’t know what I want to do, this is my passion. I want to be a fashion designer. And it took me four years to graduate. It was the beginning of this career in Venezuela. And it was a lot of work. It was very expensive. There were times that I couldn’t sleep, doing all the drawings, the designs, and making all these dresses, this yellow one, and the one along here, I made them. And I was so inspired, because that’s exactly what I wanted to do.

    Elena Rodriguez: But then something funny happened during this practice — is that every time my friends called me and asked me for a dress, because they chose the fabrics, I have my [inaudible] they chose what they liked. And I made the dresses. Then when they came home to pick them up, I didn’t want to sell them! I was like no, I keep them. So I decided that’s not for me.

    Elena Rodriguez: It took me a while and I was thinking, you know [inaudible] what am I going to do? We are almost through this and I need to make a decision. I needeed to plan because I had a strong pressure from society, my country, and I made a decision — I thought it was time for me to have a family. And that was a decision that really, I thought about it a lot, because I know what it meant for me at the time — that I had to give up some things that were important for some time.

    Elena Rodriguez: But those changes, I always ask myself — once I start with passion to adapt to a new reality, because I had that question on my mind. And the answer is definitely no, I was just growing up. And it was time for me to make that decision and get prepared and be responsible for the decision that I have made.

    Elena Rodriguez: In 1995, it was a huge revolution in Venezuela because that’s when Internet arrived to our country. It was the time also when my boy was born, he’s 25 right now. And I remember I was taking care of my son and I was hearing all this noise outside — my husband and his friends talking about Internet — let’s go, let’s navigate, let’s check — They were looking for some topics and they were celebrating and I was feeding my baby and I was thinking, Oh my gosh, I think I’m losing something, something’s happening here, and I don’t know, I don’t want to sound selfish, but I had that on my mind. You know, so what am I going to do with technology, but I don’t know if I can even think about that! Would I ever touch a computer again? I had all these questions at that time. [inaudible] years things turn to be kind of difficult in my country. And I had to work. I had to live outside definitely my [inaudible]. And I had to go outside and find a different job, something because I needed to bring money to… because I had a family and things were difficult, and I was ready to get back on track, but I wasn’t ready for the technology. I had missed one year of all these changes! So selling was becoming more challenging, new terminologies, services, a new way of communicating… communication skills.

    Elena Rodriguez: The first job I got out there was for selling ads for the magazine called Computerworld with names like Microsoft, Sun Microsystems, IBM, HP, and those that were never familiar to me — it started to be new and that was nice — I was into a completely different world. This job was the one that allowed me to meet the people that helped me, that guided me, that inspired me to be in this field. And to be honest, selling was never had never been so much gratifying for me.

    Elena Rodriguez: Five years later, I had to make a very difficult decision that by the way, this week when I was practicing this presentation I found out how, I mean, how your country, your family, your culture really touched you. And I was like, I didn’t realize before, it’s like I was keeping that into myself, but it was a big decision. It wasn’t something that I was prepared for, but that was the time where the political situation in my country was unsustainable and started to be not sustainable even worse. I had a job offer in Mexico and I didn’t think twice. I moved here. And as you can see, the picture was… I think that was my first week here in Mexico. And you can see all the disaster. And remember I was asking if I would ever touch a computer again.

    Elena Rodriguez: Well, here is a computer, but I was only able to touch it because it was impossible to carry, so heavy. Everything has been changed as we know, that’s funny. So that’s when I started. It was like, for me, that was my own revolution, geospatial, learning new terminologies. It was such an exciting world. I was working with geographers, engineers, and so many people that I met in the industry. I really was in love with this new market. I was, like, wow. And I’m very proud because I participated in the first high-resolution satellite sale to the Mexican government. And I had all these questions from people. I mean, what is that you do? Are you a spy? What is it and that was very funny. But every time I had more challenges, it was time for me to learn more.

    Elena Rodriguez: And that’s really… That was very interesting. I don’t regret. I’ve been doing this for more than 20 years now. I still live in Mexico. I’ve met such interesting people, nice people, being in this environment. And I feel the pride to sell something, that I know that it’s going to go there to help people, to make people make good decisions. And this is something I feel so proud about it. And I’m here. This is what I do now. The geospatial world got me. I’ve been doing this work for, as I said, for more than 20 years, I’ve been in the drones industry, as well. I learned how to fly the drone. I was so proud about it. This picture here — in the mining, it was something very scary because I was in Peru and I had to sleep there. So, many nice adventures. I am so happy that I got… That I decided to stay here. I don’t [inaudible] change from fashion designing to the geospatial world. I can always be creative and I use the fashion designing for myself. So I like clothes. I like that. I mean, that’s inevitable. I can’t leave that behind, but this is, the right decisions brought me here. No regrets how I did it. I don’t know.

    Elena Rodriguez: As you see, sometimes we need to do what we need to do. I’ve been humble. I know that I’m not an expert. I’ve been learning and I always learn. It’s very challenging, this work. I rely on those experts that are willing to teach me and I take that very seriously. I understood that there are ways, many interesting ways to explore different options. I learned that we have to capitalize the knowledge because after you invest so much time in learning about something, changing probably is not such a good idea.

    Elena Rodriguez: Well, I don’t want to discourage the people that are doing this, but for me, I said, no, this is what I’ve learned, took me a long time. I want to be here. I wanted to be… to decide to be part of the change was very… That’s something that really pushed me as well. So that keeps me investigating and asking. So I’m curious about the technology and especially about the things that I do. Every time I made the decision, of course, I had to ask myself how it was going to benefit or affect my loved ones and understanding that it’s not always about me, that I have to care for my family. The company that I work for, there’s a world outside.

    Elena Rodriguez: I have faith in people. Trust me, I believe in people. I think we can always… We are a big team and I have a real engagement for environment. And I don’t know, I take care of my garden, my little dog, and I actually care about that. And, well, that’s it. Thank you. I think we don’t have time for questions. Thank you for listening.

    Sukrutha Bhadouria: Thank you so much, Elena. That was amazing. We learned so much from you. So our next speaker is Sarah Preston. Sarah is a marketing manager at Planet Labs, exploring how to use space-based imagery to improve life on Earth. Just pulling Sarah up. Hi, Sarah, how’s it going, right in front of the Golden Gate Bridge?

    Sarah Preston: Thanks. Out here in San Francisco. You can hear me alright, right?

    Sukrutha Bhadouria: Mm-hmm (affirmative), Yeah. So, welcome.

    Sarah Preston: Okay. So I’m going to share my screen and… Okay, can you all see that?

    Sukrutha Bhadouria: Yep.

    Sarah Preston: Okay, great. Thanks. Yeah, my name is Sarah Preston. I’m a product marketing manager at Planet. Now, a product marketing manager… Product marketing can mean a lot of different things in a lot of different organizations. But what I do is I work across our product and our marketing team and our sales teams to really find the right fit for our imagery and to understand what our prospects and what our audiences need out of imagery, even if they don’t know it yet. As you can imagine, narratives are extremely important part of what I do. So, I’m super excited to be here with you all to geek out about data-driven storytelling.

    Sarah Preston: Okay. First, why do we tell stories in the first place? Stories are paths to community and understanding. Think about all the stories that you loved growing up. There was some kind of connection that you made, either to a character, to the author, or to the setting that drew you in and made it really memorable. You joined that community that was telling that story. And within that story, whether it’s fact or fiction, there was information, and you got to learn from others in that community and to build an understanding about the world around you.

    Sarah Preston: What is a good story? So, “a good story is driven by emotion and balanced by fact.” That’s one of my favorite quotes, actually, that I heard. I can’t claim ownership of it, but, really, when we listen to a great story and we create a connection to a story, we’re really feeling some emotion and emotions can be extremely powerful motivators. I think, in or outside of the workplace even, an emotion can be excitement. It can be fear. It can be confusion. It can be ambition, but also a very human desire to understand the world around us. Emotions, they get us engaged in a story and interested. But facts and data, they keep us grounded.

    Sarah Preston: As an example of how you might be able to see this, Planet took this image of Pripyat, Ukraine back in April. Now this was when Pripyat was experiencing massive wildfires and this was right outside of the Chernobyl exclusion zone that you can see in the center there. It was an extremely dangerous time, already a dangerous area. Radiation levels had spiked 16 times more than usual and Ukranian officials were telling the world, basically, that these fires had been controlled, extinguished. Clearly not the case. Now hearing this, when we talk about emotions, hearing this story in the news, you can’t help but feel a sense of fear, maybe helplessness and anxiety, and all these emotions that are driving, maybe not necessarily the international community, but driving officials to understand what is happening. How can we solve it? Well, Planet came in and we captured this image and this image has a lot of data in it to help move these decisions forward, to help these move and capture these emotions.

    Sarah Preston: When we look at this image, we can see where the smoke is drifting. That tells us where the wildfire might be spreading to. We can see how far the wildfire has already spread on a grander scale. We can see how close it is to the Chernobyl exclusion zone. How radiation levels might continue to increase. And it tells us a lot about where we can deploy resources and where we can deploy flame retardant and, at the same time, keep all of our first responders safe. We had these emotions that we were feeling at the beginning, and a really good way to think about it is: Emotions, they move us forward. They encourage us to do something, but facts and data, they move us forward in the right direction. They give us an idea or an insight about where to go.

    Sarah Preston: How do we craft great stories? Great stories is really about taking our audience or, on a business scale, our prospects, on a journey from ignorance to understanding. Now, there are not three key points to creating a great story. This could be an hour long seminar and I’ve been to them before. It’s such a fascinating subject, but, given the time we have, I narrowed it down to three points that I think are really important.

    Sarah Preston: Know your audience. You want to understand what are their motivations? What are their expectations? Maybe what do they feel themselves on a daily basis? What’s their vocabulary? How do they communicate with each other and interact with the rest of the world? You want to really clarify the problem. Every story has its key conflict. You want to understand: what exactly is the conflict of the story you’re building and what is driving it, whether that is the emotions. And then you want to create some insight. What is the data showing us? This is the second half of the storytelling. How do we get past the conflict and use that data to create insight, to move us all forward?

    Sarah Preston: And here is an example, also at Planet, of how we recently used those points to create a broader story. We started work with the New Mexico State Land Office and they were looking to monitor permitting activity in the Permian Basin. You can see that on the right side of the screen, the sample image. And there’s a lot of mining activity out there, but they just couldn’t see in the way they wanted to.

    Sarah Preston: First, what we did here is we had to know your audience, right? We understood, and came to understand, how exactly the office itself functions, how it fits in with the broader civil government. What exactly is their legal mandate, who is our main point of contact and how to best really work with them in the first place. This is knowing how to communicate with them. Now once we know how to communicate with them, we can clarify the problem. Why is the office really experiencing this challenge? Why did they have very poor visibility into the more remote Permian Basin? Well, aerial photography like they’ve tried, was very slow and resource-intensive as was manned surveys. Sending people out there to actually see what’s going on, it was growing expensive. They were growing frustrated, really, that they didn’t really have a good way to monitor this land.

    Sarah Preston: What Planet did was, now that we knew our audience, and we then clarified the problem, we were able to deliver the data to really create a good insight to solve their challenge. This is sample data, again, right here on the right of the screen. We deliver near-daily imagery to them so they can see change and what’s actually happening and activity. And once they see that activity, then they can deploy resources, whether that’s people or anything else to solve that issue.

    Sarah Preston: Before I wrap up, I want to put another little plug. If you’re interested in learning more about storytelling at Planet, we actually have a customer conference coming up in October and we’re going to be featuring customers and partners talking about how they used our imagery for their own storytelling and how they’ve been able to build their own paths to understanding and building their own communities. The reason I want to feature this here is because it’s actually completely free this year and online, so very, very accessible. And before I completely close out, my last point, really, is: We are in a hugely data-driven world, and it’s really not so much about just collecting data anymore. It’s about collecting the right data and really understanding how to use it, how we get insights and go from that, go from that ignorance to that understanding to create solutions and to create great stories around our world. I don’t think I have time for questions, but that is my short brief. Again, this is a topic I could talk about at length, but hopefully you captured something out of this.

    Angie Chang: Great. Thank you so much for that, Sarah, and we are now going to be bringing up Brittany, who is a natural disaster research scientist turned businesswoman.


    Brittany Zajic: Alright. Thanks, everyone. Hi everyone. Thanks for the opportunity to speak with you all tonight. My name is Brittany Zajic and I’m on the business development team here at Planet. Business development means something different at every company. And here, we focus on strategic partnerships and the commercialization of new markets. I also lead our disaster response operations, which is part of our social impact initiatives, where we provide satellite imagery to first responders and official stakeholders in the event of a large, natural disaster anywhere in the world. And, while not exactly a natural disaster, COVID-19 is very much a global public health crisis reshaping all of our behaviors and our environmental systems. So, today I’m going to talk about how satellite imagery is helping us better understand the impacts of this pandemic.

    Brittany Zajic: By capturing a series of places in different points of time, satellite imagery is able to tell an important story. When millions of people began sheltering in place earlier this year, many looked to Planet, asking how we could help. So, how can satellite imagery help during a pandemic? Tonight I am going to showcase a few of the many applications surrounding the economic and environmental impacts of COVID-19.

    Brittany Zajic: First, we head to Wuhan, China to see the start of their shelter-in-place. In these first two comparisons, we see a stark difference of traffic patterns and these images taken only two weeks apart, with not a single car in sight starting January 28. And I’ll go back one more time. I know this is quick. We then shift to expand further beyond just the limited car transportation, and, instead, think about the closures of factories, construction sites, and all other industrial activities that had a dramatic impact on the air quality in regions of, and parts of, China. Here is a comparison over a portion of Beijing from the start of the year on the left to March 2020 on the right. We then shift to Italy, the next epicenter of COVID-19. Many media outlets spoke of the now quiet canals and the cleaner waters running through the city, which was largely captured in these series of images here. I’ll run through these one more time. This is October 2019, March 2020, February 2020, and March 15th.

    Brittany Zajic: Finally, we have the next epicenter that migrates to the United States, where it continues to remain today. New York was hit hardest and here we can see the construction of a temporary hospital in none other than Central Park, Manhattan, in the heart of New York. The rest of the United States followed suit soon after and shut down as well from the Bay Bridge Toll (that you take from going Oakland to downtown San Francisco) to the decrease in air travel (here’s a Southern California logistics airport — and just to highlight, we can see all the airplanes stacked up, not being in use), to the empty beaches (of Miami, Miami Beach, Florida) and then also the empty parking lots of Disneyworld in Orlando, Florida.

    Brittany Zajic: So, it’s pretty incredible for satellites to be able to so clearly capture this pause on life that has been experienced, that we’ve all been experiencing these past couple of months. Now, there is no question that one data set has been able to tell a great story, but Planet imagery combined with multiple other data sets is going to be able to tell us even more. So I’m going to spend the remainder of this talk today, talking about, an international collaboration among space agencies that is central to the success of satellite Earth observation and data analysis.

    Brittany Zajic: The tri-agency COVID-19 Dashboard is a concentrated effort between the European Space Agency, the Japanese Space Agency, and NASA. The Dashboard combines the resources, technical knowledge and expertise of these three partner organizations to strengthen our global understanding of the environmental and economic impacts of COVID-19. So, if we remember back to my early example in Venice, Italy, we visually saw the difference of boat traffic and water turbidity. Now, with EOS Dashboard, using information from several different satellites and sensor types, we’re able to turn that visualization into a quantitative assessment and observation, which is incredibly valuable when measuring environmental and economic indicators or factors.

    Brittany Zajic: A second example of these quantitative metrics is the air quality in Beijing. Again, deriving these insights from an entire suite of different satellites, the ability to analyze these trends from space aids the effort to fight and defeat this pandemic. I leave you all with encouraging you to further explore this Dashboard and learn more about how COVID-19 is impacting people all over the world and explore it through the lens of satellite imagery, because together we can defeat this. Thank you.

    Sukrutha Bhadouria: Hi, thank you so much. That was great. Next speaker is Nikki Hampton. Nikki is Planet’s VP of People and Talent, and she would like to share a few words on their commitment to diversity and inclusion. Welcome, Nikki.

    Nikki Hampton: Thank you. I want to thank all the speakers, even though I know all of these women, I learned so much about them and the work they do and how they got to where they are. So, I’m pretty excited about that. I mostly wanted to say that at Planet, we have always been committed to diversity, but we are doubling down on our commitment and particularly so, looking with respect to attracting and retaining communities of color. And for all of you online, we are looking forward to and eager to work with you, to tap into a broader network of talented folks that you might want to consider referring to us or applying and sharing with whom you know, but we’re super excited to have been part of this and are grateful that you all attended.

    Angie Chang: Thank you so much for that, Nikki. Now we’re going to just move into the Q&A. If there are a few questions, I think we have literally like five minutes till 8:00 PM when we kick off networking. So, if you have any questions, please ask them in the Q&A section and we will be sharing them with Planet and you’ll be getting a follow-up email with job links. They are hiring for some positions like senior corporate counsel, systems engineer, software engineer, account executives. So, you can be like Elena. Sales development reps, customer success managers, and more, and the job links are usually in our Girl Geek X Planet emails that you’re receiving. So, just scroll down and click on those links or forward it to a friend who is looking for a new role.

    Angie Chang: We will be heading over to our networking hour at 8:00 PM. It is on a platform called and you will have the link in your email, if you look in your email, or we can put it in this chat and we’ll be doing some facilitated one-on-one networking where you literally meet one-on-one with people in a non-Zoom environment. It’s going to be a little more fun and you actually get to talk to people and see their faces. So, if you can hop-

    Sukrutha Bhadouria: And I wanted to call out, thank you so much to everybody speaking and thanks to everybody who has been commenting. I definitely see that it has been super valuable for you all. I wanted to mention, because I’ve also been getting asked, how you can get your company to partner with us to do a virtual Girl Geek Dinner. Definitely reach out to us, through the website, — that’s our email — and if you want to reach out individually to Angie or I, our emails are listed on the website as well. The other thing I wanted to say is, if you do get your company to sponsor, you must sign up to be one of the speakers, own it, use the stage that you are creating for everyone else to promote yourself as well. So, that’s all I had.

    Angie Chang: Great. So thank you all for being so good at the chat, and we’ll see you over at so we can chat one-on-one with everyone. Thank you all and we’ll see you there. We’re going to keep this on so people can see the link and click on it — and hopefully we’ll rejoin and see you over there in a minute. Alright, bye.

    Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Microsoft Hardware Lightning Talks (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

We enjoyed dinner and demos of HoloLens at the sold-out Microsoft Hardware Girl Geek Dinner in Sunnyvale, California.  Erica Kawamoto Hsu / Girl Geek X

Transcript of Microsoft Hardware Girl Geek Dinner – Lightning Talks:

Angie Chang: Well, thank you so much for coming out tonight to Microsoft Hardware Girl Geek Dinner! My name is Angie Chang. I’m the founder of Girl Geek X. I’ve been doing this for about 12 years in the San Francisco Bay Area, and I’m really glad to see all of you out here tonight for this sold out event in San Jose —

Gretchen DeKnikker: Sunnyvale!

Angie Chang: Sunnyvale — sorry, I live in Berkeley! Thank you so much for coming out. Please talk to us. If you’re interested in hosting one of these at your company. The hashtag tonight is girlgeekxmicrosoft. If you want to tweet something really cool tonight, please do, share pictures, and share some of the awesome words that’ll be spoken by girl geeks tonight.

Gretchen DeKnikker: Yay, Angie. Yeah. She’s on tour right now, and she just can’t remember what city she’s in. It’s just like night after night, new city. It’s rough. Right? She’s livin’ that. Okay. How many people, it’s your first event? Cool. Welcome. We do these a lot, like several times a month, so you should definitely keep coming. I’m going to show you something right now. If you have been to four Girl Geek events, raise your hand. Keep them up if it’s five. Six, seven, eight, nine, ten, eleven. Okay. Oh, 12.

Gretchen DeKnikker: Okay, so you get these cards. It’s actually my little pixie on them. You get to carry me around in your pocket. How awkward is that? It’s pretty great. Okay, so we also have a podcast. Check that out. We’re just about to launch the new season where we’re answering your user questions. We sent out a survey. So that one will be really fun. We’re going to try some new things. Rate it, please. Give us feedback. Let us know because we don’t want to make stuff that nobody wants to listen to.

Gretchen DeKnikker: We also have a YouTube channel. Every time you can’t make it to one of these, you should make it because obviously, all of these awesome people come all the time. But if you can’t, they’re always on YouTube, subscribe to that. Then coming up on March 6th to kick off International Women’s weekend, because I’ve just extended it from a day to a weekend, because why not, we’re doing an all day long virtual event. It’s going to be epic. We have the Chief Diversity Officer of Workday. We’ve got the CTO of Intuit, the CMO of Twilio, the VP of Marketing from Intel AI. I can’t even list all of the amazing women that are going to spend the day with you and share all of their information, and also that will be available on YouTube later.

Gretchen DeKnikker: Self-promotion over, but this is all just for you. Please join me in welcoming your emcee for the night, Aaratee.

Microsoft Group Engineering Manager Aaratee Rao gives a talk on diversity and her career at Microsoft Hardware Girl Geek Dinner.Microsoft Group Engineering Manager Aratee Rao welcomes the audience and talks about her career at Microsoft Hardware Girl Geek Dinner. Erica Kawamoto Hsu / Girl Geek X

Aaratee Rao: Thank you, Angie and Gretchen. Good evening, everyone. My name is Aaratee Rao. I’m a group engineering manager at Microsoft Silicon Valley. I’m also the executive sponsor of diversity and inclusion for Microsoft’s Bay Area region. As your host for the evening, I would like to welcome you all and thank you for taking the time and joining us tonight. It is amazing to see so many like-minded women in the same room. I hope you all had a chance to mingle and network with each other. If not, don’t worry, we have some more networking time after the talks.

Aaratee Rao: We have a number of interesting talks to share with you this evening. But before that, I would like to take few minutes to introduce Microsoft Bay Area to you all, who we are, what we do, and how we work together to build innovative products at scale for our customers. I’ve been at Microsoft for only 14 months. So I wanted to start with a short story about my journey to Microsoft, and why I decided to join this company.

Aaratee Rao: I’m a recent hire into Microsoft, but not to the tech industry. I’ve been working in the tech industry for over 17 years now in a wide range of companies, from a startup, with less than 50 people, to a hyper growth company like Uber, where I worked for close to four years, and some organization grow from few hundreds to few thousands of employees. I’ve also worked at some large Fortune 500 companies like Visa, Intuit, and

Aaratee Rao: I started my career as an engineer, and then grew into leadership roles. Working at such a diverse set of companies for so many years gave me exposure to different technologies, products, industries, and also different company cultures. This exposure gave me clarity on what is really important to me as I’m exploring a new role, or a new company for my next career move. While I was working at Uber, and Microsoft approached me with a new exciting job opportunity, I applied that same criteria to Microsoft, which can be summarized into three things. Number one is people, number two is product, and number three is growth.

Aaratee Rao: Let me explain these three areas further and my decision to join Microsoft. My number one requirement was people. I believe that the most important driver of any company’s success is its culture, and the people who help build that culture. For many of us, a large portion of our day is spent at work. In fact, there is proven research data that one third of a lifetime is spent at work. So it is safe to say that our job and the people we work with can have a big impact on the quality of a life.

Aaratee Rao: Like many of you in the audience, I personally thrive in a workplace where people are not only passionate about what they’re doing, but they also create a supportive and respectful environment for everyone around them. I found all the Microsoft employees that I met as part of my interview process to be smart, humble, open to new ideas, and inclusive in their thinking, which I really liked. Microsoft employees are encouraged to apply growth mindset to their work every single day, which is a mindset shift from know it all to learn it all. It starts with a fundamental belief that every person can learn and develop.

Aaratee Rao: My number two requirement was product. Now, it is important to me that I’m working on a product that helps create a positive impact in people’s lives. Microsoft creates technology so that others can create more technology. In today’s world, every walk of life in every industry is being shaped by digital technology. Microsoft’s mission becomes even more important. I was also super thrilled to learn that Microsoft Bay Area teams work on a wide range of products, from the intelligent cloud offering Azure, which is using cutting edge technologies like AI and machine learning, to a product like Microsoft Teams, which is reinventing productivity and collaboration, and also Microsoft hardware teams that you will learn more about tonight from our speakers.

Aaratee Rao: My number third requirement was growth. Microsoft has seen tremendous growth over the past few years under Satya’s leadership. This growth has created more opportunities for employees to make an impact. Besides this, the company has also undergone a major culture transformation under the new leadership. Diversity and inclusion is a core priority for the company, and part of employee performance review. Microsoft leaders believe that for a company to be successful, and keep growing for a long period of time, we need more than a good idea and a good strategy. We need a culture that fosters growth and enables employees to build new capabilities. I was super happy to see Microsoft adopting open source technologies, and also giving back to the open source community.

Aaratee Rao: Clearly, Microsoft met all my requirements and exceeded my expectations. Here I am, and it’s been a fun and amazing ride so far. With that, let me introduce our Bay Area teams to you all. Bay Area is popularly known as a hub for innovation all around the world. Microsoft’s presence here, and all the product development that we do here is also rooted in innovation. Our presence here means that Microsoft can participate in conversations with startups. All employees, Bay Area employees embody that startups [inaudible] off the Silicon Valley to drive a company through innovation. We have offices in three locations: San Francisco, Berkeley, and Sunnyvale.

Aaratee Rao: This is our company’s mission. Our mission is to empower every person and every organization on the planet to achieve more. There is no way we can achieve this mission without representing the world. That means diversity. Diversity when it comes to gender, diversity when it comes to ethnicity, and diversity when it comes to skills, all of this is required for innovation. But it does not stop here. We believe that having diversity is not enough innovation, but we must foster a culture where people who are coming from diverse backgrounds can do the best work. That is why inclusion is so important, as it stimulates creativity and innovation.

Aaratee Rao: We also believe that having a deep sense of empathy is extremely important for innovation. As a primary job is to meet the unmet and unarticulated needs of our customers. At Bay Area, we are investing in multiple programs, which are specially designed for a diverse group of individuals. We value and celebrate diversity in a variety of ways. We have multiple employee resource groups that celebrate others, educate our allies, and ensure that all employees continue to learn and grow along the journey at Microsoft.

Aaratee Rao: We also encourage enthusiast, hobbyist, and creative people to enrich the experience of Microsoft. We have multiple community groups for folks interested in cycling, running, music, dance, and community service. We also have a company-sponsored corporate program called The Garage. The Garage offers classes to employees to learn new technologies. They also regularly invite external speakers to come in and share their perspective on a new technology.

Aaratee Rao: This is my last slide, and with this I’m giving you a sneak peek into our new Bay Area campus that we all are very excited about. This campus is being built in Mountain View location and will be ready this summer. This will bring all the South Bay employees under one roof, which will improve the employee interaction and will definitely improve innovation. Also, this the greenest yet building of Microsoft, and has been built with employee-centric design in mind. It has a lot of natural light and movable workspaces.

Aaratee Rao: These days, we talk a lot more about work-life integration more than work-life balance. This site will have multiple recreational facilities on site so that employees can seamlessly move from their work into life. With that, I would like to conclude my talk and invite Safiya for the next talk. Thank you for listening.

Safiya Miller: All right, good evening, everyone.

Audience: Good evening.

Safiya Miller: Oh, you can do better than that. Good evening, everyone.

Audience: Good evening.

Microsoft Strategic Account Executive Safiya Miller gives a talk on the first 90 days of a new job at Microsoft Hardware Girl Geek Dinner. Microsoft Strategic Account Executive Safiya Miller gives a talk on what to do in the first 90 days of a new job at Microsoft Hardware Girl Geek Dinner.   Erica Kawamoto Hsu / Girl Geek X

Safiya Miller: Didn’t you guys have awesome drinks and food outside? Come on. Well, everyone, my name is Safiya Miller. I’m super excited to be here. This is my first Girl Geek experience. I changed out of my Microsoft digs, but I am a Microsoft employee as well. By day I’m at Microsoft as a strategic account executive, Adobe’s my client. By night and by early morning, work-life integration that was just there, I am a fashion designer. Thank you. I’m wearing some of my pieces right now as well. If you have questions about how you can take advantage of pursuing your passions and really making the most out of what’s most important to you, definitely speak to me afterwards.

Safiya Miller: Today, I’m going to speak about the first 90 days. This doesn’t just mean the first 90 days in a new job, it means the first 90 days, particularly maybe at the same company, but in a new team. There’s three things you need to know and to make them count in Silicon Valley. They are managing yourself, managing team and colleagues, and managing the person that probably has the biggest factor of priority on your success at that company, managing your manager. Surprising to anybody, these three things? Make sense? Okay.

Safiya Miller: I know we’re at a Girl Geek hardware session, but these are critical for every portion industry of where you are in Silicon Valley for success, and I’ll tell you a little more about that right now. Managing yourself. The key to success is to start before you are ready. What does that mean? We talked about culture earlier. It’s one thing to read about a culture, to read about what Satya is doing, to hear what growth mindset means, but are you actually seeing it? Have you spoken to the Microsoft employees today and talked to them about what that means for them day to day? Was it a driver in them coming to this company? These are important things that you can figure out before you start, and you certainly should make a priority as soon as you’re on the job.

Safiya Miller: For me, this was important because I studied psychology and Spanish at Harvard undergrad, went into finance, a traditional analyst’s route after undergrad, and this managing yourself piece is important because I knew that working abroad was important to what I wanted to do in my career. As soon as I started, I was able to clearly identify something that was important in my career trajectory, which was an international experience. Managing yourself means you should have a blueprint of what’s important to you and your career, and where you want to see yourself.

Safiya Miller: That’s really important to identify in this first 90 days. You should also be able to identify how can this company, or this role, this team help you achieve those goals? Have you read the 10K? Have you listened to the latest earnings call? Have you spoken to anybody on your team about what the street really cares about for Microsoft, or for the company that you’re interested in, or the team that you’re on? What’s really moving the dollar, the needle? Those are the questions that sometimes get overlooked. But that’s really what’s keeping the lights on.

Safiya Miller: When you ground yourself in those things, this is how managing yourself sets you up for success. [inaudible] have a power outfit. I happen to be wearing one. You know what’s funny, because … and I know there’s a lot of allies in the audience, which is amazing. Can all the women raise their hands, all the women? [inaudible] raise the roof. Okay. All right. I just can make it clear here. I think we get a lot of feedback about what you should wear as a woman, specifically in tech, and how style doesn’t matter, or what you wear doesn’t matter. But if you think about it, can you easily identify what Steve always wore, audience?

Audience: Yes.

Safiya Miller: Okay. What about Scott? Scott Guthrie for our Microsoft employees, what is he known for?

Speaker 1: Red T-shirt.

Safiya Miller: Red T-shirt. Did I just say we were in an industry that said they didn’t care about style? Now, I’m not saying that it has to be glitz and glam, but they have something that’s predictable, something that makes their day to day easy on managing yourself. There’s so many speaking opportunities, there’s so many opportunities for women to thrive. I really feel like your brand, and what you’re wearing and presenting is just as important as what you have to say, and what you bring to the table.

Safiya Miller: This is just an example of a power outfit. I personally developed my fashion brand around statement pieces when I was speaking to women who were struggling with the most revered resource, time. They couldn’t think about what they could just pull out of their closet or travel with, to just have on the road and be ready to go on stage and command a room. So I made these statement pants.

Safiya Miller: But it doesn’t have statement pants for you, right? But I’m just giving you an example because pants for me are easy. I love color, and now I have a statement outfit that is a go to, when people think of Safiya, they know that when she commands a room, she’s going to have on a statement pants, she may have on a blazer, a fun pop of color, and she’s also going to tell you some awesome things about fashion. She might talk to you about what Adobe’s doing. It’s starting to build that story and predictability. Again, think about things that are manageable, that make the stress out of your life removed, because you have some routine that makes sense.

Safiya Miller: Let’s switch to managing your manager. I love these little cartoons. Who read any of these growing up? Yeah. All right, Career 101. This one stands out because there’s such a long list of priorities. But do you know the definition of priorities? Can you have a long list of priorities, and they really be priorities? Probably not. But this is important, because your manager only knows what you’re telling them. Right? There’s a variety of things that motivate each of us in this room to come and do our jobs day to day. It’s important to manage your manager so that they know what’s important to you. What’s the driver to you? Is it the money? Is the project that you work on? Is it creativity? Is it growth? Those kind of things are important for you to take ownership of and share with your manager so that they can be an advocate for you.

Safiya Miller: Force yourself to have those hard, but necessary, conversations with your boss. I know it’s hard, and I know that a lot of us procrastinate. I’ll raise my hand. Sometimes I do too, especially on those harder conversations. But guess what, the longer you wait to happen, the worse it is. Whether it’s a vacation that you already had planned, whether it’s through thinking through growth around the company, and maybe wanting to explore another team, but you need their advocacy, these kind of conversations that are important to you, that may seem challenging for your manager, a good manager is here to be an advocate for you, and really see you grow into an amazing employee, and potentially another manager if you want to be one yourself. But again, they only know that if that is something that you’ve expressed to them. Managing your manager, being clear, speaking up up front, those things work in your benefit.

Safiya Miller: Managing your team and colleagues. I’ll give you guys a second to just take this in. Does this girl talk about hardware? Why are you taking career advice from me? This is a good one because I think sometimes when we talk about mentorship and sponsorship, we get caught up in what that needs to look like. Do I need to be mentored specifically by Satya to make it to the top? There’s probably a short list of people who are going to actually have that opportunity.

Safiya Miller: But if you look around, right, about people that work hard, I’m not saying don’t work hard, you absolutely should, but work smart. Think about the people on your team that are working smart and are being acknowledged for what they’re doing. Right? Those are the people that you might want to take some time to spend with. Doesn’t necessarily have to be someone that’s two skip levels above you. Could be someone that’s right on your team. I think we under-evaluate sometimes our own peer network, and how powerful that is.

Safiya Miller: This comment speaks to it a bit. Networking horizontally. There was a study on LinkedIn where it says 70% of people that get positions in jobs already knew somebody at that company. Could have been a colleague or a classmate. Probably not the CEO. I’m just stating the facts here and the numbers.

Safiya Miller: Can everybody take out their phone if you don’t have it out. Okay. Go to the LinkedIn (mobile) app. Give you guys a minute, as you’re thinking about who you’re reaching out to, and turn on the Bluetooth. Okay? Bluetooth is already on? Great. Some of you may already know this hack, but I’m setting you up for success when I finish. Okay? Go to the bottom screen. There’s the five buttons. Five GUIs here, my network. Click on “My Network” and then on the bottom right, there’s an icon with the figure and a plus. You can either click “Find Nearby”, “OK”, or “QR code”.

Safiya Miller: Click on “Find Nearby”. You’re activating this entire room right now. Okay? I’m helping you save so much time for later. You’re welcome. This is fantastic. Honestly, the reason I’m sharing this is because, again, networking horizontally … No one’s on? These people next to you aren’t on? Just give it a second. Okay. Use this later when you go and connect outside as well. But this is fantastic when you’re in sessions where there’s a lot of people group like this.

Safiya Miller: The other thing you can also do is to find the QR code. Okay? Everybody has a QR code, you just scan it. Those are the two options. But this piece, before you go on and start adding everybody, this is huge. Can we have coffee, because I’m trying to do X, and I’d love to hear your advice on Y? I put this up here because can I pick your brain? Can we catch up, with no indication of time or when? Those are time sucks. You should be really intentional about the people that you want to network with. What specific skill set do they have that you want to learn more about? How are you trying to grow?

Safiya Miller: Be specific, be intentional, do your research. Trust me, the other person, the other side will be appreciative and more likely to take the time to meet with you and have that coffee. Do the homework. Follow up intentionally when someone gives you advice. Keep that connection open and going.

Safiya Miller: I gave you guys the gift early, but because you’re in … because I’m awesome. Thank you. Because we’re in the Bay Area, I’m going to also give you guys another gift, right, because I want to know who’s in the room. East Bay, can I hear East Bay? Okay. Berkeley. North Bay, Marin? Nobody? That’s kind of far. Okay. San Francisco, the city. South Bay. All right. They’re rollin’ deep.

Safiya Miller: Again, I’m trying to help you get these obvious things. Where do you live? Where’d you come from out the way? Most of you guys are in South Bay. Okay, you live in the bay. Get specific. You came here tonight. You have so much potential in the audience. What do you want to grow? Where do you see yourself at the end of the year? I’m certain someone in here can share something with you that will make that impactful and valuable. Make the most of your first 90 days, manage yourself, manage your team and your colleagues, and most certainly, manage your manager. Thank you so much.

Microsoft Senior Director of Silicon & System Architecture Elene Terry gives a talk about how to leverage your silicon expertise to move into a category that lets you do your best work at Microsoft Hardware Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Elene Terry: [inaudible]. Let’s see if this is working. Thank you. I’m going to start by taking you back to 2011. At the time, I was working at a semiconductor company, and it was working on super cool products. I was working on Xbox, I was working on graphics cards, and I was doing the work that I really love. But something was missing. I would go home, I complained to my husband about everything. I was unmotivated by work. I did an Iron Man. If you’ve ever trained for an Iron Man. It’s like a full time job just training for an Iron Man.

Elene Terry: My husband, he would say, “Elene, go fix it, go find another job, go find something that inspires you.” I did. I went out, I got several job offers, but I sat on them. In fact, I sat on one for almost six months. As I was thinking about them, I knew something was missing, but I couldn’t figure out what it was. Then I got my offer from Microsoft. As I contemplated the offer, I probably was thinking about it maybe the same way you’re thinking about it. Why would a hardware engineer go to Microsoft?

Elene Terry: I’m an ASIC design engineer by training, and at my previous company, there were thousands of people just like me. But as I thought about it, I thought, “If I go to Microsoft, I’ll be kind of unusual. I’ll have some opportunities that I didn’t have before.” I got really excited, and so I came to Microsoft. I took this risk and I came to Microsoft. I’m going to start by showing you a video. This was not put together for me, but it really resonates to my message. Let’s see if it start.

Elene Terry: When I came to Microsoft, I started by working on Xbox. This was what I was working on before. Thank you. Strings and cost downs. I worked on bringing 4K content to Xbox, pretty similar to what I was doing before. But then things started to change, and I worked on HoloLens as [inaudible]. The HPU, I’ll talk about a little bit more later. Silicon for the display, and then bringing that same technology to IoT devices. I think there’s an Azure Kinect out in hall, to go explore with. Then silicon for the data center. I think we have an Azure Stack Edge presentation later too. I’ll talk a little bit more about that. We can see it’s taken on a lot of different forms while I’ve been at Microsoft.

Elene Terry: You can see it’s been a totally exciting seven years for me. I’m going to start by showing you some examples. This is the HPU. I love this picture. It’s beautiful. The HPU is the Holographic Processing Unit. This is the main piece of silicon that’s in the HoloLens. When I came to Microsoft, I worked on HoloLens 1, the HPU 1. I used some of my expertise to work on interconnects and memory controllers. As time progressed, we worked on HPU 2. It’s a pretty small team, there were only five of us.

Elene Terry: Now, I had an opportunity to become the SOC architect. What that meant is I was responsible for trying to figure out everything that went in this piece of silicon. I would never have had this opportunity at my previous company. Remember, there were thousands of people like me, but at Microsoft, I had this opportunity. For instance, as I was working on the HPU, I was introduced to new techniques, things like low power, analytical models for low power, power projections, power modeling.

Elene Terry: At my previous company, there were people that did this. It was an entire team that did this. But at Microsoft, there was no one. I had to go and learn it, so there was just a small team of us trying to figure it out. It was super exciting.

Elene Terry: Then I got to work on the entire device. Remember, Microsoft is a vertical company, so meaning we build the entire device in-house. You can see the HPU is demarked by the heart. So I started to get to work on the device. This is what we call the MLB. I call it the crab board. You can see all these little notches cut out from it in order to fit in a thermally constrained environment. It was super cool, right? Because I was at Microsoft, I got to see this vertical integration. I got to work on things that I just didn’t have the exposure to prior in my career.

Elene Terry: As I started working on this, I started working on more and more different types of trade off analysis. At the time, I had no idea what they were called. I now call this work systems engineering. It meant I was working with all kinds of different teams. User experiences team, algorithms team, firmware team, silicon teams, mechanicals, ID, thermal teams, electricals and interconnect, system validation, sensors display, and I was doing trade off analysis between all of them.

Elene Terry: I’m not sure if people are familiar with the other picture of Microsoft on the internet. But this is how it’s like for me. What I found is that people, they wanted to work with me. They had not previously had this exposure to the hardware trade off analysis. So people from all of these disciplines wanted to work with me. They wanted to understand how their part of the system worked together. Then most recently, I’ve been pivoting to work on silicon in the data center. Taking all of those same experiences, trying to figure out how we can build silicon that leverages the experiences we have, and is able to go to scale in the data center.

Elene Terry: When I talked about all of this in the past, people have come up to me and said, “Elene, how did you have the confidence to make all of these transitions? How did you have the confidence–How did you convince your boss that you could do this?” The short answer is I didn’t. I would go home to my husband all the time, almost every night, and I would cry, and I’d tell him, “I don’t know what I’m doing. I am bad at my job. I don’t know what to do next.” But what was important is that I showed up at work with confidence.

Elene Terry: I adhere to fake it till you make it. I’m not sure if people are familiar with Amy Cuddy’s research. She has one of the most watched TED talks of all time, and her research is on power positions, and how power poses change our behavior. Why not? A superwoman pose. But what really resonated with me in her talk was when she talks about having a car accident when she was 19. When she was 19, as she’s recovering, she discovers that her IQ has dropped almost two standard deviations. She talks about how she recovers from that, how every time she goes to a new role, she feels like she has to fake it.

Elene Terry: She says she just kept faking it one step at a time until she becomes a Harvard researcher. She says, “If you feel like you shouldn’t be somewhere, fake it, do it not until you make it, but do it until you become it.” The reason that really resonated with me is because you have to fake it, not just till you make it, not just until you are able to do the job, but you have got to fake it until you become it, in the sense that I had to fake it until I felt comfortable doing the job, that I didn’t go home and cry every evening that I couldn’t do my job.

Elene Terry: What does this mean for you? For me, it meant that I was able to leverage my unique expertise to really step out of my box, out of my comfort zone, and be able to leverage that for new experiences. I’m now running an organization that works on all of the roles that I talked about today. For me, I’ve so much more motivated. I come to work present and excited. I have no more time to run an Iron Man. I’ve just been so lucky to be able to identify where I fit in. What I hope for all of you is that you’re able to leverage that, your own unique talent, to find your own niche, to find something that motivates you and allows you to bring your best self.

Elene Terry: Thank you so much. I’m going to be outside answering questions about silicon, hardware, and I’d love to talk to all of you about anything in Microsoft.

Microsoft Mechanical Engineer Carolyn Lee gives a talk on HoloLens at Microsoft Hardware Girl Geek Dinner. Microsoft Mechanical Engineer Carolyn Lee gives a talk on HoloLens at Microsoft Hardware Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Carolyn Lee: Hello, how’s it going? All right. Mics are good to go. Hi, everybody. I’m Carolyn, and I am an engineer on the mechanical team for HoloLens. When Josh asked me to speak at this event, at first, I wasn’t exactly sure. This isn’t something I normally do. I realized I get to stand up here and talk to you guys for 10 minutes about something that I look forward to waking up and working on every day. So let’s just get right to it.

Carolyn Lee: A little bit of background about myself. I started off at Microsoft as an intern during the summer of 2017. They were crazy enough to let me to come back the summer of 2018 to intern again, and I started as a full time engineer here last August during the summer of 2019.

Carolyn Lee: Before we get going, how many of you guys know what HoloLens is? Nice. Awesome. How many of you guys have gotten to try HoloLens 2? Great. For those of you that haven’t, I would highly recommend trying out one of the demos outside before you go. We have Silicon Valley’s finest out there leading all of the demos. For those of you that don’t know what HoloLens is, HoloLens is an augmented reality or mixed reality device that projects laser images onto your eye to allow you to overlay holographic images on your real world.

Carolyn Lee: Unlike virtual reality in which everything that you’re seeing around you is fake, augmented reality actually allows you to see the world around you, put objects onto that world, and it enhances the way that you can interact with your space as well as people, both near and far from you. HoloLens 1 was actually released as a dev kit. It was released for developers to come up with software and create programs that would run on this device, which created a really interesting environment because it was now being used across a wide variety of industries in which developers thought it would be most useful.

Carolyn Lee: One of these industries was the medical field. Doctors can actually wear this device and overlay CAT scans on their patients to know exactly what they’re operating on before they start an operation. I was actually talking to my sister on the phone the other night, and she had mentioned how one of her friends in med school uses HoloLens as their main training device for one of their classes, which I thought was super cool, one, for the use, and also because he just mentioned this in passing because he thought it was so cool, not because he knew that she had a sister that worked on HoloLens.

Carolyn Lee: What does our product design team here actually do for HoloLens? Our product design team creates all the parts that you can actually feel and see in the product. That’s everything from design to manufacturing, to assembly, to troubleshooting later on. We’re working cross-functionally with our sensors team, our optics team, our EE team, human factors to make sure that we’re taking in all the user research into account, to try and create a device that’s going to meet everybody’s needs and requirements, and create the best experience for the user itself.

Carolyn Lee: A little bit about my career here. I started off as an intern back in the summer 2017, like I said before, and my first summer I was working on scaling the fit system prototype. The fit system is how the device actually goes on to your head, and scaling being taking it from one device to say 20 prototypes that we could then use in user studies. This is particularly relevant because one of the main points of feedback that we got from HoloLens 1 was that the device needed to be more comfortable. This is important because when a user puts on the device, we want this to be an immersive experience. We want them to transition from reality to mixed reality without even knowing that they’ve put this device on their head. That’s why comfort was so important.

Carolyn Lee: I got to work with a great manager who had did a bunch of research into what the center of gravity of the device was, what moment was this putting on your neck, how is this affecting the user experience, how heavy was the device, how could this device actually be worn for an extended period of time? I got to work on trying to scale a prototype that was going to be then be used for human factors research. Then with that, I also got to work with a super experienced engineer who was my mentor throughout my two internships, and pick his brain on how he did design, and what was important to him, and what were things that he was looking for.

Carolyn Lee: With that, it was a very hands-on experience, because with prototyping, comes actually creating the prototypes. HoloLens has a great resource here in the Silicon Valley. Brian Golden in the back leads our machine shop, and I got to work with them a lot. Yes, big round of applause for Brian. I got to work with them a lot throughout this process, and really what it did for me in my first internship ever, that summer after sophomore year of college, was take academia and make it real. It made it tangible, and it made me excited to continue on the mechanical engineering track, and made me really excited to come back the next summer.

Carolyn Lee: The next summer I came back, and I actually got to own my own part that summer. It’s a very small part, so I could actually run it through the whole design cycle in that three months. Designing it, working with vendors overseas to get it manufactured, bringing it back here, working in the shop to run it through some lifecycle testing to see how this is actually going to perform over the span of its time, and use this to inform our design later on. I really enjoyed the responsibility of getting to own my own part and work with different teams.

Carolyn Lee: I got to work with the reliability team a lot that summer to understand a broader scope of the design cycle, which became really important when I was working as a full time engineer, because right when I started we hopped into our first, or I hopped into my first full fledged design cycle. There wasn’t really much in the way of bringing up time, but I actually liked that because it made me feel like I was coming in and making an immediate impact that I was going to get to be able to work on meaningful work right away, which I really enjoyed.

Carolyn Lee: My first internship, it made academia real. My second internship gave me a dose of what the design cycle is like, and being here full time, I think I’ve started to realize how much the people are super important. The first two summers, I got to work with a great manager and a great mentor that gave me a little taste of that. But coming back full time, I realize how important it is to be surrounded by people that want to help you learn, want to help you grow, and are all working towards the same goal.

Carolyn Lee: An example of that is Edwin, who was one of the other engineers on our team, had plenty on his plate to keep him busy during this design cycle. But he was working on parts. He had worked on parts in the past that were similar to what I was working on now, and whenever I needed help with anything–I had a lot of questions starting out, and whenever I needed help with anything, he was always right there to help me. “What can I do to help you? What resources can I provide?” If he didn’t know the answer, he knew who to tell me to talk to, to find out that answer.

Carolyn Lee: I think the most impressive part about that was, I never once felt like he was rushing to get back to his own work, even though he had plenty there to keep him busy. He was there to make sure that I could be as successful as possible. On the other side of it, not just in terms of technical support, being early in my career, and not always knowing where to go and what to do, Teresa, who shares the office next to me is in the back, she’ll love the attention, was really great about making sure that I knew what to look for in my career at this point. She said, “I was in your shoes four years ago, and here are the things that you should be looking for, and this is what you might want to look out for in terms of your career, and what do you want to do.”

Carolyn Lee: It was really nice to be able to have resources both on the technical side, and I felt like my peers were looking out for me in terms of making sure that I felt as supported as I needed to, which as a young engineer looking for a job and trying to figure out where exactly I want to take my career early on, I think the thing that was most important to me was that I was going to be somewhere I felt like I could develop, and that I could learn, and I was going to be pushed to grow as an engineer.

Carolyn Lee: I think that one of the most exciting things about working on HoloLens is that it’s challenging. This is something that I remember from my very first interview back in the winter of 2016, when I was sitting in the room with Roy, who leads our mechanical team. He had said, “When you’re designing a product, you start off by looking at what’s been done before. You work on there and see which parts of this do we want to keep, which parts of this are we going to move away from?” He said that when they were creating HoloLens as an AR device, and they looked for examples, there were no examples. AR hadn’t been done before.

Carolyn Lee: That was exciting. It was challenging, there was no example to look at, but it was exciting because we get to be the people that create that example, that later on one day, a company is going to look at how they’re going to do this, and they’re going to look at HoloLens. We get to design the track that AR is going to take in the future. HoloLens, I felt was a place where I was going to be pushed as an engineer, I was going to be surrounded by bright and hardworking individuals, and it’s an opportunity to work on cutting edge technology, cutting edge technology that’s expanding industries and paving the path for what AR can do and will do to change our future. Thank you.

Microsoft Product Manager Shivani Pradhan gives a talk on Edge Computing at Microsoft Hardware Girl Geek Dinner.

Microsoft Product Manager Shivani Pradhan gives a talk on Edge Computing at Microsoft Hardware Girl Geek Dinner. Erica Kawamoto Hsu / Girl Geek X

Shivani Pradhan: [inaudible]. Oh, it gets better. We start with a very nice ad that they made. (music)

Shivani Pradhan: That’s the Azure Stack Edge device, and I’m a PM Azure Stack Edge team. I’ve been around for almost 19 years with a lot of engineering and business side experience at this point. I’m pretty new to Microsoft. I’ve been here, actually, just like Carolyn, I joined full time in August 2019, so almost six months now. The best thing I feel about Microsoft today is people. It feels like home.

Shivani Pradhan: My team was building these two products, and so they’ve been working really long, really, very hard. Being new in the team, when you approach somebody, you’re being mindful of not wasting their time. Also being conscious that you don’t want to take any dumb question to them. But everyone has been so embracing, so welcoming, not a frown on anybody’s face that you’re wasting their time. That’s very, very supporting. That is really encouraging at the same time. I’ve really enjoyed my ride last seven months, and I would encourage all of you to apply to Microsoft.

Shivani Pradhan: What is Edge Computing? Think of cloud computing and the cloud capabilities. Capabilities like artificial intelligence, machine learning, and pushing them from a public cloud to all these physical devices that are connected. Cloud ability on the Edge is basically Edge computing. You may ask, “Why bother, because a lot of these physical devices do not have great connectivity? In fact, a lot of them don’t have any connectivity at all.” In those circumstances, you want the compute power on the Edge, very close to the data because data has gravity.

Shivani Pradhan: That’s where Edge computing comes in. Microsoft over here has a wide range of devices that they bring to you for the intelligent Edge, literally from hyperscale cloud, where they have availability in 56 regions and over 140 countries to small integrated chips that they’ve put in every coffee machine with extremely high security, mindful of all the capabilities that they can bring to a coffee machine that is connected to the cloud, bringing the cloud capabilities to that machine, but the same time, making sure it’s super secure.

Shivani Pradhan: Right in the middle is the Edge device. That’s the team that I work for, and that’s one of the Azure-managed AI-enabled compute appliances that we build. It comes with hardware accelerated FPGAs. Those are integrated circuits, or Nvidia, supported Nvidia’s GPUs that you can put in there to really put high amount of boost power behind whatever compute you’re doing. You can run VMs on it, you can run Kubernetes on it. The best part is it’s completely Azure-managed, which means you go to Azure portal, you deploy your device, you can completely manage it without worrying about your IT. You can create a custom app and just push it to all your physical devices.

Shivani Pradhan: In addition to that, it is a storage gateway, which means in your disconnected mode, you have petabytes of storage to store data locally, and then you can push it to the cloud at your own pace, at your own schedule. Edge devices cover a variety of use cases. Some of the most popular ones are machine learning on the Edge. One of the most common cases that we are seeing is running intelligent AI and machine learning inferencing on the Edge.

Shivani Pradhan: For example, let me give you an example of Kroger, which is trying to look for shelf spaces which are empty. They run an AI model to detect those empty spaces. But what they found was that if they are last couple of boxes remaining, those shelves do not detect as empty. Interestingly, there’s a psychology behind when we go to pick a box, and that’s the last box, we don’t pick it. We’re like, “There must be something wrong. Why didn’t anyone else pick it?” There’s the last box of Cheerios. You look at it, and you’ll put it back, and you will not walk away with it. That shelf is not empty because it has one box sitting there.

Shivani Pradhan: They developed an intelligent AI model to actually detect that now there’s only one or two last boxes left. So instead of a customer walking and saying, “Hey, you’re out Cheerios,” or somebody walking up and down the aisle, and saying, “Okay, Cheerios out, this out, that out.” The model detects and right away informs, and so suddenly, your supply chain is working better. You’re keeping it stocked.

Shivani Pradhan: The second popular use case is Edge compute and IoT solutions. I have a full slide on that one, and the network transfer where you can actually decide your own pace of transferring your data to the cloud. Machine learning on the Edge is another very popular use case with drone footage. But I have an even better example. We all have seen or got messages on our phones when cops are looking for a specific car, where we see say, “Okay, this car, if you see it, please text.”

Shivani Pradhan: Think about it. We have tons of traffic police cameras all over the city. They all are collecting that feed. That feed gets collected, sent to the cloud. Six hours later, it tells the police saying, “That car passed over there, over there, over there.” Six hours later. Come on. In 2020, you want it to be instant. It should have said, “The car is passing this now, now,” so you can track it. Now, instead of blasting millions of people on their phone saying, “Did you see this car?” Right? That’s the immediate results of Edge processing, right, that camera could have directly, just on that quick processing on the Edge. It didn’t need to collect 20 petabytes of data, it just needed to do that quick inferencing and react to that. That reactivity, that quick response comes with reacting on the Edge, being closer to the data.

Shivani Pradhan: Similarly, filter with AI analysis. That’s near collisions. That’s actually something that state of Washington, couple of cities in state of Washington are already doing, where they’re collecting only one minute of data. They have AI models to figure out that a collision happened, or almost a near collision happened. They try to cut off the video feed 30 seconds before and after, and just that one minute is sent for further analysis, and figuring out, and influencing the traffic engineering. That is pretty cool.

Shivani Pradhan: Then lately, a lot of influence around privacy. We could actually do a lot of identification and blur it, blur the license plates, blur people’s faces. As your private data is anyways being shared, you at least feel a little at peace that it was not my face, that all the Google cars are collecting all over Mountain View. The last of the three cases that had [inaudible], the Edge compute and IoT solutions. You have, if you look at your phone today, you have tons of apps. But if you go and turn your WiFi off right now, 90% of the apps stop working. That’s because they are all cloud-based applications, and that’s where the world is headed.

Shivani Pradhan: Sure, we all have cloud-based apps. But that said, you want your cloud-based apps to work when you are in the basement, or when you are going through a tunnel, or in a deep forest. That’s what the Edge does for you. You actually continue running all your business cloud applications on the Edge, even in a disconnected mode. But at the same time, there’s certain legacy business apps, which were always made for the native applications, which do not run on the cloud. Your entire 90% of the portfolio has already migrated to the cloud. But now, you have these native apps that won’t run. Edge comes perfectly in the middle to connect the two places over there.

Shivani Pradhan: Then you have the perfect scenario where you actually want to take your applications down into the field. Like you were seeing in that very nice, fancy ad, things have broken down. Everything is not there, and you still need your maps, and you go to your online maps, they won’t work because the wires are down. But your Edge would work, and you can still do the overlays, you can still run your AI models, your drones can still fly around, take pictures, create overlays on top of that model. You update your model live on the Edge, and then you distribute at least to the disaster recovery teams, and they can keep working. So that’s taking applications into the field.

Shivani Pradhan: Then the most popular case, that’s how most of the Edge solutions started, was to do with transferring your data to the cloud. As more and more companies were trying to migrate, a lot of them constantly make a lot of data, and they want to keep pushing. But then there are some that want one large migration to go all of a sudden, and then those who know the Big O notation comes in over here saying, “Don’t send it over a pipe for 300 years. Right? Just put it on a plane and ship it, and that would be faster and cheaper.” That same way, you can decide your different models that works the best for you, and you could manage first because you only have a 10 millimeter pipeline, versus a big migration, versus constantly sending. You have all your options. It’s up to you. It’s your custom solution.

Shivani Pradhan: Esri is a company that actually works on providing maps, specifically in map-based technology for disaster situations. One of the previous examples I was giving was actually, that’s what Esri does. They, in a disaster situation, they load up a typical truck with sensors, cameras, drones, and an Edge, and they drive into the disaster zone. These drones fly around and take all kinds of pictures. Those pictures come back. Now, you have all kinds of junk as well in that picture. You run AI models on it and find the points of interest. Then you create overlays on the fly, and then you merge those overlays with your existing maps. Then you have a new map, which says, “Okay, two kilometers from here, you have a bridge broken. From here, there’s a fire, which is literally 0.5 miles away.” You can convert it on the fly of what data is important to that team, what are they looking for? Then you update the maps, and you’re able to actually do really effective work. This is something Esri is using Edge for today.

Shivani Pradhan: Let me tell you the story about this. This was a last minute slide. I’m so sorry. Doesn’t have a title. A few years back, there was a huge Ebola outbreak in Liberia, and USAID response team was put together and deployed to go work on a response for this. The team, when they landed on the ground in Liberia, their first task was to just find information and categorize information. That was not easy because they needed to go find out the state of healthcare centers, hospitals, find out the state of WiFi, find out the population density centers in that area. It was a very challenging task.

Shivani Pradhan: As they started piecemealing all that information together, this is a real whiteboard of that team that they put together. If you look at this, this is such a horribly complex and convoluted map to figure out how they’re going to provide support to healthcare centers in that environment with Ebola all around you. This was their index file. This was their index file to figure out things. The Edge team took on this mission saying, “Okay, how could we have helped them?” We did exactly that. We created an app in Azure on the cloud to actually just go and find information, and categorize information. But then, just what Edge is supposed to do, we decided to use cloud capabilities and enable all the cloud capabilities to it.

Shivani Pradhan: This is an Edge, which is actually running in a disconnected mode, and we uploaded a bunch of maps to it. Once you uploaded all the maps, it processed all those maps, and so you have some default information, PDFs, pictures, JPEGs, documents that have already been uploaded. Now, when you start enabling all the cloud cognitive services, so first thing that we would do is search for, it’s Ebola, healthcare centers. We could just search for the word hospital, for example. When you search for hospital, a healthcare facility comes up, various PDFs come up, and everything. But you look at that, that’s a JPEG. Okay. When you look at the JPEG, and you look specifically, enable the OCR on it, it can now convert the JPEG into readable doc. It can find text in it, and it has been able to detect all the hospital words in it.

Shivani Pradhan: Not only that, it actually found a French document, which also had a translation of the word hospital. I can’t see any word hospital over here. But when you go into translation, you actually see that it found the word hospital in the French translation of that. Like, “Okay, that was cool. I didn’t know French, but I did find that there is some hospital, which French organization found over there.” Now, when I go and look up the word Lofa, now, Lofa is where the Ebola had originated. It was the ground zero for that. When I looked at that, at that time, this map comes back. Why did it come back? Because the OCR technology in the cognitive services has a feature where it can actually read handwriting.

Shivani Pradhan: Not only that, it changed the JPEG into a readable format. It detected handwriting, and was able to read the word Lofa on that picture of the whiteboard. That was pretty enabling, and that was pretty helping. That’s all. Thank you so much.

Aaratee Rao: [inaudible] you hear me? All right. What an amazing set of talks. Can hear one more round of applause [inaudible].

Thank you for joining us at the sold-out Microsoft Hardware Girl Geek Dinner with HoloLens and Garage demos, great talks and even better company!
Thank you for joining us at the sold-out Microsoft Hardware Girl Geek Dinner with HoloLens and Garage demos, great talks and even better company!  Erica Kawamoto Hsu / Girl Geek X

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Bloomberg Engineering Panel Discussion, Fireside Chat, and Lightning Talk (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Team (Gretchen DeKnikker, Rachel Jones, and Angie Chang) and Bloomberg Engineering (Mario Cadette and Bailey Frady) welcoming the crowd at Bloomberg Engineering Girl Geek Dinner in San Francisco, California.  Erica Kawamoto Hsu / Girl Geek X

Transcript of Bloomberg Girl Geek Dinner – Lightning Talks & Panel:

Bailey Frady: All right. Hello everyone. How are you all doing? Good.

Audience member: Good.

Bailey Frady: Good. How was the food?

Audience member: Delicious.

Bailey Frady: Great, glad to hear it. Well, my name is Bailey and I just want to officially welcome you to Bloomberg Engineering. We are so glad to have you here. I know there’s a lot of places where you could spend your Thursday evening, so we’re really thankful you chose to invest your time here. Like I said, my name is Bailey. I’m a project manager here and I have been working with the phenomenal Girl Geek team to put this event on for you. So without further ado, please help me welcome Angie, Gretchen, and Rachel.

Angie Chang: Thank you. Hi, my name is Angie Chang, founder of Girl Geek X. I wanted to say thank you for coming out to check out Bloomberg Engineering tonight in San Francisco. If you haven’t seen the sting rays, you’re adorable. And I’m so glad that we’re here to hear from some really amazing Girl Geeks tonight.

Rachel Jones: Hi, I’m Rachel. I’m the producer of our podcast and if you haven’t listened to it before, I would encourage you to check it out. We have a lot of really great episodes. My favorites, we have one on branding, one on self-advocacy. They’re really great. Season two is starting really soon. We’re going to be trying some new stuff. Our first episode of season two, we’re actually answering your questions that you sent in through our survey. So yeah, give it a listen.

Gretchen DeKnikker: I’m Gretchen, thank you guys. Who’s, this is their first Girl Geek event? so we have a lot of returning. Welcome back. Thank you for keep coming. Most of you know that we do these almost every week. The little known secret is you can do one at your company also. So if you want to find out what it’s like, find Bailey who’s been working so hard on this has been our interface and yes.

Gretchen DeKnikker: And Noor is around somewhere also. And then I’m sure there are a ton of other people that have been working on this, but ask them what it’s like and what it’s taken to put it together and think about doing one of your own. And then if you guys have seen our emails lately, I’m trying to stop saying you guys and I did it.

Gretchen DeKnikker: If y’all have seen our–Be a proper feminist when you’re on camera! Okay. So if you’ve seen emails lately, we just launched registration two days ago for our annual virtual conference, which is called Elevate. And we have amazing lineup. We have Carin Taylor, who’s the chief diversity officer of Workday. We have the CTO of Intuit, Marianna Tessel, just an amazing, amazing lineup.

Gretchen DeKnikker: It’ll all be targeted for like mid-career women. So not just as much early stage content, but like for everybody else too. So register, it’s free. If you’d like to get involved, tell your company. It’s a really great sponsorship opportunity too. And without further ado, let’s kick this off tonight. Okay, cool.

Narrator: Go. Two letters. One syllable, a revolution, a world of potential in a single keystroke. The central nervous system of global finance was imagined and engineered more than 30 years ago. In 1981, Mike Bloomberg and his partners saw an opportunity to bring digital innovation to an industry where information was transmitted slowly and inefficiently.

Narrator: They built the Bloomberg Terminal, one computer system that allowed investors the same real time access to financial information at the same time, no matter their location. It was a product of the future willed into existence, a continuously evolving system built upon pioneering technology that transformed global capital markets forever.

Narrator: We empower people to make critical, transparent, and informed investment decisions while reducing risk and creating the tools of tomorrow. At Bloomberg, we are constantly thinking about and investing in the future. Always going where others aren’t, can’t, or won’t. We’re rolling out hundreds of new products and enhancements every day with our ears to the ground and an eye towards the future. We connect people in ways and at speeds no one else can. We process 100 billion market data messages daily, peaking at more than 10 million per second.

Narrator: Our 15 million distinct streams of financial data transmit in 13 milliseconds, 27 times faster than the blink of an eye.

Narrator: Our reporters break news from locations other news organizations have yet to visit. We have the largest business new staff producing more stories from more places than anyone else in the world, 120 countries and counting. We work around the clock in every time zone, never shutting off, never powering down because that’s what our customers require, access from wherever they are, whenever they want, however they choose to connect.

Narrator: We have over 5,000 technologists and computer engineers, a full 25% of our workforce, designing new functions and products before customers even know they need them. Innovation and collaboration are the reasons for our continued success. It’s how we’ve always worked and it’s what will guide us forward, with over 175 locations we are investing in our employees by building the workplace of the future.

Narrator: We go further. Stretch our impact farther. We use our power to connect people to create positive change for the entire planet, not just our bottom line. Through Bloomberg Philanthropies, we invest almost all of our company profits to address the most urgent public challenges generating the greatest good for the most people. It’s our purpose.

Narrator: We are vigilant in organizing and interpreting information in a complex, ever changing world. Looking decades into the future and engineering what our clients will someday need has been our mission from day one. We’ll never stop building, growing, and staying true to our original innovation. Go deeper. Go where others aren’t.

Mario Cadete: Hello. Hello. Hope you enjoyed the video of our company. Thank you, Girl Geek, for making tonight possible. Thank you all for coming. Thanks, Bailey, for putting this together. My name is Mario Cadete. I head up our Bloomberg San Francisco engineering office. A little fun fact about our office. It was custom designed for software engineers. So we really like that and we were all engineers and we like to have it as our little-

Audience Member: Sting Rays.

Mario Cadete: Engineers like Sting Rays, I’m told. We have this floor, the floor above us. It’s a little smaller, cozier than our other offices. But we like it that way. We’re due to get another floor later this year and we’re really excited. That’s going to allow us to add another 50 engineers to our workforce here in San Francisco. Personally, just a little bit about myself. I’m fairly new to the Bay Area, so I’m looking forward to meeting many of you after the program.

Mario Cadete: I started my career in Bloomberg engineering in 2000, and I’ve seen some of the 20 years. I get that facial expression a lot, especially when you interview candidates that come in. Yeah, it’s a long time. During that time, I had great opportunities to work on many challenging projects in New York, in London, and now in San Francisco.

Mario Cadete: What kept me at the company over these years are really three main areas. And they’re should… they’ll come out tonight in our agenda. First I love tech, and you’ll hear more about that in our first panel on how to thrive in open source. So that’s going to be really exciting. Secondly, I care deeply about our commitment to D&I. I know I’m in a role that I can be a key ally to women in technology and I don’t take that lightly.

Mario Cadete: I think about it often and I hope it shows in my leading of this office. And you’ll hear more ideas to make your workplace more inclusive in our fireside chat, taking an employee resource community from idea to impact. And lastly, I love as a company how we give back. It’s in our DNA.

Mario Cadete: As a company we donated almost a billion dollars to charity in 2018, $1 billion. So a lot of money. Also in that year myself and almost 20,000 of my colleagues donated over 150,000 hours to charity and communities where we live and where we work. But most importantly to me is how we invested in our employees. I take great pride in seeing our people develop both professionally and personally.

Mario Cadete: So as an office, in addition to the project work that we do, we hosted over 100 events that range from professional development to clubs like Bloomberg Women in Technology to tech community events like this.

Mario Cadete: Our culture is one of the main reasons that my colleague Dobs decided to join us a couple of years ago. You’ll hear more about that during her lightning talk and how to find a dream job in tech. So enough about Bloomberg for now, if you have any questions, please ask me or somebody in one of these stylish blue t-shirts, ‘cuz there are a couple of them around, after the program.

Mario Cadete: So let’s move on to what you came here for. Valuable insights to advancing your career and meeting other incredible women working in Silicon Valley. Without further ado, I’m proud to introduce my colleague, Danica Fine, who will lead a panel discussion on how to thrive at open source. I hope you enjoy. Thank you.

Bloomberg Engineering Software Engineer Danica Fine moderates Stephanie Stattel and Paul Ivanov in a panel conversation on how to thrive in open source communities at Bloomberg Engineering Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Danica Fine: Can you hear me? Yeah. Okay. I said it earlier today, I was really excited about the director chair. This is great. Thank you so much for giving me a director chair. Well, hey everyone. Welcome to our panel on how to thrive in open source software communities. We have a great program for you tonight, but before we get started, I really want to see a show of hands, how many of you are familiar with or already involved in using open source technologies?

Danica Fine: All right. Good. That’s good. You’re on the panel.

Paul Ivanov: Please.

Danica Fine: Who are you? How many of you are participating in these open source communities already? Or maybe even actively contributing code back to these open source projects? All right, quite a few. So I think we have a good mix in the audience tonight. I know that some of you didn’t even raise your hands, so I hope like by the end of this you’ll know what we’re talking about. So hopefully, our panelists can shed some light on the subject.

Danica Fine: So as Mario, mentioned we have with us tonight three of our star engineers. We have Stephanie Stattel, Paul Ivanov, and Kaia Young. Before we dive into questions, why don’t we introduce ourselves. Paul, let’s start.

Paul Ivanov: Hello. I’m Paul. I’ve been at Bloomberg for three and a half years. I work largely in open source on the Jupyter Project. So I’m one of the steering council members and was fortunate enough for the project. If you don’t know Jupyter Notebooks are a way to do data analysis in different languages and to communicate your results with colleagues so that you can rerun it and so that they can rerun it. And so I’ve been working on that since before the project existed as Jupyter, as IPython, and we were fortunate enough to win the ACM Software System Award in 2017. So it’s great to be able to contribute to this tool and give back to the community and continue to do that here.

Danica Fine: Stephanie?

Stephanie Stattel: Hi. Yeah, my name’s Stephanie Stattel. I’ve been at Bloomberg going on nine years now. I moved out to San Francisco two years ago to work on the team build- working with Jupyter, building a data science platform on top of Jupyter. And right now for the past year, I’ve been working on an infrastructure team, so I’m sure many of you saw the terminal demo. The team that I’m on works really closely with Chrome and the windowing stack that supports the terminal. So happy to chat with anybody about that after the panelists and talks.

Kaia Young: And my name’s Kaia Young, I’ve been with Bloomberg also about two years, here in the San Francisco office. and I’m an engineering manager here for a new team that’s focused on data visualization and tooling for a new data science platform that we offer. So my team develops data visualizations and some of the platform related to that, largely built on a lot of open source technologies like D3, Vega, pandas, NumPy, a lot of the kind of general Python data science stack that you all may be aware of.

Kaia Young: So we do develop tools for internal use as well as contribute to those libraries that we do use.

Danica Fine: Thanks. All right, let’s get started. Stephanie. So you mentioned your involvement with project Jupyter. Can you tell us more about how you got started in the Jupyter community and like what was that journey like for you?

Stephanie Stattel: Yeah, sure. So I can say that when I started on the Jupyter team, that was my first exposure to the open source world and communities. So needless to say it was a little bit intimidating. When you go to a github page and you see a list of issues and a lot of activity in terms of pull requests, it’s really hard to know where to get started. And so something that I really appreciated about the Jupyter community in particular, there’s so many in person events, conferences, workshops, hackathons, and studio days. And so for me, that was my real entry point, getting to know the people behind the community.

Stephanie Stattel: And it was a really great way to find the projects that I was interested in working on and what lined up with what the community was developing. So in something like a full studio day event, you find people of all levels of expertise. People like Paul who have been with the project for over 10 years. People who like me had never used Jupyter, made an open source pull request before and we’re all working together. So I think for me it was a great mentoring opportunity.

Stephanie Stattel: And I think when you’re looking for open source communities to engage with, it’s really important to find ones that have a really welcoming environment where it’s okay to ask questions and be new at things. And I think it really speaks to the growth we’ve seen in a project like Jupyter where it really takes into people with a lot of different viewpoints and is open to kind of pursuing different avenues. And I think that’s why I’ve stayed active in the community for as long as I have. Yeah.

Danica Fine: I really appreciate hearing your perspective on that. ‘Cause like, I’m sure a lot of us didn’t realize how simple it could be to get involved. And, as someone who’s kind of outside of the community like you’ve actually made it sound a little less daunting, a little more welcoming. So thanks.

Danica Fine: So Kaia, your Bloomberg product is built on top of open source technology. Could you give everyone an idea how you’re able to leverage this technology and your team? And as part of that, how are you interacting with that community?

Bloomberg Engineering Team Lead Kaia Young (right) talks about open source communities at Bloomberg Engineering Girl Geek Dinner.   Erica Kawamoto Hsu / Girl Geek X

Kaia Young: Yeah. I mean working with the open source community in the context of business product is a little bit different than doing it as an individual contributor or just as anything else. So there are kind of some interesting challenges there as well. But even besides that, I think there’s a lot of advantages to working with open source software even in the context of business. Like for example, you can get to market a lot quicker. Why spend a lot of time making something that other people probably already made better definitely than me.

Kaia Young: So also with that it kind of gives company this ability to focus more on our core competencies. Like for example, Bloomberg, we’re very, very interested in the financial side of things. So leveraging a lot of good open source technology gives us a way of kind of getting those products out there a little bit faster so that we can focus on the particular value that we add.

Kaia Young: I think interacting with the community is a very, very kind of interesting thing. Mainly because I think one of the areas that we’ve been able to be successful in is having good relationships with those communities. So some of the strategies we do there is we try to really build an understanding of who’s using the open source software. I think sometimes it can be really, really easy to kind of be focused on the particular thing that you want to do.

Kaia Young: Whereas some of the technology we’re using are used for all kinds of, of things. Like Jupyter itself is used for academics, for research, all over the place. So really spending the time with the community and the stakeholders in that community to really kind of gain an understanding of who’s at the table? What are people using it for? So that then we can position ourselves to understand what the roadmap is, and then how we can actually be a part of that.

Kaia Young: One of the things we do want to avoid is obviously saying that, this is something that we want to contribute to. How can it help us? I mean, that’s not what we want to do at all. So from our perspective it’s really important to kind of understand where the community is so we can see where we can act.

Kaia Young: Essentially it’s kind of a forced multiplier. So by understanding that we can identify expertise that we have that may be valuable to the community and then work together to make a product that everyone can be used and used for. I think it’s really interesting to hear kind of Paul’s perspective on it. Jupyter in particular, having gone for so long and being used by so many people. I’m not saying you’re old-

Paul Ivanov: Thanks.

Kaia Young: … but it’s like, [inaudible]. But some of the Jupyter events that I’ve been at, it’s like really, really amazing to see how some of the software that’s being used. So like for seeing some of the stuff that I’ve developed being used, I think at the last event there was being used to predict weather, there was a government demo on fluid dynamics. They’re using it to find new planets. And then like, I just made a thing that puts some stuff on the screen, but it’s like really, really cool to be able to see that we can also contribute back.

Kaia Young: So rather than just being focused on the needs of our consumers and our clients that we can actually kind of give something back to the community that’s used for research and all these other things.

Danica Fine: That’s awesome. I think it’s like really interesting to see how does that go back and forth rather than your team just taking this product and utilizing, but like it seems that there’s like a lot of effort on both sides to make this build and maintain this sort of partnership. So, Paul, as someone who is a leader in the Jupyter community, as so many people have alluded to. You’re great. Could you speak to how you maintain the community space that both fosters inclusiveness and mentorship, and then also supports these external partnerships such as the one that Kaia had mentioned?

Paul Ivanov: Right. Yeah. I think it’s useful to sort of take a step back and make the point that like, even though we’re talking about open source, like it’s one thing, it’s no monolith. So there’s different scales. And so maybe I’ll just go through some of the history of like how Jupyter came to be here and how I’ve participated in it. And that’ll help sort of shed light with how I think about this.

Paul Ivanov: And so I think the, the best way to get involved with open source to scratch your own itch. So if you have something that is bothering you, whether or not it’s making your own project around that, or finding a project that’s already helping you somewhat and then changing it for your needs, I think is a very good way. And that is the way that I started with IPython, which then led to IPython Notebooks.

Paul Ivanov: So when I was in graduate school, we were using these tools for ourselves to do our data analysis. Okay? And we knew that we wanted to share that with other scientists and with the world at large, but we didn’t have resources for that all we… it was entirely volunteer run.

Paul Ivanov: And so then in 2013, I think we got the first grant from the Sloan foundation, where for the first time, we had seven paid positions to work on this tool, IPython notebook, which already existed but was rough around the edges, full time. So we were able to continue that work, but now we sort of started to shift away from being users of the tool. We were still using it, but now we… like our jobs were to make the tool and not necessarily just use a tool.

Paul Ivanov: So it’s sort of another iteration of that. And so we were still very close to our users and we were still users ourselves. But as more people and companies started to come on board, so it’s not just funded in academia anymore. We have companies that are joining the efforts and resources and more engineers that are joining the efforts. We needed to come up with a governance model and that’s always a struggle.

Paul Ivanov: At our level, that’s one of the big issues is like which way do… which direction do we go? How do we go? And how do we keep the stream of people coming in? And so one of the ways in which… and so to me it was like going from, “Oh, this is the thing I do for fun and nobody pays me to do it because this is awesome,” to, “Somebody is paying me to work on this fun thing that I am doing,” to like, “Oh man, lots of people are actually using this thing.”

Paul Ivanov: I need to make sure that we keep people coming in and thinking that this is fun, and so that it’s not just the job. Because we now we have contributors and leaders that for their entire involvement in the system, they were paid to do that work. That’s just like weird for me. Because for me it was like… it was all of our friends that were just, “Yeah, anybody can contribute. Like we’re clearly going to use this.”

Paul Ivanov: And then there’s some people that have always been paid now to work on Jupyter and that’s great. It’s like it’s weird. It’s like a family that grows and then that also is its own employer. Like it’s a family business. I don’t know.

Paul Ivanov: All right. But what’s happened is as we grew, and this happens to large open source projects, is that there kind of isn’t necessarily room for people to be able to plug in and explore new ideas.

Paul Ivanov: Like, we’re, lots of open source projects have this notion of sprints where there’s work to be done and you can show up and we can hand you out tickets and it’s a bite-size ticket that you will be able to do either on your own or with a little bit of handholding. And I thought that, well, when we were just using these tools on our own, we used to just be very close to it and we used to explore stuff. We did stuff that nobody… we didn’t have to justify. We didn’t have to have a business justification for doing things.

Paul Ivanov: And so that’s why for about a year and a half now, I’ve been helping with my colleagues at Bloomberg running these Jupyter open studio days. So it’s a two day event where anybody of all levels, experience with tech or not, can come to our office here. And it’s kind of like a house party. It’s kind of like a hackathon, but it’s unstructured. It’s deliberately unstructured so that we can plug you in wherever it is that you want to plug in and we can have a conversation about things and to sort of have more of this incubation period. And so that’s sort of… I’m very fortunate to be involved in this.

Danica Fine: This has borderline become the Paul… Paul Ivanov show. Anyway…

Paul Ivanov: Sorry. I did not want to do this-

Danica Fine: I’m really glad that there are leaders in the community though, that are like you, who are making these opportunities more accessible to people. So I really do appreciate that. That’s the end of our deep, heavy questions, lightning rounds. I’m so excited. One to two sentence answers, please.

Danica Fine: You go over and I will come after you later. Stephanie, what advice would you give to someone looking to get involved with the community?

Stephanie Stattel: I’m going to do longer sentences and [crosstalk] junctions.

Paul Ivanov: [inaudible] on this.

Stephanie Stattel: I think for me, something I would say is don’t be afraid to dip your toe into the pond of open source and really look for a community. And I think I’ve definitely found that in places like Jupyter and Electron that really thrive on bringing new people and fresh ideas into their ecosystem.

Stephanie Stattel: I think that’s really important when you’re deciding where to spend your energy. You really want to work with people that are open to new thoughts and kind of like you’re saying, exploring where a platform can go. I think it sort of, for me sort of red danger zone if there’s sort of a timeline that’s mapped out because in reality I think projects evolve in really creative and surprising ways, and so I think you want to find sort of a tribe of open source communities that are open to where a project is going to go. Because I think I even Kaia mentioned this, you really have no idea what you’re building, who’s going to end up using it.

Stephanie Stattel: And I think being open to the possibilities really broadens the horizons for where what your work can do can have an impact. And so that would be my advice kind of…

Danica Fine: You have one more sentence.

Stephanie Stattel: Two sentences. I do?

Danica Fine: Oh that was [inaudible]. Okay, we’ll end it there. Kaia, what do you wish you had known when you started working with open source software?

Kaia Young: What do I wish I would’ve known? It’s kind of interesting to go back to something Paul said earlier, what’s really interesting about open source software is that there are so many different flavors of it. Like some is just companies open sourcing their own software. You have like academics making things and then sometimes just one person wanted something and then put it out there.

Kaia Young: Previous to my career as an engineer, I was a musician and one of my least favorite things in the world was like the unsolicited email of someone saying like, “Hi Kaia, here’s everything that’s wrong with your entire body of work.” And so I find this really… it’s one thing that is really important to bring to open source is kind of a mindset of respect, humility. These things go a long way because it’s really, really easy to look at an open source project, get on there and say like, “Hey, why don’t you have this feature? This should be designed this way instead,” when you don’t know the story about how that project got there.

Kaia Young: It could have been just one person working on it constantly and sacrifice quite a bit for it. So little respect and humility goes a long way. It’s a lesson for me.

Danica Fine: I have learned tonight that our engineers can’t count to two. Okay.

Paul Ivanov: It’s two in some base.

Danica Fine: Paul?

Paul Ivanov: [inaudible]

Danica Fine: Okay. Last question for you Paul, and it’s a doozy. Are you ready? When is the next Jupyter open studio? Is it true that anyone can get involved?

Paul Ivanov: Yes and yes.

Audience Member: Yay.

Danica Fine: Great. We’re done. It’s fine. It’s fine.

Paul Ivanov: It’ll be probably early Spring and so we’ll probably not make the February… late February cutoff, but it’ll probably be early March, somewhere around there.

Danica Fine: We’re good.

Stephanie Stattel: Will people go to see the announcement? Sorry.

Paul Ivanov: Uh-huh (affirmative).

Danica Fine: You can ask questions. This is my official job. It’s fine.

Stephanie Stattel: Sorry.

Paul Ivanov: Tech at Bloomberg will definitely retweets me whenever I tweet about it.

Danica Fine: Oh, do they?

Paul Ivanov: So yeah.

Danica Fine: I didn’t know that. Cool.

Paul Ivanov: Because I know a few people that work at Bloomberg, so it’s really great.

Danica Fine: You’re working? Okay. Great. Yeah. Awesome. Those are all the questions that we had planned for tonight. I’m sure you have more questions for our panelists. So afterwards at the networking session, please reach out to them, pick their brains, clearly they have nothing else to do, so that’d be great. Have fun with the rest of the program. It’ll be wonderful.

Paul Ivanov: Thank you.

Bloomberg Engineering Team Lead Cheryl Quah speaks with Software Engineer Rebecca Ely about taking an employee resource group (or community) from idea to impact at Bloomberg Engineering Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Cheryl Quah: Hello. Good evening.

Rebecca Ely: Hi, everyone.

Cheryl Quah: Hello.

Paul Ivanov: Hello.

Cheryl Quah: Good. Danica. This is my first time sitting on this chair. Feels pretty great. You’re lucky. No, I actually I kind of prefer standing up, but we’ll see. Anyway, welcome everyone to Bloomberg to our little corner of San Francisco with our little stingrays. My name is Cheryl. I’m an engineering team lead at Bloomberg. I’ve been here I think coming to eight years now, so not quite as stretch as Mario, but still getting there. I started out in New York and moved over to San Francisco about three and a half years ago.

Cheryl Quah: And I’m very privileged to introduce Ely. Ely started out as a peace and justice studies major. Thank you. And had a career in government contracting before joining the Hackbright Developer Bootcamp and then leaping… Yeah. Wait, where are the woos coming from? Anyone in the audience? There we go. And then we’ve been so lucky to have Ely with us for the past three to four years at Bloomberg. More specifically to the topic at hand tonight. Ely has been active in essentially all of the communities, or what we call Employee Resource Groups, that we have at Bloomberg in the San Francisco office.

Cheryl Quah: I don’t know where you find the time for that. I’m not going to ask. But and in particular she’s been part of the steering committee for the Bloomberg Women in Technology Allyship Group. And so also a little bit about me is that I’ve been very fortunate when I was in New York to be part of the exciting journey of helping to start the Bloomberg Women in Technology Community that is now being taken over and led by many wonderful other people here like Ely, like Stephanie, and all the other wonderful folk here.

Rebecca Ely: Sorry. Cheryl is downplaying it. She’s basically a celebrity at Bloomberg.

Cheryl Quah: That’s not true. But so why are we here today? We’re here today because clearly, creating and sustaining an employee resource group or community is something that’s very close to both of our hearts. And I guess just to take a step back, how many of you here are involved in an employee resource group at your organization? A good number. Not as many as I thought, but that’s interesting. How many of you who are involved or have found that your community, your employee resource group has been impactful to you personally, either you’re in your career or just in your overall happiness? All right.

Cheryl Quah: All right. How many of you are interested in getting more involved with starting an ERG at your company or figuring out how to increase the impact of the ERG at your company?

Cheryl Quah: Good. All right. So that gives us a few people to talk to tonight. So I think the reason why I’m here and, why we wanted to chat tonight as well was because if you have been actively involved in a community or an ERG then you probably are aware of how much work it takes. Yeah, I see a few nods there. It got you. You’re aware of how much work it takes, how much effort goes into running the community just to organizing a single event, shout out to Bailey, again, shout out to all the organizers of this event, shout out to the Girl Geek organizers.

Cheryl Quah: It’s just… it’s a massive amount of effort. And I think for me personally over the years as I’ve gained experience, sort of what I’ve come to realize and what one of the driving questions for me nowadays is, I always ask nowadays, “How can I be sure, sure that the effort and the work that I’m putting in is paying off? What are the specific outcomes that I actually want to achieve? And is the effort that I’m putting in going… actually moving the needle in some way on those specific outcomes that I’m interested in achieving?”

Cheryl Quah: And so for tonight, we wanted to share some stories from our personal experiences regarding that. And I think in your abstract it says something about launching, growing, and sustaining an ERG. Nobody else remembers what the abstract says, but in this spirit of saying what we advertised, we’re going to start with those questions. So in terms of launching an initiative. Bear with me and the Hamlet moment that Ely and I came up with a short while ago.

Cheryl Quah: So when we are launching an initiative, the three questions that I sort of encourage everybody to ask themselves and that we ask ourselves nowadays is, why are we doing this? Why are we doing this? Why are we doing this?

Rebecca Ely: Yeah, so the answers to those questions from an allyship perspective, at least for me, there’s an entire steering committee, in terms of why are we doing this? I think that there are endless reasons to care about diversity and for allyship more specifically, a lot of the work that happens in companies to improve the environment that folks come into and to improve statistics and to improve outcomes, that falls on the communities that are experiencing the gaps themselves much of the time.

Rebecca Ely: And so allyship very… people have a lot of opinions about the word ally, but it is… we were kind of seizing this swell of support that we have within the women in tech community that is not people who identify as women in tech to really try to shift some of the burden of the work to be done to move towards equity onto people who are already benefiting from the system.

Rebecca Ely: In terms of why are we doing this? I would say so, there’s a lot that companies can do to bring in sensitivity training or stuff like that from outside. You can do surveys and try to take the temperature of the company. But at the end of the day whereas on the ground initiative that was just started by individual contributors who cared. We have access to a lot of information that we’re sort of uniquely positioned for. And so we do a lot of workshops and trainings that are a content we designed based on… What did I call them? Based on like sessions we hold with employees to find out what gaps they’re personally experiencing and what would matter most to them to cover in these trainings.

Rebecca Ely: So we are sort of synthesizing what we’re learning from the people that we really care about supporting and then disseminating that across the company. And we also have a lot of really great access to senior leadership. If I get in a room with a senior leader, I’m not just saying, “Can you do this, this, and this for me?” I’m also saying, “I know what people are thinking. I know what people are talking about. What would you like to know from me? How can we work together to fill gaps? What are you already working on? Where, what are we already working on? What still needs to be done?” That sort of thing.

Cheryl Quah: Thank Ely for talking a little bit about the allyship initiative and I guess… Sorry, go ahead.

Rebecca Ely: Just one more thing on the why are we doing this, which I kind of already addressed, but just there’s also… on the topic of who gets involved in this kind of work most of the time. Mostly it’s not people who are benefiting from the way the systems already are. And so doing trainings on gender equity in the workplace that are attended all by people who already believe is definitely worthwhile in its case. But I think we can have a really solid impact by focusing on people who aren’t necessarily already bought in, who haven’t thought about this stuff much, who are learning for the first time from our workshops, what they could be doing better.

Cheryl Quah: So thank you, Mario.

Rebecca Ely: Thank you.

Cheryl Quah: I got a clap there. I thank you. And so just putting on my… in a former life maybe I would have been a professor, so I get a chance to do that occasionally but nobody else wants to hear that. But anyway, so just to sort of rehash what we were trying to say, it’s that if you’re thinking about getting involved in effort you know is going to take energy and time on your part, think very clearly about your objectives. Think, why are we doing this? Think, why are we doing this? Meaning what is your specific value add here?

Cheryl Quah: And then why are we doing this? Meaning that for the specific outcome that you want to achieve, there are many different paths that you can take to get there. What are the paths that maybe have the highest return on investment? Because all of us have a finite amount of time. All of us have a finite amount of energy. What are the options that you can pick that would really move the needle for what you want to achieve.

Cheryl Quah: I got a five-minute signal over there. You might be going a little bit over. But the second part of the abstract said, growing an initiative. If you think about the word growing, there are two ways to think about it. One is sort of the more intuitive thing, which is just thinking purely about numbers. For instance, my employee resource group had 200 members last year and now has 400 members this year, or my community hosted six events last year and hosted 12 events this year.

Cheryl Quah: So that’s one way to think about it. But the way that I like to think about it, is how are we growing our impact? Ely, can you tell us a little bit more about how you think about that with regard to the allyship initiative?

Rebecca Ely: For sure. So I think that they’re both are important, if you’re having a really phenomenal impact and changing hearts and minds, but you’re changing two hearts and minds, that may not be worth as much as having less impact, but changing lots of hearts and minds. On the other hand, you’ve got to find a balance. I spend a lot of time thinking about if I’ve got possibly too much time, possibly hours, if I’ve got one hour to work on this upcoming workshop, am I publicizing the workshop? Am I making sure we get as many people in the room as possible? Or am I improving the content of the workshop?

Rebecca Ely: Am I making sure that the people who are in the room are walking away with the growth that we’re looking for?

Cheryl Quah: And so the last part is how do we sustain the impact of a community? Or an employee resource group? Or really any initiative that you want to get involved in? And for me, this is pretty personal because when you think about sustaining the impact of any initiative or organization, really, it’s all about the people that are involved in helping to run the organization, helping to run any sort of initiative that the organization sponsors.

Cheryl Quah: And so for me, sustaining the impact of any community over many years means for any individual who’s an active member there, are they doing this in a sustainable manner. So if I’m asking you… I heard the lady in red, who nodded early on, if you’re actively involved in an ERG, are you doing this in a sustainable manner for yourself? Because it takes a lot of effort. It takes a lot of energy.

Cheryl Quah: So, thinking about for any given individual, are you maximizing your impact if you had multiple different options to choose from, which option are you going to pick to invest your energy in? And also, how do you start acting as a force multiplier. Somebody used that term early on as well today. But how do you get new blood into the organization? How do you grow new leadership? So that over time it’s not all resting on the shoulders of a few core people in the organization.

Cheryl Quah: So, Ely, tell us a little bit more about… you’ve been involved in this for a couple of years now, tell us a little bit more about that.

Rebecca Ely: Cheryl is intimately familiar, I would say, with how this played out for me last year. I, as Cheryl mentioned, have been involved in lots of ERGs. And little over a year ago was asked to join the allyship initiative as a steering committee member, which is a pretty big commitment, and was really having a great time with that and also was working to like give away some of the other responsibilities that I’d taken on over the years that were sort of causing me to split my time.

Rebecca Ely: And then I was asked in the middle of last year to become a co-lead for the San Francisco Sustainability… I’m sorry, I was already doing that, for the San Francisco…

Cheryl Quah: Too many communities.

Rebecca Ely: Be Proud chapter and Be Proud is Bloomberg’s queer employee resource group. And so that was something that was a really exciting opportunity. And it was really, really hard to decide what to do. Cheryl and I had many conversations. Did you mention that you’re my team lead? But also you have a lot of experience in this world.

Rebecca Ely: And it was so hard because Be Proud was an organization that… it was the first one that I joined at Bloomberg and it really was where I felt like I sort of found my home. I was going to all these great events through Be Proud. I met people across the company, across the globe, who I just really connected with, still some of my best friends at the company.

Rebecca Ely: And so it was hard to say no to this organization that meant so much and had done so much for me personally, but after a lot of reflection with Cheryl, I came to the conclusion that my background and my sort of positioning with the allyship initiative and the connections that I already had there, and sort of the potential I saw for that community to make a big difference in the things that mattered to me was the most valuable use of my limited time.

Rebecca Ely: Because I still have to be an engineer by day, and I have a life and I like to sleep and a lot of responsibilities. And so yeah, I did ultimately say no, and I have no regrets about that. But it is really hard. And some advice that Cheryl gave me that was really valuable at that point was to turn the times when you feel like you need to say no, or you should say no into opportunities for other people. So suggesting people who you know have been really involved and or have been really interested and would like to get more involved in making it a chance for them to get that networking and show that leadership and stuff like that. So thanks, Cheryl.

Cheryl Quah: Sure. Thank you, Ely. So hopefully everyone has taken the opportunity tonight to meet new people. And thank you again for taking your night to spend it with us. If you don’t remember anything else, remember our little Hamlet moment, which is why are we doing this? Why are we doing this? And why are we doing this? So on that note, thank you, everyone.

Alexandra Dobkin: Hey guys? Is my mic on? No. okay. Oh, now my mic is on. Yeah, that did it, asking the crowd. Okay. Yeah, I like that. Second round of applause.

Audience: Yay.

Bloomberg Engineering Software Engineer Alexandra (“Dobs”) Dobkin gives a talk on how to find your dream job at Bloomberg Engineering Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Alexandra Dobkin: Yeah. Yeah. I love this crowd. You guys are great. I’m going to take you guys with me everywhere. I’ll be like yeah, follow the sequins. Yeah. All right. So as you can see, it says Alexandra Dobkin. That is my real name. As many of you might know, I go by Dobs, I will respond to Alexandra, I promise. But feel free to call me Dobs. So today I’ll be talking about finding your dream company.

Alexandra Dobkin: So what I want to do is go over the 10 questions to ask every future employer so you can figure out is this going to be the right company for me? So let’s go through a little history lesson. So for those of you that haven’t met me, I’m…?

Audience: Dobs.

Alexandra Dobkin: There we go. Yeah. I’m a software engineer working on Python API and the BQuant team. And if you’d like to know more about what that means, come talk to me after I’ll be the one in the sequins. So in case you can’t tell or can’t guess, I’ve been having an awesome experience here at Bloomberg. So quick show of hands or shout outs if you’re really excited, who’s been having an awesome time at their jobs?

Audience: Yay.

Alexandra Dobkin: Okay, so a lot of people. So seems like you guys have kind of figured out the secret sauce as have I, that… how to figure out what’s going on? I feel like a lot of people at Bloomberg just raised their hands. Yeah, okay. Yeah. So what’s it that’s giving me such an awesome experience? Part of it is the work that I’m paid to do that I find exciting.

Alexandra Dobkin: But that’s not everything. It’s how I’m treated, the attitudes that my coworkers have, the capacity for me to grow and progress in my career. I learned to appreciate my time here because, well, frankly, my previous work experiences were not the right fit for me. I used to work in management consulting, as well as finance, which had a vastly different culture to tech and especially a different culture from that at Bloomberg.

Alexandra Dobkin: So while programming is definitely cooler than these jobs, Bloomberg has definitely been a much better employer for me, as well. And an example of how Bloomberg has been better is this is what I wore to work today. I could get away with that in my previous careers. Obviously, that’s a problem. So I’ve been thinking about this, what’s been the difference between my previous employment that wasn’t the right fit, and my current employment, which is awesome? Aside from the sequins? So I’ve distilled my experience down to 10 facets that I realized I care about.

Alexandra Dobkin: I’ve talked to others about my findings, and they seem to agree. Let’s start talking about what are my 10 questions? So the first question I’d want to talk about is customer service. And the question is that you can ask is, how does the company treat its customers? So what is the customer? Who are Bloomberg customers? Can you take a guess?

Alexandra Dobkin: Yeah, okay, okay. So who’s a customer of our IT department? Yeah. Or of our HR department?

Audience member: Everyone.

Alexandra Dobkin: Yeah. So click, click. So how a company treats its customers is as important because it’s an indicator of how you will likely be treated at the company too. the ingrained attitude towards customer service translates into how you’re treated by much of the company. I know that Bloomberg prides itself in a first in class customer service experience. While that sounds great as an actual customer, paying customer, that’s really meaningful to me. I’m not a paying customer. I’m getting paid by Bloomberg sort of it.

Alexandra Dobkin: So often, and especially in larger companies, many team’s clients are actually internal. So the attitudes surrounding customer service will directly affect your interactions with your colleagues. So if a company does not treat its paying customers well, how can you expect them to treat their employees well?

Alexandra Dobkin: Yeah. So now let’s talk about philanthropy. So this begs the question, what is the company’s commitment to its community? How a company serves its community and the world at large is important because it is an indicator of its commitment to being kind. So moreover, people like to work for companies with similar values to their own. So if a person, say me cares about philanthropy will be more excited to apply to a company that promotes philanthropy. Pretty simple, right? Yeah.

Alexandra Dobkin: You guys are all smart here. So it’s pretty simple. But let’s take it one step further. There’s another reason why I care about working for a company that prioritizes philanthropy. It also draws other people to work that share those same altruistic values. And what I found is that people with altruistic values tend to be really nice, kind people. So in my professional opinion, it’s really nice to work with nice people. You can quote me on that.

Alexandra Dobkin: So a company that cares about philanthropy can lead to really kind coworkers. Love you guys. Okay, all right, health. What is the company’s commitment to health and wellness? An employee is an asset to a company and should be treated as such. How a company demonstrates its care for you beyond how it compensates you affects your quality of life. Because life happens.

Alexandra Dobkin: If you want a company that cares not only about your health care policy, but your overall health too. And it’s really important to know the difference between what perks are listed in your benefits package versus the culture around taking advantage of these perks. So raise your hand if you’ve ever heard a story of someone taking a three week vacation at a company that offered unlimited vacation, they come back and they’re canned. Oh, yeah, we got a few hands. Yeah, yeah, that kind of happens.

Alexandra Dobkin: While what is on paper can look attractive, it is not uncommon for there to be retaliation at companies for enjoying benefits, such as unlimited paid time off or taking a much needed unlimited sick days. Companies that talk the talk need to also walk the walk. It is crucial to know the benefits package is not only great but what you’re being offered on paper you’re actually truly entitled to in your experiences. So make sure you talk to employees, get anecdotes about people using benefits consequence free.

Alexandra Dobkin: I don’t have time for it, but oh boy, do I have an anecdote about how I have really, really appreciated having unlimited sick days and having a company that really cares about my wellness, calling to make sure that I’m feeling better and saying do not come back until you do. Diversity and inclusion. What would this talk be if I didn’t talk about diversity and inclusion, right? So hopefully this is an easy one that we can all agree on.

Alexandra Dobkin: Clap if diversity is important to you. Yeah. Okay. Love that sound. So good. So, I will blow throough this one quickly, because I’m pretty sure we’re all on the same page. How a company treats its under represented employees matters for all, not only for members of that community. There are definitely challenges that underrepresented groups face, microaggressions, biases, marginalization, exclusion, disrespect, inequality. I’m sure you guys can name a lot more. But a company that supports hiring diverse employees invariably supports diversity of thought. And this is a benefit for everyone, from minorities, non-minorities, to the company as a whole, is it allows for a more inclusive culture that welcomes different ideas.

Alexandra Dobkin: Diversity and inclusion makes… supports making workplaces a safe space to be yourself, whether you’re identify as minority group or not. Freedom from conformity allows you to bring your best self to work. In my case of sequins. All right, moving on. So let’s talk about culture. So when we think about culture, how many of you have heard the phrase, “work hard play hard”? Yeah. What’s your company like? Oh, yes. Some useless…

Alexandra Dobkin: So that’s the absolute worst way to define company culture. Because it really tells you nothing. Let’s put up a better quote. Okay, that’s better. So how do you define a company’s culture? Because culture is hard to talk about. It’s really big. Its leadership, it’s the seasoned employees. It’s the new hires, it’s the initiatives, it’s the goals, it’s attitude, it’s the customer service, it’s the attitudes towards philanthropy, the investment in health, the promotion of diversity. So everything that we just went over goes into it.

Alexandra Dobkin: Work should not be your life, but how you’re treated daily will affect your life. So take care to find a place that shares your values, will treat you how you want to be treated and have realistic expectations of how you should balance life and work. And I find this question, what are some examples that illustrate company culture really important? Because if you ask someone to give anecdotes, to give stories about, the brown bag lunches on Tuesdays, and how someone found their mentor, it’s a lot more telling than someone just listing the mission statement of the company or the values that the company subscribes to.

Alexandra Dobkin: All right, so this is something that we heard mentioned before, impact. So what’s impact? What’s an impactful role? And that means something different for everyone. So it’s important to figure out what does it mean to the company and what does it mean to you and where are those two relating. So, for example, when I was in finance, I was managing a billion-dollar portfolio that I was in charge of. I executed trades against it, made all investment decisions. Now does managing a billion-dollar portfolio sound impactful to you?

Audience member: Yeah.

Alexandra Dobkin: It wasn’t impactful at all to me. I was extremely bored. It wasn’t analytical. I was done with my job like the first 10 minutes… the first hour of the day and then I spent the rest of the day just, on BuzzFeed, I did not feel like I was making an impact at all. So, the impact that your job makes emanates from the challenges you face that becomes learning opportunities. Just because the company’s making waves in an industry, it does not necessarily mean that your job will be exciting.

Alexandra Dobkin: However, the converse is also true. You can be at a company making a splash and have a super thrilling job. So figure out how you define impact, what you want to achieve on a job. Does it mean working with large sums of money, like a billion dollars, affecting thousands of customers, maybe. Working with cutting edge technologies. Whatever you need on the job to feel like you are making an impact should be aligned with how the company representatives answer this question.

Alexandra Dobkin: All right. Let’s move along. Okay, feedback. So you definitely want to ask about feedback because the only way to know how you are performing and how you can improve is if it’s communicated to you via feedback. So most companies have a formal annual review process, pretty standard to find that. While that’s good, it’s not the most effective feedback sessions because frequency is a key part of an effective feedback loop. In order to have full transparency into your performance, it is the informal feedback you accrue throughout your day to day performance that will ultimately help you grow the most in the year.

Alexandra Dobkin: It’s important that how your work is perceived by your team and your management because that will become your performance review, affect your pay, I like to get paid, and ultimately your future opportunities. You and you alone are responsible for your professional development. Part of that responsibility means knowing how you are doing and having a plan for where you’re headed. You should have full insight into both. The way to get that is through quality and timely feedback.

Alexandra Dobkin: So just a recommendation, I like to have bi weekly check ins with my manager to make sure I know how I’m doing. All right, let’s talk about tools and technologies. So the tools offered to help you perform your job will directly impact your quality of life at work, especially if you’re in tech. Efficient tools and automated processes allow you to spend more time doing your work and less time doing manual processes, which I personally find very boring. Moreover, staying up to date with industry leading and current technologies gives you more transferable skills and will make you more competitive as an applicant for your next role.

Alexandra Dobkin: It is important that where you work positions you for success by maximizing your time spent doing the work and minimizes the time spent doing manual processes. Especially as a software engineer, where automating things is our passion and manual stuff is just the worst. I’m preaching to the choir here though, right? Yeah. So optimal work environments are a moving target. So companies need to prove to you that they’re aware of this and constantly striving for a best in class work experience.

Alexandra Dobkin: All right. Trainings. How a company trains its workforce demonstrates its investment in people. Quality trainings improves workplace learning and workforce effectiveness. It also builds your repertoire of skills, which make you more of an asset to the company and sets you up for success beyond the current role. A company’s investment in your professional growth and development makes you a more valuable employee. I value learning and growing my career, don’t you? Yeah. Then a great hallmark of your learning potential is measured by the number and quality of trainings a company offers.

Alexandra Dobkin: All right. Number 10, career potential. When evaluating a position it is important to assess the job as a building block to your career. A job should open doors for you and give you access to more opportunities at your own company, as well as externally. If a company is offering you a job but you but cannot see how your career will progress there, you’re looking at a dead end. To have a career at a company, you need to see other opportunities for professional development now, as well as in the future. So just to be clear, you don’t need to have a whole 10-year-plan mapped out. You don’t need to go like overboard with that.

Alexandra Dobkin: You just need to be able to have evidence that you’ll be progressing in your career. Even if you have no clue what your next step is. If you’re not going to retire anytime soon, then you want to make sure that the job will open doors for your career. So just to recap, 10 questions. One, oh the animation’s still working. There we go. That’s what’s up.

Alexandra Dobkin: So in everything that we covered, so one through six, we talked about customers, community, health and wellness, diversity and inclusion, culture, impact. Seven feedback, tools and technologies, trainings, and career path options. And then just as an aside, talking about the 11th question or the 12th, and 13th, and 14th, and how you’re going to carry on the rest of your conversations. When I was reflecting on my own experiences, and coming up with these own questions, a friend actually recommended a site to me. I don’t know if you guys have heard of

Alexandra Dobkin: Yes, no, maybe so, okay. Really cool site. And if you go /culturequeries, they actually have a lot of really great questions and kind of ask you questions to help you figure out the questions you should be asking. I personally feel it’s a really valuable experience to come up with your own questions based on analyzing what you value, but definitely check out the site for some inspiration. So with that, thank you. Yeah. All right. And one last note.

Alexandra Dobkin: So just as a final note, I just wanted to say, I’m so excited that Bloomberg is hosting a Girl Geek Dinner, not to take all the credit, but I totally came up with the idea and proposed it.

Audience Member: It’s true.

Alexandra Dobkin: It’s true, it’s true. But only because I personally attended a number of Girl Geek Dinners and I really thought the experience was so awesome and so amazing. For me, I’ll share that at the height I was going to have my early dinners… The height of my Girl Geek Dinners attendance was when I was job searching. I don’t know if you guys are job searching? For me, my whole tactic was I’ll go, I’ll network, obviously, eat the good food. I’ll network and I wanted to make sure I had a really solid conversation with at least one person, it didn’t have to be more than one, but a really good solid conversation.

Alexandra Dobkin: Got that business card and I got a first round interview, if not further, with every single Girl Geek dinner company that I attended. So I just want to say make the most out of tonight. Eat the food, it’s really awesome, and feel free to come talk to anyone, blue shirt, sparkles, whatever it is. So thank you.

Audience Member: Yay.

Narrator: What impact does extreme weather have on oil production in the North Sea? How is the one peso tax helping save an entire generation of children? If 70% of everything we buy is delivered by truck, what happens to your grocery bill when there’s a severe driver shortage? How can bread scarcity spark a global political revolution? Our planet is alive and interconnected, continually shifting, adapting, and growing. Every event bigger or small results in other events.

Narrator: At Bloomberg, you’ll investigate, examine, and interpret these unique and seemingly unrelated connection points in real time. The success of our business relies on people just like you… Who can look into the future and create groundbreaking technology… Research… And expert insight to answer the world’s most complex questions. When we solve problems with a greater sense of purpose… Change begins… Dots connect… Society excels…

Narrator: The world transforms when work has meaning. Your career thrives when you feel a deep connection to it. That’s why at Bloomberg, we work on purpose. Ready to find yours?

Mario Cadete: Great. Thank you everybody. Thanks speakers. I couldn’t have said it better myself. Not even close. Thanks to my team. Thanks to Girl Geek. Again, thanks to Bailey. Please come talk to us. I think we’re here till 8:30. Have some more food, drink and so on. It really has been a pleasure. Hopefully, you come and speak to me. I’d love to meet as many of you as possible. Thanks again.

Thank you for coming out to Bloomberg Engineering Girl Geek Dinner with VR and Terminal demos, talks and networking!  Erica Kawamoto Hsu / Girl Geek X

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X + Indeed Lightning Talks and Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang, Allison Dingler

Girl Geek X founder Angie Chang and Indeed Global Diversity & Inclusion Program Manager Allison Dingler from Austin, Texas kick off an Indeed Girl Geek Dinner in San Francisco, California.  Erica Kawamoto Hsu / Girl Geek X

Transcript of Indeed Girl Geek Dinner – Lightning Talks & Panel:

Angie Chang: Hello, and thank you all for coming out to Indeed Girl Geek dinner tonight. My name is Angie Chang and I’m the founder of Girl Geek X. We’ve been doing these Girl Geek Dinners at companies in the San Francisco Bay area now for a very long time, but I’m really glad that you’re here tonight for Indeed’s second Girl Geek Dinner.

Angie Chang: I’m really excited for all the talks that we’re going to hear tonight and please enjoy yourselves and meet someone new. At least one or two, maybe even three new people, get their LinkedIn, exchange LinkedIn information and maybe grab coffee later, ask about jobs, ask about jobs here. There’s always opportunities to level up and that’s why we keep doing this is because we like learning and hearing from other women in tech and other industries about things they learned on the way, and also what are the cool things they’re doing.

Angie Chang: So, please feel free to network afterwards. We also have things like a Girl Geek Podcast, in case you would like to find us on iTunes and all of the different podcasting services, we have a podcast. We also have a conference coming up. It’s a virtual conference we do every year for International Women’s Day. That’s going to be in March, so stay tuned. But I want to turn the microphone over to Alison, but say thank you so much for hosting us, Indeed.

Allison Dingler: Thank you. Thanks, Angie. All right, awesome. Thanks so much, everybody, for coming. This room is so packed, I love it. Yes, everybody excited? Yes. Ooh, energy. I’m about it. I’ve had a lot of caffeine today. I’m going to kick it off and start with our first tech talk of the evening.

Lindsay Brothers

Product Manager Lindsay Brothers gives a talk on “A/B Testing Pitfalls and Lessons Learned” at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Lindsay Brothers: Hello. Beautiful. Hi, how are you doing? Good. It’s Tuesday. Okay, so I’m going to be talking about “A/B Testing Pitfalls and Lessons Learned”. Experimentation. It’s how we learn about the world around us. It’s something we started doing very early on as humans. It’s something we start as babies. How do we learn as babies? Well, we run the original A/B test, which is stick things into our mouths. And we learn, maybe we have a question, a hypothesis, is this edible? I can eat this.

Lindsay Brothers: Unfortunately, sometimes not so successful. It turns out to be dirt, and that test was not a success. Some tests fail. And other times when babies run this experiment, this A/B test, congratulations they’re tacos. Yay, it was a successful test. And we get to celebrate, we’ve learned something new and we ate tacos.

Lindsay Brothers: I’m Lindsay Brothers, I’m a product manager at Indeed. You can follow me on Twitter @LindsayBro. You may know Indeed, number one job search site worldwide. Who here has heard of Indeed? Everyone raise your hands. And who’s gotten a job through Indeed. Yes. That’s awesome. I love it. Yeah.

Lindsay Brothers: Okay, a little bit about Indeed so you have context for this talk. We help people get jobs. This is our mission. This is something we cared deeply about. And some context about just how big we are. We have 250 million unique visitors, 150 million resumes, 600 million salaries, and 25 million jobs. So, a lot of job seekers looking at a lot of jobs, millions of job seekers, millions of jobs.

Lindsay Brothers: We’re a very data driven company. We want to learn about job seekers and how do we do this? Well, A/B testing. What is A/B testing? A/B testing is a randomized experiment in which a new variant is tested against a control A to measure how they perform relative to each other. I just realized I don’t have a timer. Cool. So at any given time, we’re running hundreds of A/B tests to learn about job seekers, which leads to thousands of experiences. We’re running hundreds of A/B tests, which leads to thousands of experiences at any time.

Lindsay Brothers: Which means the person sitting next to you likely sees a very different Indeed than you do. This leads to many different lessons learned. We’re constantly running A/B tests and we’re constantly learning. Now this test, this talk is really about things that have gone not so well, pitfalls. We run a lot of A/B tests and we’ve made some very expensive, painful mistakes that I want to share with you.

Lindsay Brothers: The first pitfall, your metrics don’t matter. Second pitfall, big test, big failure. Pitfall three, most tests fail. And pitfall four, where does vision fit in? As we go through these pitfalls, I’ll share lessons learned. Let’s dive into the first pitfall.

Lindsay Brothers: Your metrics don’t matter. Or in this case my metrics didn’t matter and no one cared. This is a job alert, and you’ll hear more about Job Alerts in the next talk, as well. But this is the most common email we send out. It’s new jobs based on a query that a user has run. In this case, it’s a email developer jobs in San Francisco. Done a search on Indeed, signed up for this Job Alert, and they’re getting new jobs in their inbox, so they can apply to them as soon as they’re posted.

Lindsay Brothers: I was on this team as a product manager on this team and I was highly motivated to get more subscriptions, to get more people to sign up for these Job Alerts. This is an Indeed apply form. If a job is posted on Indeed, you go to apply for that job, you’ll see this form and you’ll apply directly on Indeed. What I wanted to test was adding a simple checkbox. Notify me when similar jobs are available. A job seeker will just check this check box, and the next day begin to receive new jobs in their email inbox. Pretty simple, right?

Lindsay Brothers: A versus B, no checkbox, that’s standard form, versus the test group with that checkbox. Now, it looks simple. It’s just a small change. But this test had a tremendous amount of data. Sure, it’s just one variable in a single test group, but look at all the things that are on this form. You can be logged in, you can be logged out, you can use your Indeed resume, you can attach your resume, you can attach a cover letter, you can add a phone number. So there’s all these different things you could do, just in a single form. So we had a lot of data to analyze.

Lindsay Brothers: And the first metric was really bad. We saw 0.1% decrease in applies. So just adding that single checkbox, less job seekers were going to fill out that form and apply to that job. And this was really scary. Not good. So there’s another team that exists, is not just the Job Alerts team. There’s a team called the Indeed Apply team and they own this form, and their metric for success was the completion rate of this form. They wanted job seekers to apply to jobs on Indeed and to finish this form.

Lindsay Brothers: So, 0.1% may not sound like a lot, but we’re talking about millions of job seekers, millions of jobs. That actually can equal hundreds of thousands of applies lost by adding a single checkbox. So this team says to me, “What are you doing? Okay, we’re here to help people get jobs, not email subscriptions.” Yeah, I got some angry emails about that. This was our secondary metric. This was a metric I was keeping an eye on during the analysis of the tests, but it was their primary metric. It was what they cared most about, and they didn’t really care that I was getting more email subscriptions. I was getting a lot. Come on, I can put this on my eval. 50% increase in subscriptions. That looks really good, but they don’t care.

Lindsay Brothers: It felt like a massive win but we were losing applies. So, the question was, are we helping people get jobs or are we just helping them get email subscriptions? So, what is the impact of this test? Now keep in mind, they click that checkbox, they’re getting that job alert, they’re getting that new subscription, and there’s a lot of jobs in here. So, they got to be doing something with those jobs.

Lindsay Brothers: Well, let’s start looking at the standard email engagement metrics. They’re getting this new subscription, how are they engaging with it? Well, okay, they’re less likely to open it. That’s awesome. So, job seekers were checking this check box, maybe less likely to apply and less likely to open this email. And they were less likely to click through by 12%, so open rate was 12% lower, click through rate was 12% lower. Am I tricking people into this email? Is this spam? What am I doing?

Lindsay Brothers: But we had to look further down the funnel and what we found was the apply rate was actually higher. So, job seekers were less likely to open it, they’re less likely to click through, but they were more likely to apply to a job in that email by 0.25%, and this was very, very exciting. This meant that we were actually helping people get jobs. But we had to figure out the total impact. So had to do some math and we had to figure out the total downstream applies, more subscriptions and looking at that higher apply rate, how many additional applies were we getting? Was it making up for those lost applies? And it was, we were getting millions of additional applies from that email subscription. So, it was a success. Yeah, it looked real good on that eval.

Lindsay Brothers: Something to keep in mind is that looking at only short term metrics, that makes [inaudible]. Looking at only short term metrics can mean missing downstream impact. And they really didn’t care about getting additional email subscriptions, which of course they shouldn’t. But we were really aligned on the impact of applies and the power of applies. So, the first lesson I’d like to share is that downstream analysis is a powerful tool, but it does take time. It took time to figure out that we were having a higher apply rate in that email and we were getting those downstream applies. Cool.

Lindsay Brothers: So, second pitfall, big test, big failure. In 2017 this is what looked like. And we were due for a facelift. The UX design team really wanted to update this and improve it. This was the vision. This is where we wanted to go. We wanted to modernize Indeed. Again, this is 2017.

Lindsay Brothers: Generally, when you’re doing an A/B test, you have a change and you have a result. You’re changing something on the product, a small variable change, then you see results. Well, okay, now keep in mind this is the entire search results page of Indeed. There’s a lot of things going on, and it was a really, really massive test. We were looking at, we were testing this old version versus this new version, and lots and lots of changes. Okay, lots and lots of changes going on on the site and then more changes we’re starting to see lots of metrics, and then there was lots of results, and more results, and more results and we couldn’t quite figure out what was going on and what changes were causing what metric going up or down. And it got really messy. It looked like that.

Lindsay Brothers: We had changed too many things at once. We wanted to run this massive A/B test where we were updating the old Indeed with this new, beautiful, massive, redesigned, gorgeous. Let’s skip ahead. Let’s go with this big vision. But we had changed too many things at once. And that meant our A/B tests were losing, metrics were going down, but we didn’t know why. And this is really expensive. Now, keep in mind, this is redoing the search on Indeed, that involves a lot of engineering effort.

Lindsay Brothers: We had to go back to the drawing board. 2017, we wanted to test this brand new, beautiful redesign, let’s modernize Indeed. So let’s start out where we started. We had understood when we were doing this test that too many changes at once meant we didn’t really understand what was happening. So we had to start from scratch. We had to really just redo this whole thing. What we had to do was test a single element at a time.

Lindsay Brothers: Now, this is a job card, so you do a search on Indeed, you’re going to see these job cards, which is job title, company location, maybe salary, some additional details. And this is an example of how we could test a single element at a time. You’re like, what’s changing here? It’s the spacing. So this is spacing as a single element. So a single variable in A/B test.

Lindsay Brothers: Another thing we had to do, like I had mentioned, we have this old older design that we wanted to update and originally it was just, let’s test all these elements at once. Something we also had to do was we had to switch to multi-variate tests. And so I say multivariate test, what do I mean? An A/B test in which all possible combinations of variations are tested at the same time.

Lindsay Brothers: Now let’s go back to that job card. We wanted to test a single element at a time. We want to break up all those elements so we can understand their impact. But we also want to A/B… the multivariate testing so we can go through all these different combinations. Now this is a job card. We got job title, company, location. There’s all these different elements to test. Well, let’s dive deep into a single element, which is salary. Let’s look at salary. Salary important. We like to make money at our job.

Lindsay Brothers: This is a single test where we’re just testing the element of salary, but what we’re going to do is a multivariate test to really dive deep into the UI of it. This is control, font size, 13.33 pixels. It’s not bolded and the color is gray. So, one element of the multivariate test is the font size variance, 12 pixels, 13.33, 14, and 16 pixels. Another part of the multivariate test is font weight, regular versus bolded, and finally color: gray, black, orange, and green.

Lindsay Brothers: That’s a lot. So we got four sizes, two weights, four colors. This is 32 groups. And were there spreadsheets? You know there were spreadsheets. I love spreadsheets. So, you’re like, whoa, that’s a lot going on there. Now, if we had not done a multivariate test, if we’ve just done A/B tests, so color as a single A/B test, font weight as a single A/B test, size as single A/B test, it would… like, here’s the control. Okay, that’s control. These would just be the groups.

Lindsay Brothers: So colors, one group, size, one group, you would only have eight different groups. But when you do multivariate testing, you get, you miss 24. So, 32 groups, well that’s a lot. But you get to explore how these elements play against each other. And we would have lost our winners. So if we had only done tests around those single UI elements one at a time, we would have totally missed these. And we would have not picked these. No UX designer would have picked these, because this one looks like Hulk. We call this one Hulk. It’s big. It’s green. It’s bold. No one was going to go with that. But we learned from this and we learned a lot.

Lindsay Brothers: Now, multivariate tests, of course, have some challenges. Well, you need sufficient traffic. I mentioned 32 different groups. Okay, you need enough traffic to learn anything from those groups. Also, significantly more complex analysis. So there’s a really good blog post. Robyn Rap is a data scientist on the Indeed engineering blog. You can look it up. And she actually talks about this specific test, and there’s literally equations in the blog posts about how to analyze it. It’s a lot of math. There’s some really strange combinations that you can get from multivariate tests, like the Hulk. No one was expecting that. We’re like, “All right, what do we do with that?” Okay.

Lindsay Brothers: But it really helps you optimize your UI at a very high velocity, which is really, really cool. We’re learning fast, we’re moving fast. And since August of 2018, we’ve run over 50 tests with over 500 groups, just on that search results page alone, and we’ve learned some really surprising things. Now remember when we started this test, it was old design, new design. All these changes and we couldn’t see what was going on. Metrics going up, metrics going down, not understanding the impact. But we saw really surprising things when we broke it up via by element, and then broke it up into multivariate tests so we could really learn quickly.

Lindsay Brothers: And something we totally missed was that changes were way more impactful on mobile. This data point lost, totally lost in that confusing analysis where everything was going up and down. But we had missed this, so when we broke up the elements, we learned more about the UI changes and their impact.

Lindsay Brothers: So, the lesson here, again, this was a very, very, very expensive mistake to make. We thought we could just skip ahead, come up with the new design. Let’s go there. Oh, we’re so modern. Wrong. Very, very expensive. So when we changed to testing small changes and testing them quickly, we learned a lot more, faster. Which goes directly into the next pitfall, which is most tests fail. And at Indeed, 70% of A/B tests fail. That means there’s not a clear winner. We know we put out this test group, it’s not amazing. It’s not going up. Whatever your metric is, it looks bad.

Lindsay Brothers: It’s really sad. It’s like, what’s the point of this? Why am I here? Why am I running all these tests if they all fail? Why did I make my engineers build this or build this test if it just… It’s just sad. Want to hide under a blanket. There are three different reasons, really, to run A/B tests. We can get faster wins, we can get a better design and we can get a better understanding of the product. So how exactly do we get to these things?

Lindsay Brothers: Well, let’s say we run a test and the test is positive. Your metric goes up. Well, awesome, you have a KPI win. Woo hoo. That’s great when that happens. Or okay, let’s say the test is neutral. It did not change. And this was actually quite common in that test I just discussed, where we were testing all these around elements, all many, many multivariate tests, spreadsheets galore, lots of neutral stuff. That actually means there’s a lot more design flexibility. So if it’s neutral, it means you can play with that element. You can maybe make it a little more prominent or you can do things with it. It’s not a fixed element. It’s more, you can be creative with it. And so that’s actually really exciting.

Lindsay Brothers: And, of course, if something’s negative, what we found is, okay, don’t touch it. Do not touch that element. Like I said, we broke up all these different elements. We’re testing many different variations of these elements, and some things came back consistently negative, and we’re like job seekers like that, we’re not going to do anything with it. And so you’re still winning. Even if a test is neutral, you learn that you can do things with that element. If it’s negative, don’t touch that element. Move on, try something new.

Lindsay Brothers: So, it is successful. Sometimes it’s disappointing when it doesn’t go the way you planned or your hypothesis is totally off, but we do get wins from this. And the lesson here is to change what winning means. So, even when we’re running all these different A/B tests, the metrics aren’t looking great. You’re still learning about your users, are still learning about, for us we’re learning about job seekers and how they interact with Indeed and how they look for jobs.

Lindsay Brothers: So that goes directly into pitfall four, where does vision fit into all of this? We have this big vision. Okay, so let’s go back to that test. This is where we were. So this is again that job card, you do a search on Indeed. You see all these jobs. This was the first iteration after testing. There’s a lot of stuff going on there. And this was the vision. We like to empower design at Indeed. We love to have visions. We love to think about where are we going with things. But when you’re doing a lot of A/B testing and, where does this all fit in? I’m running all these different, I’m testing all these elements. I’m trying to learn. I have these hypotheses. But where does the vision fit into this?

Lindsay Brothers: Well, even failures can inform design vision. Let’s go back to… Now which one is this? Okay, so this is the vision. This is the vision we originally started with, 2017. This is our vision. This is where we wanted to go in 2017. And we started to remember that first big test, not so good. Then we started to break up elements, did multivariate testing, lots of spreadsheets, and we started learning. We got lots of negative and neutral and positive, and we learned things like no blue or underlined needed on the job title. Doesn’t matter if you have that, job seekers are still going to click on the job.

Lindsay Brothers: We can add more spacing. Like I mentioned, spacing was a single A/B test where we’re looking at that element and playing with spacing. We can add more white space. We can do more there. Salary needs to be more prominent. Like I talked about that salary test, we could make it big and green and bold and people love it, they’re clicking through. Do not touch this. If you can apply on Indeed, that Indeed apply little tag, if we changed it, negative. Any change, negative. Do not touch it, keep it there. Font size. We played with font size, of course, the font size was way too small.

Lindsay Brothers: So again, this is where we started after that first iteration, that first iteration of testing, this was our vision, but then we ran all these tests, testing all these different elements, many variations, and we ended up here. Now, you can see salary’s a lot more prominent. There’s more spacing. We don’t need that underline. We played with underline, you don’t need that. So, as we learned about these different elements, tests were negative, tests were positive, tests were a lot were neutral, we were able to take these learnings and put it back into the vision. So we had a vision that more aligned with how job seekers were using our site.

Lindsay Brothers: Sometimes it can feel like a vision is just this beautiful thing you create and it doesn’t always align with how people use your products. And with A/B testing and with learning from many, in fact, failed tests, we were able to take that back and have a vision that aligned with how job seekers use Indeed. So it’s not just a design win, it’s also a business win. We had a lot of KPI wins, many tests not so great, but we did have ones that were delivering KPI wins and we’re able to implement that back into the vision.

Lindsay Brothers: So the lesson here is both, of course, the design vision guides your testing. We had UX designers who were thinking about where Indeed should be, where we could go, and that led us to define some of these tests. But also those A/B tests, as we’re running them, went back into the vision. So, to recap, your metrics don’t matter. Remember we ran this amazing checkbox test and it looked horrible. At first, applies were going down on that page. So, we had to align our metrics, we had to agree on those downstream applies. And downstream impact can take longer. It can take much longer to pan out, but it can prove really valuable.

Lindsay Brothers: Next up, big tests, big failures. That’s when we ran this massive redesign of the Indeed search, and we thought we could just skip ahead from this old design to this new beautiful design. We were wrong. That was big mistake. So, breaking up tests really helps understand impact. So breaking up these tests into different elements, multivariate tests, those multivariate tests really help us understand the different UI elements. And most tests fail. So indeed 70% of tests fail and that meant that we had to redefine winning.

Lindsay Brothers: So, failed test just means there’s lack of flexibility. That element is important to users. They care about it, maybe don’t change it. And a neutral test means that there is flexibility. So, if something’s neutral, maybe we can do more with it, maybe we can play with it. And then finally, where does vision fit in? So, that big redesign test, we had a vision of where we wanted to go. We’d tried testing it all by itself, didn’t work, but we were able to use it to guide those single element tests, the multivariate test, and it helped us define a test plan. But also as we ran those tests, it helped us inform the vision and we adapted our vision to what made more sense for our users and our job seekers.

Lindsay Brothers: So my question for you is, where will your testing failures take you? If you run A/B tests, if you run tests with your users, they will fail and things will go wrong, and you will run tests and will feel like an engineering waste of time. Oh boy. But there’s still learnings to be had, so where will those failures take you? Thank you so much for your time. Please clap. Here’s my information, so feel free to shoot me an email, or Twitter, @LindsayBro. If you tweet, I get metrics. Awesome. Thank you so much. So Allie is going to give a little intro.

Allison Dingler: Thanks, Lindsey. Everyone give it up one more time for Lindsay, A/B testing. Yes, I’m here for it. Awesome. Everyone having a good time so far. Getting some good food in your belly, some good drinks, some good friends, yes? Awesome. So I’m going to kick it off. We have our next tech talk that’s about to get started. We have Janie and Rohan here, so I’ll have y’all come on up and get you going. Do you want another microphone?

Janie Clarke: Yeah.

Allison Dingler: Microphone.

Janie Clarke: Thank you.

Rohan Kapoor: [inaudible].

Janie Clarke: Can you get a timer?

Allison Dingler: Timer.

Janie Clarke speaking

Senior Product Manager Janie Clarke gives a talk on “AMP for Email” at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Janie Clarke: Hi everyone. My name is Janie Clark and I’m a product manager here at Indeed. I’m here today with Rohan Kapoor who’s a software engineer on my team, and we’re here to talk about our adventures with AMP for Email. Did you click? All right, there’s a little bit of content here that might be repeated from Lindsay’s talk. As you probably just learned, we are very data driven company here. We test a lot of things. And Rohan and I work on Job Alerts, which Lindsay also used to work on.

Janie Clarke: It’s an important product for Indeed. I assure you it’s not the only product, even though we’re talking about a lot tonight. Job Alerts is a big product. We have over 250 million active Job Alerts subscriptions. We send it to over 60 countries in 28 different languages. And we do a lot of A/B testing within this email. At any given time we have dozens of A/B tests running.

Janie Clarke: You’ve also seen this before. This is a Job Alert email. Basically it’s a way for job seekers to get the newest jobs emailed to them. We’re here today to talk about our adventures in AMP. So first, I’ll give a quick background about traditional email development and some of the challenges that we had faced with it, and we were hoping AMP would solve these challenges for us. And then Rohan will go into some technical considerations and obstacles that we faced and how we solved them. And finally, we’ll share some of the results of our testing so far.

Janie Clarke: So first up, traditional email development, or why email development sucks. Do we have any email developers in the room or anyone who has worked on it before? Woo! Email geeks. So you may have heard, email development can be difficult. Because emails are sent out as static HTML content, it has to be able to support whatever old arcane email client that your users may be using to read their email. Outlook 2007, being a good example of this.

Janie Clarke: Many different email clients have their own special rules about what kind of content and markup they require and can render. And if your inspect the source of an email, that in your inbox, you may notice that it’s a mess. Lots of nested tables, inline CSS with special rules for different Outlook versions. Again, some of you may have noticed this before. But Outlook aside, even modern email clients like Gmail have issues of their own.

Janie Clarke: So remember, at Indeed our mission is to help people get jobs, and on our team we do that by sending Job Alert emails to millions of job seekers every day, so that they can get the latest results for their search. We know from extensive experimentation on the site, as you know, and also within our email, that job seekers get a lot of value from personalized contextual information about the jobs that they’re looking at. Providing them with this information helps them make better decisions about which job they want to click on. It helps them apply to the right jobs and it helps people get jobs.

Janie Clarke: But whenever we do a test in the Job Alert email to add this information, we run into a problem. And this happens. Have you seen this before in Gmail? The email clips, since traditional email cannot load any external content, no JavaScript, no external CSS files, all the content has to be static and contained within the HTML. And along with all those nested tables that you need for Outlook and custom inline CSS, for the older email clients, it can be a real challenge to squeeze in all of the information that you want.

Janie Clarke: So it’s something we’re constantly trying to find the right balance on our team. I’m including as much information and as many jobs in the email as we can, while also not running into clipping. So there’s a certain size limit that Gmail hits, when the email will clip if the HTML is over that limit. So we’ve tested adding more jobs to the Job Alert, more content always leads to more clicks, more engagement, more applies. But the more jobs we add, the more it clips. Right now, about 5% of our emails clip.

Janie Clarke: Another problem we’ve run into with Job Alerts is how to include the most Up-to-date information. I’m going to go into a little bit about sponsored jobs. A sponsored job is one where the employer is paying to promote that job and get it in front of job seekers. When we’re sending Job Alerts, we send out both sponsored and organic jobs. We will not send the job alert if there are no new organic jobs, but we do allow the sponsor jobs to be a little bit older.

Janie Clarke: And the thinking there is that, if the employer is paying to get this job promoted to more job seekers, they’re actively trying to fill that position. So, we want to help them do that by including the older job. But it’s also a challenge because job seekers can open their email hours or even days after we send it. And if a job is older, it’s more likely to be closed by the time they see it. If you look closely at this screenshot, you might notice a difference between the job at the top, which is sponsored, and the organic job below it.

Janie Clarke: In our Job Alerts, we worked around this problem in a pretty clever way. When the user opens their Job Alert, a request will be sent to our server to fetch the latest sponsor job for that slot and that alert, and a screenshot would be taken and we would render it into an image. So we call this image ads. Image ads were really clever workaround for the problem of how to show the latest sponsored jobs, but they came with their very own problems. So, every email renders HTML a little bit differently. Many of them render poorly, and different clients have handled different images in very strange ways.

Janie Clarke: So some problems that we’ve faced with this are giant sponsor jobs or tiny sponsor jobs, grainy images, you name it, we’ve run into it and fixed it. So getting image ads right was something we had struggled with on our team for a long time. And because of this, when Google first announced AMP for Email, using it to replace image ads was the first thought that we had. We are also very excited about how AMP for Email wouldn’t necessarily need to include all of the older markup that’s required by older clients because it’s only supported by a certain newer clients. Now I’ll give a little bit of background about AMP.

Janie Clarke: AMP is a web component framework that is used to help create interactive websites, stories, emails and ads. AMP is designed to create a user first experience, which they define as being mobile first and loading fast. And it’s especially helpful for users on poor quality connections because they try to load the most important content first. AMP also does not allow for content to change positions once it’s loaded. Meaning that all the page elements have to have a fixed width and height.

Janie Clarke: The reason for this is they want to avoid the page jumping around as it loads, which you’ve probably noticed a lot on the internet is the thing that happens, especially on slower connections. And AMP achieves these goals by providing a set of predefined components that you can use to build web pages. AMP for Email is a way to offer email users an interactive experience within email, by allowing certain AMP components to be used in email. It brings a lot of modern app functionality directly into emails that has been impossible before.

Janie Clarke: If you use Google Docs, you may have seen this email, which allows you to reply to a comment directly from your inbox without having to leave. It’s pretty amazing. I use it every day. It’s a killer use of the AMP functionality. Some of the benefits of AMP for Email are, it can dynamically load content from a remote server, not just images, but also text. Users can also submit forms and information to a remote server. So users can submit feedback and content to you. And lastly, the layout can change as the user interacts with it.

Janie Clarke: So in some ways this allows email developers to build an entire web application inside of an email. AMP for Email allows for a level of customization that has literally never been possible before in email. It’s very pretty exciting if you’re an email person like me. So back to our use case. Google opened up the AMP for Email preview in April of 2018, and we jumped on the opportunity to participate in the developer preview. The timing was really good because we had an intern starting for the summer in May, and at the time Google expected their launch to be around September. So it would be perfect for an intern project. He’d be able to see his functionality go live, sound really great. So we assigned him the project of creating an AMP version of the Job Alert that loads the sponsor jobs using AMP.

Janie Clarke: Now the reason we assigned it to an intern, it was hard to justify putting full time resources from the team onto this, due to the large time frame before launch and the general unpredictability. And this has been consistently a challenge when working with AMP, which we’ll go into more later. So now Rohan is going to talk about what it was like to actually work with AMP.

Rohan Kapoor: Before jumping into AMP, I wanted to take a second to talk about emails traditional development. Traditionally emails have had two MIME types, the text MIME type and an HTML MIME type. So a modern email client, such as a Gmail or Outlook on the web, that can support HTML will read HTML, and text only clients that run into terminal, something like Mutt, will read only the text part.

Rohan Kapoor: So then came AMP, which was implemented as a new third MIME type. Clients that support the AMP MIME type will read the AMP part and display that, while other clients will fall back naturally to HTML and text. So, there’s full backwards compatibility, there’s no risk that an email client will display incorrect content or garbage just because you start sending AMP.

Rohan Kapoor: As Janie mentioned, one of the biggest considerations that Google had when they built AMP was mobile first experiences. In a mobile world, it’s quite common for the user’s device to lose their connection as they’re moving around from place to place. And on the web, AMP works around this by using AMP caching and caching some components on the user’s device. This reduces the network traffic required to load pages. But, we’re talking about email. And in email, developers have to send fallback content like above, which can be used if the network request fails to return data. So in this case, this email failed to load this data, and so this data that was preloaded as fallback content shows up so that there’s not a giant white space where it should be.

Rohan Kapoor: With AMP’s emphasis on user first design and development, there’re some imperatives that we found particularly tricky to work with. For example, AMP requires that all content has an integer valued width and height. For an email system, that means that at the time you’re sending it, the system needs to know what the width and height of all of your dynamically generated content will be, even though that content doesn’t exist yet.

Rohan Kapoor: So, in our case, we fetch text like the job snippet dynamically. And it can lead to situations where the content is too long and gets truncated. See the text sponsored on the image on the slide. Or too short with extra padding on the ends. And on this slide we can see that the difference in height is pretty remarkable between the job in the middle and the one above and below it.

Rohan Kapoor: Now AMP also suffers from the clipping problem that Janie was mentioning, but it behaves very differently than the traditional HTML email. If the AMP MIME type is larger than 100 kilobytes, the email client silently drops it and falls back to the HTML MIME type. There’s also additional limitations on the size of the entire MIME tree that can cause the whole email to just disappear into the void. It’s also important to keep in mind that this is still an email. Everything is still happening inside an email client, inside the user’s browser. And UI elements such as lightboxes may not work exactly the way you expect them to.

Rohan Kapoor: So we had created mock ups for replicating an Indeed view job page inside our Job Alert email. This was a little bit of a slimmed down version, just because you know it’s running inside an email. And the idea would be that the user clicks on the job and instead of leaving the email, inside the email themselves, inside the email itself, they can view the job description. However, it didn’t quite work the way that we expected it to.

Rohan Kapoor: So the video loads, so you can see, when you click on the job at the top, you get a normal job description. But as you scroll down and then click in, it never scrolled up. The job description is up there. And if you go all the way down to the bottom and then click in, the job description is gone. But actually it’s all the way up there. Because of the way AMP content is rendered, for security reasons it all runs inside an iframe. It has no idea what the viewport is and so lightboxes don’t quite work the way we would think they would.

Rohan Kapoor: So we emailed the team at Google, filed a ticket, and a little bit of back and forth happened, and then they closed the ticket, and said that they’re going to remove AMP lightbox from the list of email approved components because they couldn’t find a way to make it work. So, at Indeed, we’ve built our own email service provider or ESP. And so our journey with AMP begun by adding support for sending the AMP MIME type through that platform. As I said earlier, the AMP MIME type, oops. The AMP MIME type is backwards compatible, so any Indeed application that doesn’t support sending AMP would have no change in behavior. But any application that is sending AMP is received by a client that supports AMP, everything will work fine.

Rohan Kapoor: We also ran into a bunch of specific challenges while working with AMP, and there’s some interesting workarounds that we wanted to share. As Janie mentioned, we started working with AMP during a developer preview period, and at the time AMP for Email was considered bleeding edge. As many of you may know, when you’re working with bleeding edge software, sometimes you have to be ready to bleed. One of the biggest challenges that we ran into was that AMP specifications changed a lot during this time, and the documentation frequently lagged behind. And here’s one such story.

Rohan Kapoor: One day a QA in our team reached out to me and he was telling me that none of our AMP emails were working anymore. And keep in mind that these were emails that were in his inbox from yesterday and worked fine yesterday, but it’s dynamic. So what worked yesterday may not work today. What happened was that where the sponsor jobs were supposed to be, there was just a large white boxes. So we opened the developer tools, looked in the error console, and it’s full of incomprehensible red error text. And all of the error text is minified, so no idea what any of it means.

Rohan Kapoor: So we reached out again to our developer contacts at Google, and they told us that, “Oh yeah, they have now enforcing Gmail-specific CORS headers. The documentation doesn’t come out yet, but it’ll be there in like a week.” And they sent us a quick and dirty version, but basically we had to now add the Gmail-specific CORS headers. That’s an example that we use, and without those all the Ajax requests to fetch the content failed.

Rohan Kapoor: So AMP initially required that the domain that it sends requests to matches the email sender domain, and at Indeed we use when we’re sending Job Alerts. However, jobs can live on a variety of different domains depending on what country they’re for. So jobs in the UK live at, for example. You’ll notice that, where the email is coming from, and where the job is hosted don’t match.

Rohan Kapoor: So, one possibility that we had was to change the sender email address to match the job domain, but this raised another potential issue for us. It could dilute the sender reputation of the domain. So, for those of you that send a lot of email, you’re probably familiar with the concept of sender reputation. Basically, all of the incoming email providers like Gmail, Outlook, Yahoo, have a score that they assign to emails coming from your domain. And switching to a new domain like would have no sender reputation and we’d lose all of the reputation we had from So this was not really something we wanted to do.

Rohan Kapoor: What did we end up doing? Well, the most straightforward solution that we could come up with was creating a proxy web app. This web app lived at an sub domain. In our case it was called, and it accepted a base 64 encoded URL that told it which domain to actually go fetch the job from. And it handled all of the AMP security validation, handled the CORS headers, and then did the request to the real domain to get the data and passed it back. We called this solution AMPXY for AMP Proxy, and it effectively allowed us to perform cross-site origin requests while masking them, so that as far as AMP and the Gmail client were concerned, we were still hitting

Rohan Kapoor: Ironically, a few weeks after we built this solution, the team at Google reached out to us and told us that based on our feedback, they were removing the same domain requirement. Again, bleeding edge software developer preview all of that. Well, fortunately there was some other benefits that AMPXY gave us, so it wasn’t all just wasted effort. AMP for Email doesn’t allow for any redirects, and if redirects are present, requests fail automatically with no error. AMPXY allows us to proxy the redirect on the server side and once again hide it from Google and make it look like everything’s fine.

Rohan Kapoor: Second, QA testing. So, since AMPXY exposes a single endpoint, we can put that through the QA firewall and have a very simple, straightforward way to test in QA rather than having to open up a bunch of different end points for various functionality. But there’s one significant downside: URL lengths. Because we’re using a base 64 encoding, your URLs ended up almost twice as long as before they started, which makes email clipping much, much worse. In the future we’re planning on building a URL shortening system, which we will integrate into AMPXY, which will hopefully allow us to have shortened URLs and use them in AMP and everything will hopefully work, but it hasn’t been built yet.

Rohan Kapoor: Unlike regular HTML, email AMP does have strict validation requirements. If AMP content validates, it shows up. If it doesn’t validate, you get HTML content instead. And this doesn’t actually show up in any way that you can see. So, one such scenario here is that the AMP spec doesn’t allow you to import functions and then not use them. We use AMP list components for sponsored jobs, and every time we have a sponsor job, it goes in its own unique AMP list.

Rohan Kapoor: So if we have no sponsor jobs at the time, we’re rendering an email, then we don’t use any of our AMP list components, but the import was still there. We learned the hard way that we had to remove the import, and so now it’s wrapped in a nice if-statement to make sure that we don’t fail validation that way.

Rohan Kapoor: As I mentioned, when emails fail validation, there’s no reporting back from the Gmail system as to why it failed or the quantity that failed or anything like that. There is however, this very convenient developer sandbox where you can copy and paste in your entire email, and it will tell you line by line what you did wrong. So, in this example, the tag image is disallowed, because AMP doesn’t allow you to use images, you have to use AMP images instead. Some dynamic caching magic stuff, I’m sure.

Rohan Kapoor: So as a result we are adding AMP validation to our internal ESP because as you’ve probably heard, Indeed is a data driven company. We love our metrics. We want to know how many of our emails that are leaving fail validation and why and hopefully correct them.

Rohan Kapoor: One last thing with AMP is that there’s additional security requirements than traditional email. It’s a trend here. You take traditional email and then add a bunch more requirements and then you get AMP. So to pass AMP validation, you must always pass DKIM, SPF, and DMARC, and email is must always be encrypted in transit using TLS. If not, they magically disappear into the ether. We learned that one the hard way, too.

Rohan Kapoor: So, rounding up the list of important considerations with AMP is that users probably aren’t reading your email exactly at the time you’re sending them. So Gmail supports rendering AMP up to 30 days after the email has been sent, at which point it will permanently switch over to the HTML content forever. So this means that as a developer, when you’re sending out your emails, all your URL endpoints, all of your paths, all of that has to be valid for 30 days, otherwise the email fails. Another use case for fallback content, which can be displayed if those requests do fail.

Rohan Kapoor: So, few technical takeaways. Working with bleeding edge software is hard. Specifications change frequently and you have to be willing to adapt at all times. You have to plan for fallback content with AMP. You are in a mobile world, network connections change all the time and you don’t want large holes in your email where the dynamic content was supposed to be. And you have to find a way to work around the fixed width and height limitations. Basically, you want to make sure that parts of your email don’t clip internally. There’s no giant white spaces, so find some sort of an easy medium. Make sure all of your content that’s coming dynamically is capped at that limit.

Rohan Kapoor: And now I’m going to hand it back to Janie to talk about some of our tests results and conclusion so far.

Janie Clarke: Thank you. How is our test going so far? You learned all about A/B tests before. We test everything, and AMP is no different. We are running AMP in an A/B test right now, that’s targeted at users only. So the control group does not send the AMP MIME type at all and the test group does send the AMP MIME type. When we are also testing a few other little functionalities, but the main thing we’re testing right now is the sponsor jobs. So far when we look at our test at the aggregate level, we’re not seeing much change in our test group, and there’s a reason for that.

Janie Clarke: When we compare emails that were opened as AMP to emails that were not opened as AMP, they were just opened as regular HTML, we do see two times more clicks in the AMP opened emails, which is a really great promising early results. So, the reason for this is mobile. Google has not started to roll out AMP for mobile yet–for the Gmail app. And in Job Alerts, 82% of our opens are on mobile. So that means most of the users that are getting that AMP MIME type aren’t seeing it. So since AMP is currently not supported on mobile, our test results are pretty limited.

Janie Clarke: It’s hard to spot many behavior changes when we look at the test at the aggregate level. And so we’re waiting right now for Google to start rolling out the AMP functionality for the Gmail app, so that we can really see how it does at full scale. We’re in a little bit of a holding pattern right now, just watching and waiting. We do have some future ideas for features that take advantage of AMP that we’re really looking forward to testing and we’re working on them right now. One of them is an interactive NPS survey as shown here, so we can show NPS right at the bottom of the email and users can answer the question, even type in some feedback for us without even leaving Gmail.

Janie Clarke: We also working on an interactive unsubscribed surveys, similar idea, someone can unsubscribe and tell us why they’re unsubscribing right there. So, it’s a great way to capture some user feedback from your email users. So here are a few things to consider about AMP from a product manager’s point of view. Firstly, be aware of the current limitations when you’re planning. As I mentioned before, there are some challenges with mobile support and with how certain components behave.

Janie Clarke: And secondly, you have to plan out how you’re going to measure your test. We didn’t go into much detail about that, but since AMP adds interactivity to your emails, you need to know what actions you might want to measure and make sure that you’re logging those, tracking them and whatever system they use, so that you can see what’s happening inside the email. And lastly, designing for AMP brings new challenges. So it’s different from designing for a traditional email and it’s also different from designing for the web or mobile. Like the lightbox case that we mentioned, the design that you have in mind when you first do it might not be how it actually works in real life. So you have to be aware of that and be ready to adapt.

Janie Clarke: So we’re very excited about AMP but cautiously. So as I mentioned, it totally changes email. It lets email do some things that have never been possible before. It brings the whole conversion funnel directly inside the email. In short, we think it’s awesome. If you’re excited about AMP as we are, we recommend giving it a try. You do need to have some development bandwidth to work on it. But there may be dragons. So there are some challenges when it comes to working with AMP, so just make sure you’re prepared. Thank you.

Allison Dingler: Everyone hyped? Everyone ready? Yeah? Can I get woo hoo? Y’all can do better than that. Here we go, awesome. I’m going to pass it over to Galina.

Galina Merzheritaskaya, Nitya Malhotra, Erin McGowan, Alison Yu

Indeed girl geeks: Galina Merzheritaskaya, Nitya Malhotra, Erin McGowan, and Alison Yu speaking on women in leadership at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Galina Merzheritskaya: Hi, everyone. My name is Galina. I’m a QA Engineer on the data science platform at Indeed. I have a few question we got from panelists tonight, and once I finish them, I would like to move to the audience and hear your questions. Before I start with my questions I would like all panelists to introduce themselves.

Nitya Malhotra: Hi everyone. My name is Nitya. I’ve been at Indeed for about five years now. I’m an Engineering Manager. I transitioned from an IC to engineering manager about two years ago. Before that, I was a product manager with Merrill Lynch, Bank of America, and then Indeed. And yes, that’s where I’ve been ever since.

Erin McGowan: I’m Erin McGowan. I’ve been at Indeed for three and a half years. I’m our Associate Site Lead in Seattle. That’s part of our chief of staff organization, so we’re looking at overall site health and ensuring engineering growth across the site.

Alison Yu: Hi everyone. I am Alison. I am the Open Source Community Manager here at Indeed. I’m part of the Open Source Program office and I report directly into the Engineering Capabilities Organization. I’ve been here a year and a half. I think I said that. I have a little bit of a cold, so if I start to cough, I’ll exit to stage left.

Galina Merzheritskaya: Thank you. I think all of us know that tech is a male dominated industry statistically. You have this number. Does anyone know the ratio between men and women in tech for the past five years? Okay.

Audience Member: 10:1.

Galina Merzheritskaya: 10:1. Close. So, Catalyst is a nonprofit organization collects a lot of data to help women at workplace, and they provide the data that 30% of women in tech industry, only 20% in a leadership position, and Forbes also did their own research showing that you have some growth coming from 3% to 6%, from average 15 to 17. So there’s some work done, there are some changes, probably it’s why you’re here.

Galina Merzheritskaya: So I want to ask panelists, what do you think can be done to change the situation? To make it better and have more female, the women in a male dominated industry?

Erin McGowan: [inaudible].

Nitya Malhotra: Hang on for a second. I think something that in my mind would really make a difference is A, seeing more women in leadership positions. This is something that I personally find motivating, or demotivating sometimes, to not see. But how do we do that? How do we actually get to that place? Now, I know that from Indeed, I’m again speaking from Indeed’s perspective, I know that support, having either great mentors or great support within your organization has also been really helpful.

Nitya Malhotra: Another thing that Indeed has been doing is we’ve actually been partnering with a few programs. I’m actually more familiar with the program in the Seattle office where we’ve been partnering with the Ada Developers Academy. The Ada Developers Academy is a training program for women who have not necessarily been through the traditional CS program in school, and it’s basically a bootcamp after which they go through an internship and then end up joining a bunch of tech companies.

Nitya Malhotra: So the Ada program in Seattle has actually been pretty amazing. I actually ended up working with a lot of ADs as we call them in Seattle, and it’s been great to see so many women in the Seattle office, thanks to the Ada program, and they’ve all been doing such an amazing job. So that in itself has been pretty encouraging to see more women in general in a lot of the offices. So, I think just it’s a numbers game out there and seeing more women in tech is I think what is going to solve the problem in the end.

Galina Merzheritskaya: And [inaudible]. Is it a major program for anyone who works at Indeed or anyone from outside [inaudible].

Nitya Malhotra: It’s actually outside Indeed but they end up partnering with Indeed and a bunch of other tech companies as well, and do internships at Indeed and end up getting them full-time offers with tech companies.

Galina Merzheritskaya: Thank you.

Erin McGowan: And in San Francisco, Indeed is also partnering with Techtonica. They are hosting a cohort of 15 on site now. It’s the second cohort. The first cohort has four interns that are currently interning here at Indeed. The other from that cohort are in interning at other tech offices. So San Francisco is also helping to increase the funnel to help grow women in these engineering roles.

Alison Yu: And I can expand on Techtonica. I am based on the San Francisco office so I work closely with them. One of Techtonica’s missions is to make sure that women and non-binary individuals who have non-traditional tech backgrounds have a way to join the tech industry. So they put them through a bootcamp, essentially, and then help place them with jobs. At Indeed, we not only host them but we, let’s last quarter, also help get them to Grace Hopper. So we did sponsor travel and passes for Tectonicans who went and hosted a project at Grace Hopper’s open source day, which the Open Source Program office did sponsor.

Alison Yu: So, we’re trying to make sure that their name is getting out there, that there is more ways for people to get involved with tech. And then just being a part of the Open Source industry as well is, one thing that we like to stress is that technology isn’t all about codes. So there’s so many different jobs within tech that don’t require you to be a hard engineer or a coder, even. So there’s so many different facets of tech that have roles open. So marketing, legal, et cetera. So, I think there’s a lot of other ways that people can evolve in tech, which aren’t traditionally thought of.

Galina Merzheritskaya: That’s right. Thank you, Alison. You all [inaudible]. Great. Based on the value you see right now, have you, having mentioned of leadership roles, how did you land there? How do you get to this current destination where you’re right now?

Erin McGowan: Sure. I’ll take this one first. I didn’t always start in tech. I actually started in hard science as a lab tech, and it was awful. If you think tech has low men to women ratios, hard sciences has nothing on that. I was the only woman in both labs that I worked in. And looking at it and as we’re using computers, running instruments, it was interesting and fascinating way more than the hard science. So at that point I transitioned into tech largely in quality assurance. I learned how to become an [S debt 01:09:45] at Microsoft way back when. And I always said, “I don’t want to be a manager. I don’t want to be a manager. That’s awful.”

Erin McGowan: And then I started mentoring people and seeing them grow and I was like, “Oh, this is what management is about. I can help drive careers.” At one of my jobs prior to Indeed, I took the plunge and I accepted a leadership role. I had an awesome, awesome female advocate who was big on ensuring that women had access to those leadership roles, and I loved it. When I joined Indeed, I came in as a quality assurance manager, actually in Austin, and I grew my team from four, and before I transitioned I had a team, teams in Tokyo, Austin, and Seattle, and tried to help them grow.

Erin McGowan: And also growing the next generation of leaders really helped fill my bucket. So even if you think you don’t want to be a manager right now, you never know, don’t close that door, leave it open.

Galina Merzheritskaya: You’d like to add anything?

Nitya Malhotra: Yeah, I can go. I actually started as a product manager in, I was working with Merrill Lynch. So I ended up being a product manager. Now after school, despite good grades, et cetera, I had a computer science degree, I wasn’t really sure if I was maybe cut out for software engineering. And I know a lot of women who have gone through similar experiences, it’s amazing how many women I know who have basically had similar experiences. Anyways, so at that point, it was when I missed getting my hands dirty and actually coding. And that’s when I realized that, “Oh, this is maybe something that I do want to get into.” And at Indeed is when I would say I completely transitioned back to a software engineer.

Nitya Malhotra: And again, I think my journey, even at Indeed was, it took a lot of work, a lot of, I would say a lot of coaching and a lot of mentoring from really great managers, really good mentors, that really helped. Because there was always this, [inaudible] I did this, but is it really that great? Everyone could have done this. And it required a lot of my manager be like, hey, you know what, this is actually something good that you’ve accomplished, et cetera.

Nitya Malhotra: So, I would say it was a lot of that. It was a lot of help from my managers and mentors to actually get me to a state where I felt that I was confident enough to go ahead and then ultimately make a switch from IC to tech lead to then engineering manager. And I think that transition off of that has happened really smoothly.

Alison Yu: Yeah. I started out in clean tech in solar, right, if anyone remembers when [inaudible] happened. That was a fun time. I really got thrown into the tech bubble in a sink or swim situation. From there I transitioned from clean tech into tech. I was really lucky. I had a great manager who I actually followed from one company to the next, and she really encouraged growth. And here’s some different ways you can help expand what you’re doing.

Alison Yu: Manage some vendors, manage different contractors, figure out how your style is, and that’s how I’ve gotten to where I am. I don’t currently manage any people, but I do manage many different relationships cross departmentally and within my own program. So, external and internal. That’s how I’ve navigated the waters.

Galina Merzheritskaya: It sounds like you can come from individual contribution to a manager. Maybe you can advise something, what steps look like to come from IC to manager.

Nitya Malhotra: I can take this one one first. When I was thinking about moving, becoming an engineering manager, I think my biggest question was, am I going to lose technical focus? My goal was to be an engineering manager who was also really technically strong. And I was really worried about either plunging into management a little too early and losing the technical focus.

Nitya Malhotra: Fortunately that has not happened. What I have learned over time is that being an engineering manager has given me, I might not be directly making a lot of technical contributions, but it’s given me the chance to make those contributions by influencing others. And that has been the big switch in my thinking. So I can still be involved in really big, really technical changes.

Nitya Malhotra: The only difference is I’m influencing those changes rather than actually executing on them. And that has also been pretty, I still feel technically involved, at the same point in time, I’ve still felt that I could influence others careers. Coaching has also been really, really rewarding and I have no regrets.

Erin McGowan: I think what I would tell someone who is interested in going from being an IC into management, is to try mentoring. See if there’s an opportunity to mentor an intern or a new hire, and see how you like that. As that is successful, working with your manager to say, you know what, I want to do this full time. Where do you see my skill gaps? What do you see that I need? Is there any training that you can attend, books that you would suggest? And making sure that your manager is aware.

Erin McGowan: If your manager is not receptive, finding someone who has seen some of that leadership, your mentorship to get as an ally. Unfortunately, sometimes we need to have that external ally who’s not our direct manager. But the biggest thing is doing it, showing leadership, stepping up in meetings, stepping up to volunteer to take on some of that extra unofficial leadership. As people see you in that role, it is a lot easier for them to see you in a full time management role.

Alison Yu: Yeah, and I would expand on that as well. Not only just finding one-on-one mentorship, but if you can try to step into a leadership position in what we would call here a ERG, like Lena is leading the women in tech group at in the San Francisco office for Indeed. If you can find different programs where you can help lead and bring people together, it makes a big impact. People see you. It’s a highly visible role and it’s really easy way to also find other people who can help mentor you.

Alison Yu: Because otherwise, unless you’re asking for mentorship or asking for help, if you don’t raise your hand and look for those opportunities, people don’t know. So I think the first step is really asking and looking for those opportunities. Prior to joining Indeed, for example, I led the philanthropy efforts at my last company for about three and a half years prior to joining Indeed, but I didn’t have a formal role there. But that gave me the tools to actually be able to step into more of a leadership role where I had the experience behind me.

Alison Yu: And it was something that, even though it wasn’t official, I could put on my resume, I could talk about my experience. So, I think as Erin said, getting your hands dirty and actually doing it is really 80% of the work.

Galina Merzheritskaya: Yeah, I can probably [inaudible]. You try and you like it but we have [inaudible]. You probably had a lot of advice in your career path. Can you guys discuss for some areas, some industries that at least have [inaudible] of sharing? can you share maybe one of the advice with us?

Alison Yu: Sure. I’ll start. One of the things, one of the pieces of advice that I’m still grappling with is, don’t focus so much on perfection and don’t burn yourself out. I am probably a Type A type of person. If you know me, I very much focus on perfection. But one thing I’ve learned over the years is, if you’re burned out, you’re not putting out your best work. You might be running for perfection but you’re not going to get there if you’re completely burned out. So, take time for yourself and rest. I know it seems counterintuitive, but recharging is really one of the most important things that you can do.

Galina Merzheritskaya: That’s great

Erin McGowan: Sure. I think the piece of advice that has stuck with me the most is, if you’re not uncomfortable, you’re not growing. It can be easy to get into a role or into a position where you’re comfortable, you know what you’re doing, you know you have this. But when you’re in that steady state, you’re not pushing yourself and you’re not growing. So it’s okay to get comfortable for a little bit, but then bump yourself to that next level and look for that harder challenge, because that is when you’re going to grow and take your career to the next step.

Nitya Malhotra: Completely second that, by the way, completely. My end, I would say the biggest career advice that I have received is–something that I’ve also really followed, is I’ve really looked out for mentors outside my manager as well. Whether it’s peers or managers on another team, basically looked around the office for people that I know I can go to with questions about various different things. I have a mentor that I go to for technical questions. I have a mentor that I go to if I have questions around being a manager.

Nitya Malhotra: There’re so many different ways. There’s so many different people that I know to reach out to if I have questions on so many different things, and having that these are often the people who will end up supporting you and end up giving you that additional visibility as well within the office. That’s something.

Nitya Malhotra: Another thing that this applies specifically to mentoring, and this advice that I received while mentoring which I’ve always moved–paid forward is, when mentoring, always think about the why rather than, focus on the why rather than the what. No matter what it is that you’re teaching someone, focusing on why something is important rather than what, actually really helps a concept stick. I think this is just in general, good advice that I’ve always passed on.

Galina Merzheritskaya: Okay. So really great advice. I don’t [inaudible]. This is a good time to take questions. If you have a question, they have a mic over there [inaudible] so we can [inaudible].

Vanessa: Hi [inaudible]. Is this working?

Galina Merzheritskaya: No.

Vanessa: Hi, can you hear me?

Galina Merzheritskaya: Yeah.

Vanessa: Cool. My name is Vanessa and I was wondering if you could give me some advice for a situation I find myself in pretty often, which is I oftentimes find that when giving my opinion on a situation, it’s either implied or explicit that I back my decision up with data. Whereas I find a lot of male colleagues can just give their opinion and not back it up with data, and be considered credible. When I back up my decision with data, sometimes it’s met amicably and sometimes it’s met with skepticism, even if it’s based in reality. So I’m wondering if you have any advice for those kinds of situations.

Galina Merzheritskaya: Tough one [inaudible].

Erin McGowan: That’s a tough one. I am really, really happy that Indeed does not have a culture like that. I think the biggest thing is to be consistent. And I would also, with those men being questioned, ask. Be the one to stand up and ask them, what are you basing this on? What did you see that led you to this result? And be consistent with that and they can come to expect the same level of questioning as yourself.

Nitya Malhotra: I would ask the other questions, are there others in the room who probably feel similarly? Are you the only one in the room who is feeling that way or there may be others who are feeling similarly. Maybe this is a wider problem as well and needs to be addressed in a wider way, but that’s the other avenue that I would go down, and the me–

Alison Yu: Yeah. I would also say, if you feel like this is the only… you’re the only one on the team for example, bring it up to your manager, talk about it. If they don’t realize that it’s an un-bias–or an unconscious bias, then how can they address it? So, I think that’s something that we should be talking about more in the workplace, anyways, I am very happy that we don’t have that culture here at Indeed. But I think that until you raise that issue with others and even talking to your teammates one-on-one, pull them aside, say, “Hey, why are you questioning me about this?” Until you have those open conversations with them, I don’t think the situation can change. But I think that’s your first step.

Nitya Malhotra: And I think it’s fair to ask for data, but I think it should be applied consistently. I think it’s absolutely fair to ask for data when you’re making a statement. But the inconsistency is the issue. And the other thing that I would say is don’t let that stop you from actually providing your input. Even if you have to back it by data, go ahead, keep going strong with providing your opinions, even if you have to back it with data. But don’t stop doing that.

Vanessa: Thanks.

Galina Merzheritskaya: Anyone else? Everyone seem to ask big questions, tough questions. Okay it’s 8 o’clock already.

Audience Member: I’m not sure if anyone, any of you, y’all said this already, but can you tell me how long it took you to get into leadership and are there any other steps that could take to be there?

Erin McGowan: Sure. I’ll go first. From the time that I really decided I wanted to move into leadership, was probably about 12 months, and that included doing an extensive six month leadership program that had some very intensive training sessions, talking with other leaders in the organization. This was prior to Indeed. It’s really going to vary, depending on your organization. I’ve had people on my team that I was able to get from an IC into a management, that was anywhere between say six months and 18 months, depending on where they were at on their career when they started addressing interest and wanting to go to management.

Nitya Malhotra: And I was actually similar situation once I figured out that I wanted to be in a leadership position, was about asking, “Hey, what are the next steps that I can take, move into a tech lead role?” And tested that out for a while, then moved on to engineering management. It was a similar 12 month period for me. The advice would be if that’s something that you’re interested in, I would say ask for it and test it out, see how it goes.

Galina Merzheritskaya: One time I got advice that if you’re solving problems, you lead something. It can be project, a team, a team has issues and so you have to handle them, overcome them, failure. Then it’s just next step if you want to make them official or if you want to just [inaudible]. If you want to make official you speak with the manager and [inaudible] talk with what you want and how fast you want it.

Nitya Malhotra: Yes.

Galina Merzheritskaya: What should I do to make manager? Steps just what can [inaudible] and what can make a [inaudible]. If it’s two months, three months [inaudible] you solve the basic problem [inaudible]. For [inaudible] company, tomorrow you [inaudible] [inaudible]. Yeah.

Alison Yu: Yeah. It seems like it really does vary depending on where you start in your career path. If you start very early on, knowing that you want to manage someone, it’ll take you much longer than if you’ve been in your career for five, 10 years. So I think it’s just knowing what you want and going after it. When I manage people, I actually did a job switch and looked for particular roles that had managing positions and where I would hire on a team. I know that’s not always the most ideal way to do it, but sometimes the company that you’re currently at lacks resources. Here at Indeed, we thankfully don’t lack resources. I just decided that that wasn’t the path that I wanted to be on at the moment. So, many different ways.

Galina Merzheritskaya: [inaudible]. Anyone else?

Audience Member: Hi. I just wanted to ask, what advice do you have for somebody who is starting off in their career but wants to have influence within the team? What kind of strategies or what did you do when you were in that position to influence the decisions that your team makes or just to have some influence, because as a leader, obviously you have to make decisions and have influence over your team. So do you have any advice for people and they’re starting their careers?

Nitya Malhotra: So, actually this is a good question back to you. I would ask yourself what is the thing–What is the… there’s always, no matter what the team, what the company, there is always some room and some scope for improvement. And finding things that either you are passionate about getting… about the team getting better about team’s process improving, about how the development processes. Or if you want to let’s say, if you’re really passionate about this particular, it could even be at this particular class in your service that you think is not tested for example. Or you see a bunch of errors and no one’s caring, but there’s so many.

Nitya Malhotra: I would say pick something that, let’s say, that you are passionate about that, let’s say, bothers you and go ahead and fix it. Things like these, the tiny, tiny things that you spot and as you keep improving these, people are going to notice that you’re taking the initiative to go ahead, find something that you don’t like, and improve it. I think that is a great leadership quality in itself. And once you start doing that in the small level, you will start doing that on a larger scale, as well.

Erin McGowan: I would say one of the mistakes I made early in my career, is I was afraid to speak up. I would be sitting in a meeting room and I would be afraid to actually voice my opinion. I would have the thoughts and I would just be afraid to actually put them out there. Put yourself out there. Don’t be afraid to say, what about this? Have we considered this? You know what, I tried this and it really didn’t work. They’re not going to bite. At least if you’re in a good workplace, they’re not going to bite. So I would say just go ahead and speak up. They want to hear your voice. You’re in that room for a reason, so don’t be afraid to use it.

Nitya Malhotra: Also to be fair, that never gets easy. Today there have been times where I am like, “Oh, should I see this? Everyone else seems to know.” But that never gets easy and that’s always a struggle. That’ll continue to be a struggle, times you just have to push past it.

Alison Yu: I would say try to find gaps and try to become a subject matter expert in one of those areas. I was on a marketing and communications team across multiple different companies. I specialized in social media for a while, then I became more broad. And because of my expertise in one area, people came to me from many different departments, from different business units. Even though I was in the marketing team and I sat in marketing, people from different engineering teams would come to me and ask, “How do I market my product better?”

Alison Yu: So once you get your name out there and you’re proven that you’ve done the research, you’re doing a good job, people will seek you out anyway. So focused on something that you’re passionate about, that really fires you up because that’s something that will be recognized that you’re doing a good job and then they will naturally follow.

Galina Merzheritskaya: I can add the two pieces. One as jealous [inaudible]. If you think I don’t know team, I was like, “Oh my God, she loves this team, loves the product, what can I improve? Like everything’s great.” And then, “Oh is it [inaudible] feels great.” And he’s [inaudible] today [inaudible] every day since 5:00 PM, do you guys use [inaudible] and then he’s trying to give [inaudible] aspect. Not [inaudible] but [inaudible]. Whereas where I can help to improve this. And as it little by little you have this guy like, “Hey, this is actually not a big problem.” And [inaudible] somewhere you can show the need to do such.

Galina Merzheritskaya: You’re also independently and [inaudible] are like this is my research and definitely how can you grow. And I’ve learned [inaudible] goals, that sometimes will follow when you lose everyone else, you just have your own goals, but for [inaudible] before breaking them, bring on some rules that exist. And knowing how everything works. So like okay then you can figure out different ways that basically can prosper in that area.

Alison Yu: I’m just going to add to that I think different perspectives and the way that different people will look at a problem or a situation, can only make a product or a team stronger. So even if you think, “Hey, my opinion, what is it really valid?” Your opinion is valid so never question that. But also know that the different ways that you look at a problem is a different way that someone might have never seen it, and you can be revealing something about a weakness that maybe no one else on the team had thought about and that can be a really great thing for you and for the team.

Galina Merzheritskaya: That’s true. Okay.

Audience Member: Hi there. I have a question about how do you grow on the job? I know that you mentioned that there is a women in leadership program that you went to to get a lot of training before you became a manager. What if there’s no training like that or you’ve thought about becoming a manager maybe down the road, how do… what kind of things have you done, maybe in the past, to prepare yourself to get to this road? Do you read books? Do you listen to podcast or do you go to meet up to meet mentors? Yeah.

Erin McGowan: Yes. All of the above. I think the biggest thing, if you’re looking for some leadership training and your organization doesn’t offer it, look external and ask. Often time your organization will pay for external training. Most of the time there are budgets for that and people don’t even know to ask. So ask for it. Perhaps they can bring some in. You’re probably not the only one. There are a number of books that you can help read. I [inaudble] in the management before podcasts were a big thing, but I know that a lot of people now are using podcasts to help learn and having a mentor. Having a manager mentor that is not within your organization can be huge.

Erin McGowan: You’re able to really have good conversations with them without any fear of bias or… I asked this question, is my manager going to think I’m not ready? So having someone who’s outside, whether they’re outside just your group or your company, is invaluable to really help understand where you as a person need to go.

Nitya Malhotra: I’m not great with podcasts or it reading outside work. For some reason after work, I just switch off and I… it’s really hard. So a lot of my growing I feel happens, majority of it for me has happened at work. Something that I found useful is putting myself in, let’s say my manager’s shoes, and thinking about A, how would I have done this differently? How would I have done this better, or do I like what they’re doing? As time has gone by, this has created a mental model of how it is that I want to be as a manager, and that has really helped me model my experiences with my reports as well.

Alison Yu: Yeah, I am also the same way. I do not want to read or listen to a podcast about work after work. I work enough, so I think one of the things that I try to do, is at work or when I travel for work, so speaking at conferences, et cetera, one of the things that I try to do is make sure I go to the networking events at those things. Talking to my peers in the same industry, seeing where they are in their careers, how they’ve been able to progress, and then finding other people who are already in other higher management roles and seeing and talking to them about what was their career path, if they’d be open to mentoring.

Alison Yu: I think that’s really important that you want to see what other people in your same industry are doing, and not just only stick to what’s the norm in your company. Because sometimes when you’re so isolated and so siloed in your own company, you might not notice that the path that your management runs, it’s not the same as the rest of the industry. So I think that’s also very important to keep just a tab on.

Galina Merzheritskaya: Yeah. Sometime it’s called shadowing, if you think you would like to try some role and be like a measurable, you speak with your and you’re like, “Hey, I want to see what you do so I can know what you do from seeing what you’re doing.” And then you will have questions and then you will have a pass while you actually [inaudible]. And then you always say, “Oh, I did [inaudible].” And if you have an opportunity to speak with the manager be clear what you want, or how will the manager help me.

Galina Merzheritskaya: Or like if it’s not in your company, there are so many programs outside. We have so many mentorship right now. There are many [inaudible] to step up higher and you learn. It’s a great time to do it now because, if you can do more then you can do tomorrow [inaudible]. It makes it so that managers who are [inaudible] can draw a time just to help you to grow as well. Yeah. I think that’s fine. And we have time for a few more questions if you have any. Yes, okay.

Audience Member: Hi. Just wanted to firstly thank you guys for sharing your experience. My question was around conflict and some strategies that you use when you experience conflict in the workplace. I know sometimes women can be more of the supportive or the accommodating role, where men can be more dominant in that conflict situation, so just wondering what tips and tricks you have for dealing conflict.

Erin McGowan: I think the first tip is to make sure that both parties are leaving emotion at the door. If either party starts to escalate with emotions, asking for a timeout and table, and have that conversation at a later time. You’re not going to have any kind of a productive conversation if someone is angry, yelling, or on the flip side, if someone is upset, perhaps crying. You’re not going to be able to have a good healthy conversation, table it and schedule time. If you need to involve a neutral third party, either your manager or that person’s manager, or both, to help, if you are afraid of emotions escalating again.

Erin McGowan: I think the other thing is that some conflict is healthy. It’s okay to disagree, but you want to make sure that you are disagreeing respectfully, and making sure that you’re not crossing any of those professional lines disagreeing. Having data can really help. This is where I am on this because one, two, three, can you help me understand how you came to this?

Erin McGowan: So, really understanding their point of view can help bridge. You might be closer than you thought, but you’re talking sideways. You’re just in a different place, but you might still actually be really close to each other. So taking the time to understand where they’re coming from and how they got there. And helping them to do the same on your side.

Nitya Malhotra: I might have slightly different opinion, especially around the emotions. I know emotions in general are turned on as this bad term. I actually don’t think they’re bad. I think that a lot of times when there are strong emotions associated with how someone’s feeling, there’s generally a reason behind it, and it’s always, I think it’s important to figure out what that reason is. So even if there is emotion around it, obviously we don’t want things to escalate, but that emotion is stemming from something and it’s really important to figure out where that’s coming from.

Nitya Malhotra: So stating, let’s say in a conflict, stating exactly why you’re saying something, you’re stating where you’re coming from, what your intentions are. My intentions are not to disagree with you, but I really think that this is the right thing that we should bring. I’m just giving a stupid example. But stating your intentions, where you’re coming from, I think that really helps clarify and make someone understand that you’re not actually attacking them and that you’re coming from a place of logical reason.

Nitya Malhotra: Crucial conversations was this class that actually I took at, it was one of the trainings that was offered at Indeed. I thought it to be super helpful, not just in my work life, but also in my personal life. I would recommend, I would look into it as well.

Alison Yu: I think for me it’s making sure that everyone’s on a neutral playing field. Making sure that someone doesn’t feel ganged up on. For example, taking any of those external situational feelings that could happen if you were to say, “Hey, I want to address this.” But you’re doing it, for example, in the cafeteria or in a meeting room full of other people. I think there’s a time and place for everything, so if you are feeling those emotions, I don’t think they’re necessarily bad, but I do think that you do need that time to have everyone simmer down a little bit, so you have the root of the issue that you’re actually trying to get to, versus the feelings that are really getting there.

Alison Yu: I do agree that when you have those charged feelings, there is something that is sparking that and you need to get to the issue and resolve that, but you need to also do it in a cool, calm, collected manner, because you going in there guns blazing is not going to help with anything, neither you or the other party. But making sure to have it on a neutral ground, I would say before you really escalate it to managers, try to see if, hey, can you work this out with your peer? Is there a way you can take this out? Try going for a coffee and just going and talking it out. Usually that, I think, helps it.

Alison Yu: It’s not necessarily in the office or in front of others, but it gives you a neutral ground and it’s time away where other people can’t really hear what’s going on, and I think that’s very important when you’re trying to hash out the details when there’s a serious conflict.

Galina Merzheritskaya: And I’ll just maybe say super [inaudible] personal. If you personally got offended or someone’s [inaudible]. All my [inaudible] and game matters. I’ve been thinking about it and [inaudible] up there. You guys [inaudible]. And I [inaudible] solve problems, do it once I think. I’ve been told if you have to do [inaudible] five times breathe in, and breathe out five times [inaudible], you will get more oxygen to your brain and you will be feeling a little more rational. You cool down and then you can move on to the next step. You can also agree to [inaudible] partner [inaudible]. Thank you.

Audience Member: Thank you.

Galina Merzheritskaya: And I think one more question then we’ll close. Okay. One, two, three. Okay, I [inaudible]. Thank you so much for your time.

Allison Dingler: Awesome. I won’t take up any more time. Another round of applause for our amazing panelists up here, yes.

Alison Yu, Erin McGowan, Lindsay Brothers, Janie Clarke, Rohan Kapoor, Galina Merzheritskaya

Thank you to Indeed’s Alison Yu, Erin McGowan, Lindsay Brothers, Janie Clarke, Rohan Kapoor and Galina Merzheritskaya for speaking at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Indeed Girl Geek Dinner attendees

Thanks to all the girl geeks who came out to Indeed for dinner, networking, talks, panel discussion and more networking!  Erica Kawamoto Hsu / Girl Geek X

Our mission-aligned Girl Geek X partners are hiring!

Toyota Research Institute Girl Geek Lightning Talks (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Kelly Kay, Rita Yau, Suzanne Basalla, Jen Cohen, Carrie Bobier-Tiu, Ha-Kyung Kwon, Steffi Paepcke, Fatima Alloo

Toyota Research Institute (TRI) girl geeks: Kelly Kay, Rita Yau, Suzanne Basalla, Jen Cohen, Carrie Bobier-Tiu, Ha-Kyung Kwon, Steffi Paepcke, and Fatima Alloo, at TRI Girl Geek Dinner in Los Altos, California.  Erica Kawamoto Hsu / Girl Geek X

Transcript of TRI Girl Geek Dinner – Lightning Talks:

Angie Chang: Okay. Hi. Thank you all for coming out tonight. My name is Angie Chang and I’m the founder of Girl Geek X. If this is your first time, welcome. I’ve heard several stories of women who have got tickets for Girl Geek dinners over the last decade and missed one, two, three and four and they’re here tonight. So thank you so much for coming after a long day of work and having fun with us at Toyota Research Institute. I’m really excited to be here and learn all about the cool things. I wanted to point out at these actually really fun stickers. They have the robot arm and the car. If you want to pick them up, they’re not business cards. You can pick them up and take them home with you.

Gretchen DeKnikker: Hi everybody. I’m Gretchen also with Girl Geek. How is that food? Can we just all applaud for how amazing that was? You guys, Toyota has just killed it tonight unlike every level. It’s [inaudible] and I’m not jinxing anything. I swear. And if this looks fun, you can do it at your company too. We do these almost every week and a different company hosts. And so grab someone in a red t-shirt who can put you in touch with someone who actually did the planning, who can tell you how much work it takes, that it’s really fun, that you can bond with your colleagues, that you’re elevating women in your company. You have a cool video to submit to Grace Hopper next year. All these awesome things that can come out of it. So with that, how many of you it’s your first time? Okay, so we want to see you. You can come all the time now. You can do this all the time with all these awesome women. Okay, perfect. I would like to welcome Rita from team Toyota.

Rita Yau: Thank you, Angie and Gretchen. Hi, everybody. Thank you for coming tonight. Hopefully you guys are full with all the food and have a beverage in hand. A few shout outs. We have a few of our executives here this evening. We have our SVP of our autonomous driving organization, Ryan Eustice, as well as his two VPs, Wolfram Burgard and Steve Winston here with us as well. And then we have Max Bajracharya, who is head of our robotics organization. And so we’re really excited to have you guys and have them supporting us. And with that I just wanted to give the platform to our Executive Vice President and CFO, Kelly Kay, who’s going to go over our mission statement.

Kelly Kay speaking

EVP and CFO Kelly Kay gives a talk on “TRI’s Mission: Improving the Quality of Life” at Toyota Research Institute Girl Geek Dinner.

Kelly Kay: Welcome everybody. I’m super excited to have you all here today, and it’s been so much fun to wander about and hear everyone talking and learning more about what we do here at TRI. It feels like we’ve been living in the shadow and now that so many wonderful people have come to learn more about what we do, I’m hoping we could do more events like that here at TRI. I need the clicker. Okay. So I want to talk a little bit about improving the quality of life and TRI’s mission, but I’m going to actually start with myself and my journey that actually brought me here to TRI.

Kelly Kay: I actually went to Ohio State, go Buckeyes IO. So really excited, especially since we have partnerships with the University of Michigan and when I go to Michigan, I have to wear my Ohio gear. Sorry Ryan. I also went to law school, so I’m actually a lawyer. I practiced law for over 20 years before I made it to TRI and I did so kind of in a unique way. My life was actually really focused around taking traditional products and turning them into online products. So I started in the banking industry, so and I don’t want to date myself, but back in the day before you could actually log into a website and look at your bank balance or pay your bills online or apply for a credit card, I was working at a bank trying to make those products possible.

Kelly Kay: So it kind of progressed through my life of kind of taking these really old fashioned, old school regulated industries and turning them into something that was cool and new and innovative. And it kind of ended most recently at Lyft, where I was working as the VP of Operations and helping them actually take their product around America and working in negotiating with different regulators on how you actually take what’s traditionally a taxi industry and turning into a ride sharing industry. So it was really fun to help Lyft grow. And I came to TRI, actually, to be the Chief Operating Officer almost three years ago, and it hit me at just the right time.

Kelly Kay: So Lyft is an amazing company. I was doing amazing things, in my opinion, changing the world as it came to transportation. And when I was approached by TRI, they came to be in the first conversation was more about the mission at TRI. And I was in a really unique place in my life where my stepmother had just had two strokes. My father had fallen trying to help her get up. They ended up on the floor or calling me and calling 911 and they’re like, “Wow,” I’m like, “What’s going to happen to my parents when I live so far away and I have to find a way to actually make sure that they can take care of themselves at home because we can’t always be there and it’s really hard and expensive to afford a home to put them in.” At the same time, my dad could no longer really drive well because he’s disabled and I was thinking, what am I going to do?

Kelly Kay: And when I was talking to the recruiter, they were telling me about the vision and the mission at TRI and what it was all about. You guys have learned quite a bit about it tonight and I was thinking, wow, this company, I could go work at a company that’s actually going to change the future and enable people like my father and my stepmother to actually be able to stay home and age in place and not have to worry about caregivers coming in and being embarrassed by caregivers coming in and taking care of them all the time. And my dad’s not going to have to worry about how is he going to get to his doctor’s appointment or how is he going to get to the grocery store because he can’t drive anymore, because we’re going to have autonomous cars and I’m going to work for a company that can help solve these problems for people in my life that I care about, and ultimately for myself.

Kelly Kay: This probably isn’t going to be around when my dad is still alive, but it’s going to be around when I need it most. I’m an only child. What happens when I’m by myself at my house and I need to get to the doctor? So our vision and mission is really about that. We are actually envisioning a future where Toyota products dramatically improve the quality of life for everyone. And our mission is to develop automated driving robotics and other human amplification technology for Toyota in this space that will enable us to actually allow people like my father and your parents to age in place gracefully, to be able to still move around the home and have robots help them move around the home, and help us transport ourselves from point A to point B in a safe way through autonomous vehicles. So, our leadership team was another thing that really inspired me to come to TRI.

Kelly Kay: We’ve got some of the most amazing minds when it comes to autonomous driving and robotics here at TRI. And the best part about them, is I was very scared, I’m a lawyer, I’m not an engineer. What am I going to do? How am I going to sit at the table with these people because they are so smart and they don’t have the egos that you think these people would have. There’s a lot of doctors that you’re seeing. So most of these folks are PhDs and they really bring a lot to the table without the ego that usually comes with it. And I found that to be one of the other amazing things when I was thinking about coming to TRI, is the people I’m going to be working with really matter.

Kelly Kay: And we just spent three days together, everyone on this screen, actually, learning and thinking together about how to design the future of TRI and what should we be thinking about? How do we even be more innovative than we’re being today? It’s a constant question that we’re asking here everyday at TRI is how do we do more? How do we think about the next great thing that we’re going to do to help transform Toyota and the world today to be more mobile. And our values are another reason why I wanted to come to TRI. When I came here there were… Toyota has principles and it’s got these 10 principles that they bring from Japan and they exist and it’s really, how do you think about working at Toyota? But I wanted TRI to have its own values.

Kelly Kay: And what I did with the team was really think through what we want them to be. And I wanted to put a lot of myself into this because at that point in a lot of what my role is, is really about enabling the company as a whole to be more effective, to helping them design the future, to work with the HR team on the type of people in the culture we want here, to work with the engineering team to get things done, which should the processes be that we have to get things done, building all of that within the company. And the values really are the contract that we have. And when we think about how we work together at TRI, the be yourself value is the one that is the most important to me. And I’m kind of the sponsor of that value, if you could possibly sponsor a value.

Kelly Kay: And we spent some time during our retreat with a famous actor, a Japanese actor, and if anyone watched the TV show Heroes? Yes. So, the Japanese character that reads the comic book and goes back in time. So he came to our retreat with us and worked with us on actually being yourself even more and kind of almost being a child. And we were doing improv and he’s like, “Make a blue doctor machine.” And we’re like, “What?” So we all had to act out a blue doctor machine. It was really like anything you could think of to really be yourself. And he’s like, “Everyone is a genius.” And he really got us to think of our internal genius. So even though we all bring different skills to the table and we all have different levels of education, we actually are in each individual a genius in and amongst themselves.

Kelly Kay: So what my genius is may not be the same as somebody else’s, but we each have individual characteristics and at TRI, it’s really important that everyone has a seat at the table and a voice at the table to bring their unique characteristics to what we’re doing here. The next is respect one another. And that one is just as important as being yourself. And I think they balance each other out really well because if you’re a jerk, you still have to respect somebody else. So we think about these of kind of making sure that we’re really thinking about all of these as a whole and respecting one another is another one. It’s super important. Again, we might not all have the same opinion, but we want to hear everyone’s opinion.

Kelly Kay: We want to think about it, we want to debate, we want to make informed decisions. So we have to sit back and respect everyone’s opinion here at TRI. And the next is assume best intentions. And this picture is mine. And I will explain the picture to you. I foster kittens throughout the kitten season here in Silicon Valley. We do about 2,400 foster kittens every season here. And I have a German shepherd and he loves to play with the kittens. He’s always licking the kittens and things like that. And the question is, is this dog eating the kitten or is he saving the kitten? He’s in fact saving the kitten. And even dogs are basically good. So we’d like to assume here at TRI that people are basically good. We need to assume best intentions.

Kelly Kay: We’re working in an interesting environment. We have offices all over the world. A lot of email and email can always lead to miscommunication and misunderstanding. So we’d like to make sure that when we’re like, when we get upset, when we read that flaming email, it could be a cultural issue because people are speaking many different languages at Toyota. It could be an issue someone’s just grumpy because they worked late and they just sent something out they shouldn’t send. So we like to think about things of always step back, don’t get upset, assume best intentions, and that really allows us to interact in a different way here at TRI than you would find in other companies where you’re like, people come in very aggressively into meetings like, “Well, why did you send that?” We are like, “Hey, now I assume you didn’t really mean this, but this is how I took it. Let’s talk.”

Kelly Kay: So we just take a different approach to things here. And then thinking globally, again, we’re a global company. TRI’s a small part of a really large company. There’s 360,000 people at Toyota. We are 350, 340 people. So we need to make sure we’re thinking globally on everything we’re doing when we’re designing the car, when we’re designing a robot, we’re actually looking at it from a broader perspective than most companies would. And then finally, make it happen. We’re here, we’re in Silicon Valley, we need to move quick. Japanese companies are historically slow. TRI I was created to actually move faster than a traditional Japanese company. So we’d like to think of ourselves as kind of this company of making things happen.

Kelly Kay: That means taking risks, doesn’t mean jeopardizing safety, but it means taking risks and making decisions and things like that to allow us to move as quickly as possible. I probably completely failed to talk about what I was supposed to talk about, which is what I do here, but these are the things that I put together that are a main part of what I do here at TRI. I have CFO in the title. It’s about this much of what I do. I spend a lot of my time in meetings, just making sure things actually get done. Sometimes it’s coaching people, sometimes it’s helping with reorganizations, sometimes it’s figuring out what should we do strategically? How should we design a program? Why should we do things in a certain way?

Kelly Kay: So I spend a lot of time with the executive team, with anyone who wants to come and talk to me. I believe in an open door policy, and I work really closely with the CEO as well. So, if you want to know anything about TRI, I probably know it. If you wanted something technical you should probably talk to somebody else. But I am a work in progress. So a lot of the professors have taken me under their wing and have been teaching me a little bit more about the technology behind what we do. But again, I think we have a great lineup of some amazing technical people who are going to come up and talk to you and some of our really good leaders here at TRI. So thank you for all coming out. And again, if you have any questions, feel free to ask me or anyone on the staff. We’re super excited to have you here tonight and hopefully you learn a lot about TRI. Thanks. And I didn’t do everything and that’s what really matters.

Rita: Thank you, Kelly.

Kelly Kay: Do you want this?

Rita: All right. So I also just wanted to add to that, one of the reasons why I joined TRI was because of the amazing, awesome things that we do here. And everybody here at TRI really does have the same mission and goal to better the world. And so I’m super privileged to be working with a bunch of amazing, awesome people. And with that we are going to kick off our next round of lightning speakers and we’re going to have the next couple of speakers come up. And Carrie Bobier-Tu, who is our manager of the… Or one of our managers in our autonomous driving organization is going to come up and give us her overview of why she’s here.

Carrie Bobier-Tiu speaking

Manager of Control, Planning and Control Driving Team Carrie Bobier-Tiu gives a talk on “Building the Uncrashable Car” at TRI Girl Geek Dinner.

Carrie Bobier-Tiu: Thanks everybody for coming. I’m really glad to see so many people here. My name is Carrie Bobier-Tu, and I’m the manager of our control team, which is part of our autonomous driving team. And I’ll talk a little bit more about what that means in a minute. But first I’m also going to tell you kind of how I got to TRI. So, the first kind of engineering project, hands on project that I worked on was, I was a member of the Solar Car team at Stanford University when I was an undergrad. So it’s the picture of upper left for you guys.

Carrie Bobier-Tiu: But I worked, as you can see, on a team of me and a bunch of guys, pretty much, and we built this solar car and raced it from Texas to Canada–cutting out–over a week one summer. And that was kind of my entry into cars and my love for cars. A lot of hands on experience there in engineering and also met who would become my advisor for my PhD and kind of build the next many years of my life at Stanford. So Chris Gerdes is a professor at Stanford in vehicle dynamics and control, and I started talking to him about suspension design for the solar car, really got along with him and his students [inaudible] lab and ended up staying in the lab for the next eight years to do my masters and PhD there.

Carrie Bobier-Tiu: The two cars that you see that kind of looked like black dune buggies are the vehicles that I worked on there. The top one I used for my research, which was based around safety systems and advanced stability control and how we can kind of take the stability control that we have and vehicles on the road to the next level by having advanced sensing capabilities. And I also worked on building a new platform, which is the car on the bottom, two students and many after us, kind of, we worked on this car together and I worked on designing a suspension system that can enhance friction estimation capabilities. So I really got deep into vehicle dynamics and engineering here. I was really hands on with it.

Carrie Bobier-Tiu: I liked building cars in our test beds and working with all the sensors and computers, but when I graduated not that long ago, even, there weren’t really any autonomous driving or vehicle control jobs in the Bay Area. So I went to work at HGST, which is a hard drive company. It’s now part of Western Digital. But I am really glad that I went and worked there because I got the experience of designing a control system for a product that you had to have this controller that worked on millions and millions of devices that were going out to customers. So I think having that product experience early in my career was really helpful to me in seeing how to design a robust product and a robust controller. I also got into CrossFit there and got to work with my husband, who’s the second from the left in that picture with all of the red shirts.

Carrie Bobier-Tiu: They had a CrossFit company, or CrossFit gym, at the company, which was really cool and kind of has spurred my interest in health since then, which I’m really appreciative. But I really missed working on cars. So after a couple of years, the autonomous driving business was kind of starting to pop up and I went to work with a few of my old lab mates at Renault, which is the sister company of Nissan for those of us in the United States who aren’t familiar with Renault. But they have a small outlet here in Sunnyvale. So we built this a test vehicle, Callie, the white car with the stripes, and it was just a team of three of us. We were working on controls research. It was basically an extension of my PhD. We were all kind of coming from that same background.

Carrie Bobier-Tiu: So I had kind of the research experience there that I was really enjoying, but it wasn’t going anywhere into a product or out into the company. And we didn’t have a lot of support from the Renault or Nissan itself to do that. So I was starting to look for something different. And what drew me to TRI, it was a couple of things. One was that continued ability to do the research that I was really engaged with, and continuing on for my PhD and bringing that experience with me. And the other piece of it was the ability to work on a safety system that would go into real vehicles. So around the time that I was at Renault, I had a son. So it became very important to me.

Carrie Bobier-Tiu: It’s kind of like figuring out how to make cars safer for my family and for everyone that I know and since I have that background and expertise, I was really excited to come here and work on the guardian system that we have, which I’ll get into in a minute. The other thing that’s really great about TRI is our ability to do research with universities. That’s been mentioned a few times before, but this is the bottom of the last picture is me with some of the students at Stanford. We went out and got to drivers around with one of Toyota’s Drift drivers.

Carrie Bobier-Tiu: So, TRI, we have kind of two approaches to automated driving. So we’ve talked, think of it as one system, but two modes that are built on the same technologies. The first one is Guardian, which is a parallel autonomy system where the driver is still in control of the car. So we try to follow the driver’s intended commands but with minimal and intuitive interventions to maintain safety. And then the second half of our automated driving stack is the one that more people are familiar with, in general in the industry, which is the fully autonomous system where the autonomy system is determining some kind of policy for the vehicle to drive.

Carrie Bobier-Tiu: And we calculate commands that can maintain the safety of the vehicle, like for steering, acceleration and braking. So how does my team’s work fit into this? In the autonomous driving stack, as we call it, which is kind of the full set of software that runs the autonomous vehicle, we have some large groups of kind of algorithmic expertise or design. So there’s a perception system, which is taking in all the information from the sensors of the vehicle and figuring out what does the road look like, who’s on the road around us? And then on top of that comes the prediction system, which is, we know what’s happening now, but what’s going to happen in the future?

Carrie Bobier-Tiu: And finally, planning. So what should the car do, given what the environment looks like or in the Guardian case, what do we think the driver’s going to do in the near future? And control is a part of this planning problem. So what my team does, is something that we call Envelope Control. It’s something that I started developing, as I mentioned when I was doing my PhD. Envelope Control is a holistic control scheme that keeps a given system inside a safe operating regime or envelope. So we have a few things that we have to do. One, is stay on the road and don’t hit anything.

Carrie Bobier-Tiu: Two, is to maintain stability of the car. So, for example, if you’re driving on an icy road, you can lose control of the car and spin out. So we’re trying to prevent situations like that or situations where like a kid runs out in front of the car and you might not have time to stop, so you have to swerve around them. We also don’t want to ask the car to do something it can’t. So if it can’t swerve around for some reason, there’s something in the way, don’t ask it to do that.

Carrie Bobier-Tiu: If we don’t have enough steering capability or enough or the ability to brake fast enough, we can’t ask the car to do that. So we need to know what the limits are. And finally, for the Guardian system, we want to give the driver as much control of the car as possible, but help them maintain the safety of that vehicle. So the technology that we build kind of incorporates all of these things together. And that’s how we, here at TRI on the control team, are trying to create an uncrashable car. Thanks.

Ha-Kyung Kwon speaking

Research Scientist Ha-Kyung Kwon gives a talk on “Accelerating Materials Discovery by Helping You Fail Faster” at TRI Girl Geek Dinner.

Ha-Kyung Kwon: Hi, my name is Ha-Kyung Kwon, research scientist on the Accelerated Materials Design and Discovery or AMDD team here at TRI. So I’m also going to talk about my path to TRI, but I’m going to start from the very beginning. So I was born in Seoul, South Korea. Spent most of my childhood and adolescent years in Manila, in the Philippines, where my dad’s job took us. And it was fantastic growing up in the Philippines. There’s warm tropical weather, great food, white sandy beaches, which was on the picture there. But my favorite part was meeting friends from all over the world.

Ha-Kyung Kwon: And in fact, when I’m not here talking about science on a Thursday night, I’m out there playing flag football with my high school friends, their college friends, and their friends. So anyway, when I graduated from high school, I went to Princeton to study chemical and biological engineering. And my first year at Princeton I started doing research in an organic solar cells lab, and I loved it so much that I continued to do research in the same lab for the next three years. When I graduated from Princeton, I decided that I wanted to do even more polymer science research.

Ha-Kyung Kwon: So I went to Northwestern to get a PhD in material science and engineering. My PhD work was on the face behavior of ion containing polymers. These are polymers that are great candidates for polymer or plastic batteries. Plastic batteries, you might ask. But think about all the plastics or polymers that you know, and their rich properties, like styrofoam, which is light but rigid, nylons and polyesters, which you can wear, and Kevlar, which is extremely tough. It’s Bulletproof. Think about this wide array of properties that polymers have and imagine the possibilities. A polymer battery that is flexible, lightweight, safe, and even recyclable is not out of the question, but it’s going to take many, many years before we can get there and that’s because the scientific research process is incredibly slow.

Ha-Kyung Kwon: First, you have to understand the problem. What are the technical challenges of making a polymer battery? What’s been tried, what hasn’t been tried? Then you have to formulate a hypothesis. Maybe this material has a mechanical strength in the ionic conductivity that’s relevant for a polymer battery. Then you have to figure out how to make that material. Then you test it to see if it has the properties that you want. Once you have the results, you analyze them. Did your hypothesis work? Yes, no, maybe you don’t know. So you have to repeat the entire process. And this process is slow, not only because each step can take a long time, but because many scientific hypotheses end in failures, and that’s part of the process.

Ha-Kyung Kwon: It’s what enables us to learn, to refine our understanding of science and to take a step in the right direction. But this necessary process is unnecessarily slow. In a traditional industrial lab, the R&D cycle can take anywhere between five and 25 years, even more. And we don’t have this kind of time to solve the challenges that we face today. As of 2017, transportation accounted for more than 29% of greenhouse gas emissions in the United States, and more than 82% of that came from driving. The technologies that we have in our cars and trucks today simply aren’t cutting it. For a sustainable future we need new materials and new technologies, and we can’t wait tens of years. So how do we accelerate this materials discovery process while enhancing scientists’ ability to learn, to discover, and to advance scientific knowledge?

Ha-Kyung Kwon: And that’s where my team comes in. Using big data, machine learning methods, and high throughput automated experiments that are driven by these methods, we develop tools to accelerate the design of advanced materials for zero emission technology, such as batteries and fuel cells. Our tools accelerate materials discovery by helping researchers fail faster. And what does that mean? Here are some of the projects that our team’s been working on. Matscholar uses natural language processing that contains over… Sorry. It contains information from over four million scientific abstracts. In the matter of minutes, it can help you discover whether materials that are similar to yours in composition, property, or application have already been studied.

Ha-Kyung Kwon: Something that without this tool could have taken you years, if not decades, to achieve. Using machine learning methods, we’ve also created beep, which can predict the lifetime of a battery from just the first hundred cycles. And this saves a ton of time on the battery development cycle because currently in order to test a battery’s lifetime, you have to cycle it until it dies, which can take more than thousands of cycles. In addition, using optimal experiment design, we can recommend the next set of experiments to run, given the last set that you ran and its results. We can even teach a scientific tool the scientific method in CAMD.

Ha-Kyung Kwon: Given an objective, such as discover a new stable material, it can formulate its own hypothesis, launch simulations according to that hypothesis, refine its hypothesis according to failed simulations, and keep running them until it discovers a new stable material, and it does the leg work so that you as a researcher don’t have to start from ground zero. As you can see, our work in AMDD really spans many materials, many applications, and many scientific disciplines. And this brings me to my favorite part, the people. Our team really brings together scientists and engineers from a diverse set of backgrounds anywhere from computational physics, applied math, to software engineering, who are passionate about discovering materials and accelerating materials discovery for zero emission technology.

Ha-Kyung Kwon: Our tools are interdisciplinary because they’ve been developed, tested, and used firsthand by researchers with diverse backgrounds: us. We also maintain deep connections to fundamental science. Our tools really draw from and build upon the intuition and knowledge of scientists and seek to empower scientists in their learning. And to do this, we work very closely with our consortium of more than 10 university partners and 125 researchers from those academic institutions. Our goal is to accelerate materials discovery, not just for the autonomous vehicle industry, but for the scientific community as a whole. By building tools, we’re building connections and communities because we believe that working together and failing together is the fastest way to sustainable solutions. Thank you.

Rita Yau: Thank you, Ha-Kyung. All right. Okay. To kick off the next section, we are going to be having Jen Cohen, our VP of Operations to tell us about her journey.

Jen Cohen speaking

VP of Operations Jen Cohens gives a talk on “I am the IT Guy” at TRI Girl Geek Dinner.

Jen Cohen: Hi, everyone. I’m so glad you’re here tonight. Thank you, Rita, for the introduction. My name is Jen Cohen, I’m VP of Operations for TRI. You’ll notice my presentation is I am the IT Guy. I promise I’ll explain that. How many people here are in IT? All right, I guess I kind I am. Ian, did you raise your hand back there?

Ian: Halfway.

Jen Cohen: Okay. Halfway. How many of you have had to say, “I am the IT guy?” Okay, so it’s not just me. I feel better. So I’m going to talk to you a little bit about my journey, a little bit about operations, and some of my hacks for high performance teams. So over the years I’ve had to say I am the IT guy. And recently I had to do a presentation talking about my career, and I realized that not only have I had to say it a lot, but I’ve actually learned to love it. I love that surprise when they realize that it’s actually me who’s going to be giving them the answer. And, you know, Facebook’s great about reminding us of what’s happened in the past. So I found this memory recently from 2011.

Jen Cohen: So apparently I’ve been saying the same thing for a while, “Wow, just had another vendor say, ‘Your IT guy needs to…’ It’s funny and sad that they never assume it could be a woman.” But at the end of the day, I love it now. I’ll share that way back when those were popular phones, I started my career as a sys admin at Cisco and I grew my career to do IT management at companies like Smith and Hawkin, which is gone now, but Birkenstock, anybody rocking the Birkenstocks tonight? The big question is do you have socks on? And then I grew from there to do technology development and platform management.

Jen Cohen: So platforms in the convention and real estate industry and the gift about doing technology, the gift about being in IT is that you get to see problems across the entire organization. And I really love to problem solve, but I realized technology wasn’t going to be enough. And so recently, a few years back, got into operations management at Line2, and then here at TRI. I will also share with you that I’m a mom. I have a daughter, Sabrina, who is a junior in college. She’s a computer science major, but she thinks she’s totally different than me. And I have my son Logan, he is a senior in high school and he’s an artist. And based on these slides we know he didn’t get it from me.

Jen Cohen: And then I’d like to talk to you a little bit about operations at TRI. So the teams that I am either responsible for or support some of them, IT of course, because I am the IT guy. But there’s more than me and the amazing team that does that work. CyberSecurity. I co-sponsor the Infrastructure Engineering team. So, that’s essentially a fancy way of saying dev ops. Facilities, Internal Comp, I’m not going to go through the list, but a good group of people who support a lot of fun things here. We have three sites in California, in Michigan, and where am I missing? Massachusetts. I can’t believe I forgot that. I was born there. I should remember.

Jen Cohen: We manage over a hundred key systems, and we support over 300 employees, contractors, and interns. And I decided to show you a picture of some of the amazing folks we have in our ops team because while I say I’m the IT guy, these are the really, the folks who make it happen. And the challenges that we have as Operations at TRI are speed. And while I’d like to have that car, that’s not actually the speed I’m talking about. We move really fast. Our researchers, they need things. We need to make sure that they have what they need at the right time, and we need them to be able to move quickly. We need to be, as has been mentioned, able to fail fast, and we need to get it done at my absolute favorite deadline, which is yesterday.

Jen Cohen: Ask anyone on the team. But we also have to have balance. Part of our job is to protect the company, to protect ourselves. And so we need to make sure we have things like cybersecurity. At the same time we need to enable people to use the technology and get out of their way. We need to have freedom within constraints. And we have these amazing, really smart researchers and software developers. And so we want to make sure that they have the time to work on the things that they’re there for and don’t have to build the technology. But at the same time, some of them know far more about the tech at their desk than we do. So we want to make sure that they have the self-service.

Jen Cohen: So we’re not blocking them. And finally we want to make sure that we’re flexible because there are things that are failing fast, and we need to make sure that we can pivot when the direction of research changes. And we need to build platforms that don’t pin us into a corner. So flexibility. So that’s a little bit about TRI, and the fun things that we do in our Operations team. And then I will briefly share with you three of my hacks for building high performance teams. So how many people here have heard of the concept tank in relation to support teams? Lauren, you don’t count. You know what that is. Anyone else?

Jen Cohen: So learned about tank, a few companies back at a PagerDuty summit, and the idea comes from video games, and I hate to say it, but it’s the person who takes all the hits in a video game, right? So from a support perspective, we have tier one and we have tier two and we want to make sure that our tier two teams who are handling escalations also have time to do the projects that they need to do to help our researchers be successful. And so if they’re handling escalations all day long, they’re not getting the chance to do their project work. So the idea of tank is that one day a week, each sys admin takes on the escalation, they get all the interruptions, they have to deal with it, they don’t get to do their project work, but they get four days the rest of the week relatively uninterrupted.

Jen Cohen: The nice thing about this is it absolutely forces cross training. So if Ian is the only one who knows how to fix the mics and Ian is on vacation, somebody else knows how because they’ve had to handle it on their escalation day. So that’s probably the most powerful thing I think we’ve brought to IT and Infrastructure Engineering here at TRI. My next hack is to celebrate wins. Now I know this can sound a little bit Pollyanna, but I will say, how many people here are problem solvers, and how often do you think about the ones behind you that you finished? Are you mostly looking at the ones coming forward? Yeah, so the problem with that is it’s easy to burn out. So I think it’s really important that we celebrate our wins, not just look forward to the next problem.

Jen Cohen: So in our weekly meetings, our teams list their wins first, so we get a chance to memorialize them and then we list our challenges, so that we have that moment of really acknowledging the work that we’ve done. The other nice thing about this as we put it into a deck that we can go back and look at, because I don’t know about you, but I have no idea what problems I was working on in January, February, March. Anybody here remember theirs? So the nice thing is because we memorialize this, we can go back at the end of the year and look back at what we’ve done and have that moment to remember. And I think that’s really important for support teams, especially, to keep moving forward.

Jen Cohen: And my final hack, and I am the IT guy and I love my technology, but is get off your keyboard. How many of you have been on the email, the GChat, the Slack that has turned into a book? Yeah, not just me. One of the things I found a few years back, I was working with these two developers who were, I think, on GChat and they were going at each other, but they were saying the same thing. They just didn’t realize it. We got them on the phone and within five minutes, they realized they were saying the same thing. The argument was over and they were coding the solution. And I realized at that point how about some part of my job is making sure that people connect.

Jen Cohen: So how many of you have seen the keyboard warrior at work? How many of you have been the keyboard warrior? I’ll admit it, I’ve been there, I’ve sent a flame email I probably shouldn’t have. So the reason I put this up here, and whether you’re a manager or you’re an individual contributor, doesn’t matter. Get off your keyboard. If that’s starting to happen, get on a call. I hate the phone too, but get on a call, get on a VTC, go to somebody’s desk. Because that really will help to work out those problems. And I will also say use Kelly’s tip. Assume best intentions when you do. All right, that’s it for me. Thank you guys so much.

Fatima Alloo speaking

Legal Counsel Fatima Alloo gives a talk on “Navigating the Intersection of Law and Technology” at TRI Girl Geek Dinner.

Fatima Alloo: Hello? Looks like it’s working. Thank you. Hi, everyone. My name is Fatima Alloo and I’m part of the legal team here at TRI. Thank you for coming. So many of you. So, I actually will be talking about navigating. It’s actually a less daunting presentation that’s, than my title might indicate. But what I really want to do is share a little bit about what it is that I do here and some of the awesome issues that we get engage on here as part of the legal team at TRI. And, first I’ll go ahead and start with my career background. So for me, it all started a long time ago when I graduated from law school. And there I am with my parents who are super proud and excited at the time.

Fatima Alloo: After law school, I went into patent litigation, and essentially, I was defending clients in patent infringement lawsuits. So that meant that I had to get quickly up to speed on the mechanics of various technologies, including fun topics like semiconductor fabrication and audio and visual signal processing. And while I loved it, the more I interacted with various tech companies, I started realizing that I was more interested in how the technologies that I was defending were actually developed. So I knew someone at an augmented reality startup called Meta, and it turned out that they needed some legal support. So, I convinced my law firm to second me there, part-time.

Fatima Alloo: And in short, I absolutely loved it. And since that time, I was just so eager to find a way to work full time for a cutting edge tech company with a heart. And that’s how I ended up here at TRI. Now, from all in working with these clients and companies on various, on existing and new technologies, what I realized is, I actually discovered something about the law. And what I realized is that while it’s obviously really important as a lawyer to know what the existing laws are, the law is actually a pretty dynamic and adaptable and can actually be shaped by individuals in this space. In short, the law can actually be fun. Surprise, surprise.

Fatima Alloo: So let me take a quick poll. Who gets excited when you hear the words, “legal’s involved”? Wow, thank you. Wow. It’s more than I expected. Most of you probably think something closer to this. And don’t worry, I’m not going to take any offense. Sometimes these feelings or thoughts are justified, but at TRI, as part of the legal team at TRI, we like to see ourselves a little bit differently instead of trying to attack or pounce on your project, we’re here to support it. And while I can’t help you with the technical side of things, what I can do is amplify your voice in how the next generation of tech is received. I can enable you to partner with other players in this space and I can help ensure that all of your hard work is properly secured.

Fatima Alloo: And ultimately for me, this is what makes it super rewarding to be part of legal team here at TRI. Now, I’m sure all of you are just itching to know, what does my day to day role look like as a lawyer at TRI? Fear not. I have put together three words to describe what I do here, and my team does here. Pioneer, partner and protect. First as a lawyer at TRI, we get to pioneer and all of you have heard Carrie’s presentation, all of the awesome work that we’re doing in the automated vehicle space. And we all have some sense in terms of how automated vehicles are going to disrupt the automotive industry. But the big question on the legal side of the equation is what should the laws and regulations that govern automated driving look like?

Fatima Alloo: What standards should manufacturers that are making automated vehicles adhere to? So, imagine with me for a moment that you are in an automated vehicle and it’s taking you to your destination, but for some reason you need to stop abruptly. Where is the stop button? What does it look like? What color is it? What shape is it, where is it located? And is it located in the same place across vehicles made by different car manufacturers? These all might seem like trivial questions, but it’s important to build a consensus with commercial players in the space for the industry to flourish. Now, we’re super lucky here at TRI because we’re part of Toyota and one of the biggest automobile makers in the world.

Fatima Alloo: And because Toyota is also part of the automated vehicle space, we actually get a seat at the table in determining how these laws are developed. And as a lawyer, consensus building, negotiation, drafting laws and regulations and standards, those are right in the wheelhouse of my skillset and our skillset. But what we need to do is hear and understand from our engineers on what they think the solutions to be to issues like this. And once we do hear from them, we can actually advocate on their behalf. The second thing that we get to do is partner kind of like C3PO and R2D2. Anyway, so as you’re on your way to bringing, onto building groundbreaking technologies, you’re going to need some support. And while we have many brilliant minds here at TRI, many of whom you’ve heard from and will hear from, no company can do this alone.

Fatima Alloo: So you might need to find support outside of our company. Maybe you want to partner with a university or a consultant or a startup that’s developing a component that you just need to have to make your solution come to life. Our job then becomes to make that partnership happen, support the development of your tech, and then to think through about whether this project is really in the business’s best interest. So let’s say, for example, you’ve decided to partner with one of our universities and, as Carrie mentioned, we partner with so many universities, and Ha-Kyung. And so let’s say one thing you might want to think through is, what does each player want to get out of that deal?

Fatima Alloo: The university might want to ensure that they own the IP that’s generated in a joint collaboration. TRI might then want to ensure that we have licenses to that jointly developed technology in case we want to commercialize the tech down the line. The point is, that as lawyers, we often have to think through these situations and then memorialize these agreements in writing. The last thing that we do is protect. Now while you’re engaging with different partners, one big question for TRI is how do we make sure that we’re protected in the process? So I often, am asking several questions, basically. For example, does the partner have access to our systems, data, or code?

Fatima Alloo: Has a partner agreed to be liable if they fail to protect our systems, data, or code? And, sometimes it’s more along the lines of are their cybersecurity standards strong enough to actually guard our code? Occasionally I ask the question of–sometimes the questions are very different in, might be something closer to, as a TRI employee, you think copyrighted images that they don’t have a license for when giving a presentation before hundreds of people. The point is that because we’re in a very hot space, that being self-driving car research and robotics for mobility, we’re subject to a lot of both malicious and inadvertent threats that could cause a company like ours to lose their competitive edge.

Fatima Alloo: And for you Black Panther fans out there, I like to think of our role as protecting the secrets of Wakanda. So, now that you’d have a better idea of what it is that I do here at TRI, and what a lawyer does more generally at a tech company in terms of pioneering, partnering, and protecting, I hope that the next time you have a project you might be the one to get legal involved, and see how we can help you get to wherever you’re going. Thank you.

Rita: Thank you, Fatima. All right. With that, we’re going to welcome Steffi Paepcke who is a Senior UX Designer on our robotics team to the stage.

Steffi Paepcke: All right. Am I on? Can you hear me?

Audience Member: Yes.

Steffi Paepcke speaking

Senior UX Designer of Robotics Steffi Paepcke gives a talk on “Designing Robots to Serve an Aging Population” at TRI Girl Geek Dinner.

Steffi Paepcke: Yes. Okay, great. Cool. Hi everyone. My name is Steffi Paepcke. I’m a Senior UX Designer here. I work on the robotic side of the world and I don’t want to leave you hanging. I’m going to tell you how I got here as well. I started by studying psychology at UC Santa Cruz and after that I kind of didn’t really know what to do. I thought about being a therapist. That had been sort of my goal for a long time. And I wound up at Willow Garage as a research assistant.

Audience Member: Oh.

Steffi Paepcke: Oh, someone’s heard of Willow?

Audience Member: Yes.

Steffi Paepcke: Yeah, cool. Willow Garage, for those of you who don’t know, it was a now defunct privately funded research company. We did all kinds of really exciting work. The PR2 robot, we made turtle bots. We made what now is beam telepresence robots and we did a lot of the maintenance, the primary maintenance on ROS robot operating system and it was at Willow Garage, it was really a pivotal position for me is where I realized that I can combine my interest in humans and how they think and feel and interact with other people and objects. I could combine that with technology and in this case robotics.

Steffi Paepcke: And that was a really big sort of turning point for me where I kind of found robotics is the field that I wanted to work in. I realized also that I needed more training. So I went to Carnegie Mellon and received a Masters in Human-Computer Interaction. And then after that I came back to the Bay Area where I grew up and co-founded Open Source Robotics Foundation, which is now just called Open Robotics. And they are now the primary maintainers of ROS and Gazebo, which is a physics-based robot simulator. One of the biggest projects OSRF worked on was the DARPA Robotics Challenge.

Steffi Paepcke: Which was a little while ago now, but it was a very impressive program put on by DARPA where teams competed in search and rescue tasks with an Atlas humanoid robot or with a robot that they had built themselves. And the program manager of that project was Gill Pratt, who is the TRI CEO now. So I ended up here and have been working on really exciting robots since arriving about three years ago. So I’m part of the UX team. We have user experience researchers, designers and industrial designers. And our main goal is to help TRI figure out what sorts of robotic capabilities to make to improve the quality of life for an aging population.

Steffi Paepcke: So you’ve probably heard that the population is aging relatively quickly in the world right now, approximately 8% of the population is 65 and older. By 2050, that’s supposed to double to 16%. And in Japan, this problem is the most pressing. That’s where the population is aging the fastest. Currently about a quarter of the population in Japan is 65 and older. And by 2050, that number is supposed to be one third. If you think about it, that is staggering, it’s one out of three people will be 65 and older in Japan in 2050. So it’s critical that we find solutions also to the shortage, the caregiver shortage.

Steffi Paepcke: The goal is to make robotic capabilities that can support older adults aging in place longer, taking care of tasks they don’t want to do anymore or can no longer perform. And also alleviating some of the sort of day to day tasks that caregivers need to take care of. So alleviating the chore-like tasks so that they can focus on the human to human interactions that really make caregiving what it is. So that’s our main goal as a UX team here and user experience as a field has become a lot more prevalent in tech companies over the years. It took a little while for companies to really understand that UX was a critical part of creating a successful product.

Steffi Paepcke: And it’s been similarly slow now with robotics as more and more robotics companies crop up. Some of them have user experience teams, a lot of them don’t. I think hardware is obviously very challenging and takes longer than software in terms of development process, but it’s really critical that we have UX in the workflow from the very beginning because you can spend a whole lot of time creating a hardware solution and then you get to the point where you realize you were solving the wrong problem or a problem that doesn’t even exist and then you’re really sort of in trouble.

Steffi Paepcke: So I’m going to walk you through some of the methodologies we use to combine user experience with robotics. What you probably have heard of is just interviews and focus groups. So we do those. Those are pretty standard in UX and we also do participatory design sessions, which is when you work with your target population, in our case, older adults, to come up with solutions together. So you’re not just doing the research and then going back to your office and coming up with the solution. You’re actually sitting down with an older adult and designing something together and co-creating it. Another really valuable methodology we use is called contextual inquiry, which is when you follow someone around and observe them doing a task that you want to learn more about without really interrupting, just sort of asking questions so that you understand the process.

Steffi Paepcke: And in our case we wanted to understand the grocery shopping process for older adults. So we followed them, we met them at their house, followed them in their car to the grocery store, did the whole loop around the store, came home, watched them bring the groceries in, put them away. And it was very illuminating. You can see on the bottom there’s one kitchen that we saw, which has pretty much no mess in it. It’s pretty sparse, a lot of cabinetry, really spacious. And then compared to the kitchen above, it was a very small little apartment with items stacked on the walls and the cabinets were very full.

Steffi Paepcke: So it’s important for us as designers to understand the workspace that our robots will be functioning in, but also very important for the engineers to see what sort of dynamic environment their robot needs to be successful in. You can also see the white cabinetry is pretty reflective, which can cause problems for certain sensors on robots. This is all really important information to bring back to the engineering teams and we try to bring at least one engineer with us when we do these visits so that it’s not just us sort of regurgitating what we saw, but really bringing them along for the ride. Another valuable insight we got was the image with the fridge.

Steffi Paepcke: So we opened the fridge and notice that the woman we were chatting with had kept all of her items at the very edge of the shelves. And when we asked her why that was, she said, “Oh yeah, I just can’t bend over and reach in.” And it’s not something that I would have asked about, “How far back do you put your items on the shelf?” But by being there and really observing it firsthand, we’re able to understand that, that’s one of the problems that comes up a lot. And that turned out to be a really big trend. Being able to stoop safely and without pain is something that is challenging for a lot of older adults, which is sort of common sense, but it helps to see this in the context of people’s lives over and over again. It really drives it home.

Steffi Paepcke: Finally, we do a lot of home walkthroughs, which are probably my favorite. We find people who are pretty open to sharing their lives, which not everyone is, but we meet them at their home and we go into every single room in their home and talk about the tasks that they do in their home, the challenges that they face and the goal there is to figure out if there are any things that we can create solutions for to help them. This again, is really good for context setting for us. There’s one, I apologize that it’s so small, but there’s a person, she wheeled an office chair onto her little patio and she weatherproofed it with plastic bags and whenever she needs to reach the hose that’s on the ground to water the plants, she sits on the chair, lowers herself down, reaches for the hose, raises herself back up and does it that way.

Steffi Paepcke: And that’s not ever something I would have thought to ask about. How do you reach the hose on the floor? But we, by being there with her, we got to witness the trouble she goes through, right? To do something as what I would consider simple as picking up a hose. It can be a real challenge for some people. She also is the owner of the closet next to her and she said that anything above about shoulder height she just pretty much consider as lost and she doesn’t ever expect to get to it again. Yeah. So you really learn about the challenges very viscerally that people face. And then finally, I really like the dishwasher down below.

Steffi Paepcke: This was another participant who doesn’t generate enough dirty dishes to need to run the dishwasher. So she stores her plastic bags and her plastic containers up there. And again, she doesn’t use the lower rack because that’s too far down. So we learned a lot about the importance of designing robots that can reach areas that older adults are not able to, or people with different physical abilities. And yeah, so these are some of the methodologies that I think are really critical in getting on the right track to making a robot that actually solves real problems. You can get pretty far with the interviews, but the data is just much richer when you can actually follow people around. And whenever I have a captive audience, I like to make a plug for getting more diverse folks into robotics. Robots are going to be everywhere in our homes, all over.

Steffi Paepcke: We’re going to be riding around in them. And if robots are not designed by a very diverse group of people, they’re not going to serve people equally and fairly. And we’re at the point now in robotics where it’s really starting to pick up. And if we don’t have diverse designers working on these challenges now, it’s going to take a really long time to catch up in the future. So there’s a niche for everyone really in robotics. You can come at it from the law perspective, mechanical software, electrical engineering, design, psychology. There’s so many ways that you can contribute to the robotics field. And if you’re thinking about making a change, I really encourage you to consider the robotics field and just getting involved. It’s a really exciting time to be a part of this industry, and that’s all I got.

Suzanne Basalla speaking

Chief of Staff Suzanne Basalla gives a talk on “2020 Olympics Showcases Mobility and Inclusion” at TRI Girl Geek Dinner.

Suzanne Basalla: Good evening everybody. I’m Suzanne Basalla. I’m Chief of Staff here at TRI and I’m going to talk to you about mobility and inclusion and really talk about inclusion from two different lenses. Like the speakers before me, I want to tell you a little bit about my path to TRI, which is a little bit different. I did actually take a fair number of STEM courses when I was in college, but I majored in Asian studies and the reason I had STEM courses is because I joined the Navy right out of college and spent 13 years as an intelligence officer in the Navy. And so I took the engineering and physics classes you need for that.

Suzanne Basalla: And the Navy is what gave me the opportunity to go to Japan. And what you’ll see about my career and what brought me to TRI is I’m very passionate about working with US and Japan, both countries, and bringing the best from both countries to solve problems and the issues that are super important to both of our countries, whether it’s our economy or our national security or issues like that. So the Navy, I was with the Navy in Japan for four years and I really fell in love with Japan. And more importantly, I found my passion, which was to really work at that intersection between the United States and Japan, and committed my career to alliance management, which is really focusing more on the national security side first of the relationship.

Suzanne Basalla: And so through my career I’ve had a chance to work between Japan–Tokyo and Washington, DC, mostly on the relationships. So I had a chance to work for brief time at the White House, worked at the department of defense where I was the Japan director working on our defense relationship. And then I also had a chance to serve as the Senior Advisor to our ambassador in Tokyo. And that was a really pivotal time in my life because I was in Japan on March 11th, 2011, which if you may remember, was the triple disaster of the earthquake, tsunami, and the nuclear disaster and was part of the US government’s response on that.

Suzanne Basalla: But I actually then really realized that I wanted to get involved in the nonprofit side of… I’d gotten [inaudible] nonprofit in order to really help the people because fundamentally it’s the people of our two countries that make the relationship strong. So I spent five years as the COO, EVP of a nonprofit, the US-Japan Council. And the bulk of my work that I was doing was helping the next generation for US-Japan relations, particularly a lot of work on women and girls empowerment, which was really exciting for me. Now working in US-Japan relations, Toyota… I, of course, got to know Toyota. Toyota is a global brand.

Suzanne Basalla: If you’re in Japan, Toyota is such a dominant company. There’s even the headquarters is in Toyota City, which gives you a sense of how important Toyota isto Japan. It’s also really important the United States. We have plants in 10 states in this country and many, many jobs are created through Toyota. So when I was invited to come and work at Toyota, at TRI, for me it was a huge opportunity to continue to my work on really solving the most important problems before our countries working in an exciting space of AI and thinking about the important economic and social issues from robotics and automated driving that you heard about earlier tonight.

Suzanne Basalla: So I am Chief of Staff and a lot of people ask me, “What is Chief of Staff?” And I usually tell them that means I’m a Jack of all trades. But today I’m excited to say I’m a Jill of all trades. And so I want to talk to you about two areas that I get to focus on, what my job is to really follow the priorities and strategic issues that are important to my CEO. And the two I want to talk to you about, one is the Olympics. So I hope you all know that the Olympics going to be Tokyo next summer, Tokyo 2020, and Tokyo plans for that to be the most innovative Olympics in the history of the Olympics. You may or may not know that Toyota is the most… The largest sponsor of the Olympics and Paralympics and Special Olympics in history.

Suzanne Basalla: And we’re going to be sponsoring the Olympics all the ways to at least 2024, the Paris Olympics. Here at TRI we’re really privileged because we’re working on a lot of the technology that’s going to be shown in Toyota’s demonstrations at the Olympics. So the bulk of the driving team and much of the robotics team and others in the ops and PR part of TRI are focused on getting ready for Toyota’s presence at the Olympics next year, which is really exciting. My job as Chief of Staff includes kind of helping the leadership team organize across the company so that we are prepared and doing what Toyota needs. And then kind of my sweet spot is engaging with the external stakeholders, especially in Toyota.

Suzanne Basalla: But I also get to work with the Olympic committee. I get to work with other companies that are sponsors, such as Visa or Intel. I also get to work with the broadcasters, and really cool, as occasionally I get to interact with some of the athletes and the Paralympians and they’re super inspiring. Toyota is sponsoring the Olympics because it wants to transform ourselves as a company into a mobility company. And what I’ve been really proud of is, through my work, seeing how much Toyota is using its sponsorship to lift the Paralympics and really focusing on the topic of mobility for all, which aligns with the things you’ve already heard tonight, and accessibility and inclusion.

Suzanne Basalla: The second area that I get to work on is be a champion for diversity and inclusion in the company, which is very important to our CEO. And you heard from Kelly who… Our Executive Vice President. Diversity and inclusion is important throughout the company. It’s not my job to do it. I’m the champion for it within the company, but it’s the responsibility of all the leaders. And you saw that all of our leaders from… Who are here in the headquarters today are here showing their support and that is very typical of the company. But I have the privilege of being the champion for diversity and inclusion in the company, which means I get to be in a lot of different conversations and continue to help us to think about how we can do better and do more.

Suzanne Basalla: Because we want to create as inclusive environment as possible here to attract diversity and get the most out of diversity that’s here. People call employee resource groups different things at different companies. But we at TRI have started three employee resource groups and these are initiated by employees. We’re still a pretty new company. I don’t know if that actually came across in the speeches yet, but we’re pretty new company. We’ve only started our ERG program just a little over a year ago and we’re really excited that we have three employee resource groups already up and running. One is Women and Allies. Yeah. One is LGBTQ, sorry. Plus.

Suzanne Basalla: And then one is Parents, which is our newest ERG and were just started and it’s focusing right now mostly on new parents or parents, young children, but bringing together people who are facing issues as working parents. And so we call these resource groups for a couple of reasons. First of all, they’re a resource for the members of the affinity group that belonged to it. It’s a place for them to get together and share their concerns and work together to find creative ways to address those concerns. They’re also a resource to the TRI leadership, to the CEO and the leadership team because it gives us a way to hear about what are the issues that are important to those communities and they are a resource for the company.

Suzanne Basalla: Because they advance a caring community in a respectful environment in the company, which aligned with the values that Kelly talked about earlier. So hopefully you’ve had a chance to learn about some of the ERGs. If not, ask us more questions. We love to talk about the activities in the ERGs but of course, part of the emphasis we place on diversity is why we’re so pleased to be hosting Girl Geek X tonight. We are so grateful that you’re here tonight. So I want to thank you. I’m going to turn it over, I think to Rita for some final remarks, but thank you very much.

Rita Yau: Thank you. Did all of you guys have a good time tonight? Yeah. Awesome. We again, we are so thankful for Girl Geek. Thank you guys for coming tonight.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Mode Girl Geek Dinner & Lightning Talks: “Limitless” (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Meeting people is fun and easy at Mode Girl Geek Dinner in San Francisco.

Meeting fellow girl geeks is fun and easy at Mode Girl Geek Dinner in San Francisco’s Design District.  Erica Kawamoto Hsu / Girl Geek X

Transcript of Mode Girl Geek Dinner – Lightning Talks:

Heather Rivers: Oh my gosh, there are so many of you here. This is very exciting. Welcome to our very first Girl Geek Dinner, I am Heather. I am Mode’s CTO. Let’s see if we can get the technology to work. Yup, there’s me. Yeah. When I started at Mode, I didn’t have gray hair. This is dyed so long ago. So yeah, really, really excited to be hosting. You may have noticed that the theme for tonight is limitless. That can mean a lot of different things in different contexts.

Heather Rivers: Let’s just do a quick poll, why do you think we chose limitless? This is room of self-selected geeks. I am also a geek. Who here thought we meant the SQL LIMIT keyword. Anyone? Okay, not too many. Yeah, we got some Mode employees, definitely. Okay, raise your hand if you thought we meant the Bradley Cooper media franchise? Yup, okay. Yes, you’re all correct. We meant both of those. We also meant it in a third way.

Heather Rivers: So in 2008, at the Democratic National Convention, Michelle Obama famously said, “The only limit to the height of your achievement is the reach of your dreams and your willingness to work for them.” You’re about to see seven incredible women who fully embody this quote every day. I work with them, so I can say that.

Heather Rivers: Nobody joins a startup because it’s easy. Some startup people in the audience? Yeah. Any of you join startups because it was easy? No? Cool. Yeah, me either. Or obvious, or because success is guaranteed? No, not seeing a lot of yeses there. So, you join a startup because your dreams are high and because you’re willing to work for them. That’s what all of… Oh, it’s lo-res, sorry. Enhance. Enhance. No, enhance. Okay. Technology.

Heather Rivers: That’s what all of these women have done, along with the rest of the team day by day. They took a chance on the startup, they dreamed big, they worked hard, and as a result, they’ve set both themselves and this company on an incredible growth trajectory.

Heather Rivers: So in the six years that I’ve been in Mode, again, the hair. I’ve seen it go from a pre-seed proof of concept, in a super crowded market, by the way, to a simple but promising little app with a few customers, to a real product with traction and revenue, to a leader among data science platforms.

Heather Rivers: And it’s been really exciting to watch us win power users among data scientists and analysts, but we’re not done. There’s still so much more we can do. We don’t have to limit ourselves to just serving data science teams.

Heather Rivers: And that’s why just a couple weeks ago, we launched the latest step change in Mode’s trajectory. We call it Helix. So Helix is an instant responsive data engine that lets not just data scientists, but anyone, run analysis on huge data stats, up to 10 gigabytes at a time. All in the browser, and all without writing a single line of code.

Heather Rivers: Helix lets you explore your data without limits, SQL or otherwise. And I can’t be 100% sure, but I’m pretty sure that’s what Michelle Obama was talking about in her talk. Don’t try to look that up, that’s not verified.

Heather Rivers: So, in one way or another, everyone you’re about to hear from played a huge part in building, launching, and supporting Helix. So let’s give them a huge round of applause.

Kaitlin Hart speaking

Senior Enterprise Account Executive Kaitlin Hart gives a talk on “Sales is Life, The Rest is Just Details” at Mode Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Kaitlin Hart: I’m super excited to be here, this is my first Girl Geek. I’d like to kick this off with a really quick question. Who here is in sales, can you raise your hands? Okay. All right. You saw the topic title there, I see that.

Kaitlin Hart: I want to start by exploring why I believe everyone in this room should raise their hands, but first I want to start with a little confession. I’ve been field testing this talk for years, usually in ride shares, as weird as that might sound, but it happened just yesterday. So it’s still very relevant.

Kaitlin Hart: And it happens when people ask, “What do you do?” And I tell them I’m in sales. What happens next is this crash between perception and reality that I get to explore for whatever the duration of the ride might be. Because culturally, we perceive sales to look like this. Or maybe this.

Kaitlin Hart: And we associate salespeople with the traits of being aggressive, competitive, maybe selfish and untrustworthy. And apparently, as men with slicked back hair.

Kaitlin Hart: So while it’s nice to know I’m not any of those things, it’s kind of sad for me to hear because none of these things reflect what I love about sales. In reality, sales to me is much more like this. Two strangers trying to devise a plan. A parent listening to their child’s problems. Or two robots trying to form a relationship.

Kaitlin Hart: I know exactly what you’re thinking right now. These looks like everyday interactions, except the robot part, bummer. And you’re absolutely right. There’s a ton of research on how sales and life are intertwined. Daniel Pink is one of those folks, you don’t have to take my word for it. He wrote a book on it, and he said, “If you spend time persuading, influencing, or convincing others, you’re in sales.”

Kaitlin Hart: So, regardless of what business unit you’re in, you might be a PM, you might be in Dev, you might be in marketing, doesn’t matter. Because about half of your job is still spent on sales related activities.

Kaitlin Hart: So, are you reconsidering yet keeping your hand up? The point here is that sales is just life. You don’t need a special degree. We don’t need to learn any special language. And forget about it being your job title, it doesn’t even need to be in your job description. That’s how ingrained it is into your everyday life.

Kaitlin Hart: Basically, you’re all in sales, and now hopefully you know it, congrats. Comp checks are going to be at the end, [inaudible]. But we’re not going to end there. Because the details are also really important. And what really separates us is how we spend our time and focus. I spend my time focusing on developing interactions and trying to make them more effective. You probably spend your time on something else. And maybe, until one slide ago, you didn’t even know you were in sales. So that’s okay, I’ll give you a pass.

Kaitlin Hart: In the meantime, I’d like to help you get up to speed by sharing some specific skills, aka details, that we know lead to success and growth over time. And I’m not just saying this, we have data to back it up. It’s called revenue. So let’s just dive in.

Kaitlin Hart: The first one is being curious, and this one’s super close to my heart because I was born curious. Over time when I started my career, I realized this was much more of a skill than it was a trait. Because when you approach conversations with a genuine curiosity, people feel that. And when you learn, or when you ask questions that are based on understanding them, and then you listen to their responses instead of thinking about your responses, there’s this feeling of trust that’s built in your conversations.

Kaitlin Hart: And then to take it a step further, you’re going to replace judgment with curiosity wherever possible, and you’re going to avoid assumptions by, again, being curious instead of diving into your assumptions. And this is both for your career and your personal lives. Knowing nothing about someone, this is how you build a relationship rooted in respect right out of the gate. If I don’t know you, but I ask you questions that are thoughtful, and I ask and I listen to understand as opposed to respond, that’s the start to a very fruitful relationship. And then you practice this over time and you see as it grows in other areas of your life.

Kaitlin Hart: Another thing people in sales love, plans. We have account plans, territory plans, comp plans. Name it, we probably have a plan for it. But what we know is that there’s no such thing as a perfect plan. So instead, I like to take the approach of being prepared. Because when you think of being prepared, you can think of it as an outline as opposed to a filled out plan to perfection.

Kaitlin Hart: So, as you outline what it is that you want to achieve and you think about your desired outcomes, think about the how. And then you adjust by collaborating with others, being flexible to changes as they might come, and over time you learn. It’s definitely okay to fail here. That’s part of the learning process. And over time, you’ll naturally learn what leads to more successful plans and you’ll be able to grow from there.

Kaitlin Hart: And then third, we have storytelling. Anyone here read Sapiens? Or listened to the audiobook, that counts. Okay, cool. So basically, Yuval says that stories are the reasons humans rule the world. And he even says that society was built by stories. So look at politics, religion, societal norms. And so without stories, we’d be living in a very different world today.

Kaitlin Hart: But the reason stories are powerful is because they tap into emotion or imagination. Data and facts simply can’t do that. But you don’t have to take my word for it, I have a couple examples for you.

Kaitlin Hart: Here’s an ad that uses facts. Okay, this is a shoe that is breathable and supportive. How does it make you feel? Let’s compare. Here’s an ad that uses a story. Note that there’s no features, there’s not even a product clearly defined here. They’re 100% relying on storytelling and the feeling this imagery plus words are making you feel.

Kaitlin Hart: Maya Angelou actually said it best. It’s not about what you feel, I think I’m missing a slide here, that’s okay. It’s not about what people say that you remember, you remember what people make you feel. And when you think about telling a story, then you should think about how it is that you want to make someone feel. Because there’s a lot of power there.

Kaitlin Hart: And so in order to do that, you just apply this really simple framework. Know your audience, have a clear point, and use either emotion or imagination to deliver a why that connects with the audience. Then you practice. And then you field test, I hear ride shares are great for that. And over time, you’ll discover how to deliver your own powerful stories.

Kaitlin Hart: Ultimately, my hope is that you adopt curiosity, preparedness, and storytelling, and then you develop them over time, both separately and together, to unearth your own limitless opportunities. And selfishly, maybe next time when someone asks if you’re in sales, you’ll raise your hands. Thank you.

Heather Rivers: All right, next up we have Senior Product Designer Sam Novak.

Sam Novak speaking

Senior Product Designer Sam Novak gives a talk on “R, Rice Chex, and Re-usable Frameworks” at Mode Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Sam Novak: Hi everyone. I’m excited to be here today. I’m going to be talking to you about rice Chex, R, and reusable design frameworks. Here’s a photo of me with lots of pixels, but to really set the stage for this talk, let’s go with something a bit more historically relevant. There we go. Let me paint the scene. But be warned, you might need to prepare yourselves for a bit of nostalgia.

Sam Novak: The year is 1996. Pokemon has just been introduced to the world. Independence Day is the largest grossing film. The Macarena is a song and dance beloved by all. And I am almost seven years old. My favorite food is, and I quote, “white rice with butter.” But when I’m not eating buttery carbs, you can find me playing video games on my Windows 95. But alas, this story is not about me. This story is about cereal. Chex cereal.

Sam Novak: In fact, I was merely one of about six million children to fall in love with one of the most ingenious computer game strategies in all of history. That’s right, I’m talking about Chex Quest. For those not familiar, Chex Quest was the largest single mass replication of a CD-Rom ever. 6,000,000 free video games delivered as prizes in boxes of Chex cereal. How did they do it?

Sam Novak: The team was six people. The budget, $500K. The deadline, six months. They were tasked with the objective of creating an educational video game with the ultimate goal of reinvigorating the Chex cereal brand. So they set off to invent a video game from the ground up, to teach users about Chex cereal that kids would want to play. In six months, no big deal.

Sam Novak: The original game concept involved navigating around a cornfield with a flashlight, looking for ghosts. But despite their efforts, the game was just not landing with children. Until leadership came to the team and said, “Look, you’ve got 24 hours to come up with a better idea.” Enter Doom. For context, Doom is a first person shooter game that had been released three years prior. The style of gameplay was really landing with kids, and even today it is still often cited as one of the greatest video games in history.

Sam Novak: Now you may have heard this phrase. “Good artists copy, great artists steal.” Well, the team did just that. They relicensed the Doom engine to build Chex Quest. Now, the Doom team was pumped. They actually thought this was a really creative use of the engine. And the Chex team was happy. It was Doom with a facelift. The gameplay was largely unchanged, and this decision sped up the decision making process tremendously. They were now able to focus on creative ways to make the game nonviolent by redesigning the weapons, and by having the main character, yes a piece of Chex cereal, save the world by sending aliens back to their home planet.

Sam Novak: Everything started to come together. Finally it was time to release it to the world. All six million copies sold out in 6 weeks. Chex cereal sales went up 248%. It received major awards for advertising effectiveness and promotional achievements, and despite a bit of initial heat from video game critics, it developed this huge cult following really quickly. All in all, the project was a hit.

Sam Novak: The thing I love about this story is that the team had no pride or fear around leveraging existing technology. And reusing a style of gameplay that was already resonating with children. And as a result, they ended up creating something pretty inventive and magical. By applying this huge limitation, the results became that much more limitless. “Good artists copy, great artists steal.”

Sam Novak: But what does any of this have to do with modern product design? Well, there seems to be this never-ending debate in design that if you merely copy what others are doing around you, you will never truly innovate. And yet, here lies Chex Quest, one of the most innovative promotional strategies of the 90s. So how can we reconcile these points? How can we take this success story and apply it to modern software development?

Sam Novak: After all, relicensing a video game engine isn’t exactly the same thing as copying an interface, and stealing the user experience workflow. But what if it was? What if we weren’t afraid to get up here and talk about our justifications for stealing, when it led to great design schemes?

Sam Novak: So I’m going to use one more recent example from [inaudible]. The introduction of the [inaudible] notebook interface. I’ll justify stealing from two angles. First, you need to have the right intent. And second, you need a goal of building user trust. Are you ready?

Sam Novak: The timeline for the R project was three months, which was a super aggressive deadline. And the goal was to add support for R, a statistical programming language, in addition to Python while fitting in as many design improvements to the interface and experience as we could. Make no mistake though, this was a redesign of our notebooks. A redesign that would involve a fair amount of stealing.

Sam Novak: So my first justification for stealing is having the right intent. What do I mean by that? Well, you could have argued that our goal was to simply add support for R to our existing UI, but in reality RStudio in IDE was far more popular than writing R in a notebook interface among our user base. So in the same way the Chex team looked to Doom, we stepped back and asked ourselves, why do people love RStudio so much, and how can we recreate some of that passion in Mode? So we asked. Not what features do you like, but what makes RStudio a great experience for you? We documented ideas that were resonating and time and time again, in-app documentation came up as being particularly valuable. So we built in-app documentation. It didn’t matter that our interface wasn’t the same as RStudio, or that they had built documentation first. Adding documentation was just a clear user improvement. Now, the intent here was not to check a box. It was to help both Python and R writers learn about having to leave the context of our notebooks.

Sam Novak: My second justification for stealing is building user trust. Predictability and dependability are two of the largest foundations of building trust with your user base. Now, our old UI resembled a notebook, yes. But it didn’t look or work much like Jupiter Notebooks, the most widely adopted notebook interface. And as a result, the switching costs and the cognitive load, the mental energy required to learn our notebooks increased. It felt different, it looked different, and that difference didn’t necessarily lead to immediate user trust.

Sam Novak: Now, imagine trying to get a ride at the airport in a hurry, switching over from Uber to Lyft and having to learn an entirely new paradigm. But you don’t need to do that, it’s extremely easy to jump between the two. The point I want to make here is that there are workflows and patterns out there that are understood, that are resonating with users. You should have really strong reasoning to completely reinvent something new. Significant change will almost always increase the cognitive overhead required for users to adapt a product.

Sam Novak: What I’m not saying here is that there are never good reasons for introducing newer, better ways of doing things because of course there are. What I’m talking about instead is avoiding an NIH, an acronym that stands for “not invented here” syndrome. Don’t be afraid to reference design patterns that are working well just because you yourself didn’t design them. So, we re-skinned our interface to make it more trustworthy. Better accessibility, better usability, and frankly a familiarity you should come to expect after using other notebook products.

Sam Novak: So in closing, I would challenge you to keep these two justifications in mind when you’re looking to steal. First, don’t just steal for the sake of stealing. Your aim is not to win a feature [inaudible] contest or skip the design process altogether. The goal is to recognize great ideas and innovate on them. Your intent should be to learn and improve. And second, know when to steal. Borrow when it helps to build user trust. By creating something that feels familiar, dependable, and predictable, you reduce both cognitive load and switching cost to your platform.

Sam Novak: And finally for the sake of innovation, I’d like to make one slight improvement to Pablo’s phrase. Good artists copy, great artists steal, but the best artists eat Chex cereal. Thanks, y’all.

Heather Rivers: I will immediately apply the lessons I just learned and steal the mic to introduce our People Operations Partner, Josee Smith.

Josee Smith speaking

People Operations Partner Josee Smith gives a talk on “How To Ruin Your Team’s Effectiveness in 5 Easy Steps: A Guide To Eliminating Psychological Safety” at Mode Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Josee Smith: Hi everyone. My name is Josee. As Heather said, I’m the People Operations Partner. Today I’m going to talk a little bit about psychological safety. So here at Mode, I spend my days empowering our managers and building out people programs. In that work with my managers, I bet you all can guess the number one question that I get from our managers here at Mode. How can I make my team less effective?

Josee Smith: Okay, quick story. So, before coming to Mode, I worked as a paralegal at a law firm. And as part of this role as a paralegal, we would have performance reviews every six months with the HR manager. So she would go around to all the different attorneys and ask for feedback on our work, then we’d go over it during the performance review.

Josee Smith: So in the session, she proceeded to tell me about a mistake I had made about four months previously on a project I wasn’t working on anymore. I had some follow-up questions for her, such as was I still making the mistake, did other attorneys think I was making this mistake, or more specific details about the mistake. And she gave me nothing. She had no additional information. And I was really frustrated because here was this person sitting there telling me about this problem, this mistake I had made, but not giving me any information to adjust it or feel like I was being set up for success. I lost a lot of motivation in my work because I felt like they weren’t trying to help me be better at my job.

Josee Smith: So this brings us back to this question. But clearly, this is not what we’re going to talk about today, because no one wants to be less effective or less successful. But I can guarantee that there are companies out there doing things to make their teams less effective.

Josee Smith: So just some ideas of what this can look like. Asking employees for feedback, and then doing nothing with that feedback. Who here has experienced that before? Okay. Making big changes, and then not informing employees affected by those changes. Who’s experienced that? A few more people. My personal favorite. Inconsistent, vague feedback. Anyone, anyone? I think we should all, should all put our hands up. Because I think this is something, it’s a serious problem. A lot of us have gone through some of these things. I’ve experienced a lot of these things, including at the law firm, and I’m no longer at those companies because these actions not only make teams less effective and less successful, but they’ve been shown to drive away diverse talent.

Josee Smith: We’re a values driven team here at Mode, and underlying a lot of those values, there’s this idea of psychological safety. So, for those who haven’t taken a psych 101 course or if you don’t work in HR and think about this all the time. Psychological safety is created when team members feel comfortable taking risks and being vulnerable with each other. Here at Mode, we also see it being created when team members feel comfortable bringing their whole selves to work. Of course, in a way that is respectful of their teammates.

Josee Smith: A climate of psychological safety makes it easier for people to speak up and share their different thoughts and perspectives. And not feeling comfortable sharing your thoughts, or not feeling safe in that environment to speak up, can be a powerful barrier to collaboration and good decision-making. Psychological safety is particularly important in regards to underrepresented groups as a lack of the safety can lead to the kind of undermining behaviors that can drive these groups out of tech, such as feeling excluded from meetings or social events, feeling talked over, or feeling like your thoughts and perspectives aren’t being heard.

Josee Smith: A lot of research has been done on this topic, including a 2015 report from Google summarizing their findings from a two year study of their highest performing teams. And so I’d like to go over some of those traits. At a high level, successful, psychologically safe teams foster curiosity. So just encouraging teammates to study topics outside the scope of their role.

Josee Smith: Taking and encouraging risks. Skydiving, that is me up there. It doesn’t always have to look like skydiving, of course. It can be starting a new project that might fail for the sake of learning from it.

Josee Smith: Promoting respect throughout your company and your team and being thoughtful about how teammates talk about each other. And it also looks like encouraging candid conversations, such as managers asking employees for feedback and then actually doing something with it.

Josee Smith: So, as I mentioned, we’re a values driven team here and I see psychological safety being created through some of those values. I’d like to focus in on one specific value that has been instrumental to my success here at Mode. Honest words, kindly delivered. So I’ve been at Mode for about two and a half years. In that time, I’ve had the same manager, her name is Bailey. Maybe you’ve talked to her tonight. And one of the many great… Obviously one of the many things I can count on from her is consistent, constructive feedback. I know that as soon as I make a mistake, but also as soon as I’m doing really well, she’ll tell me about it because it’s important for her, it’s important to her to make sure I understand how my performance is doing. And that makes me really happy. My performance is not a secret to be talked about every six months.

Josee Smith: Okay, so, you might be sitting there and thinking, “Well great, Josee, that’s excellent for Mode. So happy for you that you found this place, but how do I practice it? How can I go about creating a more psychologically safe team?” Don’t worry, I have some tips. Here at Mode, we make it a habit of appreciating when someone is vulnerable. It can be hard to express yourself and take risks, especially if you don’t know how it’ll be received. So, when someone speaks up in a meeting when they’re normally silent, or if someone says they’re nervous about a project or presentation and then they go in and then absolutely crush it, give them some kudos. Let them know that you appreciate their efforts and you’re proud of them for stepping outside of their comfort zone.

Josee Smith: So I learned this next tip from Heather, actually. She’s somewhere. Oh, there she is. Be mindful about meetings. Not everyone likes to speak up during meetings, nor should they have to. So pay attention during meetings to who is and isn’t speaking, what is and isn’t being said, and encourage your teammates to expand on their thoughts. Consider sending a follow-up message after the meeting summarizing your thoughts, and asking your teammates to chime in with their opinions. You might unearth a perspective that didn’t come through during the meeting, but could be vital to the task at hand.

Josee Smith: So in my experience, the number one way to create a psychologically safe environment is to change your mindset around failure. To some, failure is the worst possible outcome and something to be avoided at all costs. In a psychologically safe environment however, failure can instead be viewed as a stop on the road to success or as something to learn from. So, when considering how failure plays out in your own work, don’t view it as something to be avoid, just the worst thing that could possibly happen. But instead, think about how it can be something to learn from or how it can get you one step closer to the right solution to a tricky problem. Sometimes, you have to fail to get there.

Josee Smith: I encourage all of you to think about how psychological safety plays out in your current workplace. Do you think you could bring up the topic or these ideas with your manager? If you don’t feel like you could bring this up because you think your manager won’t listen, or you worry they’ll think you’re a low performer for caring about this subject, think about how, what kind of workplace you’re going to thrive in, and what role psychological safety will play for you, like I did at the law firm. And, if you feel like you’re not getting anywhere, that your manager isn’t listening, lucky for you, Mode is hiring. Thank you.

Heather Rivers: We have one last talk before a quick break, and this is from Back-end Engineering Manager Max Edmands.

Max Edmands speaking

Backend Engineering Manager Max Edmands gives a talk on “Constructing Feedback Loops for Fun and Profit” at Mode Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Max Edmands: Hi. This is me, kind of. It’s a really pixelated version of me. This is an animation of someone pouring milk for a cappuccino, but this talk is not about being a barista. This is actually a talk about feedback loops. So, I want to start by breaking this term down a little bit.

Max Edmands: Feedback generally means conversations between humans. Someone noticing something about what you’re doing and giving you an opinion about it. Totally true, but feedback is actually more than that. Here is an example of extremely valuable feedback. As the barista’s pouring, they can see and feel the results of what they’re doing. Are they keeping the espresso crema intact? Is the milk the right ratio of liquid to foam? Is the pattern on the top of the milk glass the pattern they wanted to create? Are they filling the glass to top? Is everything the right temperature?

Max Edmands: All of this can be generalized into three attributes for great feedback. One, great feedback is high bandwidth. Lots of information coming in as quickly as possible. There’s the weight of the cup and the milk jug. There’s the temperature of the liquid in the cup. There’s the pattern that the milk is making on the surface. There’s the sound that it makes when it’s pouring. There’s so much there.

Max Edmands: Two, great feedback is relevant. There’s very little distraction here. Everything is signal and there’s no noise. You’re seeing it and you’re holding it, and all of the senses you’re getting are relevant.

Max Edmands: Three, great feedback is timely. It’s actually all in real time. The barista can change the angle of the cup and immediately they see a change in the surface area of the crema and the resulting change in the milk pattern they’re creating. So that’s feedback.

Max Edmands: Now let’s talk about feedback loops. A feedback loop is when you can take the feedback you got and try to use it again, or sorry, use it to try again. But this time, a little bit more effectively. And then use the results to get more information and then do it over and over again.

Max Edmands: So, there are a lot of great examples of feedback loops in video games. This animation is from a game called Celeste. Definitely recommend this game, by the way, it’s super great. Every feedback loop follows five steps.

Max Edmands: So step one, identify a goal. In this case, the goal is get the strawberry and bring it to the top left hand, right hand corner of the screen. Two, take concrete actions toward that goal. So jump on the block and ride it to the other end, and then try and jump off of it onto the platform and, oh no, falling into the spikes. So three, step three, evaluate your feedback. Ask yourself questions like, what did I learn just there? In this case, really clear information, if you jump in that way, then you’re probably going to fall onto the spikes and that’s bad.

Max Edmands: So then four, adjust your approach and try again. Maybe this time let’s try a dash jump when we’re in the air so we jump a little bit higher, so we can get onto that platform. And then see what we can do when we’re up there. And it works. Cool. Do it over and over again until we reach the goal. But now we’re on the platform, we have to figure out what to do next. So now, probably, we’re going to start a new feedback loop with a slightly different challenge. Great. So that’s games.

Max Edmands: But what about stuff you’re probably doing every day? Here’s an at-work example for those of you who write code. Test-driven development. So, here we have two sides of the screen. One of the sides has test results, and the other has the code. We’re adjusting the code in order to make the test suspect something that isn’t true yet. Then, we’re adjusting the code to make the test pass again, and then repeat. We know that something’s, I guess, needing to be fixed when stuff is showing up in red, and we know that stuff needs to be made to fail once stuff is showing up in green. It’s a very clear set of what do I do next.

Max Edmands: There’s another, smaller loop going on there too. As we’re editing the code, the editor is underlining certain things in red to let us know that the syntax isn’t quite right. The moment that we finish typing a line, or the moment we fix the syntax error, the red goes away to let us know it’s fixed. And then repeat.

Max Edmands: Working together with other humans is another great way to create an immediate feedback loop. I think this is a super cool photo. Two early programmers, collaborating on one of the world’s first computers. Early pair programming. I’m not 100% sure what they’re doing here. In my imagination, Esther is holding a specification that says what patch cords need to be connected to which ports. She’s reading the list out to Gloria, giving her time to connect or verify each one, and probably doing a visual check too just to make sure. Esther’s also got a bundle of extra cords ready for when they move onto the next one.

Max Edmands: So together, they’re able to keep track of where they are and move from one step to the next. They’re much more likely to notice and correct mistakes early. They’re somewhat less likely to get distracted, since they’re both concentrating on the same thing at the same time. And they’re way more likely to come up with new approaches or make adjustments to their process as they go.

Max Edmands: Which brings me back to conversational feedback. Retrospectives, one on ones, coffee walks. Words are an incredible way to fit lots of information into a really small space. Setting up regular places to have more of those conversations between either teams or between individuals, you and your manager, you and a peer, gives you way more opportunities to get and give feedback.

Max Edmands: So, how do you go from no loop to a feedback loop? Well first, we have to define the goal. Let’s say I want to draw an owl. So, now we need to figure out how we’re going to do it. I already have a process for drawing an owl. It’s something along the lines of flail along the page with a pen for a while and use a lot of white out. Eventually we got somewhere interesting. If I put it on a timeline, it might look something like this.

Max Edmands: So, next up is we identify specific decision points that’ll get us there. In this case, every time that I’ve scribbled on the page a little bit, I take a step back to figure out where to start adding the next round of details. But which details specifically should I add? This is the perfect place to start getting feedback. So, what feedback would be good here? Feedback could be comparing it against another implementation of the goal and figuring out what tweaks to make. It could also be user testing. Show your picture to another human, ask them what they think. It could also be, try to sell it and see if people will buy it. Would you buy this owl?

Max Edmands: Then iterate. Keep thinking of ways to increase the number of decision points and increase the quality of the feedback you’re getting at each point. Warning though, make sure that the additional process you’re adding is worth the cost you’re paying for it. Too much process is busy work. Nobody likes busy work. Too little process is confusion, doing the wrong thing. There’s a really fine line between the two, and staying in that balance itself is actually pretty tricky. Which is why I recommend, build feedback loops out of the quality of your feedback loops.

Max Edmands: It sounds like a joke, but I’m being completely sincere. The best way to figure out if you’re balancing cost versus benefit of process is to think about the process in exactly the same way that you’re thinking about the thing that you’re doing. Be continuously learning if there’s too much or too little, and be continuously adjusting as you go.

Christin Price speaking

Senior Manager, Business Strategy & Operations Christin Price gives a talk on “Ops, Table for 1” at Mode Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Christin Price: Hey everybody, thanks for coming out tonight. I am Christin Price, and I work in the finance and operations department at Mode. Tomorrow actually happens to be my two year anniversary. So, I joined Mode shortly after we raised our series B and I was our first in-house finance hire. And that sounded extremely cool to me.

Christin Price: So at the time, Mode was experiencing some of the typical growing pains you might see at a series B startup. For example, we’d grown out of our office space. At the same time, I was going through some of my own growing pains. I was getting whiplash from how quickly my job title kept changing. I went from leading an annual planning cycle to doing a deep dive audit on a revenue number to prepare for a series C, and I even inherited a sales ops function.

Christin Price: As these demands kept mounting, I felt like I always needed more time or more people to get anything done. Everything felt like a fire, and I didn’t feel like I was getting to do my best work. As this persisted across multiple months, I began to wonder if I’d made a mistake. My career trajectory felt like it was getting buried under the number of tasks it took just to keep the lights on.

Christin Price: Historically, I may have taken this as a sign that Mode wasn’t invested in me. And a mentor challenged me on this line of thinking. She asked me if I knew what the difference was between a fixed mindset and a growth mindset. A fixed mindset was believing that the situation was permanent, and that I had no power to influence or mold it into something different than what it was. Whereas, a growth mindset was to develop a true love of learning and believe that the best learning opportunities are presented through challenges.

Christin Price: Here are some examples of fixed versus growth mindset. Wow, that feedback really hurts my feelings. I’ve been working on this for months and clearly I’m not valued here. Versus, that’s an interesting perspective this person brings. I wonder if I incorporate that feedback into my work how it will change my work product. Or, I’ve never gotten along with this person and we just shouldn’t work together. Versus, these are this person’s strengths and these are my strengths. I’m really interested on iterating on them together to figure out how we can best work together.

Christin Price: So tonight I’d like to share a framework with you that I use to develop a growth mindset while also ensuring my career trajectory doesn’t get buried beneath the day to day. First off, do I have an executive sponsor? A mentor is someone we rely on and learn from their experiences to shape our own viewpoint. A sponsor is someone who will fight for us behind closed doors. I encourage you to ask your direct manager to be this for you. Ask them what would it take for you to have zero hesitation fighting for me?

Christin Price: The second question I ask is, am I soliciting continuous feedback? Y’all, feedback is exceptionally hard. Sometimes I feel like I’m walking to the edge of a cliff and asking someone else to push me off. But, with time and practice, I’ve gotten quite comfortable being uncomfortable. The best way to solicit feedback is to make a verbal contract with everyone you work with. Say, “I’d like to solicit ongoing feedback. Are you able to do this?” And then as you work together on projects, check in frequently, and I’m talking a couple times a week, and say, “Hey. What do you think is going well and what could I be doing better?”

Christin Price: Am I advocating for myself? As a society and especially as women, I feel like we’re pre-conditioned to believe that hard work in and of itself pays off. And I haven’t found this to be particularly true. Now that I’m comfortable being uncomfortable, I practice stepping outside my lane. I ask to be in the room.

Christin Price: Last month, there was a strategy meeting about how we hit our revenue number for the remainder of the year. It was 8 pm on a Tuesday night and I was asked to put together a model for the meeting the next day. I did so and I got my boss up to speed on it, and then I thought, “I have a valuable contribution here and I’m an expert on the subject.” I asked to be in the room. Not only did I join the meeting, I ended up leading it and one of our co-founders chased me out of the room with follow-up questions. It ended up being one of my most productive meetings in my two years at Mode.

Christin Price: Asking for public recognition. This past spring, I did a reboot on our commissions policy for our customer success function. And it took a lot of hard work, and the head of that team thanked me, privately, for the work I’d done. I asked him if he’d stand up at our Thursday all hands meeting and give me that recognition publicly. Not only did he agree to this, he thanked me for asking him. These small asks will increase your exposure to others within the organization, and also increase your level of influence.

Christin Price: Building multi-threaded relationships. This is actually a sales strategy. Imagine you’re working a deal, and your single point of contact leaves the company. It makes that inherently risky. Similarly, by building multi-threaded relationships with all different people at all different levels and in all different departments of your company, it ensures that there’s no single point of failure. Our CEO left on maternity leave earlier this year. If I relied exclusively on him to give me a voice within Mode, I would’ve been starting from scratch. Instead, I had many strong relationships to lean on during that time.

Christin Price: Don’t try to be everything to everyone. I had an epiphany about a year ago. I have always considered myself a direct person who establishes clear boundaries, but reflecting on my time, I’d realized I was trying to prove my worth by being a yes woman. Telling people I need more time, or that a project isn’t high priority, and then subsequently not doing all of the late work necessary to find that project a home is a really good practice. Others respect my ability to prioritize, and more importantly, I have the energy to bring the intellectual and emotional intelligence to the work that does fall within my purview.

Christin Price: Am I giving myself room and grace to make mistakes? I had a pretty serious miscommunication with a senior leader at Mode. Instead of accepting that I burned that bridge and beating myself up over it, I decided to apply a growth mindset. I apologized, I collected feedback, and I incorporated that feedback to rebuild our relationship. Today, I can gladly say we have a great working relationship. And furthermore, I don’t regret that mistake because of how much learning I got out of it.

Christin Price: So yes, this framework is a work in progress and yes, it takes serious energy to execute on it every day. And no, by no means have I mastered it. But I choose to apply a growth mindset and believe that with time and practice, I can continue to improve. It will become second nature. And I do truly believe there is something to be learned from every situation, especially the tough ones. And every day, I see the dividends of this practice.

Christin Price: Today, I am no longer on an island, as other people have joined the department. I was promoted to be a people manager, and I even have two open recs to continue growing the team. So, if you are a pace setter with a growth mindset who is hungry to learn and step out of your lane, Mode rewards that. And come find me after, because I want you on my team.

Christin Price: So today, I’m glad to say my relationship with Mode is mutually beneficial. It’s both give and take, and my career trajectory continues to crystallize. And with my growth mindset, I see limitless opportunity.

Heather Rivers: Next up, we have Senior Product Manager Nishi Patel.

Nishi Patel speaking

Senior PM Nishi Patel gives a talk on “Limitless Success: Influencing without Authority” at Mode Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Nishi Patel: All right, hi everyone. As Heather mentioned, my name is Nishi and I’m a PM here at Mode. I’m going to start off with a little story. So, it’s 2013, I had just landed my first PM gig after many interviews and I was so excited. I was bright eyed, bushy tailed, had done all of the reading I needed to do, and walked in on my first day. Within the first few weeks, I had been given the perfect project for our team. I’d been working with the design team, the engineering team, we’d done customer research, we’d built a bunch of prototypes, done testing. And we thought we had the ideal solution to bring our company’s first app to life.

Nishi Patel: So, in the following weeks we were going to have our first planning meeting. The CEO was going to be there, there was going to be a lot of stakeholders, some people that I hadn’t really even talked to yet. But I felt really confident in our solution and I was ready for it to be applauded and praised, and just feeling really good about it. What actually happened was that the CEO, amongst others, questioned every single point that I brought up. He pretty much shot down every single one of our ideas, and he couldn’t really connect the dots between how what we were doing and what we were proposing was going to get us to our revenue goals in three months, which is what was really on his mind.

Nishi Patel: I left, feeling pretty defeated. I went home that night, I remember, and was just circling all of the thoughts in my head and thinking what could I have done better, I thought I did everything I was supposed to do. But what I didn’t realize at that time was that our solution was actually pretty spot on to what we would end up building in a few months. They just didn’t resonate with the audience, and fell flat in that meeting that we had.

Nishi Patel: So what could I have done better? In one answer, instead of trying to explain a bunch of tactics around how we were going to build a solution, I could have used influence. So bringing us back to the point, why is influence important?

Nishi Patel: A lot of us here today are in tech or at startups, or maybe both. And we find ourselves working more and more cross functionally. On top of that, orgs are getting flatter, and so there’s a better chance that we may or may not have direct reports to help us ease into influence.

Nishi Patel: Daniel Pink, which apparently is popular amongst our group of speakers, I’m going to bring him up again. He said that we spend about half of our time at work trying to persuade others to part with resources. Resources in this context can mean time, someone’s ownership, someone’s decision making, or maybe even money. So, if it’s something that we all need to do, what are some ways to get there?

Nishi Patel: I’m going to talk through a few tactics that have worked for me, which is by no means exhaustive, but a few that I’ve had a great experience with. And also I’m going to talk a little bit about why sometimes we fail, and things that we can do to combat those failures.

Nishi Patel: So here’s one of the first influence tactics. Know your audience. I’m sure we’ve all heard this, but it’s something that’s really easy to glaze over when you’re really excited about something. What are the things that they care about? What are the things that get in the way of them doing their job? What are the things that keep them up at night? What are the things happening in their day to day that maybe affect them that you don’t even realize? I think most importantly out of all of this is really understanding how what they want, their incentives, and their motivations, can really align what you’re trying to bring to the table.

Nishi Patel: So going back to my story from earlier. I could have been much better at influencing and getting my message across if I was to understand better who was going to be in that room. I could have socialized the idea beforehand, and probably learned that the revenue goals were huge for our company and I could have better framed my story, to better connect the dots between why our solution was going to get us there and make our users happy.

Nishi Patel: Next, build trust and be vulnerable. This is easy to say, but pretty hard to do. I think the things that have worked best for me are just showing that I care and empathizing with the people that I’m talking to. And most importantly, being vulnerable. Definitely scary at first, but once you learn to put yourself out there, you can really show everyone that you’re talking to that you’re human. Consistently showing up is an amazing way to build trust and show that you care, because people can clearly see it.

Nishi Patel: So something I could have done in that situation is instead of just walking into that room with this really great presentation and this really great solution, or so I thought, I could’ve built a relationship with some of the people that were going to be in that room and really gotten their trust prior to entering and presenting.

Nishi Patel: And lastly, be clear about what you’re proposing. Be clear about how it impacts them, what you potentially need from them, or from my quote earlier, what resources they need to part with, and how it could positively benefit both their day to day and make their lives easier, and benefit the company. And also, stay true to you. If you’re not convinced about what you’re saying, they’re not going to either.

Nishi Patel: All right, so this is all good and great, and you might have even seen some of these, heard some of them, I know I have an inbox full of blog posts and newsletters that I could probably find even more tactics. But, sometimes we have every intention of doing all of these things and we prepare, and our message just falls flat. And we have to ask ourselves, why? So for me, the reality is we get in our own heads. I know in that situation, I was thinking, why would the CEO believe me? What if I fail? I’m new, why would that person even want to believe what I’m saying?

Nishi Patel: And a lot of this is fear of failure, and a lot of this is imposter syndrome. It’s a vicious loop. We don’t want to fail, so we don’t put ourselves out there, and we don’t put ourselves out there so we can’t even set ourselves up to succeed or even to fail. So at this point, we’re kind of just stagnant and we’re not doing anything at all. So if we get in our own way, how can we get out of it?

Nishi Patel: Here are a few things that I’ve come up with. Socializing your ideas. Pressure test your idea, and share it with others. This is a really simple way to start small, especially if you’re not this comfortable with everyone you’re going to be with in that room. And it’s a great way to get a signal of the things that are on people’s minds and how people feel about things. It’s also a great way to get advanced feedback, make sure no one is hearing it for the first time, and also to learn the opposing viewpoints that can help you in advance to shape your message when you walk into that room.

Nishi Patel: At Mode, we have a culture that’s pretty open and we have lunches. And so we all try to eat lunch together, and that’s a great way to have some of these casual conversations. Or, we also do a lot of coffee walks, and this has been a great way for me to kind of understand what’s going on around the company.

Nishi Patel: Secondly, observe and adapt to what works. So Sam talked earlier about not necessarily needing to reinvent the wheel if something works. So if there’s someone that you look up to, or someone in your company that you see that’s really good at influencing or maybe even in your life, build that into how you influence people. Analyze and pick up the things that work and put that into your message. Like Sam mentioned earlier… Sorry, not Sam.

Nishi Patel: Another story with Sam is earlier this year, I was actually doing a talk where I had to really incorporate the audience and really influence them with the case study that I was presenting. And there was a lot of things that I had observed with her when we practiced with each other that I was able to incorporate into my own talk.

Nishi Patel: And lastly, as a PM, I have to put in a shameless plug for learning and iterating. So in true product fashion, learn what works, learn what doesn’t, and iterate on this. And apply this thinking so that you can continuously improve.

Nishi Patel: So if there’s a couple big takeaways, it’s these. Find what works for you, and know that not all of these tactics are equal. There’s not one good formula and perfect formula to use, but the more you put yourself out there, the more you can try and figure out what works for you. And for those times where maybe the message doesn’t land, or you don’t influence the way you want, that’s totally okay. We’re all human, and the one thing that we have control over is that we can always and forever learn and iterate. Thanks.

Heather Rivers: All right, we have one final talk by our director of back-end engineering, Ushashi Chakraborty.

Ushashi Chakraborty speaking

Director of Backend Application Engineering Ushashi Chakraborty gives a talk on “Limitless Growth: Practicing Inclusion in Performance Reviews” at Mode Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Ushashi Chakraborty: Thank you. Hello everyone, Ushashi. I’m going to be talking about limitless growth, who doesn’t want that? Practicing inclusion in performance reviews.

Ushashi Chakraborty: So for those of you who are people managers, hopefully this talk is going to help you practice performing reviews in such a way that they are inclusive and they incentivize engineers with different kinds of stress. For those of you that are not people managers, hopefully this talk sparks an idea that helps you to ask for things from your manager when you’re sitting across them, and being deliberate in performance review.

Ushashi Chakraborty: This talk is going to have three takeaways. Let’s begin with the first one. That’s not a takeaway, that’s me. That’s the takeaway. Every engineer is different. Think about that for a second. Think about yourself, then think about your peers, various engineers that you have worked with. Seniors, juniors. Think about their strengths. Think about their ways of working. You’ll find that every engineer is different from each other even if they have largely to similar strength areas, for example, both of them are good at JavaScript. Even there, you will find nuanced differences.

Ushashi Chakraborty: Very recently, I was at the Grace Hopper conference, and there was a lot of chatter about bringing more underrepresented folks into computer science. It is great that we are at a place where we are encouraging everyone, everyone that is interested to come to this industry. Yet, don’t you think it’s absurd that we still think that people that are good at math and science, or people that are coming from traditional computer science backgrounds are the only ones that can make it well in this industry?

Ushashi Chakraborty: That is a flawed ideology. And if you take that flawed ideology, you are going to have biases. And if you start building a performance framework, you’re going to end up having a flawed performance framework.

Ushashi Chakraborty: Takeaway two. Don’t look at only one type of data. Data is great, but if you look at only one type of data, you will end up incentivizing engineers with one kind of strength. And if you incentivize engineers with one kind of strength, ultimately you will be left with an org where the engineers can solve only one kind of problem. And we don’t want that.

Ushashi Chakraborty: Most reviews that I have been part of have focused heavily on delivery. Things like code reviews, the number of code reviews that you have given. Number of comments, code quality are often given a lot of emphasis. I understand, because there’s a very easy metric to attach to these skills. And they are important skills to have. But sometimes, a different value that you add to an org, for example mentoring and interim. Or perhaps, sitting with your coworker and helping them debug a problem. Or perhaps writing a blog post for your eng blog.

Ushashi Chakraborty: We have to find ways to incentivize those skills, because all of these skills are important to excel as a software engineer. As an engineer myself, I have had reviews where those four skills are put together in one group, one bucket. And these skills are different from each other. And hence, those skills need to be talked about.

Ushashi Chakraborty: Takeaway number three. While a review conversation walks you through how your past performance has been, it is incomplete without a conversation about your future growth. How many of you here have gotten a performance review that scored you as does not meet, or meets, or exceeds? You’re familiar with that framework, right?

Ushashi Chakraborty: Now, something that has happened to me in the past is that I would get a great review that has been, a couple of times, in the past where I have gotten an exceeds. And I would be sitting there across from the manager waiting for the promotion to happen, very excited. Only there would be no promotion, there would be no talk about it at all. And I would be too uncomfortable at that point to ask for it, or ask why I didn’t get it.

Ushashi Chakraborty: When I look back at my career today, I can understand why I did not get it. Even though I was doing very well for my role at that time, I still had gaps for the next level. And hence, while the meets, not meets, exceeds framework is great, and its giving you context about how you are doing, that context is not complete unless you know how far you are from the next step. And hence, managers need to have that conversation when they’re giving you your performance reviews.

Ushashi Chakraborty: So now that we have learned about those three takeaways, let’s talk about how we do engineering performance reviews at Mode. We have adapted heavily from a framework built by Medium called Snowflake. It’s open source, you can check it out. And we rely a lot on robust conversations from managers to employees about their performance, as well as future growth. And we also take into account the inclusivity, such that engineers with different kinds of scripts are able to thrive.

Ushashi Chakraborty: The framework has four main tracks: build, execute, support, and strengthen. We will look at our favorite engineer’s performance review last quarter. Yeah, we are not embarrassed to say we have a favorite engineer. That’s our favorite engineer, Marshawn.

Ushashi Chakraborty: So let’s look at Marshawn’s performance review. So right now, Marshawn has not yet gotten a review, I’m going to review Marshawn very soon. First, let me explain the framework. So, on the right hand side you see a flake. We will start coloring that flake up as Marshawn gets some points. On the bottom, don’t worry if you can’t see, or if you can’t read what they say. The colors depict the different tracks I talked about. The building track, the executing track, the strengthening track, and the supporting track. We’re not going to get into the details of those tracks, but each track has about three to four skills.

Ushashi Chakraborty: Building is all about your code. Executing is everything that you do to get that code to production, for example in project management, communication. Supporting is the skills that you need to be supportive of your team, for example, their well-being. And strengthening is about building community inside and outside, for example, evangelism, recruit, those kind of things.

Ushashi Chakraborty: So each of these skills go from zero to five, and your manager evaluates you on those. At present, Marshawn is at zero and is an Engineer I, and total points zero. We’ll be walking you through two different scenarios of two different personas of Marshawn, and see how Marshawn plays out in these.

Ushashi Chakraborty: In the first persona, Marshawn is now having some depth of skills, or some really good skills around building and executing. You see those colors pop up, those are like getting two and three numbers in those skills. You see now, Marshawn’s title has changed to Engineer II, and Marshawn has 18 total points. So here, Marshawn is getting incentivized because of their deeper skills in building and executing, and they have shown depth in a portion of the flake.

Ushashi Chakraborty: Let’s talk about a different persona. Marshawn as a different engineer. This flake looks different. In this flake, Marshawn has a different kind of skill set. Once again, Marshawn is now in Engineer II with 18 points, but is a more holistic skillset that encompasses larger breadth of the flake. So perhaps lesser on the executing and building side, but still there, decent amount of skills. But they’re also having skills on the supporting and strengthening side of things.

Ushashi Chakraborty: Now let’s go back to those takeaways that I talked about. First, every engineer is different. So we see those personas. Those are real life engineers that we perhaps work with, having those skills. And now we are learning how to incentivize all kinds of skills while also having a way of our framework where we can provide feedback for the other kind of skills that they need to grow or hone.

Ushashi Chakraborty: The next thing, don’t look at only one type of data. Had we only focused on [inaudible] focusing on the blue and green that is the building and executing skills, and then Marshawn in persona two wouldn’t have been as successful.

Ushashi Chakraborty: Takeaway three. The framework should force the conversation about future growth. So in this little block here, the points to the next level which is 18, we see that Marshawn has 18 more points to get to the next level. So while your manager will be having a conversation with you as to how many points you are at today and how did that add up, they’ll also be having a conversation with you about how far you are from the next level and what you need to do to get there. And build this strategy with you to help you progressively get to there.

Ushashi Chakraborty: In conclusion, [inaudible] whichever side you’re sitting on during the performance review conversation, it is a challenging space to be in. I get it. And handling it with inclusivity is going to help you build an org that has all kinds of engineers that can thrive there and have professional growth that is wide, that is limitless. Thank you so much.

Heather Rivers: That was our last amazing lightning talk for the evening, but the party’s not over. Feel free to hang out here until 9:00. If you’re interested in talking to anybody about Mode, we have these green shirts, or if you’re interested in learning more about our product we have a little demo booth over there, very cool. And finally, we’re hiring in all departments, so feel free to ask any of us about our open roles. And yeah, let’s just hear one last huge round of applause for all the amazing speakers.

Mode girl geeks attending

A warm round of applause for all of the speakers at Mode Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Our mission-aligned Girl Geek X partners are hiring!


Intuit Girl Geek Dinner: “Powering Prosperity for Small Businesses” (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!


Transcript from Intuit Girl Geek Dinner – Lightning Talks:

Angie Chang: Thank you to everyone for coming out tonight to another Girl Geek dinner at Intuit. I know we were here five years ago and had a terrific time and this time it’s going to be even better. In case you missed it, there are amazing demo stations with engineers and product managers speaking and giving talks about what they’re working on – and donuts. So, they’re out there and thank you so much to the folks at Intuit for hosting us again. This is an amazing welcome. The campus is even more beautiful than I remembered and thank you so much for bringing these amazing women together.

Gretchen DeKnikker: Hi, I’m Gretchen. In addition to doing these almost every week, which if this is your first event, raise your hand. Okay, so now you know that we do these almost every week, so you’ll be on our mailing list, come check out more of them. We also have a podcast, we’ve done like 20 episodes now and we would love some feedback because we’re thinking about what we want to do for the next season.

Gretchen DeKnikker: So, mentorship and career transitions and switching job function and the definition of intersectionality, like all sorts of stuff. So definitely go check that out and rate it and please give us your honest feedback because that’s the only way we’re going to get better. Then we also have … We’re going to do our virtual conference, our all day online conference. I don’t know how many of you guys have come the last two years but it’ll be just before International Women’s Day on March 6th.

Gretchen DeKnikker: So keep an eye out if you think you might want to have your company participate or if you might want to be a speaker, then start thinking about those topic ideas and then we’ll let you know when we’re sort of ready for all of that. If this looks like a good time, you could do it at your company too. So, just email us or grab us. Our information’s everywhere on the web and we’d love to talk to you about it. So, thank you.

Tracy Stone: Thank you. So welcome to Intuit. We are so thrilled to have all of you here tonight and thank you to Angie and the Girl Geek team for partnering with us on this wonderful event. For those of you who we haven’t had a chance to meet, my name is Tracy Stone and I lead the Tech Women Intuit initiative here at Intuit, and that initiative is sponsored out of our CTO’s organization as an initiative to attract and recruit, retain and advance women in technical roles.

Tracy Stone: So an event like this is so wonderful for us to partner with Girl Geek and to be able to bring the community together as we build connections and empower our women in technology. For those of you that aren’t familiar with Intuit, I hope you got a chance, as Angie said, we had some amazing demos from our technologists of some of the technology that we’re developing as we are in our mission.

Tracy Stone: Our mission at Intuit is to power prosperity around the world. We are a global financial platform company, makers of QuickBooks, TurboTax, Mint, and so I hope tonight you’ll get a chance to learn more about Intuit, about our products and the technologies. In addition, we featured some of our small business customers today, so I hope you got a chance to interact with them, get some of the swag to take home, some cool stuff.

Tracy Stone: So I hope you got a bag and got to take home some of those, some swag and the treats from our small business customers. So tonight we have amazing program in store for you. We’re going to start with a fireside chat with our CTO, and then we have some of our technologists and leaders across the company going to share some of their lightning talks with all of you. In the middle of all that, we’ll offer raffle prizes throughout the evening.

Tracy Stone: So, we’ll go ahead and get started. I want to introduce our first talk, which is a fireside chat with our CTO, Marianna Tessel. Olga Braylovskiy will be leading the chat with Mariana. Olga leads HR for our technology teams and partners closely with Mariana and all of her organizations on talent related items. Mariana is our CTO and as our CTO she oversees technology strategy and leads our product engineering, data science, information technology and information security teams worldwide.

Tracy Stone: She joined Intuit in 2017 to lead product development in our small business and self-employed group. Before Intuit, she was an executive vice president of strategic development at Docker and she’s held engineering leadership roles at VMware, Ariba, And General Magic. We’re so thrilled to have both Olga and Marianna with us tonight.

Olga Braylovskiy: All right, welcome everyone. What a turn out, incredible. Lots of power in this room. So, let’s start, Mariana, with you sharing a little bit of your career journey and how did you get you this amazing role of being CTO at Intuit?


Marianna Tessel speaking

Intuit girl geeks: CTO Marianna Tessel shares her career journey with VP Olga Braylovskiy at Intuit Girl Geek Dinner in Mountain View, California.  Erica Kawamoto Hsu / Girl Geek X


Marianna Tessel: Wow, well, thank you. First of all, thank you everybody for coming in today. So wonderful to see so many geek women, I love it. So, a little bit about my journey. I’m an engineer in my background, so I studied engineering and then I worked as an engineer for quite a long time. I actually started, I’m from Israel. Anybody from Israel here? Whoo, hi, shalom.

Marianna Tessel: So, I started in the Israeli military as an engineer. So I worked there for some time and then I came here to the US after that and I worked in a company that was trying to do devices like iPhones and we weren’t successful, but there was a documentary on the company, it’s called General Magic. You should look it up. It basically says that almost any device today, you can trace back to the roots of that company because the people that worked on iPhone and on Android both came from General Magic.

Marianna Tessel: So that was actually a real pleasure to work with an amazing set of people and doing something so like envelope pushy. Is that a thing? Yeah, pushing the envelope and then just thought I’ll use it in a different way, why not? Then actually I worked in other companies. I worked at Ariba, et cetera, but actually at General Magic I became a manager. I didn’t expect that, they had a need for a manager and they came to me and I said, what me? And then I said, actually, I really like it and I started my leadership career then, and from there on I was a leader in multiple companies.

Marianna Tessel: Then, just before actually coming to Intuit, I spent a long time in infrastructure. So that means that VMware and Docker. VMware by the way, I learned way too much, more than I ever wanted to know about storage, network compute and whatnot, really down in the guts of systems. Then when I came to Docker, that was amazing because that was like a start-up and we were changing the world and that was a really great experience.

Marianna Tessel: But I decided to join Intuit and to your question, Olga, I’m trying to answer your question. To your question, how I became a CTO, I think that I spent a long time as actually an engineer and then I became a leader and kind of grew with different roles, and because I had really different roles, actually I think I had like different perspectives and sometimes it’s luck, sometimes I grabbed the luck when I had it and here I am and I’m really happy about that.

Olga Braylovskiy: Luck is largely about preparation and clearly you had a good one. You should also mention how passionate you are about people and technical talent, which leads us to another question. Intuit is a really special company, a little plug-in for Intuit and we have amazing culture and we have very unique engineering culture that I know you’re super passionate about. So what’s unique about our engineering culture?

Marianna Tessel: Yeah, I think Intuit is really known for its culture and I knew it before coming to Intuit, but I think what’s really unique about Intuit and for engineers here is that you actually get to work on things that are really meaningful for lives of people, and that is a great feeling. You heard our mission is to power prosperity around the world and you saw some of our customers here, and when you work on something that you feel fundamentally touches the lives and people and really help them, that is really, really powerful.

Marianna Tessel: At Intuit, we are very good about looking at it this way and not just building awesome technology, which I’m really passionate about, but also thinking about how it helps our customers. So I think that intersection of really working on great technology and being able to, across the stack, really exercise your craft as an engineer, then working on a mission that is meaningful and then as a company having this culture where it’s really welcoming and nice, and just kind of not that very cut throat, et cetera.

Marianna Tessel: So, it’s kind of unique in its culture of how it’s a very welcoming of people and I think that’s why we actually have also actually high percentage of women relatively because we’re a very welcoming culture where you can feel like you can come in and be yourself, a lot more than I’ve seen in other companies.

Olga Braylovskiy: Awesome, so we use the word awesome a lot and that applies to our culture.

Marianna Tessel: That’s awesome.

Olga Braylovskiy: You touched on amazing, kind of use of amazing technology and being part of our culture. What are some cool and interesting uses of technology, especially more modern tech really as we try to fulfill on our mission of powering prosperity around the world?

Marianna Tessel: Right, and as all you guys you know, we declared our strategy to fulfill that mission to be an AI driven expert platform and what that means … And by the way, this is one of the things about Intuit that I’ve learned. You are very clear about our mission, our strategy, our values. In the beginning, I was like, wow, that’s really heavy, but I actually learned to really, really appreciate it and the clarity that it brings.

Marianna Tessel: So, I really appreciate it now. But our strategy is to be an AI driven expert platform, and what that means is actually it’s a combination of technology and being a platform, both in terms of how we build product, as well as interacting with other entities and actually people because what we have also coming to our platform are real people experts and bless ya, accountants, et cetera. So we are allowing, not just we’re building a great platform but we’re allowing people to be very, very productive on our products.

Marianna Tessel: The AI part is the one that I’m recently very excited about because this is where we really use innovation and a lot of kind of industry buzzwords and applying them to customers to again, like in a real life changing way. We actually we’re … One of the things and nice things we’ve done, we actually defined AI for ourselves, and we said, when we talk about AI, what we mean is machine learning, knowledge engineering and natural language processing.

Marianna Tessel: Now, you’ve probably heard about machine learning and natural language processing, but just to give you a bit of a taste of knowledge engineering, it’s actually about taking rules and relationships and turning them into code. Where it becomes very interesting for us is as you know, we have a leading tax product and other products that have to do with compliance, and what it enabled us to do is really encode compliance in a super efficient way. So that’s kind of one of the things we’re super excited about.

Marianna Tessel: So here is like, on the surface, a problem that could be like sounding to an engineer, slightly boring like compliance and you go ahead and you end up with a technology that’s actually really amazing and completely revolutionizing that field. So, that’s some examples that I’m excited on.

Olga Braylovskiy: Awesome, so if you reflect back to earlier in your career, what is something that maybe a true that you held at the time that you no longer hold true? Like you kind of reevaluated, your perspective shifted.

Marianna Tessel: How early? Like yesterday or?

Olga Braylovskiy: When we’re saying earlier, think back maybe 10, 15 years and I know that our perspective shifts all the time. Something that would be useful especially to this group of geeks who aspire to be the CTO.

Marianna Tessel: Right, I actually … There’s a lot of things that I change my perspective about, but let me kind of touch on a few that came to mind when you asked. The first one is how much do I want to plan my career or not? So, early on I was like this, I would say, like a leaf in the wind that I was like, oh, whatever that takes me. I’m like that sounds interesting, that sounds interesting and I didn’t really plan my career.

Marianna Tessel: I was like, oh, whatever is the next thing, if it sounds good, I’ll just go with it. What I realized at some point is that I need to have a little bit more direction to my career and not necessarily that I have to decide that I want to be like a CTO or whatever, but just kind of think about how the combination of my experiences is actually adding up and where am I going is not just like, oh the cool people. It’s also like, hey, what does it mean? How is it all adding up as a path?

Marianna Tessel: So that will be like one thing that at some point I was like, I’ve started to think … So I’m not a huge planner of my career, but I will be thoughtful about, you know what, I don’t think I’m learning anything here or I don’t think this is like … Doesn’t sound like the traditional step, but I think I’m going to learn a lot, so I want to go and do that. Like an example would be at VMware and it would be at Docker.

Marianna Tessel: I did end up doing a lot of business development while still having an engineering role and I’ve learned a ton from it. That’s something that early on I would be like, no, I’m an engineer, don’t talk to me about anything else. Another thing that I changed my mind about is around leadership. Early in the leadership … And maybe again because that’s kind of a little of what was expected from me, at least I felt, is like I was really focused on the people leadership side.

Marianna Tessel: This is an area that I’m gravitating to anyways. So I would really think myself as a leader of people and I just focused on that as in my leadership, and what I’ve noticed is that I actually really need to continue to develop my craft and I also need to be a technical leader, not just kind of a people leader. So that’s another thing that I changed my mind about how I lead and now I really focus on making sure that I follow technology, I understand technology, I understand the craft to a really big depth, not just focusing on leading people and that’s just like its more fun, but it’s also like, I feel like I add a lot more value to companies when I do it this way. So, I can also talk about other things about learning to be more assertive and things like that, but at a high level, those are a couple of examples.

Olga Braylovskiy: Awesome, last question. I think we have time for one more question. This is super powerful event. We’re all here to learn, share ideas, network. What’s your perspective on leveraging this type of event to the fullest? And just advice on how to make it most effective as a contributing factor to developing relationships and career.

Marianna Tessel: One of the things that actually I’ve learned in, again in my path that your network is one of the most important things that are going to help you in your career. Sometimes you find it in an unexpected places, so just to give you a few ideas around it. When you think about network is like the people you know then later on they will go places, or you need something, or they need something and then you have that connection to really make a dent for them, for the companies, for you, for your career, for your company or sometimes even just getting advice when you need it or sometimes it’s finding that next job when you need it, whatever that is.

Marianna Tessel: So developing a network, if you take one thing away today to at least kind of from what I’m have to offer, developing a network is something super important and I will really focus on that. Then the … What I will say that it’s really important to develop a network, it cannot be like this give and take. You can’t say like, oh I have a bunch of people that when I need them I’m just going to call them, is you need to think about it.

Marianna Tessel: You want to give your network more than what you are taking and you want to really develop great relationship and really, really think about it. Not just like something that you, a tool but something that you’re really caring about. So again, I always focus on am I giving more to my network than I’m taking, because that’s what I think is like a best set up. Of course you don’t want to have people that just always take, take, take from you and will never … There when you need them.

Marianna Tessel: That’s not good part of your network, and then how you develop your network is through events like that. You meet people, you exchange maybe information, then you can follow up. Maybe you have like, you decide to have a coffee or et cetera, but invest time in that. When you think about your day and when you think about your week or your month or whatever, make sure you allocate time to develop your network and to again, make sure you meet with people, you continue getting advice, you continue offering advice, and remember the golden rule, you give more than you take. So anyway, that will be my advice.

Olga Braylovskiy: Awesome, thank you so much for all the insights, Marianna. One and only Marianna Tessel.

Marianna Tessel: Thank you, Olga.

Olga Braylovskiy: You’re welcome. We like hearts in Intuit. That’s true though.

Tracy Stone: You can put those there. Thank you, Marianna and Olga, we appreciate you being here and your wonderful words of advice. Okay, so as promised we’ll do our first raffle and our raffles tonight are some goodies from our small business customers. So, our first raffle is from the Basik Candle company up in South San Francisco, and so I need to draw a ticket. Everybody have their tickets ready? You didn’t get a ticket?

Audience Member: I didn’t get to.

Tracy Stone: Okay, so ready? It’s 691156. Right here? No worries, there you go. From Basik Candle-

Audience: Is this right, 69?

Tracy Stone: Yeah, you are winner, congratulations. Thank you. Okay, so next up we’re going to hear from Rajashree. So Rajashree leads our engineering team for Intuit’s external developer platform and third party app experiences, enabling an ecosystem of thousands of applications that connect to Intuit’s QuickBooks platform. She’s passionate about building purpose-driven engineering teams with a customer first thinking and an inclusive culture. Before Intuit, Rajashree held various engineering roles at PayPal. Thank you, Rajashree.


Rajashree Pimpalkhare speaking

Director of Product Development Rajashree Pimpalkhare gives a talk on “Building Solutions with 3rd-Party Developers to Serve the Needs of Small Businesses” at Intuit Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X


Rajashree Pimpalkhare: Thank you. First of all, let me say this is amazing to see all of you guys here and then secondly, how do I get to follow that? That’s really not fair, but I’ll do my best. So, I lead engineering for Intuit developer group and my focus and my team’s focus is empowering developers that want to work with our customers, deliver solutions to our customers that work with QuickBooks, which is our main product for small businesses.

Rajashree Pimpalkhare: I always like to start with customers. Actually, it’s the first thing I learned when I joined Intuit, that the customer is always in the house, and for us customers are three main type of customers, small businesses, self-employed and consumers. The key thing about all of these personas is they have their financial lives mingle between their business world and their personal world. So most small businesses will have bank accounts that they keep moving money from their personal account to their business account and so on.

Rajashree Pimpalkhare: So we always think of all of those customers in one breaststroke while we may provide different products for them. Then at the bottom you see our partners, so accountants, developers, financial institutions, mega platforms like the Amazons and Googles of the world, educational institutes and governments. These are all the partners that we work with. So, at Intuit we really believe that we want to power prosperity of our customers and we want to bring our partners along, and one big constituent in this part among the partners, are the third party developers.

Rajashree Pimpalkhare: So this is a little bit of geeking out. When you think of third party developers working with QuickBooks, they want to connect their applications to QuickBooks in one of three different ways. Before I go into the specific ways, let me put your mind to who are these developers, what are these apps and what is QuickBooks?

Rajashree Pimpalkhare: So, QuickBooks is our product that does accounting for small businesses. We offer them a payment solution. We offer them payroll solutions, we offer them capital, so we take it off of their financial lives. But these same customers think of like a food truck that you might have seen when you entered building 20. The food truck has things that they need to order. They need a POS system that they need to run credit cards through. They might have some discounts or coupons that they want to give out and those go through social networks and so on, so they use many different apps.

Rajashree Pimpalkhare: A typical small business uses anywhere from four to 15 apps to run their business and it is really critical that their data from these apps flow seamlessly into QuickBooks. Because at the end of the day, this is where they go to understand whether they’re making money, they’re losing money, or how are they going to pay for their dinner the end of the week, right. So there are three ways in which the platform enables us to or enable the third party developer to connect their app with QuickBooks.

Rajashree Pimpalkhare: The first one is data connections. So, this is what most of our developers use. So an eCommerce app, or a POS app, or any kind of other app that’s doing financial transactions for the developer can write that data into QuickBooks. So that can be reconciled into our books if it’s inventory for a product based business. Somebody is selling something on Shopify, the inventory can reconcile with QuickBooks.

Rajashree Pimpalkhare: If it’s a set of customers and a general contractor that’s doing 10 different jobs or a plumber that’s doing 10 different jobs for 10 different customers, those customer names can all come in here and they can understand where they spend money versus where they invoice their customers. So that data flowing in and out. So everything works together is the primary use case that our platform enables. The second use case is for certain types of experiences.

Rajashree Pimpalkhare: When small businesses are into QuickBooks and they’re trying to run their business, they don’t want to have to go and open up another app. So this might be something like you are in QuickBooks and you want to pay your bills. Now, we don’t offer bill pay as our own capability, but we do power it through a partner. If you are a small business in the UK and using QuickBooks, you can actually run payroll through a product that’s run, that’s developed by a different company but the experience is seamless and the platform underneath enables that. Then the third piece is we want to be where our customers are.

Rajashree Pimpalkhare: So, if you are in Google and if you are responding to some emails from your customers, you are able to invoice your customers right from there, from Gmail and that invoicing is powered by Intuit. So the three kind of ways of allowing integrations or ways for third parties to integrate with us, are data connections, so everything works together.

Rajashree Pimpalkhare: Powered by partners, so seamless experiences for our customers where they don’t know where our experience starts and ends and the partner experience starts. The third piece is powering through partners and this is really critical when we want to actually go and serve customers where they are. So what do we do with this? Why is this so important? So we are the small business platform of choice in the United States and increasingly in the global countries where we are.

Rajashree Pimpalkhare: The reason that’s true is because we have over four million customers that use QuickBooks today and for all of those customers, they have needs beyond what QuickBooks provides. So small businesses say, I need additional tools beyond QuickBooks to help me run my business and I would rather get it from Intuit because I trust Intuit and Intuit knows what really is needed for my business to run, and for my business to work, and for me to be profitable.

Rajashree Pimpalkhare: Then the developers need access to customers because if you are a small developer creating a niche app for a specific segment of small businesses, it is incredibly difficult to get them to know you and get them to pick you. There are so many different channels, but if they came in through, we have an App Store that the QuickBooks customers can view. If these developers put their apps in the App Store, they have access to all of their customers and they can put forth a value proposition and reach them.

Rajashree Pimpalkhare: So the developers come to us because they need access to customers to build a business that can grow and be profitable and we sit in the middle of it being able to really power that network effect. So it’s actually really, really gratifying because we work for developers and we work for customers and it’s great to be able to connect the two. A lot of build-outs. So I love this slide because it tells you a little bit about what kind of … So it tells you multiple things. First it tells you look at how big our ecosystem is.

Rajashree Pimpalkhare: All of these apps that show up here are connected to QuickBooks in one of the three different ways that we looked at. The second thing it tells you is look at how difficult it is for small businesses to really know what they want and what they need and the QuickBooks at the center is where we really take our role really seriously. We want to recommend the right apps to our customers. We want to make sure they’re successful with those. We want to make sure they know exactly how they interact with our products and so on. So this is just to tell you where we are today.

Rajashree Pimpalkhare: So I joined Intuit five years ago. I think Tracy said I came from PayPal and I came from a world where it was all about money, and all about revenue, and all about the money that came into the pocket of the company. I love PayPal as well, by the way, but at Intuit it’s just a little bit different, right? The purpose is just a little bit higher and the passion you can bring to the table is just that much bigger. In the last five years we have gone from about 700,000 QuickBooks customers to over four million.

Rajashree Pimpalkhare: I think the number is 4.2 million today. 40 apps that were on the App Store to more than 700 and one in five customers of QuickBooks today use at least one app, and we are continuing to work on it and continuing to grow it. So just super proud to be associated with the Intuit developer group to be working here and we are the global trusted platform of choice. So, with that…

Tracy Stone: Thank you.

Rajashree Pimpalkhare: I assume you’re not taking questions now, but I love to talk to you about it after.

Tracy Stone: We would love for you to engage with our speakers after. We’ll have some time at the end of the session. So thank you, Rajashree. Now we’d like to invite Nhung up to our virtual stage up here, I guess. Nhung, you want to advance the slide. Nhung is the director of data science for our QuickBooks ecosystem and customer success data science teams. She leads the strategy and execution of applied machine learning programs that span the range of marketing product, forecasting, and other strategic capabilities.

Tracy Stone: Her applied machine learning teams build new to the world products and services backed by artificial intelligence to serve Intuit’s customers across the whole customer life cycle. So I’m excited to hear from Nhung. So let’s give Nhung a warm welcome.


Nhung Ho speaking

Director of Data Science Nhung Ho gives a talk on “Using Machine Learning to Solve Small Businesses’ Problems” at Intuit Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X


Nhung Ho: All righty, can you hear me? All right, so couple things as ground rules. I speak really quickly and I tend to pace. If I walk into these chairs, feel free to laugh at me. All right, so just a quick introduction about myself. I’m actually a former astrophysicist turned data scientist and one of the things I’m super passionate about is actually using math and code to solve real world problems. That’s why I escaped from looking at the stars, actually looking at people and turning my gaze down.

Nhung Ho: The other thing is I also love food and so if you’re traveling to any city and want restaurant recommendations, I have a running list. I made a Google maps, I pin everything and I have a list of all the dishes that I liked. So I’m happy to share that with anyone. All right, so I’m here to talk to you about how my team uses machine learning to solve small business problems because the business of running your business is actually really hard and it all starts with starting your business, right?

Nhung Ho: You would think that like Michael Scott in The Office, when he says, I declare bankruptcy, right? You can say, I declare a business and then you have a business. But actually that’s not true and this is just the first six of 25 steps that you have to take to even start a business, right? There are a lot of questions that you have to answer before you even get your first customer through the door. Once you start doing that, then you say, okay, I expect that I’m going to open a clothing store.

Nhung Ho: I’m going to be able to dress women, make them happy, make them look good and everything is going to be awesome, dandy. I’m going to go home feeling great, but what you do is you actually go home to a mountain of paperwork, right? I know this really intimately because I have 10 siblings and five of them own small businesses and every day they go home, the stacks and stacks of papers, stacks and stacks of receipts.

Nhung Ho: It is incredibly hard to manage. So, while during the day you’re satisfying your customers, you’re doing what you love, you go home at night and you’re dealing with paperwork because you actually have to run your business and make sure that you have enough cash flow to continue serving your customers. Just a simplified view of some of the things you need to do as a small business, you need to track your expenses.

Nhung Ho: You’re going to go buy supplies, you’re going to go buy inventory, and you’re going to have a stack of receipts and invoices that you have to keep track of. If you drive for a living, you have to keep track of how much you use your car because that ends up being tax deductible, right? You want to be able to track your income so now you have bank statements to take care of. If you invoice people, you have to keep track of that as well. Then, finally at the end of the year, you need to make sure you’re tax compliant, right?

Nhung Ho: So if you’re a sole proprietor, you need to know about this thing called a schedule C and you then you need an EIN. So there’s a whole host of things that you need to be able to do and know and manage as a small business owner that I think a lot of us who don’t own small businesses don’t realize. So where do I come in, right? Why do you need an astrophysicist here at Intuit for solving these problems?

Nhung Ho: It’s because a lot of these things that are really rote, and boring, and tedious, you can actually make much easier using machine learning. So I’m going to go through two examples. The first one is on receipt tracking. So I mentioned that a lot of small businesses go out and buy a bunch of things. They need to be able to track their receipts. They need to be able to take those receipts and actually transcribe every single piece of information into an accounting system. In this case, in QuickBooks, right?

Nhung Ho: So if you as a human, you have to go through and scan this and say, where did I buy this from? What is the address? When did I buy it? What are all the things that I bought? What is the individual prices? And then decompose that one receipt into individual lines in there. Think about how much time that takes and how tedious it is.

Nhung Ho: So what that ends up doing is actually causing you to say, it’s not worth it, I’m just not going to do this and then you end up leaving money on the table because you could have deducted these. So if you can imagine, what my team did was we actually married computer vision and natural language processing. You can utilize an OCR system to go through and actually pick out every single character. You can figure out exactly where the bounding boxes are for each of these fields and then begin to lift that information out.

Nhung Ho: You can then use a deep learning system to say, okay, I know that when I see Aroma Cafe, that’s a vendor, and when I see that number format, it’s actually a date, right? We can use the latest deep learning technology, scan through the system that’s already been OCR, pull out that information and then make sense of it. Then finally put that into your accounting system and you’re done.

Nhung Ho: This system is 10 times faster than what a human can do and we can go and grind through thousands of receipts for you as long as you send us the image. That’s kind of the power of what machine learning can do to help a small business succeed. So the next example I want to talk about is mileage tracking. As I mentioned earlier, how many of you here know that if you use a personal vehicle for business purposes, every single mile that you drive is deductible at 54.5 cents per mile?

Nhung Ho: Actually, quite a bit. Actually, I definitely did not know that when I started this. But if you are a real estate agent and you’re driving between showings, that could be hundreds of miles per day. That’s a lot of money that you can deduct on your taxes. We actually have a product called QuickBooks Self-Employed, that makes this really easy for you. You turn on our automated mileage tracking service, we will say, okay, we know that Nhung traveled from point A to point B on this day and this is the distance that she drives, which is super awesome, except we don’t actually know exactly what the purpose was, right?

Nhung Ho: Because again, you need to be able to say, this trip is a business trip versus a personal trip. If I drive to the grocery store, I can’t deduct that on my taxes and if the IRS finds out, it’s not going to be good times for anybody. So you have to go in and say, is this a personal trip? Is this a business trip? And if it’s a business trip, why did you actually use your car for that purpose, right? So it’s multiple pieces of work.

Nhung Ho: So to give you a real example that I have scrubbed a lot of personal information from is, meet Claire. Claire works at City Hall during the day and she actually teaches piano on the side. These are all of the trips that Claire took. You can see that some of them are for business purposes and some of them are for personal, but there’s no real pattern that jumps out here. In an average month, she takes 200 trips and you can imagine that if Claire, like I am, there’s a huge procrastinator, at the end of the month, she’s got to answer basically 800 questions.

Nhung Ho: What was this trip? When did I take this trip? Where was I going and what was the purpose of this trip? Is it for because I’m driving to teach piano or is it because I’m driving to City Hall for my job? We don’t make that distinguish. We don’t distinguish that, you do it for us, but I can build a machine learning system that can do this in less than a second for Claire and that’s exactly what my team did. We utilized frequent pattern mining and we essentially automatically learn very personalized and highly individualized rules per user.

Nhung Ho: So we can group all of the trips on the left side that she took exactly to the same destination, show that to her and say, we think that this is a personal trip and if it ends up being a personal trip or a business trip, here is a deduction that you get. For our users, they take 50 million trips per year. Now you can imagine building a system like this that is scalable, extensible, and is ready to use for any new user who comes in.

Nhung Ho: So what I hope I showed you was how machine learning can actually make the business of doing business much easier, right? And it’s not … I can tell you it wasn’t obvious to me when I started here, why we need a data scientist to work on accounting. Isn’t it solved? It’s so old, it’s so boring. But I can tell you some of the most boring, mundane things are the areas that are the most exciting because those are the areas that have not seen innovation in a really long time and that’s where you can go in and make a difference. So thank you for letting me share those with you.

Tracy Stone: Thank you, Nhung. I don’t think you tripped over the chairs either, but that might be my job then, huh? Okay, one more raffle now. So get your tickets out and this time our raffle prize is from Origaudio in the set of very cool noise canceling headphones. You like this? Yeah, all right. Let’s see if I can pull your raffle card. Okay, it is 691056. Anybody? 691056. Here you go, I believe you. Okay, wonderful. All right, so next we have Kristina Fox, and Kristina Fox is our staff iOS engineer working on the QuickBooks Self-Employed iOS app.

Tracy Stone: She has spoken internationally at over 25 meet-ups and conferences, including Grace Hopper, iOS Dev UK. Were some people at Grace Hopper a few weeks ago? iOS Dev UK, WeAreDevelopers World Congress and many others on topics ranging from Apple watch development to diversity inclusion. So thank you Kristina for being here and let’s give her a warm welcome.


Kristina Fox speaking

Staff iOS Engineer Kristina Fox gives a talk on “Using Mobile Capabilities to Save Time and Money for the Self-Employed” at Intuit Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X


Kristina Fox: Thank you everyone. Okay, so we’re going to be talking about using mobile capabilities to save time and money for the self employed, and this is really a topic that’s near and dear to my heart because, well I’m an iOS engineer and it’s basically one of the most versatile platforms out there. Okay, is there anyone here that doesn’t have a phone on them right now?

Kristina Fox: Exactly, so that’s why this is one of those platforms that’s literally always on you and it can make such a huge difference in your lives than it already has, considering the fact that none of us really travel without phones anymore. Okay, so let’s get started. Just to give you a bit of context, I work on QuickBooks Self-Employed, it was actually the app that Nhung talked about earlier, and for those of you who aren’t familiar with it, it is an app that helps freelancers and contractors help manage their business finances.

Kristina Fox: So she talked a little bit about the mileage tracking. We also do transaction and expense reporting and also invoicing. So there’s lots of really cool stuff that our app does to help all of those self-proprietors out there, the sole proprietors out there. If you think about it, the types of people that we help are people like this freelance photographer out there or maybe even the Lyft or Uber driver that took you here today.

Kristina Fox: So lots of really interesting industries that we can also help out there. Okay, so the next thing that I basically want to dive into is specifically a project that I worked on and developed called Receipt Capture. This is this is one of these most probably my favorite project that I’ve ever worked on here, so it’s really fun for me to share this with you. Okay, so let’s take a step back and say, oh, we decided to go shopping and this is just to set some context about why receipts are so important to our self employed people, right?

Kristina Fox: So we decided to go shopping and we found a really amazing dress. It looked awesome on you in the dressing room and you decided to buy it and bring it home. But once you got home, you really … It ended up changing colors on you. You didn’t really know what color that dress was. Was it white and gold? Was it blue and black? You’re not really sure anymore. So, well, okay, you decided the blue and black doesn’t really work for you and so you want to return it and what exactly do you need usually to return things to a store?

Kristina Fox: Receipts, exactly. So receipts are really important to us as proofs of purchase as consumers, but for our small businesses, they’re really important to help prove that the things that we’re buying are actually business purchases and so this is really important, especially if the IRS comes to audit at some point, you need to be able to prove that the things that you purchased at the end of the day are actually for your business.

Kristina Fox: So, let’s talk about the basic Receipt Capture experience and so I direct your attention all the way to the left side. Usually you’ll hook up QuickBooks Self-Employed to your bank account and you’ll see the list of transactions coming through from your credit card or your checking account. From there, if you have a receipt for Starbucks for example, you’ll want to tap on Starbucks and then that will pull up a detailed view of your transaction.

Kristina Fox: From there you can tap, attach receipt and then you’ll be able to either take a photo or choose from your camera roll if you decided to take that photo earlier. That’s fine, pretty simple experience. You’re really just attaching a photo to a transaction, but one thing that’s kind of interesting about this is that if you’ve ever gone into your checking account or your credit card account and you just bought something, you’ll notice that there might be a pending transaction, right? It means that it hadn’t exactly cleared yet because maybe the restaurant doesn’t have your final total or your credit card is still being authorized.

Kristina Fox: So it can take generally about one to two days for these pending transactions to clear and unfortunately until that time we can’t really attach any receipts to it because it’s not a real transaction at that point. One to two days might not sound like very much time, but really anything can happen in these receipts in that one or two days, anything at all.

Kristina Fox: So what do you do? What do you do when you lose that receipt? Well, hopefully with our new enhanced Receipt Capture experience you won’t have to find out. So I’m going to do a quick demo of the new enhanced Receipt Capture experience that I ended up building for QuickBooks Self-Employed and this is going to be fun to juggle. So hopping in, I’m going to go ahead and end the show and I bring up my phone screen here. So this is the QuickBooks Self-Employed app. I’m trying to make this bigger.

Kristina Fox: You can go into the transactions tab where you can actually snap a receipt. So if you remember before we had to go and tap into an individual transaction to take a picture, well now you don’t really have to do that anymore. So you go into here, we’ll tap snap receipt and then we bring up this new camera view that we have, and so what’s cool about this is it actually does the cropping for you. So I’m shaking a little bit too much right now.

Kristina Fox: Demos are always nerve wracking. Okay, so we’ll steady. Okay, so now we’re able to get a cropped version of our receipt photo here and if you ever wanted to mess around with it, say it didn’t quite crop correctly, you could even go in, it takes you back to that original photo. So then you can manipulate those crop points or you can even rotate that image. So, there’s lots of really cool image enhancements that you can do here, and then from here all you have to do is hit at the bottom, use this photo and it disappears off. It gets uploaded to our service in the background and that’s it.

Kristina Fox: That’s all you have to do now. So this is our new enhanced Receipt Capture experience. So what exactly is happening here? Well, in the background we start off with QuickBooks Self-Employed. From there it actually gets uploaded to our document service. So this is where it actually stores that receipt image for you and then from there it goes to our data extraction service.

Kristina Fox: So, it’s something that Nhung was alluding to earlier where we can actually go and run optical character recognition on that receipt itself and then pull out the data that we really care about. So that’s stuff like, again, the vendor at the top of the receipt, we’re looking for the date, the total, and the credit card number too. So we can pull out all of that receipt data for you, and then we do automatic matching.

Kristina Fox: So now we go back into your list of transactions. Instead of actually having to go in and find that Starbucks transaction again, this algorithm actually takes that receipt and automatically looks at the data that’s coming in from that transaction and it attaches it for you. So it’s literally you take a picture and then you just forget about it and it’s already in your account, all done. Yeah, it’s literally a life changing event.

Kristina Fox: So what are some of the mobile capabilities of this? Well, obviously we’re using the camera to take a photo. We have a lot of touch … We have the touch screen capability where we can actually manipulate the photo if we need to, and we’re also running a lot of image processing algorithms in the background in order to make sure that the photo is usable. So you saw that it was telling me to hold steady.

Kristina Fox: In some cases, if there’s not enough light, it will tell me, oh, you need to add some more light to the background’s too dark, things like that. So there’s a lot of really cool things that are running in the background, just doing image processing there. It’s also a very on the go capability, like as you can see, a lot of the people we’re supporting are Lyft drivers. They’re a freelance photographers, they’re always constantly on the go, they’re gig economists and so they need to be able to have this type of capability wherever they’re going.

Kristina Fox: Of course, on one last thing, if you’re using the iOS platform, you might be familiar with Siri shortcuts, and so if I hop out again and on my phone. Okay, so let’s tell Siri, snap this receipt. It hops directly into that camera. So now the user doesn’t even have to go in and launch QuickBooks Self-Employed. You can just add a Siri shortcut with your own custom phrase and then it will go into the exact view that you need.

Kristina Fox: So we’re really taking advantage of all those platform capabilities here too, and here’s a look at what that looks like. So you can add this custom phrase to Siri and then it will do whatever you tell it to. So this is the one of the cool Siri shortcuts capabilities that we have. That’s it for me. Thank you so much.

Tracy Stone: Thank you, Kristina, and a live demo was so awesome. All right, next up, we have Cassie Divine and Cassie’s the senior vice president of the Intuit QuickBooks online platform where she leads the business units responsible for small business, self-employed, accountant, and the Intuit developer group. She is especially passionate about driving diversity and inclusion in technology with an eye on fostering a favorable work environment for women.

Tracy Stone: Cassie’s been awarded the Silicon Valley Business Journal Women of Influence award and has twice received the Intuit CEO Leadership award. A passionate small business owner herself, she also has an Etsy shop where she sells her DIY kids’ Halloween costumes, might come in handy. So let’s welcome Cassie Divine.


Cassie Divine speaking

SVP of QuickBooks Online Platform Cassie Divine gives a talk on “Owning Your Career: Reinventing Yourself to Create Impact at Scale” at Intuit Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X


Cassie Divine: Thank you, Tracy. That glorious introduction just made my impostor syndrome in this room even worse. I’m so grateful to be here and crowned a Girl Geek with this amazing lineup of my Intuit colleagues. My topic, my lightning talk isn’t a technical one. It’s how to hack your own career and for all of you who know you’ve been working, there is so much conventional wisdom about what you’re supposed to do to move up, and advance, and create an enriching career.

Cassie Divine: By all of those conventional rules, I have done nothing right, and yet I find myself with a big fancy job, and title, and opportunity to make a huge impact. So tonight I want to tell you five quick stories. The stories are unique to me and they’re my crazy stories, but I think the lessons I’ve learned that I’ll share are applicable to lots of you, I hope. So I hope you hear something you already knew and needed to hear or something you could learn.

Cassie Divine: So story one, I started at Intuit 12 years ago and I joined as a senior manager and I had a plan to get promoted in two years. I had talked to the boss I came to work for. He had seemed to think that that was a reasonable expectation and I was at a point in my career where I was convinced that I was better than a lot of people and I needed to go. It was a sure thing that was going to happen. Of course in this story, it doesn’t happen that way and I was devastated.

Cassie Divine: I was devastated because I didn’t get this title that I wanted and the wisdom from my boss was just to work hard, to work even harder and to sort of ride the time and that it would happen in one to two years. I found myself being frustrated with the idea that I was waiting to get a label from someone else and in a moment I decided to create a label for myself. I’d been thinking about starting an Etsy store, a side gig, for a long time and I decided that this would be my moment and screw not being the title I’d wanted, I would be the CEO of my own thing.

Cassie Divine: And I kept working, but the work changed because so much of my creativity, and hustle, and anxiety, instead of worrying about some label, it went into something that was for me, and it was really enriching for me, and I actually probably stopped staying as late and stopped doing some of the things you do when you’re searching for a title and just started focusing on impact at work.

Cassie Divine: Because I had this creative outlet and don’t you know I got promoted six months later, and the lesson for me though was you define you, nobody else gets to do that. Titles can just be labels and we don’t like them when they’re bad, and I worry we give them too much power when we think they’re good and a lot of it is about the impact and the journey. Story two, I was coming back from my maternity leave and I got a lot of wisdom from women and men, which was, the plan was just to come back and show that I was the same person.

Cassie Divine: I could just, nothing had changed and I could work the same way and I remember thinking like, I don’t think I can even do that, everything has changed. I don’t want to be here the same hours. My priorities have shifted and I had never cashed in any of my street credibility or what I had had. I had never asked for anything and I decided that this might be the moment that I would, and I did my research, I talked to a lot of people who had organized creative arrangements.

Cassie Divine: A lot of people gave me advice that it was absolutely the worst time in my career to take a step back and there are a lot of studies that say women are going to earn less. I just decided to go for it and I asked to take 80% of my salary and have every Friday to be with my little girl. But I made a deal with my boss, which was, I will deliver the same impact, it just is going to look different. I did that arrangement for two years. It was so awesome and I got promoted after I came back full time.

Cassie Divine: I learned in it to ask for what you want. The worst thing anybody can say is no, but in asking for what you want, to be clear about what you’re going to deliver so people know what they get when they say yes. Third story, I was 15 years into my career, seven years at Intuit, and I had just been promoted to vice president and conventional wisdom is like, this is your career, this is what you’re going to do, this is your path. You build on it.

Cassie Divine: I had been unhappy for a long time and just going through the motions, and I loved the company, I loved the company’s impact, but I wasn’t finding it was meaningful to me, and I started working on my network. I agree with what Marianna said and I had always given to my network and this was a point I asked, I worked my network on what would someone take a bet on me to do that would be different from my job and I got my big break and it came with a demotion.

Cassie Divine: It came with giving back a team, which I had been led to believe that it was all about creating this big kingdom and you grow in giving … You grow and getting this big team and I went to being an individual contributor and people inside and outside the company thought it was a little crazy because it just seemed like it was too late to make a change and that it on paper looked a little insane to a lot of people. But I was so excited about it to get to have an impact on moving into a product business when I actually moved into the QuickBooks Self-Employed business.

Cassie Divine: I would move into a role in BizOps and I was excited because the person I was going to work for saw something in me that I didn’t even know I was capable of doing, but both of us were willing to take that bet. 18 months later after getting that job, I got the biggest job I’d ever gotten. I almost went for something bigger than what I had stepped down from, and it came when I wasn’t focusing on trying to get promoted. I was just trying to focus on the biggest impact. But the lesson that I learned in this is bet on yourself, but that has two parts.

Cassie Divine: It has the courage that you make the bet and it’s also about finding somebody who will make that bet with you, and my advice to all of you would be seek someone, your boss or mentor who sees what you’re capable of doing and suspends disbelief and isn’t just obsessed with what you’ve been able to do on paper already.

Cassie Divine: Fourth story, I was now in my big job leading this product business and it was really fun and I knew the rap on me is I had taken over from someone who was really successful. It was sort of like my was seen as the person who was just helping keep it up and running, and there was a job similar to mine that had been vacant for three to four months and it was bigger, and it was a lot harder, and it had a different profile and I wasn’t a candidate for it.

Cassie Divine: In fact, I was on the interview committee to go and search for the person who would be the right person and based on every discussion of what we thought we were looking for, wisdom said I had no business to do this. As we talked to the team and as we kept interviewing people, I started thinking, I think I could do this job and I actually think I could start to show that it’s not just about keeping the easy thing running, but I could show that I could work on something hard and lead a turnaround.

Cassie Divine: But I didn’t necessarily have the courage at the time to raise my hand to just take it, and so I raised my hand to help with it. I had said, what if I help in a capacity as we continue to search for this person and I will make it better. This team is in need, I have a great leadership team in Self-employed. They’ll step up and do more and I’ll play this dual role and six months later we decided that I actually was the right person for the job and I stepped into the role.

Cassie Divine: But more importantly, I showed that I was willing to take on the hard things. I was willing to do a lot more and as a result, I got my big job today, which is far bigger. What I learned in that is it’s okay to be your own hype woman and raise your hand and sometimes you’re the best person qualified to say you might be able to go do a job and be considered for it.

Cassie Divine: What I’d offer for you is, if you’re worried about it or if they’re worried about it, one of the best ways to get that shot is in a volunteer or rotational capacity because there’s nothing that is a downside to that except you do a lot more work in the meantime, but it shows you and it shows them that it’s a great opportunity. Last story isn’t a story, it’s something that has weaved through all of my stories, which is conventional wisdom is that career success is all about you and you answer this question, what do I need to do to get ahead and how can I show that I’m, I’m being successful?

Cassie Divine: What I found is career success is actually about making everybody else successful and investing in your teams, and your peers, and your boss, and your customers. One of my favorite books, I encourage all of you to read it is by Adam Grant. It’s called Give and Take and it is all about this idea that investing in other’s success is what creates the most success for us. Now, it feels counterintuitive when you think about what it feels like to give someone credit, God forbid it’s someone mansplaining something to you. It’s hard sometimes because it feels random.

Cassie Divine: Maybe you don’t know them or you haven’t … It’s not sort of about this give and take, but do it anyway. Real leadership is about creating impact and it is more easy to do. It is more fun to do with others, and as you do that, I promise it comes back to you. You become the person that everybody wants to work with and it has its ups and downs and I agree with Marianna, you have to worry about the people who are just taking and not giving back to that, but I encourage you to do it.

Cassie Divine: So the lesson I’ve learned there is what I’d call, what Adam Grant says is givers take all, and invest in others and raise others up and make that a part of your leadership brand. It will unlock things that are just amazing for you. To wrap up, start … I like starting with a plan. I like starting with a path. I like the point Marianna made about the things that are important. I would encourage you to be open to the idea that it can change dramatically and I would think a lot more about moving forward than moving up. Hack your own career.

Cassie Divine: You define you. Ask for what you want with accountability, bet on yourself and find the person who’s going to bet on you, raise your hand to take on work people didn’t know, but you did, that you’re capable of all while investing in others’ success. I’m so excited to find out what all of you will go and do and the impact that you create in your own careers. Thank you.

Tracy Stone: Okay, so with that, that ends our formal program tonight. I want to thank you all for coming. I want to thank all of our speakers. So these are some amazing women that work at Intuit doing some really cool work and leading teams around the work we’re doing to power prosperity for our 50 million customers and growing. You can just spend some time and interact with them as long as they’re able to stay.

Tracy Stone: Ask them questions; those questions that you had that you didn’t get a chance to ask. There are drinks out there. You can go out on the patio and network for a little while longer. I hope you all had a chance to connect with everybody and walked away with some few, those nuggets of wisdom that you’ll take as you go on to do amazing things, as Cassie said. So thank you all.


Cindy Osmon speaking

Distinguished Engineer Cindy Osmon demos “Smart Mirror” at Intuit Girl Geek Dinner. Erica Kawamoto Hsu / Girl Geek X


Nirmala Ranganathan speaking

Principal Product Manager Nirmala Ranganathan demos “Shield” at Intuit Girl Geek Dinner. Erica Kawamoto Hsu / Girl Geek X


Yi Ng speaking

Principal Product Manager Yi Ng and Senior Software Engineer Regina Garcia demo “QuickBooks for New Users” at Intuit Girl Geek Dinner. Erica Kawamoto Hsu / Girl Geek X


Intuit Tech Women at Intuit Girl Geek Dinner group of women in tech

Thanks to all the Tech Women @ Intuit for hosting us warmly at Intuit’s Mountain View headquarters for our Intuit Girl Geek Dinner! Erica Kawamoto Hsu / Girl Geek X

Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Microsoft Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang speaking

Girl Geek X Welcome: Angie Chang kicks off a sold-out Microsoft Girl Geek Dinner at Microsoft Reactor in San Francisco, California.  Erica Kawamoto Hsu / Girl Geek X

Transcript of Microsoft Girl Geek Dinner – Lightning Talks & Panel:

Angie Chang: So hi, everyone. My name is Angie Chang and I’m the founder of Girl Geek X. I want to thank you so much for coming out tonight to the Microsoft Reactor. I’m super excited to see everyone here and to introduce you to all of Microsoft’s girl geeks, to see this amazing art and tech demos. Who here signed up for a demo? I saw a lot of people interested in demos and getting tours, so I’m really excited that you are able to do that. Thank you once again to Microsoft and to all the people who helped plan this night.

Angie Chang: How many of you this is your first Girl Geek Dinner? Wow. And how many of you consider yourself like a regular at Girl Geek Dinners? Thank you so much for coming back again and again. We do this almost every week, going to different tech companies, meeting the girl geeks, and we hope you tune into our podcast. We have a regular podcast on topics from internet security, to emotional security, to management, to working in the Silicon Valley. So please tune in on iTunes or Spotify. We also have a very active social media. So if you follow us at Girl Geek X, you can also tweet and share with Girl Geek X Microsoft tonight and we will retweet and reshare.

Angie Chang: Now I would like to introduce our first presenter. Her name is Kaitlyn Hova and she is the co-owner of Hova Labs, where they have designed and produced the Hovalin, which is a 3D printed violin. Kaitlyn.

Kaitlyn Hova: Thank you so much for having me. This is wonderful. So my name is Kaitlyn Hova. I currently work at Join and I also co-own a company called Hova Labs, where we like to make a bunch of weird projects. It’s kind of like one of those like, “If I had time, why wouldn’t I make this?” kind of companies. So it’s just me and my husband and the biggest thing that we really wanted to do was to find a way to convey what synesthesia was like in real time. Who here knows what synesthesia is? Yeah, it’s not very many people. It’s all right. So synesthesia is a neurological phenomenon in which two senses are inherently crossed, causing sensations from one sense to lead to an automatic but also involuntary experience in another. A version of this is called chromesthesia, which is when people can physically see sounds.

Kaitlyn Hova: I didn’t know this was in any way unusual until I was around 21 years old when I was in my final music theory course and our professor just mentioned, “Isn’t it crazy? That some people can see sounds?” Yeah, I ended up dropping my music degree and going into neuroscience, because that’s way more interesting, right?

Kaitlyn Hova: So, ever since then, I’ve been trying to find a way to display what synesthesia was like, because when you’re discussing it with people, it tends to end up going into the more like psychedelic conversation, and it’s not really. So, how to display it? I play violin, so we thought, “Wouldn’t it be wonderful if there was a violin that we could light up with the colors that I see in real time?” This didn’t exist, so of course you have to go to the drawing board, and the first thing on our list was, “What if we had a clear violin and we just put LEDs in that?” We couldn’t find a clear violin and if we could, it was probably too expensive.

Kaitlyn Hova: So, ended up deciding like, “Well, how hard would it be to 3D print one?” It took a year and a half to figure out how not to make a violin and then to figure out how to. I think we went through about like 30 or 40 iterations because you end up getting really desperate and saying like, “Well, what is the violin anyway?” because it’s really hard to make this. It started out as a stick with strings and then kind of grew from there.

Kaitlyn Hova: So now, here it is. Once we got our first prototype, we ended up deciding that this violin on its own, LEDs aside, was a really great product, so why not release it open source for people to 3D print their own music programs? We’re still seeing a trend in schools where music is systematically underfunded, while these same schools are getting STEM grants, so why not? Seems like a connection there. Thank you.

Kaitlyn Hova violin playing synthesia

Violinist Kaitlyn Hova plays a few songs at Microsoft Girl Geek Dinner.   Erica Kawamoto Hsu / Girl Geek X

Emily Hove: Let’s hear it for Kaitlyn. Kaitlyn, thank you so much.

Kaitlyn Hova: Thank you.

Emily Hove: This is fantastic. What a great way to start off such an inspirational evening.

Kaitlyn Hova: Thanks.

Emily Hove: So thank you very much.

Kaitlyn Hova: Cheers.

Emily Hove speaking

Program Manager Emily Hove welcomes the Girl Geek X community to Microsoft Reactors around the world, from San Francisco to London!  Erica Kawamoto Hsu / Girl Geek X

Emily Hove: Welcome, everybody. Welcome to the San Francisco Microsoft Reactor and the Girl Geek Dinner.

Kaitlyn Hova: Thank you, Chloe.

Emily Hove: My name is Emily Hove. I’m part of the global Microsoft Reactor program and we have a lot of synergies between Girl Geek and the Microsoft Reactors. Similar to the way Girl Geek inspires and connects women in technology, our Reactors are all about being community hubs and everything that is related to developers and startups, giving developers and startups the tools where they can learn, connect, and build. So, we hope you all find a night that is inspiring and where you’re able to connect and build today.

Emily Hove: If you’re interested in a little bit more about the Reactor program, we’ve got some cards around the room and they talk about some of the fantastic upcoming workshops and meetups that we have. So we’d love to encourage you to check out our calendar of events and invite you all to attend. With that, I’d like to bring up Chloe Condon, who will be our MC for the evening, and help introduce some of the inspiring people and inspiring women in technology that we have for you tonight. So Chloe, cloud developer advocate extraordinaire.

Chloe Condon: Hello. Thank you so much for coming. This is theater in the round. So I’m just going to keep walking in a circle like I’m giving a very serious keynote so you all don’t see my back. Thank you so much for coming tonight. We are so excited to have you here at the Reactor. Who’s first time at the Reactor, this event? Incredible. That is so exciting. I hope we see you here a lot more. If you want to participate in one of the Fake Boyfriend workshops that I put on here, you can build a button to get you out of awkward social situations, come see me after. We are doing those all the time here. They’re so much fun. Also ask me about my smart badge. This is a little scrolling LED badge that we’re probably going to do a workshop for pretty soon, as well. So come see me after if you’re interested at all in learning about those events and we’ll get you signed up for them.

Chloe Condon: I’m going to tell a little story before I introduce our first guest. I am so, so excited to be your MC tonight. I actually met Angie because I went to Hackbright. Do we have any Hackbright or bootcamp grads in the audience? No. Amazing. So, Angie spoke at my bootcamp and told us all about Girl Geek Dinner and I thought, “That sounds so cool. I would love to go to one someday.” So it’s literally a dream come true to be here with all of you today. This is my first Girl Geek Dinner ever, and I get to be your MC.

Chloe Condon: So, I’m so excited to introduce our first speaker tonight. She is incredible. Please, please show everybody how cool your dress is when you come up here, or I’ll be very upset. I would like to introduce Kitty who is going to tell us all about the incredible technology and fashion that she uses to make things like the amazing dress that I’m sure she’s about to tell you about. So Kitty, come on up. All right.

Kitty Yeung Microsoft Girl Geek Dinner

Microsoft Garage Manager Kitty Yeung gives a talk on “Hacking at the Microsoft Garage” at Microsoft Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Kitty Yeung: Hi, everybody. Good evening. Thank you so much Chloe for introducing me. In fact, I’m not going to talk about my dress. That’s for the demo later. I’m going to talk about actually what’s behind that, all the innovation work that we’ve been doing at Microsoft. So, I’m the manager of The Garage at Microsoft. How many of you have heard of The Garage before? Some of you, some of you I’ve met actually.

Kitty Yeung: So, this is a program that drives the innovation, drives a culture of innovation and experimentation. How do we do that? We say, “Doers not talkers.” We actually get our hands dirty. When we think about something, we act on it. These are the culture pillars for Microsoft. To a lot of us when we first see them, they saw just words, but how do we actually implement these and achieve this? We have all kinds of programs and mechanism to drive innovation in Microsoft. Hacking, we have global sites, we have internship programs, experimental outlet is how we ship projects out, and we have intrapreneurs program, and we do storytelling. So I’m going to go into each of these.

Kitty Yeung: The hacking at Microsoft has become the culture. We actually organize the world’s largest global hackathon at Microsoft, and The Garage is the organization that organizes it. Guess how many people attended this year? Globally, there were 27,000 people attending our hackathon, and everyone was excitedly bringing their great ideas to the hackathon and forming teams all around the world. Whether or not you know them, whether or not you’re from the same org, same teams, you can put your skills together and build something that you feel passionate about. We had thousands of projects every year submitted to the hackathon, and The Garage helps people not only have these ideas submitted, we help them grow their ideas into prototypes, and we help them ship.

Kitty Yeung: Satya is a big supporter of our hackathon. He walks in the tent and look at the projects. He said last year, “Bigger ideas, more customers.” So, we can hack on anything we want. So it could be small things. It could be something that we use every day. It could be something that has real impact in the society, we can really help our customers achieve their industry scale ideas. So we also work with our customers and we bring our customer come here to hack.

Kitty Yeung: The experimental outlet, we also call it a ship channel. So this is a mechanism for us to get those ideas in but also provide them with the business model, idea building, how to enter the market, and we help our employees ship those projects out. So if you go to The Garage website, you will see about 100 projects that’s already in the market, and we feature our employees who came up with those good ideas. You can see all the teams on the website, everyone who put their part time together to really achieve something. So, we also have very big projects that we collaborated with industry partners and customers.

Kitty Yeung: Intrapreneurs program is kind of a internal startup program. It involves these ideas, these teams, hackathon teams, to actually pitch their ideas to the leaders and get support. So some of these projects can grow into a feature of an existing Microsoft product, or sometimes they become a product of Microsoft.

Kitty Yeung: We also run our internship program very differently. If you are familiar with traditional internships, usually students come in and they work under one manager in a big team working on a small part of a big project. Instead, our interns come in as a team and inside a team usually we hire like 30 students per site. Silicon Valley just started our first pilot program, so we only had one team, but we have six really, really good students. Usually we’ll have teams of six to eight, and they have developers, usually a PM, and a designer, forming a complete skill set. Then business teams at Microsoft pitch their ideas to our interns and the interns pick which one they like to do, and they drive it like a startup in the company for 12 weeks. Then they can deliver the projects back to the team, or even better, we can ship it directly into the market. It’s a very, very competitive and rewarding program. So if you’re undergrad, think about applying to that internship program at The Garage.

Kitty Yeung: We also engage with storytelling, those ideas, those projects got shipped out. We tell a story, we have a PR team, and you will see a lot of news articles about Microsoft innovation. Pay attention next time when you read an article like that if they mention The Garage.

Kitty Yeung: The global sites is also our feature. We have seven global locations right now for The Garage, and we are expanding. Each location has our own ecosystem, and also, each location has our facility. We have maker spaces, we have technologies that we provide to our employees. They can do prototyping, they can bring their ideas to share with their colleagues. We do startup pitching. We do show and tell and workshops to educate our people and also give them a platform to achieve their collaborations.

Kitty Yeung: So these are the seven sites worldwide. We’re in Silicon Valley and we are now called The Garage Bay Area. And as you can imagine, we have a unique ecosystem of a lot of startups, a lot of big companies and universities. So we work with all of these people in the ecosystem and we collaborate to really build projects that can impact the world. So, as I mentioned, we work with our employees and engage with all of our business teams inside Microsoft, and we work with customers. We bring them to work on projects and hack with us.

Kitty Yeung: Here are some numbers. You can see that we have very global and diverse team, but we actually only have 20 people worldwide. So, the 20 people drive all of those activities that I just mentioned. 27,000 hackers this year is an updated number. Last year, behind that 27, there was 23,000. You can see that it’s growing every year. It’s only going to get bigger. 76 countries participate and we’ve held more than 100 interns already. With the most competitive schools around our local areas. You can find more than 100 projects that’s in the market and on the global website. 19 of them became actual Microsoft products and lots of social media posts, lots of news articles about Microsoft innovation. So, make sure you follow us on the social media.

Kitty Yeung: Some of the Bay Area’s specific projects. Seeing AI, we build a lot of projects that help the people with needs, people who have disabilities. Seeing AI is a project that we shipped a few years ago that help blind people see through technology. So you can hold a phone, the camera will detect what’s in front of you and also read it out, interpret. It can also detect facial expressions and people’s age. So it gives blind people information about their surroundings.

Kitty Yeung: Sketch 360 is a project we just shipped last year, is by an artist inside Microsoft, Michael Scherotter. He had an idea of, “Why don’t we sketch 360 pictures directly?” So, we can build like a full environmental canvas and you can draw anything you want. You can also put that into VR or AR to visualize it. We also last year shipped some apps. Spend is by MileIQ team. So, lots of local projects. We’re just going through our hackathon projects this year.

Kitty Yeung: So personally, that’s why I’m also here to do a demo. I’ve build some of the projects in The Garage to satisfy personal ambitions of anyone in Microsoft can use The Garage as a resource to build their communities, can build their projects. So I have built a lot of wearable technologies. I’m doing a demo right there. We have these different dresses with different sensors and AI, machine learning functionality, and robotic dresses that I can show you later on. But I also have a passion for quantum computing because of my physics background. I’m a physicist, actually. So, I see the need to build a community of people learning about quantum. So this is a study group that I founded in Bay Area, teaching people how quantum computing works, including physics, maths, the hardware, and software, and any employee with good ideas, they can do this. So we have a lot of employees who wanted to do, say AR tech community, they can come to The Garage and do that. Or they have passion for IOT, they can come to The Garage and do that. So, these are just some examples.

Kitty Yeung: So since Girls Geek is also sort of about career, I think this will be my last slide to show you something about your aspiration. This is a guide. So see where you are in this chart of Ikigai and see where you are and figure out what would you like to be. I think for me, I can feel Ikigai in Microsoft because I’m doing something I love, something the world needs, and something I can be paid for that’s important, and something I’m good at. So, if you can get to that sweet spot, that should be your goal. Also, think about how you’re aligned to the global goals. That’s what I can do. I highlighted some of the goals that I could do in the company as well as through my personal projects. I think I would love to expand this and I think this will be a good guide for everyone, how we can do more impactful work for the world. Thank you.

Chloe Condon: Okay. Wait. You cannot leave the stage without sharing this dress. I’m going to make you model it. It is so incredible. So, do you want to say a little bit about it first?

Kitty Yeung: Okay. This is one of my designs, among the other ones I brought. All of these prints are my own paintings. This is a painting of Saturn and I wanted to simulate Saturn on the dress. How do I do that? Because Saturn has a ring, so why don’t I make a ring that when I rotate it will show Saturn. It also has an angle detector. There’s an accelerometer in here. So if it achieves a certain angle it will light up like the stars.

Chloe Condon: Amazing, amazing.

Kitty Yeung: Thank you.

Chloe Condon: Thank you so much. When you wear such a fabulous dress, we should have had a catwalk. I’m so sorry everyone. Amazing. Thank you so much, Kitty. I really, really love that and I loved that final slide. I took pictures of it so I can look at it later and map out my own plan. I am so excited to introduce our next guest that is going to tell us all about machine learning. Priyanka, come on up to the stage. I have a little … do you need a clicker? Amazing. Here you go.

Priyanka Gariba speaking

Head of TPM for AI Priyanka Gariba gives a talk on “Leading a large scale and complex machine learning program at LinkedIn” at Microsoft Girl Geek Dinner.  Erica Kawamoto Hsu

Priyanka Gariba: Hi, everyone. First off, I’m not showing off anything as cool as what the other women did, but I also want to say this is my first time here at Girl Geek Dinner and I think this is amazing. Look at the energy, like room full of women. How many times in a day do we get to see that, or even a month, right? So thank you for having me. My name is Priyanka Gariba and I lead Artificial Intelligence Technical Program Management group at LinkedIn. My talk for today is going to be how we are scaling machine learning at LinkedIn. We are one of the large and complex program that has been funded by our engineering group.

Priyanka Gariba: So, I’ve structured my talk into four different areas. I’ll give a quick introduction on LinkedIn and some of the products that are really powered very heavily by machine learning. I will then get into the problem statement of what we are trying to do in order to scale machine learning. Then talk a little bit about our technology, and then wrap it up with sure, we can scale with building a solution and with technology, but there’s also an aspect of people, and so how do we scale that, and what is LinkedIn doing about it? Okay. All right. With that, let’s get started with the vision and mission for LinkedIn.

Priyanka Gariba: Our vision is to create economic opportunity for every single member in the global workforce. Our mission is, the way we are going to realize it is of course by connecting world’s professional to make them more productive. Let’s take an example of this room itself, right? So many cool things that were shown up, so many cool people, so many cool women that we spoke to. Just imagine if we were connected to one another, there’s so much value we can bring in each other’s life, and LinkedIn can help us do that. So, how are we trying to realize our vision and our mission is through some of our products.

Priyanka Gariba: I’m hoping and I think everyone here is at least having a profile on LinkedIn, and if you’re not connected to the cool women here in the room, I encourage that before you leave, definitely connect with one another. But some of the products that really help us do that is People You May Know. This is a product line that really helps us build our connections. It understands, there is a recommendation system that runs behind it, there is machine learning models that run behind it, very heavily AI powered, and it really allows us to know who are the people, like minded people, that we need to be connected to, and the value we can bring in each other’s life by just having that connection.

Priyanka Gariba: Then of course there is Feed. Everybody who goes on LinkedIn as a platform is going to see Feed as the first product. Jobs is another product, which is very heavily powered by machine learning behind it. Why am I talking about all these products? AI at LinkedIn is like oxygen, and one thing that all these products have in common is AI. With that, what that means is we know that machine learning is everywhere. It’s powering every single product line that we build, it’s helping us bring the best experiences to all our members across the board. So, because of that one reason, we know that what we need to do is we need to enable more people to do machine learning at LinkedIn.

Priyanka Gariba: So, there are two pieces to my talk. One, which I think I’ll dive into more than the second one, is going to be technology. There’s one way we can scale technology, is by building a solution. How do we enable our machine learning engineers to really build and deploy models faster so that the experiences that they can bring to all the members is at a faster rate. The second one is by scaling people.

Priyanka Gariba: So, to tap into the exact problem that we are trying to solve, let’s look at our machine learning development life cycle. It’s as simple as any software development life cycle, right? Basically a machine learning engineer has an idea, there’s something you want to solve for, what is the first couple of things that they would do? They’ll think about what are the machine learning features that are available to them? How do you crank up all these features together? Try and test it in an offline model, train with some datasets, and once you value it and feel comfortable that this is something good, the next big piece is going to be actually serving it in production and then seeing results through AB testing and all of that.

Priyanka Gariba: I’m not going to dive too much into this. This really just is an extension of that life cycle. Basically you start with an idea and then there are different functions along the way. There is a product management, there’s dev, and the way we really make decisions on product is very heavily powered by our AB testing platform. We make ramp decisions only based on that. Once we see the results, only then do we believe that that is a model that we want to ramp further to our members.

Priyanka Gariba: Why talk about all of this? Why talk about the life cycle, right? If all these products are being built at LinkedIn and if so many people are doing it and all the teams are doing this, what that means is every single team is doing and deploying models in a very different way. There are many, many technologies, they are all on different stacks, it’s not standardized across the board, and one thing we encourage at LinkedIn is for people to move around within teams. So today if you want to work on a Feed team, tomorrow you want to work on a Job Recommendation team, how do you do that? Your stack is different. Half the days are going to be spent in just ramping up.

Priyanka Gariba: So, we introduced something called as Productive Machine Learning. Really our goal is to enable end to end experience of machine development life cycle to be more robust, reliable, and consistent, and standardized. The experience we are looking for is for an ML engineer, all you have to worry about is come up with an idea, and then there is everything else is opaque for you. There is a big box and you don’t have to worry on how you move from one phase to the other. Ideation to machine learning features to training to scoring to serving it in the introduction. You don’t have to worry about this and how are we going to do that.

Priyanka Gariba: So, we’ve put together this program, it’s to give you context, this is a really large scale program, about 6,200 engineers across the board working on it, different geolocations. The way we are structuring it is by talking about three different phases.

Priyanka Gariba: Model creation, going back to that life cycle that you saw, everything from ideation to training and evaluating your model comes under model creation. So we have multiple components that blend into that. Then the next piece for us is deployment. Once you believe that your model is really good and ready for serving, you deploy it in production. The third piece, this is not really a phase, but something that cuts across, is making sure your quality is accurate. Meaning features that you used for your offline training are very similar to what you see in online. So online, offline consistency.

Priyanka Gariba: So, I just wanted to, because I had 10 minutes, I just wanted to give you a flavor of this big undertaking that we are doing at LinkedIn and also give you a little bit of flavor of how we are structured. Typically, every time we build something, we follow a traditional model. You have a leader, you have multiple managers, you have engineers, and you come up with a goal on a project and everyone works together. This one, we wanted to do something different. What we did is, let’s bring every single person in LinkedIn who is really passionate about solving this problem.

Priyanka Gariba: So put together what’s your team, we had everyone across the board, in different geolocations too. There is someone who will be infrastructure heavy. There is someone who is a machine learning engineer who can help us really give us inputs when we are building the solution that it’s really going to work for them. Then there’s product managers, CPMs, engineers, across the board, but it’s really all of these coming together, forgetting the boundaries of management, realizing that there is one goal that we have, is to get an end to end machine learning life cycle ready, was the key thing for us. I already mentioned that, team of teams, we’re geolocated. That is also one reason why we wanted to do that, is we wanted engineers across the board because if we were solving a problem just for headquarters, which is in Mountain View, we will not be solving for everyone at LinkedIn.

Priyanka Gariba: Then of course with any product that you build in any company, there is a big piece of adoption. So, for us, the strategy that we have used is that let’s, the three big phases that we spoke about, let’s build small components underneath it and let’s allow every product team to pick up a component and adopt that depending on what their pain point is. So, for example, if a Feed team is really struggling with how do you train a model, then what we wanted to offer them is pick up that component and get adopted on that. Once you buy the idea, then slowly and gradually navigate into the adoption of the other components too. This helped both ways. This helped us get real early feedback from our customers and users, and then it also allowed us to load balance. So we could develop things while something was already being tested and we were getting that iteration loop from our users.

Priyanka Gariba: So, I spoke about the technology, and I spoke about the solution. The second thing that LinkedIn is doing, and I’m just giving a very high level preview of this, is in order for us to democratize AI or to make it readily available and to enable more engineers to do that, there’s a program that LinkedIn’s kicked off, it’s called AI Academy. There are three different types of courseworks of program, AI 100, 200, 300. As you graduate from one to the other, really the intensity of the techniques and machine learning increases. So AI 100 is really just getting a flavor of what AI is, what machine learning is, and get you familiarized with it. And then 200 you start understanding how do you build a model, and three is when you actually build your own model and put it in production. I can talk all about this and I’m happy to talk about it later on, but this is just a preview, and there’s a lot of blogs and things that we’ve already put on LinkedIn.

Priyanka Gariba: This is another blog for Productive Machine Learning for those of you who are interested in reading more about it, and I’ll share my slides as well. That’s it. Just a quick flavor. I had 10 minutes, so I thought at least I’ll come up here and talk to you and give you a flavor of what we are doing to democratize machine learning at LinkedIn. But happy to, I don’t know if I have time for questions, but I can take questions later on as well. Thank you.

Priyanka Gariba: Okay. I can take a question or two if … After. Okay. All right. Sure.

Chloe Condon: Thank you so much. All right. So, next up, I will take that from you. Next up we have a very special treat, but before I introduce our very special guest, I’m going to show you my favorite LinkedIn feature. How many people have added someone on LinkedIn tonight? Okay. Well now you’re going to add more people. So, if you go to your LinkedIn app in the very top in the search bar, there is a barcode, a scanning barcode, and if you click on that, instead of having to type out the person’s name and awkwardly ask for spelling, you can just scan their barcode tonight. So you can share that secret tip that I learned recently from someone else at a meet up that I now pass onto you to make spelling people’s names less awkward. So definitely scan everyone’s badge here tonight. My best advice always in tech is to meet as many people as you can, and tell your story and share their stories while you’re here tonight with all these amazing people.

Chloe Condon: I am going to welcome our very, very special guest for tonight, Charlotte. Come on down. We are so excited to welcome Charlotte Yarkoni to the SF Reactor. Here you go.

Charlotte Yarkoni speaking

Corporate Vice President, Cloud + AI Division, Charlotte Yarkoni gives a warm welcome at Microsoft Girl Geek Dinner.  Erica Kawamoto Hsu

Charlotte Yarkoni: Thank you. I need to start out and tell you guys, I’m sick. I really, really apologize for my voice. I’ve been told I don’t look as bad as I sound, so I thought it’d still be okay to show up, but hopefully you’ll manage to go with me this evening. It was important for me to come. So again, I hope you can work with me on the sound quality. But my problem is as I’m watching everybody on stage, I wanted one of these mics so I can put it down, cough, and anywhere I go I’m going to … somebody’s in my blast radius. So, if I come over here and stand by the post, please don’t be offended.

Charlotte Yarkoni: Anyways, good to be here tonight. Thank you guys all for coming. I thought what I would do is first share with you a little bit about my journey of being a woman in tech and what that’s meant to me in my career. I do need a clicker. My telepathic PowerPoint clicking slides are not on today due to the head cold. So, I actually go talk a lot to universities. I go to some high schools. I love talking to young girls about STEM, but I always kind of have to ground in. Let me tell you what tech looked like when I was in middle school and high school.

Charlotte Yarkoni: This was it, by the way. There were no smartphones, there were no tablets, there were no laptops. I remember when Asteroids came out and me and my brothers thought it was amazing. Right? So that’s kind of where we were. Then this was our social network. There was no Twitter, there was no WeChat, there was no Snapchat. It was pretty much a bonfire in somebody’s field when their parents were out of town in the town I grew up in. So, that’s kind of where I come from.

Charlotte Yarkoni: I actually, I grew up in South Carolina. I was super fortunate to get a scholarship to come to UC Berkeley. I’m pretty sure I’m the only person from South Carolina to ever go to Berkeley. I was actually part of an inaugural program at the time called Electrical Engineering or Computer Science, or EECS as it was known. This is what code looked like when I was coding. Has anybody ever written in Lisp? Anyone? Did anyone? Yeah. Kicking it old school. All right. So, that was sort of my education, if you will, and my real foray into tech.

Charlotte Yarkoni: Then, I got out of college and started working and figuring out how to use technology as an applied science, not just in an academic sense, and this was kind of the world I was in. Actually cell phones came out and yes, that’s what they looked like for those of you that weren’t born then, because I know there’s a few of you here. Windows 95 was all the rage, right? You remember that? Then we get to today and it’s just a very, very different world.

Charlotte Yarkoni: One of the things that I love about technology is the fact that it has actually opened up all of our worlds, in so many ways that we can have so much more impact. We can instantly connect to people that we could never connect to 30, 40, 50 years ago. I’m not that old, I’m just framing my comments. But you think about that and it’s not just connecting to those people, it’s the access to information that you also have immediately at your fingertips. It’s amazing. It’s amazing that what you can harness with that kind of resources at your fingertips.

Charlotte Yarkoni: The challenge is, though, it comes with a responsibility, and I will tell you, at Microsoft, and GitHub, and LinkedIn, we spend a lot of time on that. In fact, it’s not just about innovating, it’s about innovating with purpose, and really making sure that you’re actually leaving the world in a better place than you found it before you introduced your solutions. So it’s those unintended consequences that you have to be very thoughtful about. As we continue to get more and more technology at our disposal, how do we use it for good? That kind of brings me to really, what’s my role.

Charlotte Yarkoni: Today in my role is, at Microsoft, I run a group called Commerce and Ecosystems. You can tell I’m not a marketing person, so there you go. But I’m really here. I focus on answering three questions. The first is, how do people actually discover who we are and what we do in our products and services? And Microsoft’s a very big company, it’s a global landscape. We offer lots of different products and services across our portfolio, but there are a lot of ecosystems and communities that actually don’t know who we are and what we do.

Charlotte Yarkoni: Five years ago it was a lot about open source, and I remember I actually went to … I started at Microsoft about three years ago and I went to an open source conference. By the way, I grew up in open source, so my background actually started out in Unix and moved to Linux. I never wrote a piece of code in .NET. Would probably look and feel a little bit like Lisp to me, honestly, if I tried to do it now. So when I came to Microsoft, I went to a familiar conference, and people were like, “Why are you here, man? Azure doesn’t run Linux.” I’m like, “What are you talking about? Yeah, it does.” People need to know, right? So we had to go fix that.

Charlotte Yarkoni: Second thing I focus on is after you discover us, how do you engage with us in a way that’s meaningful to you? And most of that is online. People don’t always want to have to go somewhere to learn how to do something. They will now have to sign up for a week long course, right? Necessarily to know how to build a solution using the technology that they have. So we spend a lot of time and energy focused on that and what’s the set of tooling or resources that we can offer.

Charlotte Yarkoni: Then the final point is, how do we just get easier to do business with our customers and partners? That’s where the commerce piece comes in and it’s all about what are some of the new business models we need to create to actually, how do we run all those capabilities across all our products and all our channels today? So there is a good bit of engineering that comes in each one of these aspects, but there’s also a lot of business work that I have to focus on. And again, it comes with that overarching layer of responsibility, is to how do we think about continuing to make progress in a positive way so we can have a positive impact on the communities we serve.

Charlotte Yarkoni: So that’s kind of who I am, and I think what we’re going to do at this stage is a little bit of like an AMA, and I’m really hoping you guys don’t ask me too many questions because the more I talk I think the worse I sound, but I will try to answer everything for sure. I was going to have Chloe join me, and I was going to have Shaloo Garg join me. So, just as a reminder of both, Chloe and Shaloo are part of my team and they’re part of the drive discovery effort. So I’ll let you guys, you guys will talk a little bit more about yourselves, I’m sure, but I’m going to turn it over to our master of ceremonies. Kick us off. Do you want that mic or you want–

Chloe Condon: Sure. Mics all round here.

Charlotte Yarkoni: This one may be contaminated.

Chloe Condon: All right. I wouldn’t want to catch the virus, the Charlotte virus. Amazing. So, I figure we’ll have a seat. Have a seat wherever. We had a bunch of people submit questions earlier in our fishbowl, thank you so much for all of the questions that we got earlier. So, what I figured I would do is we would start with an introduction with Shaloo. Would you like to tell everyone who you are, what you do?

Shaloo Garg, Chloe Condon, Charlotte Yarkoni

Microsoft girl geeks: Senior Cloud Developer Advocate Chloe Condon, Corporate Vice President for Cloud + AI Charlotte Yarkoni, and Managing Director of Silicon Valley’s Microsoft for Startups Shaloo Garg answer audience questions with candor at Microsoft Girl Geek Dinner.  Erica Kawamoto Hsu

Shaloo Garg: Yeah. Absolutely. Firstly, thank you guys so much for coming here today. It means a lot. My name is Shaloo Garg and I lead the startup business growth for Silicon Valley for Microsoft, and entire California as well. It’s an exciting space to be in, and part of Charlotte’s team and part of what we do is not only engage with founders and CTOs and CIOs here of startups, but also drive meaningful partnerships, which is … this is Silicon Valley, there are a lot of partners here, how do we work with them to drive awareness of how Microsoft can help entrepreneurs there? So good to be here.

Chloe Condon: Amazing. Thank you so much. I have these randomly selected questions here.

Shaloo Garg: Those are a lot of questions.

Chloe Condon: It’s a lot of questions. I don’t know if we’re going to get through all of them. We may do kind of a rapid inside the actor’s studio type of lightning round at the end here. But I love this first one. I chose this one first and this is for Charlotte. It says, “What’s it like being an executive at one of the top companies? Do you have a life?” Great phrasing, whoever wrote this.

Charlotte Yarkoni: I’d like to think I have a life. Yes, I do have a life. I have two children, both girls, one–

Chloe Condon: Great. Are they coding already?

Charlotte Yarkoni: One is 23, just graduated. She went to Reed College, and by the way, back to Berkeley, I thought when I went to Berkeley from South Carolina, I was an enlightened liberal. And when I dropped my daughter off at Reed College, I felt like I was the most conservative person on the planet. I was a little worried about my life choices at that point. But she graduated there in linguistics and she actually is starting school this week, getting her master’s at University of Washington.

Charlotte Yarkoni: She would be very offended if I called her a developer or an engineer, yet she spends a lot of time writing programs and are doing statistical analysis on languages because she focuses on Russian, Japanese, Spanish language and language heritage.

Chloe Condon: Wow.

Charlotte Yarkoni: So, that’s my oldest. My youngest is 13, and a prolific gamer and developer. Python is her language of choice. She has lots of opinions about every other language.

Chloe Condon: As she should.

Charlotte Yarkoni: It kind of takes me longer these days to set up an environment for her to code in than it does for her to whip out a new game that she’s thinking about. So, I’m pretty sure she’s going to end up somewhere in the engineer community as a professional at one point. I also have three horses. I ride. I grew up three day eventing, for those of you who know what that is. Now that I’m older and have kids, I wondered what my parents were thinking when they let me do that. But I still ride and I still compete. Then I do my day job.

Chloe Condon: That is a fun fact.

Charlotte Yarkoni: I think the thing about today’s technology is, the good and the bad is it allows you to be accessible all the time. So, you can actually, you have to know how to be at the right place at the right time, which is usually the conflict that occurs, but you are able to go do what you need to do personally and do things professionally as you go. So that’s something I’m really, I feel privileged by who I work for in the industry I’m in and the technologies that we’ll be bringing for all the working moms out there.

Chloe Condon: Wow. That’s actually a great segue into the next question, which I’ll direct to Shaloo first, which is, how do you relax and unwind? Like with how long and tough your day jobs are, how do you get to chill?

Shaloo Garg: So, best is tennis. I love playing tennis and that’s how I unwind, and when I go out and play tennis, I try not to take my cell phone with me or my kids. So I have a 13-year-old daughter too, and a nine-year-old son who quite a handful.

Charlotte Yarkoni: Do you have any Serena moments on the court?

Shaloo Garg: I do. But that’s how I unwind, which is just completely unplug, just a moment of Zen and just go out there and hit it.

Chloe Condon: I’m very similar. I craft. I like to do like things with my hands and not look at a screen and just build something fun, like a costume or something that lights up. And you’re riding horses.

Charlotte Yarkoni: Yeah, but I could not build a costume. So, we each have our strengths.

Chloe Condon: Hit me up for Halloween. We’ll get you guys–

Charlotte Yarkoni: I’m going to hit you up for Halloween. Okay.

Chloe Condon: This one says, “What would be your advice for your past self coming straight out of college?” I love that question.

Charlotte Yarkoni: Who you asking?

Chloe Condon: Anyone can jump in. Yeah.

Shaloo Garg: I think coming out of college, I wish I was more aware of getting a coach or a mentor, which I was not aware. And during my career I sort of looked upon women leaders and requested them to be mentors and coaches. So what I try to do now is go out and coach and mentor women or young girls myself. So, I realize that they may be in the same situation as I was in, which is, “Hey, I can ask a woman leader to say, ‘Would you mind spending 30 minutes with me?'” But they don’t ask. Right? So I preemptively do that in schools, colleges here in Silicon Valley. Actually right up our Market Street office, that’s another office of ours, every month, I host open office hours for young women who are out there, budding entrepreneurs. It doesn’t have to do anything with Microsoft. So, as soon as you walk in the door, it doesn’t have to be, “Hey, you have to sign up to work with us,” but it’s just coaching, and I love it. So, wish I had that, but a part of me is just giving back, just making sure that someone out there is benefiting.

Chloe Condon: Yeah, that’s great advice. Charlotte.

Charlotte Yarkoni: I think, for me, one of the things that it’s taken me a long time to appreciate and I really, I encourage everybody to have some thought about this for their own journey, both personally and professionally, resilience is such an important thing. When I look back on my career, I feel, again, very privileged to have worked in all the places and spaces that I have. But the successes I had weren’t one success right after the other. It was a success built off of quite frankly, a mountain of failures and trials to get there. It was about taking those learnings and applying and getting better. I think a lot of what we do as an industry is about solving a problem, solving an opportunity, and getting better as we go, and iterating, and it’s really hard to do that as a person.

Charlotte Yarkoni: I’m going to go out on a limb and assume all you people here are somewhat overachievers. So every time that you have a failure, you want to prosecute the failure and you want to prosecute yourself, and that’s okay as long as you make it a constructive thing and learn from it, and the older you get and the more experienced you get, the more you start to really embrace and almost be proud of those failures for what they taught you, because you wouldn’t be wherever you are without it. That’s just a fact. I don’t know that I appreciated that in my younger age. I was certainly an overachiever and thought I knew a lot more than I knew at the time. I know that’s shocking, but it’s true. But as I went through my career, it was a process for me to understand how to really get value in the mistakes, how to really give value in the failures, and use them to move forward.

Charlotte Yarkoni: I just would encourage everybody, get out there and try. That’s step one and step two, is make sure you learn and embrace the mistakes, right? And it is about that of resilience that will just make you so much of a better person whatever you decide to do, however you decide to do it.

Chloe Condon: My advice would be, I don’t think I knew right when I graduated what I wanted to do with the rest of my life. I wish I had taken a little time to travel or maybe to explore different industries and fields that maybe I wanted to dip my toe in. Because I think what the wonderful thing about working in tech is you don’t have to commit to doing the same thing for your entire life. You can always change and learn a completely new technology or … There was a tweet that I think I retweeted this morning, which was, “Your job that you have in five years may not even exist. So try not to plan out your life too strategically,” and I think that’s really wonderful advice because technology is growing at a rapid rate and we may be working for something we don’t even know exists yet. The new, I don’t know, a new iPhone. Who knows?

Chloe Condon: Great. Next question that I have is, I love this one, “What’s the best book you’ve read this year?” Does anyone have one? I know mine. I can go first while people think.

Shaloo Garg: Go, go for it.

Chloe Condon: I read a book. Oh no, you go first because I want to make sure I get her name right, the author’s name right.

Shaloo Garg: So I think the life-changing moment for me was the book that I read by Eckhart Tolle. It’s called The Power of Now, and it teaches you a lot about what Charlotte talked about, failure. It also teaches you how to stay engaged but not attached, which is you’re really passionate about something that you’re doing. Keep that passion, but don’t get so emotionally sucked into it that you break down. So it also teaches you mindfulness and awareness. And then how to be an A player, which is you’re mindful, you’re aware of what you’re doing, but guess what? You got to go and get it. So I thought that was completely life-changing for me because I learned quite a bit in terms of just being strong, being very passionate about what I do, but not emotional, and then just chasing it, chasing the ball and just chasing the heck out of it.

Charlotte Yarkoni: Mine’s an oldie but a goodie, because my youngest was doing a book report on this one, the Life of Pi.

Chloe Condon: That’s a good one.

Charlotte Yarkoni: I just loved that. I haven’t read it in many years and so she brought it home and I brought out my copy so we could read it together. It is just an amazing book.

Chloe Condon: That is on my list. You said yours was The Power of Now?

Shaloo Garg: Power of Now.

Chloe Condon: Okay. Write that one down, everyone. I recently read Just the Funny Parts by Nell Scovell, she’s a female comedy writer, and I found … it’s an autobiographical piece. She used to write for Saturday Night Live, David Letterman, and it’s a completely male dominated field. It was the first time I had read about an industry other than tech that was similarly structured and formatted and it talked about, she’s a comedy writer, so it comes from this place of empathy and humor, and I would highly recommend it. She helped write Sheryl Sandberg’s book. She also wrote a lot of Obama’s jokes, I found out in that book. So, a lot of the things that made us chuckle from Obama came from her.

Chloe Condon: So, next one is, “Who has influenced you most in your life and why?”

Charlotte Yarkoni: That one’s actually really hard. I will tell you both my parents passed away in the last year. They were quite older. I’m the youngest of a large family. Pretty sure I was an accident, so, it’s okay. But you spend a lot of time reflecting on your nuclear family when those kinds of things happen, and they happen inevitably to everyone. So I definitely think my parents had a large influence on my life. I think my teachers had a large influence on my life. I’m the proud product of the public education system of South Carolina, which I think at the time I was growing up was like 49th in the country. But I went from there to UC Berkeley, which was an amazing school. And I had some amazing teachers to help me learn how to learn, is what I got from that.

Charlotte Yarkoni: I’ve been super fortunate to have some great mentors and what I would call guidance counselors throughout my career, that I still do lunch with and dinners with and catch up with. So, I feel like I’ve had a lot of influences and I do think for the last 20 plus years, though, my kids have probably taught me more humility and patience and resilience and all the other virtues we speak so highly of. They’ve probably been the biggest forcing function in my life in recent years.

Chloe Condon: What about the horses?

Charlotte Yarkoni: The horses are my sanity. I will tell you, we moved to Australia for a couple of years and I couldn’t take my horses with me and I was, my husband will tell you, I was a miserable person for the time I was gone.

Chloe Condon: I’m picturing you writing postcards back to your horses at home.

Charlotte Yarkoni: I came home. I came home every two months to see them.

Chloe Condon: Aww. How about you, Shaloo?

Shaloo Garg: So, parents, but I think my mom. So I lost my parents at a very young age. I remember when thinking back growing up, so I was born in India, but I grew up in Middle East, and I grew up in a community where there was lot of domestic violence and girls were not allowed to go to school. And so there were a lot of changes that were happening around me. In fact, while growing up, I went to 14 different schools between elementary, middle, and high school. So you can imagine moving from Saudi Arabia to Iraq, to Kuwait during the war zone time. But I remember going through all this, my mom always taught me and my sister is that, if there’s ever a problem in life and there is a simpler solution, and there is a hard solution, guess what? Pick the hardest one, because it’s going to make you go through that process, whereas a simpler one, you’re just going to take it and just sit with it and you’re not going to learn anything. So I do look back and I think that she’s had an amazing influence on me.

Shaloo Garg: And as Charlotte said, my kids, I keep learning from them every single day. They teach me so many things in terms of if I get upset about something, they’ll just say, “Hey mom, just relax. This is just a small thing, just move on.” I think that’s how I keep learning more and more. And of course, amazing coaches and mentors and some really amazing female leaders who I look upon to.

Chloe Condon: I would have to agree. My mother passed away when I was 16, but she was a costume designer, graphic designer, creative arts person, and I try to bring my creative arts training and background into all the technology that I do and create. So I think that was probably the biggest influence on me, would have to be my mom as well.

Chloe Condon: What is the biggest challenge we are facing in tech currently? A tough one.

Charlotte Yarkoni: I actually think our biggest challenge as a society is climate change. I think technology can be a solution for that. So, that’s an indirect answer to a direct question, but I would say that is the thing that I would love to see all of us, I don’t care what you’re doing, where you’re working, but to start having serious thoughts about how we can go reverse decades of adverse effect on the planet. It helps everybody, and I do think the real accelerants are going to lie not just in changing our behavior and our consumption, but also in having technology help us. I don’t think we’ve really gone there yet as a society at large. So for me, it’s something I’m kind of anxious to push along however I can in whatever small way that I can. I think that’s how I think about it.

Charlotte Yarkoni: With technology, you have things like quantum, which is just amazing. The beauty of working somewhere like Microsoft is we are spending a ton of research and we have really crazy people, crazy smart people working on this, and every now and then if I have to go give a talk and I need to give my five minutes of quantum computing update for the cloud, I always ask, “Are there any theoretical physicists in the audience? Because if there are, I’m not going to do this because you know way more than me,” kind of thing.

Chloe Condon: Come on up.

Charlotte Yarkoni: But it’s amazing, and in essence you take what sits in a data center the size of a football field today and you can run it in what’s in the size of a refrigerator in your house. But, the cooling you need to do that is extraordinarily more than the power we’re consuming today, and the impact that will have, by the way, if it’s not done right, either we’re not producing it correctly and/or we’re not cooling it correctly, can have a devastating effect. So how do we think about things like that, these new trends with this aspect of sustainability around the climate, I think is super important. So I apologize, I kind of rambled on that answer, but I actually think this one’s a really important one.

Chloe Condon: I agree. I actually met someone at Open Source Summit recently who works on our IOT team here at Microsoft in Redmond, and his job on the IOT team is to help offset our carbon emissions from our server center. So I thought, “That’s such an important, important way for us to help make the environment a better place with Microsoft.” So, yeah.

Charlotte Yarkoni: Absolutely, and the lady who runs our data centers, her name is Noelle, she’s a peer of mine. I love her dearly. She’s just an amazing woman. She actually grew up as a chemical engineer.

Chloe Condon: Wow.

Charlotte Yarkoni: A lot of her time on how do we run our data centers is spent in areas that you and I wouldn’t know how to go solve, because it is about how do you think about power? How do you think about new sources like geothermal and things like that. I think it’s great. I think it’s great we’re thinking that way, but we got to do more.

Chloe Condon: Yeah.

Shaloo Garg: I think the biggest challenge is the knowledge or the lack of awareness behind power of technology. So, I often see this, I keep bringing up edtech as a very common example, and in fact, here in the Valley, edtech is right now the hottest topic in the social impact circle. I can guarantee you, when I throw the word school out here and I ask you to just close your eyes and think of, tell me what you think of. You’re going to think of a building. You’re going to think of kids running, a blackboard, and a teacher. But that’s not what education is only. Education can be a seven-year-old girl sitting in Uganda who’s not allowed to go to school, but she can sit at home and do schooling at home using an iPad, right? Just because she’s a girl, she’s not allowed to go to school.

Shaloo Garg: That is the power of technology, and it kills me every single day when I read about places like Somalia and Syria, and so many other places, where easily companies, and Microsoft does amazing job, that’s one thing I’m really proud to be, which is be part of this company. We do amazing work globally in enabling this. I think we need to continue to talk about the power of technology, which we do in our jobs and outside our jobs, but we need more and more people to go out there and coach people and say, “Hey guys, education is just not about textbooks. It can be digital education powered by technology.” I think that to me is the biggest challenge right now, which is lack of awareness.

Chloe Condon: Yeah, accessibility and access to that is so important.

Charlotte Yarkoni: Can I interrupt this broadcast? Do we have any recruiters in the audience? Because I think we have our newest recruit. She did an awesome walk-in by the way.

Chloe Condon: Love the pants. Great pants. This is a very fun question. What emoji do you use most often?

Charlotte Yarkoni: I don’t use them correctly, as my children … I always send them stuff–

Chloe Condon: It’s the horse one, right?

Charlotte Yarkoni: … and they’re like, “Why did you send me this? Do you know what this means?” I’m like, “No. No.”

Chloe Condon: I think that’s part of your job as a mom, right?

Charlotte Yarkoni: Well, I have gotten in this habit of sending random ones just to freak my kids out.

Chloe Condon: Love it.

Charlotte Yarkoni: I usually am pretty clean at work with the okay and the goofball face, and the smiley face, but it cracks me up because we were just having this discussion the other day, because I sent something that apparently I shouldn’t have sent as a parent.

Chloe Condon: It’s like a secret hidden emoji language.

Charlotte Yarkoni: It really is.

Chloe Condon: Yeah.

Charlotte Yarkoni: And you, what do you use?

Chloe Condon: I would say it’s a tie between the sobbing emoji and the laugh crying emoji, because I don’t have any other two emotions other than those two extremes. There’s no in between for me. I’m either hysterically laughing or hysterically crying.

Charlotte Yarkoni: What do you use, Shaloo?

Shaloo Garg: Smile and laughter, and that’s it. For the kids, with the kids, I’ll just use hearts, and sometimes my daughter says, “Mom, just stop using those… You’re embarrassing me, mom.”

Chloe Condon: Yeah. What are the most important decisions you face every day? Or what is the most important decision you face every day?

Shaloo Garg: How to make founders successful, and especially in a market like this. I just love it. It’s an upstream market, constantly challenging ourselves. What else can we do? What else can we do in this market? I absolutely love it. It is challenging. It’s extremely challenging.

Chloe Condon: It’s a huge question.

Shaloo Garg: It’s a huge question. I’ve been with the company for eight months and when I joined initially, I was a bit nervous. I was like, “Great, I’m so excited about this job,” and when I went out there, talked to founders, everyone was like, everyone gave me a standard response, “Well, yeah, okay.” But now slowly and slowly we’ve started building it as part of the narrative that we haven’t only the meetings, which is how do we help the founders, and if we switched that, our jobs become much more easier, which is, “I’m here to help you and this is how I can help you.” So I think that to me is absolutely the most fun part.

Chloe Condon: Yeah.

Charlotte Yarkoni: By the way, as part of my team, that’s a great answer for these little startups. I think my job is really making the set of decisions that best serve our customers, our partners, best serve the team. It’s always a balance, right? We have so much we’ve got to get done. We love innovating, we love getting new capabilities out there, making sure that we’re doing that with the right sense of urgency and the right balance for the teams delivering them. Most of my day, in any one of my teams that I look at, is just making the right calls to make sure that we’re doing right by the community, as both our community that’s working on it and the communities we’re trying to serve.

Chloe Condon: Yeah. I would say for me it’s how to get people excited to learn, and what is going to get them having fun. Because I think we work all day, we work like an eight-hour plus day sometimes in front of machines using technology, and what are fun creative ways to get people excited about that and to build really cool, amazing things together that can solve these big questions and problems like the environment and getting accessibility to folks who don’t have the access to this technology. So, it’s always fun to enable that power to people.

Chloe Condon: How much time do we have? Do we want to do maybe one or two more questions? One more question. Okay, cool. Let’s see. I think this is a really good … Actually, I would love to end with your advice to all of our amazing women in this audience, and men in the audience. What would be your advice to someone who’s looking to move up in their career and have a successful career as a person in tech?

Charlotte Yarkoni: I think being you is the most important part. Whatever that means, right? Just be your most authentic self. It’s a hard thing to do. It’s a hard thing in our industry. It’s a hard thing in super competitive environments like here in San Francisco. Seattle is very similar in that regard. I have found people get the most reward and have the most success when they’re actually themselves, whatever that means. I also think being the authentic you will not just make you better, it will actually make whatever team you’re on better. It will make whatever company you’re at better, it will make whatever product or service you’re working on better. Just be you and be proud to be you.

Chloe Condon: I love that.

Shaloo Garg: So, I would say do what you’re passionate about because when you’re passionate, you bring your best. Do not be afraid to take risk, and I know this sounds like a cliche, but really challenge yourself. If there is a risk, if you want to do something and it looks very risky, just go ahead and do it. Maximum, you’re going to fail, but you’ll learn something from it. If you come out victorious, that’s great. Then the last thing I would say is just trust yourself and just believe in your instinct that you’re doing good for the business, you’re doing good for the company, you’re also doing good for those startups or customers or whoever your stakeholders are, and just go chase it. If you keep it straight and if you keep what I call the compass straight, there’s going to be lots of amazing learning in the process.

Chloe Condon: My advice is actually a great segue into our mingling and happy hour section. Mine would be to talk to as many people as you can in this industry. If you have the opportunity to get coffee with someone you really idolize or a mentor, or someone who’s doing what you want to be doing in this industry, having conversations, I think, is so wonderful and you are all about to use that LinkedIn feature that I just taught you, and meet some really amazing people. So make connections and network and yeah, have the most amazing time.

Chloe Condon: I want to thank both of our…

Shaloo Garg: Thank you.

Chloe Condon: … panelists today. Round of applause for Shaloo and Charlotte.

Charlotte Yarkoni: Thank you for hosting.

Chloe Condon: Of course. Thank you to to Kitty. Thank you to Priyanka. Thank you to everyone, to Kaitlyn who’s not here, but oh my gosh, that amazing, amazing musical performance we had to start off the evening. Please, enjoy yourselves. I think we still have some beverages and snacks here, so have a wonderful time. Make sure you get some swag and stickers and we will be around to chat. All right. Thanks everyone.

Microsoft girl geeks, Microsoft Reactor fun

Microsoft girl geeks and allies: Thank you to all the Redmond, San Francisco and Silicon Valley teams who worked together to make this happen!   Erica Kawamoto Hsu / Girl Geek X

Kitty Yeung Microsoft Girl Geek Dinner

Microsoft Garage Manager Kitty Yeung is a creative technologist with a skirt that lights up when she spins.  Erica Kawamoto Hsu

girl geek experiencing Microsoft mix reality

Principal Program Manager Lead Jane Fang and SF Academy Head of Marketing Jo Ryall demo “Mix Reality” to a girl geek  at Microsoft Girl Geek Dinner.   Erica Kawamoto Hsu / Girl Geek X

Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X OpenAI Lightning Talks and Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Gretchen DeKnikker, Sukrutha Bhadouria

Girl Geek X team: Gretchen DeKnikker and Sukrutha Bhadouria kick off the evening with a warm welcome to the sold-out crowd to OpenAI Girl Geek Dinner in San Francisco, California.   Erica Kawamoto Hsu / Girl Geek X

Transcript of OpenAI Girl Geek Dinner – Lightning Talks & Panel:

Gretchen DeKnikker: All right, everybody, thank you so much for coming tonight. Welcome to OpenAI. I’m Gretchen with Girl Geek. How many people it’s your first Girl Geek? All right, okay. Lots of returning. Thank you for coming. We do these almost every week, probably like three out of four weeks a month. Up and down the peninsula, into the South Bay or everywhere. We also have a podcast that you could check out. Please check it out, find it, rate it, review it. Give us your most honest feedback because we’re really trying to make it as awesome as possible for you guys. All right.

Sukrutha Bhadouria: Hi, I’m Sukrutha. Welcome, like Gretchen said, Angie’s not here but there’s usually the three of us up here. Tonight, please tweet, share on social media, use the hashtag GirlGeekXOpenAI. I also, like Gretchen, want to echo that we love feedback, so any way you have anything that you want to share with us. Someone talked about our podcast episodes today. If there’s any specific topics you want to hear, either at a Girl Geek Dinner or on our podcast, do share that with us. Either you can find us tonight or you can email us. Our website is and all our contact information’s on there. Thank you all. I don’t want to keep you all waiting because we have amazing speakers lined up from OpenAI, so.

Sukrutha Bhadouria: Oh, one more quick thing. We’re opening up sponsorship for 2020 so if your company has not sponsored a Girl Geek dinner before or has and wants to do another one, definitely now’s the time to sign up because we fill up pretty fast. We don’t want to do too many in one month. Like Gretchen said, we do one every week so definitely would love to see a more diverse set of companies–continue to see that like we did this year. Thank you, all. Oh, and over to Ashley.

Ashley Pilipiszyn speaking

Technical Director Ashley Pilipiszyn emcees OpenAI Girl Geek Dinner.   Erica Kawamoto Hsu / Girl Geek X

Ashley Pilipiszyn: All right, thank you.

Sukrutha Bhadouria: Thanks.

Ashley Pilipiszyn: All right. Hi, everybody.

Audience: Hi.

Ashley Pilipiszyn: Oh, awesome. I love when people respond back. I’m Ashley and welcome to the first ever Girl Geek Dinner at OpenAI. We have a … Whoo! Yeah.

Ashley Pilipiszyn: We have a great evening planned for you and so excited to see so many new faces in the crowd but before we get started, quick poll. How many of you currently work in AI machine learning? Show of hands. All right, awesome. How many of you are interested in learning more about AI machine learning? Everybody’s hands should be up. All right. Awesome. We’re all on the right place.

Ashley Pilipiszyn: Before we kick things off, I’d like to give just a brief introduction to OpenAI and what we’re all about. OpenAI is an AI research lab of about 100 employees, many of whom you’re going to get to meet this evening. Definitely, come talk to me. Love meeting you. We’ve got many of other folks here, and our mission is to ensure that safe, artificial general intelligence benefits all of humanity.

Ashley Pilipiszyn: To that effect, last year we created the OpenAI Charter. The charter is our set of guiding principles as we enact this mission and serves as our own internal system of checks and balances to hold ourselves accountable. In terms of how we organize our research, we have three main buckets. We have AI capabilities, what AI systems can do. We have AI safety, so ensuring that these systems are aligned with human values. We have AI policy, so ensuring proper governance of these systems.

Ashley Pilipiszyn: We recognize that today’s current AI systems do not reflect all of humanity and we aim to address this issue by increasing the diversity of contributors to these systems. Our hope is that with tonight’s event, we’re taking a step in the right direction by connecting with all of you. With that, I would like to invite our first speaker to the stage, Brooke Chan. Please help me welcome Brooke.

Brooke Chan speaking

Software Engineer Brooke Chan from the Dota team gives a talk on reinforment learning and machine learning at OpenAI Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Brooke Chan: Yeah. Hello. Is this what I’m using? Cool. I’m Brooke Chan. I was a software engineer on the Dota 2 team here at OpenAI for the past two years. Today, I’m going to talk a little bit about our project, as well as my own personal journey throughout the course of the project.

Brooke Chan: We’re going to actually start at the end. On April 13th, we hosted the OpenAI Five Finals where we beat the TI8 world champions OG at Dota 2 in back-to-back games on stage. TI stands for The International, which is a major tournament put on by Valve each year with a prize pool upwards of $30 million. You can think of it like the Super Bowl but for Dota.

Brooke Chan: There have been previous achievement/milestones of superhuman AI in both video games and games in general, such as chess and Go, but this was the first AI to beat the world champions at an eSports game. Additionally, as a slightly self-serving update, OG also won the world championship this year at TI9 just a few weeks ago.

Brooke Chan: Finals wasn’t actually our first unveiling. We started the project back in January of 2018 and by June of 2018, we started playing versus human teams. Leading up to finals, we played progressively stronger and stronger teams, both in public and in private. Then most recently, right before finals, we actually lost on stage to a professional team at TI8, which was the tournament that OG later went on to win.

Brooke Chan: Let’s go back to the basics for a minute and talk about what is reinforcement learning. Essentially, you can think of it as learning through trial and error. I personally like to compare it to dog training so that I can show off pictures of my dog. Let’s say that you want to teach a dog how to sit, you would say sit and just wait for the dog to sit, which is kind of a natural behavior because you’re holding a treat up over their head so they would sit their butt down and then you would give them that treat as a reward.

Brooke Chan: This is considered capturing the behavior. You’re making an association between your command, the action and the reward. It’s pretty straightforward for simple behaviors like sit but if you want to teach something more complicated, such as like rolling over, you would essentially be waiting forever because your dog isn’t just going to roll over because it doesn’t really understand that is something humans enjoy dogs doing.

Brooke Chan: In order to kind of teach them this, you instead reward progress in the trajectory of the goal behavior. For example, you reward them for laying down and then they kind of like lean over a little bit. You reward them for that. This is considered to be shaping rewards. You’re like teaching them to explore that direction in order to achieve ultimately your goal behavior.

Brooke Chan: Dota itself is a pretty complicated game. We can’t just reward it by purely on winning the game because that would be relatively slow so we applied this technique of shaped rewards in order to teach the AI to play the game. We rewarded it for things like gold and kills and objectives, et cetera. Going more into this, what is Dota?

Brooke Chan: Dota is a MOBA game which stands for multiplayer online battle arena. It’s a little bit of a mouthful. It’s a game that was developed by Valve and it has an average of 500,000 people playing at any given time. It’s made up of two teams of five and they play on opposite sides of the map and each player controls what’s considered a hero who has a unique set of abilities.

Brooke Chan: Everyone starts off equally weak at the beginning of the game, which means that they’re low levels and they don’t have a lot of gold and the goal is that over the course of a 30 to 60-minute game, they earn gold and become stronger and eventually, you destroy your opponent’s base. You earn gold and experience across the map through things like small fights or like picking people off, killing your enemy, taking objectives, things like that. Overall, there’s a lot of strategy to the game and a lot of different ways to approach it.

Brooke Chan: Why did we pick Dota? MOBAs in general are considered to be one of the more complex video games and out of that genre, Dota is considered the most complex. Starting off, the games tend to be pretty lengthy, especially in terms of how RL problems typically are, which means that strategy tends to be hard with a pretty delayed payoff. You might rotate into a particular lane in order to take an objective that you might not be able to take until a minute or a minute and a half later. It’s something that’s kind of like hard to associate your actions with the direct rewards that you end up getting from them.

Brooke Chan: Additionally, as opposed to games like Go and chess, Dota has partial information to it, which means that you only get vision around you and your allies. You don’t have a full state of the game. You don’t know where your enemies are and this leads to more realistic decision-making, similar to our world where you can’t like see behind walls. You can’t see beyond what your actual vision gives you.

Brooke Chan: Then, finally, it has both a large action and observation space. It’s not necessarily solvable just by considering all the possibilities. There’s about 1,000 actions that you can take at any given moment and the state you’re getting back has the value size of about 20,000. To put it in perspective, on average, your game of chess takes about 40 moves and Go takes about 150 moves and Dota is around 20,000 moves. That means that the entire duration of a game of chess really wouldn’t even get you out of the base in Dota.

Brooke Chan: This is a graph of our training process. On the left, you have workers that all play the game simultaneously. I know it’s not super readable but it’s not really important for this. Each game that they’re playing in the top left consists of two agents where an agent is considered like a snapshot of the training. The rollout workers are dedicated to these games and the eval workers who are on the bottom left are dedicated to testing games in between these different agents.

Brooke Chan: All the agents at the beginning of the training start off random. They’re basically picking their actions randomly, wandering around the map doing really awfully and not actually getting any reward. The machine in green is what’s called the optimizer so it parses in all of these rollout worker games and figures out how to update what we call the parameters which you can consider to be the core of its decision-making. It then passes these parameters back into the rollout workers and that’s how you create these continually improving agents.

Brooke Chan: What we do then is we take all of these agents and we play them against all the other agents in about 15,000 games in order to get a ranking. Each agent gets assigned a true skill, which is basically a score calculated on its win-loss records against all the other agents. Overall, in both training and evaluation, we’re really not exposing it to any kind of human play. The upside of this is that we’re not influencing the process. We know that they’re not just emulating humans and we’re not capping them out at a certain point or adding a ceiling on it based on the way that humans play.

Brooke Chan: The downside of that is that it’s incredibly slow. For the final bot that we had play against OG we calculated that it had about 45,000 years of training that went into it. Towards the end of training, it was consuming about approximately 250 years of experience per day. All of which we can really do because it’s in simulation and we can do it both asynchronously and sped up.

Brooke Chan: The first time they do get exposed to human play is during human evaluations. They don’t actually learn during any of these games because we are taking an agent, which is a snapshot and frozen in time and it’s not part of the training process. We started off playing against our internal team and our internal team was very much not impressive. I have us listed as 2K MMR, which is extremely generous. MMR means matchmaking rating which is a score that Valve assigns to the ranked play. It’s very similar to true skill. 2K is very low.

Brooke Chan: We were really quickly surpassed. We then moved on to contract teams who were around like 4K-6K MMR and they played each week and were able to give us feedback. Then in the rare opportunities, we got to play against professional teams and players. Overall, our team knew surprisingly little about Dota. I think there are about four people on our team who had ever played Dota before and that’s still true post-project, that no one really plays Dota.

Brooke Chan: This leads us to our very surprising discovery that complicated games are really complicated and we dug ourselves into this hole. We wanted a really complicated game and we definitely got one. Since the system was learning in a completely different way than humans, it became really hard to interpret what it was actually trying to do and not knowing what it was trying to do mean we didn’t know if it was doing well, if it was doing poorly, if it was doing the right thing. This really became a problem that we faced throughout the lifetime of our project.

Brooke Chan: Having learned this, there was no way to really ask it what it was thinking. We had metrics and we could surface like stats from our games but we were always leveraging our own intuition in order to interpret what decisions it was making. On the flip side, we also had human players that we could ask, but it turned out it was sometimes tough to get feedback from human players.

Brooke Chan: Dota itself is a really competitive game, which means that its players are very competitive. We got a lot of feedback immediately following games, which would be very biased or lean negatively. I can’t even count the number of times that a human team would lose maybe like, “Oh, this bot is terrible” and I was like, “Well, you lost. How is it terrible? What is bad about it?” This would create this back and forth that led to this ultimate question of is it bad or is it just different? Because, historically, humans have been the source on how to play this game. They make up the pro scene, they make up the high skill players. They are always the ones that you are going to learn from. The bots would make a move and the humans say it was different and not how the pros play and therefore, it’s bad. We always had to take the human interpretation with this kind of grain of salt.

Brooke Chan: I want to elaborate a little bit more about the differences because it goes just beyond the format of how they learn. This game in general is designed to help humans understand the game. It has like tooltips, ability descriptions, item descriptions, et cetera. As an example, here’s a frozen frame of a hero named Rana who’s the one with the bright green bar in the bottom left. She has an ability that makes you go invisible and humans understand what being invisible means. It means people can’t see you.

Brooke Chan: On the right, what we see is where we have like what the AI sees and it’s considered their observation space, it’s our input from the game. We as engineers and researchers know that this particular value is telling you whether or not you’re invisible. When we hit this ability, you can see that she gets like this little glow to her which indicates that she’s invisible and people understand that. The AI uses this ability and sees that the flag that we marked as invisible goes from 0 to 1 but they don’t see the label for that and they don’t really even understand what being invisible means.

Brooke Chan: To be honest, learning invisibility is not something trivial. If you’re walking down the street and all of a sudden, you were invisible, it’s a little bit hard to tell that anything actually changed. If you’ve ever seen Sixth Sense, maybe there’s some kind of concept there, but additionally, at the same time, all these other numbers around it are also changing due to the fact that there’s a lot of things happening on the map at once.

Brooke Chan: Associating that invisibility flag, changing directly to you, activating the ability is actually quite difficult. That’s something that’s easy for a human to do because you expect it to happen. Not to say that humans have it very easy, the AI has advantages too. The AI doesn’t have human emotions like greed or frustration and they’re always playing at their absolute 100% best. They’re also programmatically unselfish which is something that we did. We created this hyper parameter called team spirit which basically says that you share your rewards with your buddy. If you get 10 gold or your buddy gets 10 gold, it’s totally interchangeable. Theoretically, in a team game, that should be the same case for humans but inherently, it’s not. People at its core are going to play selfishly. They want to be the carrier. They want to be winning the game for the team.

Brooke Chan: All these things are going to influence pretty much every decision and every behavior. One pretty good example we have of this is called buybacks. Buybacks is a mechanic where when you die in the game, you can pay money in order to immediately come back to life and get back on the map. When we first enabled the AI to do this, there was a lot of criticism that we got. People were saying, “Oh, that’s really bad. They shouldn’t be wasting all their money” because the bots would always buy back pretty much immediately.

Brooke Chan: Over time, we continue doing this behavior and people kept saying, “Oh, that’s bad. You should fix it.” We’re like, “Well, that’s what they want to do.” Eventually, people started seeing it as an advantage to what we had, as an advantage to our play style because we were able to control the map. We were able to get back there very quickly and we were able to then force more fights and more objectives from it.

Brooke Chan: As a second self-serving anecdote, at TI9, there were way more buybacks way earlier and some people pointed this out and maybe drew conclusions that it was about us but I’m not actually personally going to make any statement. But it is one example of the potential to really push this game forward.

Brooke Chan: This is why it was difficult to have human players give direct feedback on what was broken or why because they had spent years perfecting the shared understanding of the game that is just like inherently different than what the bots thought. As one of the few people that played Dota and was familiar with the game and the scene, in the time leading up to finals, this became my full-time job. I learned to interpret the bot and how it was progressing and I kind of lived in this layer between the Dota community and ML.

Brooke Chan: It became my job to figure out what was most critical or missing or different about our playstyle and then how to convert that into changes that we could shape the behavior of our bot. Naturally, being in this layer, I also fell into designing and executing all of our events and communication of our research to the public and the Dota community.

Brooke Chan: In designing our messaging, I had the second unsurprising discovery that understanding our project was a critical piece to being excited about our results. We could easily say, “Hey, we taught this bot to learn Dota” and people would say, “So what? I learned to play Dota too. What’s the big deal?” Inherently, it’s like the project is hard to explain because in order to understand it and be as excited as we were, you had to get through both the RL layer which is complicated, and the Dota layer which is also complicated.

Brooke Chan: Through planning our events, I realized this was something we didn’t really have a lot of practice on. This was the first time that we had a lot of eyes on us belonging to people with not a lot of understanding of reinforcement learning and AI. They really just wanted to know more. A lot of our content was aimed at people that came in with the context and people that were already in the field.

Brooke Chan: This led me to take the opportunity to do a rotation for six months on the communications team actually working under Ashley. I wanted to be part of giving people resources to understand our projects. My responsibilities are now managing upcoming releases and translating our technical results to the public. For me, this is a pretty new and big step. I’ve been an engineer for about 10 years now and that was always what I loved doing and what I wanted to do. But experience on this team and growing into a role that didn’t really exist at the time allowed me to tackle other sorts of problems and because that’s what we are as engineers at the core, we want to be problem solvers.

Brooke Chan: That’s kind of my takeaway and it might seem fairly obvious but sometimes deviating from your path and taking risks let you discover new problems to work on. They do say that growth tends to be at the inverse of comfort so that means that the more you push yourself out of your comfort zone and what you’re used to, the more you give yourself opportunities for new challenges and discovering new skills. Thank you.

Lilian Weng

Research Scientist Lilian Weng on the Robotics team gives a talk on how her team uses reinforcement learning to learn dexterous in-hand manipulation policies at OpenAI Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Lilian Weng: Awesome. Cool. Today, I’m going to talk about some research projects with that at OpenAI robotics team. One big picture problem at our robotics team is to develop the algorithm to power general-purpose robots. If you think about how we humans are living this world, we can cook, we lift to move stuff, we add some more items with different tools. We fully utilize our body and especially our hands to do a variety of tasks. To some extent, we are general-purpose robots, okay?

Lilian Weng: That’s, we apply the same standard to our definition of such a thing. A general-purpose robot should be able to interact with a very complicated environment of the real world and able to manipulate all kinds of objects around it. However, unfortunately, most consumer-oriented robots nowadays are either just toys or very experimental or focus on specific functionalities and they are robots like factory arms or medical robots. They can interact with the environment and operating tools but they’re really operated by humans so human controls every move or they just play back a pre-programmed trajectory. They don’t really understand the environments and they cannot move autonomously.

Lilian Weng: In our projects, we’re taking a small step towards this goal and in this we try to teach a human-like robot hand to do in-hand manipulation by moving the objects. This is a six-phase block with OpenAI letters on it, move that to a target orientation. We believe this is an important problem because a human-like robot hand, it’s a universal effort. Imagine we can control that really well, we can potentially automate a lot of tasks that are currently done by human. Unfortunately, not a lot of progress have been made on human-like robot hand due to the complexity of such a system.

Lilian Weng: Why it is hard? Okay. First of all, the system has very high dimensionalities. For example, in our robot, which is as you can see this cool illustration. Shadow dexterity hand, it has 24 joints and 20 actuators. The task is especially hard because during the manipulation, a lot of observations are occluded and they can be noisy. For example, your sensor reading can be wrong and your sensor reading can be blocked by the object itself. Moreover, it’s virtually impossible to simulate your physical world 100% correctly.

Lilian Weng: Our approach for tackling this problem is to use reinforcement learning. We believe it is a great approach for learning how to control robots given that we have seen great progress and great success in many applications by reinforcement learning. You heard about OpenAI Five, the story of point AlphaGo and it will be very exciting to see how reinforcement learning can not only interact with this virtual world but also have an impact on our physical reality.

Lilian Weng: There is one big drawback of reinforcement learning model. In general, today, most of the models are not data efficient. You need a lot of training sample in order to get a good model trained. One potential solution is you build a robot farm. You just collect all the data in parallels with hundreds of thousands of robots but imagine just given how fragile a robot can be. It is very expensive to build and maintain. If you think of another problem, a new problem, or you want to work with new robots, it’s very hard to change. Furthermore, your data can get invalidated very quickly due to small changes in your robot status.

Lilian Weng: As that, we decided to take the sim2real approach, that is you train your model every single simulation but deploy that on physical robots. Here shows how we control the hand simulation. The hand is moving the object to a target orientation. The target is shown on the right so whenever the hand achieved the goal, we just sample a new goal. It just keeps on doing that and we cap the number of success at 50.

Lilian Weng: This is our physical setup. Everything is mounted in this giant metal cage. It’s like this big. The hand is mounted in the middle. It’s surrounded with a motion caption system. It’s actually the system that people use for filming special effects films, like the actor has dots on their bodies, kind of similar. This system tracks the five fingertip positions in the 3D space. We also have three high-resolution cameras for capturing images as input to our vision model. Our vision model predicts positional orientation of the block. However, our proposal sim2real approach might fail dramatically because there are a lot of model difference between simulation and reality. If your model all refer to the simulation, it can perform super poorly, the real robots.

Lilian Weng: In order to overcome this problem, we decided to take … we use reinforcement learning, okay. We train everything simulations so that we can generate technically, theoretically infinite amount of data. In order to overcome the sim2real difference, we use domain randomization.

Lilian Weng: Domain randomization refer to an idea of randomized different elements in simulation so that your policy can be exposed to a variety of scenarios and learn how to adapt. Eventually, we expand the policy to able to adapt to the physical reality.

Lilian Weng: Back in … This idea is relative news. I think they first proposed it in 2016. The researchers try to train a model to control drone like fly across furnitures or the indoor scenarios. They randomized the colors and texture of the walls and furnitures and without seeing any real-world images, they show that it performs pretty well in reality.

Lilian Weng: At OpenAI, we use the same approach to train a better model to protect the position orientation of the objects. As you can see some of the randomization looks totally unrealistic but somehow it worked very well when we feed the model with real images. Later, we also showed that you can randomize all the physical dynamics in simulations and this robot trained with domain randomization worked much better than the one without.

Lilian Weng: Let’s see the results. Okay. I’m going to click the … You really struggle a little bit at the first goal. Yes, okay. The ding indicates one success. This video will keep on going until goal 50 so it’s very, very long but I personally found it very soothing to look at it. I love it.

Lilian Weng: I guess that’s enough. This is our full setup of the training so in the box A, we generate a large number of environments in parallels in which we randomize the physical dynamics and the visual appearance. Based on those, we train two models independently. One is a policy model which takes in the fingertip position and object pose and the goal and output, a desired joint position of the hand so that we can control the hand. Another model is the vision that takes in three images from different camera angles and output the position orientation of the object.

Lilian Weng: When we deploy this thing into the real world, we combine the vision prediction based on the real images together with a fingertip position tracked by the motion capture system and feed that into our policy control model and output action so that then we just send it to the real robot and everything starts moving just like the movie shown. When we train our policy control model, we’ve randomized all kinds of physical parameter in the simulator such as masses, friction coefficient, motor gain, damping factor, as well as noise on the action, on observation. For a revision model, we randomized camera position, lighting, material, texture, colors, blah, blah, blah, and it just worked out.

Lilian Weng: For our model’s architecture, I’ll just go very quickly here. The policy, it’s a pretty simple recurrent unit. Has one layer of really connective layer and the LSTM. The vision model is a straightforward, multi-camera setup. All the three cameras share this RestNet stack and followed by a spatial softmax.

Lilian Weng: Our training framework is distributed and synchronized PBO, proximal policy optimization model. It’s actually the same framework used for training OpenAI Five. Our setup allowed us to generate about two years simulated experience per hour, which corresponds to 17,000 physical robots, so the gigantic robot factory and simulation is awesome.

Lilian Weng: When we deploy our model in reality, we noticed a couple of strategies learned by the robot like finger pivoting, sliding, finger gaiting. Those were also commonly used by human and interestingly, we never explicitly give it words or encouraged those strategies. They would just emerge autonomously.

Lilian Weng: Let’s see some numbers. In order to compare different versions of models, we deployed the models on the real robots and count how many successes the policy can get up to 50 before it dropped the block or time out. We first tried to deploy a model without randomization at all. It got a perfect performance in simulation but look, you can see it’s zero success median. Super bad on the real robot.

Lilian Weng: Then we’re adding domain randomization. The policy becomes much better because 13 success medians, maximum 50. Then we used RGB cameras in our vision model to track the objects. The performance only dropped slightly, still very good. The last one, I think this one’s very interesting because I just mentioned that our policies are recurrent units so like LSTM, it has internal memories.

Lilian Weng: Well, interesting, see how important this memory is so we replaced this LSTM policy with a FIFO or NAS and deployed that on robot and the performance dropped a lot, which indicates that memory play an important role in the sim2real transfers. Potentially, the policy might be using the memory and try to learn how to adapt.

Lilian Weng: However, training in randomized environments does come with a cost. Here we plot the number of success in simulation as a function of simulated experiencing measured in year. If you don’t apply randomization at all, the model can learn to achieve 40 success with about three years simulated experience but in order to get to same number like 40 success in a fully randomized environment took 100 years.

Lilian Weng: Okay, to quick summary. We’ve shown that this approach, reinforcement learning plus training simulation plus domain randomization worked on the real robot and we would like to push it forward. Thank you so much. Next one is Christine.

Christine Payne speaking

Research Scientist Christine Payne on the Music Generation team gives a talk on how MuseNet pushes the boundaries of AI creativity, both as an independent composer, and as a collaboration tool with human artists.  Erica Kawamoto Hsu / Girl Geek X

Christine Payne: Thank you. Let’s see. Thank you. It’s really great to see all of you here. After this talk, we’re going to take a short break and I’m looking forward to hopefully getting to talk to a lot of you at that point. I’ve also been especially asked to announce that there are donuts in the corner and so please help us out eating those.

Christine Payne: If you’ve been following the progress of deep learning in the past couple years, you’ve probably noticed that language generation has gotten much, much better, noticeably better in the last couple of years. But as a classical pianist, I wondered, can we take the same progress? Can we apply instead to music generation.

Christine Payne: Okay, I’m not Mira. Sorry. Hang on. One moment, I think we’re on the wrong slide deck. All right, sorry about that. Okay, trying again. Talking about music generation. You can imagine different ways of generating music and one way might be to do a programmatic approach where you say like, “Okay, I know that drums are going to be a certain pattern. Harmonies usually follow a certain pattern.” You can imagine writing rules like that but there’s whole areas of music that you wouldn’t be able to capture with that. There’s a lot of creativity, a lot of nuance, the sort of things that you really want a neural net to be able to capture.

Christine Payne: I thought I would dive right in by playing a few examples of MuseNet, which is this neural net that’s been trained on this problem of music generation. This first one is MuseNet trying to imitate Beethoven and a violin piano sonata.

Christine Payne: It goes on for a while but I’ll cut it off there. What I’m really trying to go with in this generation process is trying to get long-term structure so both the nuance and the intricacies of the pieces but also something that stays coherent over a long period of time. This is the same model but instead trying to imitate jazz.

Christine Payne: Okay, and I’ll cut this one off too. As you maybe could tell from those samples, I am more interested in the problem of composing the pieces themselves, so sort of where the notes should be and less in the actual quality of the solemnness and the timbre. I’ve been using a format that’s called MIDI which is an event-based system of writing music. It’s a lot like how you would write down notes in a music score. Like this note turns on at this moment in time played by this instrument maybe at this volume but you don’t know like this amazing cellist actually made it sound this way so I’m throwing out all of that kind of information.

Christine Payne: But the advantage of throwing that out is then you can get this longer-term structure. To build this sort of dataset, it involves a little bit of begging for data. I’ve had a bunch of people like BitMidi and ClassicalArchives were nice enough to just send me their collections and then a little bit of scraping and also MAESTRO’s Google Magenta’s dataset and then also a bunch of scraping online sets.

Christine Payne: The architecture itself, here I’m drawing really heavily from the way we do language modeling and so we use a specific kind of neural net that’s called a transformer architecture. The advantage of this architecture is that it’s specifically good at doing long-term structure so you’re able to look back not only at things that have happened in the recent past but really, you can look back like what happened in the music a minute ago or something like that, which is not possible with most other architectures.

Christine Payne: In the language world, I’d like to think of this, the model itself is trained on the task of what word is going to come next. It might initially see just like a question mark so it knows it’s supposed to start something. In English, we know like maybe it’s the or she or how or some like that. There’s some good guesses and there’s some like really bad guesses. If we know now the first word is hello then we’ve kind of narrowed down what we expect our next guesses should be. It might be how, it might be my, it’s probably not going to be cat. Maybe it could be cat. I don’t know.

Christine Payne: At this point, we’re getting pretty sure–like a trained model should actually be pretty sure that there should be a good 90% chance the next word is name and now it should be like really 100% sure or like 99.5% sure or whatever that the next word is going to be is. Then here we hit kind of an interesting branching point where there are tons of good answers so lots of names could be great answers here, lots of things could also be really bad answers so we don’t expect to see like some random verbs, some random … There are lots of things that we think would be bad choices but we get a point here to branch in good directions.

Christine Payne: The idea is once you have a model that’s really good at this, you can then turn it into a generator by sampling from the model according to those probabilities. The nice thing is you get the coherent structure. When you get a moment like this, you know like I have to choose … In music, it’s usually like I have to choose this rhythm, I have to choose … like if I choose the wrong note, it’s just going to sound bad, things like that. But then there are also a lot of points like this where the music can just go in fun and interesting different directions.

Christine Payne: But of course, now we have the problem of how do you translate words, how do you translate this kind of music into a sequence of words that the model can do. The system that I’m using is very similar to how MIDI itself works. I have a series of tokens that the model will always we see. Initially, it’ll always see the composer or the band or whoever wrote the piece. It’ll always see what instrument to expect in the piece or what set of instruments.

Christine Payne: Here, it sees the start token because it’s at the start of this particular piece and a tempo. Then as the piece begins, we have a symbol that this C and that C each turn on with a certain volume and then we have a token that says to wait a certain amount of time. Then as it moves forward, the volume zero means that first note just turned off and the G means the next note turns on. I think we have to wait and similarly, here the G turns off, the E turns on and we wait. You can just progress through the whole set of music like this.

Christine Payne: In addition to this token by token thing, I’m helping the model out a little bit by giving it a sense of the time that’s going on. I’m also giving it an extra embedding that says everything that happens in this purple line happens in the same amount of time or at the same moment in time. Everything in blue is going to get a different embedding that’s a little bit forward in time and so forth.

Christine Payne: The nice thing about an embedding or a system like this is that it’s pretty dense but also really expressive. This is the first page of a Chopin Ballade that is like actually encapsulates how the pianist played it, the volumes, the nuances, the timings, everything like that.

Christine Payne: The model is going to see that sequence of numbers like that. Like that first 1444 I think means it must mean Chopin and the next one probably means piano and the next one means start, that sort of thing. The first layer for the model, what it has to do is it needs to translate that number into a vector of numbers and then it can sort of learn a good vector that’ll represent so it’ll get a sense of like this is what it means to be Chopin or this is what it means to be like a C on a piano.

Christine Payne: The nice thing you can do once … The model will learn. Like initially it starts out with a totally random sense so it has no idea what those numbers should be but in the course of training, it’ll learn better versions of that. What you can do is you can start to map out what it’s learned for these embeddings. For example, this is what it’s learned for a piano scale like all the notes on a piano and it’s come to learn that like all of these As are kind of similar, that the notes relate to each other. This is like moving up on a piano. It’s hard to tell here but it’s learned little nuances like up a major third is closer than like up a tritone or stuff like that. Like actually really interesting musical stuff.

Christine Payne: Along with the same thing, given the fact that I’m always giving it this genre token and then the instrument token, you can look at the sort of embeddings it’s learned for the genres itself. Here, the embedding it’s learned for all these French composers. Ends up being pretty similar. I actually like that Ravel wrote like in the style of Spanish pieces and then there’s the Spanish composer that’s connected to him so like it makes a lot of good sense musically. Similarly, like over in the jazz domain, a lot of the ones. I think there are a couple of random ones that made no sense at all. I can’t remember now off the top of my head. It’s like Lady Gaga was connected to Wagner or something like but mostly, it made a lot of great sense.

Christine Payne: The other kind of fun thing you can do once you have the style tokens is you can try mismatching them. You can try things like literally taking 0.5 of the embedding for Mozart plus 0.5 of the embedding of jazz and just like adding them together and seeing what happens or in this case what I’m doing is I’m giving it the token for Bon Jovi, instruments for bands, but then I’m giving it the first six notes of a Chopin Nocturne. Then the model just has to generate as best it can at that point.

Christine Payne: You’ll hear at the start of this, it’s very much how the Chopin Nocturne itself sounds. I’ve cut off the very, very beginning of it but you’ll hear–so that left-hand pattern is going to be like straight out of Chopin and then well, you’ll see what happens.

Christine Payne: Sorry, it’s so soft but it gets very Bon Jovi at this point, the band kicks in. I always loved it like Chopin looks a little shocked but I really love that it manages to keep the left-hand pattern of the Nocturne going even though it’s like now thinks it’s in this pop sort of style.

Christine Payne: The other thing I’ve been interested in this project is in how musicians and everyone can use generators like this. If you go to our OpenAI blog you can actually play with the model itself. We’ve created, along with Justin and Eric and Nick, a sort of prototype tool of how you might co-compose pieces using this model. What you can do is you can specify the style and the instruments, how long a segment you want the model to generate and you hit start and the model will come back with four different suggestions of like how you might begin a piece in this style. You go through and you pick your favorite one, you hit the arrow again to keep generating and the model will come up with four new different ways. You can continue on this way as long as you want.

Christine Payne: What I find kind of fun about this is you’re actually really … like it feels like I’m composing but not at a note by note level and so I was really interested in how humans will be able to, and musicians will be able to guide composing this way. Just kind of wrapping up, I thought I would play an example of … This is one guy who took both GPT-2 to write the lyrics, which I guess is hence the Covered in Cold Feet and then MuseNet to do the music. It’s a full song but I’ll just play the beginning of it that he then recorded himself.

Christine Payne: (singing)

Christine Payne: Visit the page to hear the whole song but it’s been really fun to see those versions. The song, I ended up singing it the entire day. It gets really catchy but it’s been really fun to see musicians start to use it. People have used it to finish composing symphonies or to write full pieces, that sort of thing.

Christine Payne: In closing, I just wanted to share I’ve gone through this crazy path of two years ago being a classical pianist to now doing AI research here and I just wanted to … I didn’t know that Rachel was going to be right here. Give a shout out to She’s the celebrity here but yeah. This has been my path, been doing it. These are the two courses I particularly love, and and then I also went through OpenAI’s Scholars program and then the Fellows Program. Now I’m working here full-time, but happy to talk to anybody here if they’re interested in this sort of thing.

Christine Payne: The kind of fun thing about AI is that there’s so much that’s still wide open and it’s really helpful to come from different backgrounds where you bring a … It’s amazing how if you bring a new perspective or a new insight, there are a lot of things that are still just wide open that you can figure out how to do. I encourage anyone to come and check it out. We’ll have a concert. Thank you.

Mira Murati speaking

RL Team Manager Mira Murati gives a talk about reinformatiion learning and industry trends at OpenAI Girl Geek Dinner.   Erica Kawamoto Hsu / Girl Geek X 

Mira Murati: Hey, everyone, I’m Mira Murati and I’ll talk a little bit about the advancements in reinforcement learning from the lens of our research team here at OpenAI. Maybe I’ll kick things off by just telling you a bit about my background and how I ended up here.

Mira Murati: My background is in mechanical engineering but most of my work has been dedicated to practical applications of technology. Here at OpenAI, I work on Hardware Strategy and partnerships as well as managing our Reinforcement Learning research team alongside John Schulman, who is our lead researcher. I also manage our Safe Reinforcement Learning team.

Mira Murati: Before coming to OpenAI, I was leading the product and engineering teams at Leap Motion, which is a company that’s focused on the issue of human machine interface. The challenge with the human machine interface, as you know, is that we’ve been enslaved to our keyboard and mouse for 30 years, basically. Leap Motion was trying to change that by increasing the bandwidth of interaction with digital information such that, just like you see here, you can interact … Well, not here, with the digital space in the same natural and high bandwidth way that you interact with your physical space. The way you do that is using computer vision and AI to track your fingers in space and bring that input in virtual reality or augmented reality in this case.

Mira Murati: Before that, I was at Tesla for almost three years leading the development and launch of the Model X. That’s enough about me. I’ll touch a bit about on the AI landscape as a whole, just to offer a bit of context on the type of work that we’re doing with our Reinforcement Learning team. Then I’ll talk a bit about the impact of this work, the rate of change in the field as well as the challenges ahead.

Mira Murati: As you know, the future has never been bigger business. Every day we wake up to headlines like this and a lot of stories talking about the ultimate conversions where all the technologists come together to create the ultimate humankind dimension, that of general artificial intelligence. We wonder what this is going to do to our minds and to our societies, our workplaces and healthcare. Even politicians and cultural commentators are aware of what’s happening with AI to some extent, and politicians like this, to the extent that there’s a lot of nations out there that have published their AI strategies.

Mira Murati: There is definitely a lot of hype, but there is also a ton of technological advancement that’s happening. You might be wondering what what’s driving these breakthroughs. Well, so a lot of advancements in RL are driving the field forward and my team is working on some of these challenges through the lens of reinforcement learning.

Mira Murati: Both Brooke and Lilian did a great job going over reinforcement learning so I’m not going to touch too much upon that, but basically, to reiterate, it is you’re basically learning through trial and error. To provide some context for our work, I want us to take a look at …

Mira Murati: Oh, okay. There’s music. I wanted to take a look at this video where first you see this human baby, nine months old, how he is exploring the environment around him. You see this super high degrees of freedom interaction with everything around him. I think this is four hours of play in two minutes. In some of the things that this baby does like handling all these subjects, rolling around all this stuff, this is almost impossible for machines to do as you saw from Lilian’s talk.

Mira Murati: Then … Well, he’s going to keep going, but let’s see. Okay, now that … What I want to show you is … Okay, this is not working, but basically, I wanted you to show you that by contrast, so you have this video game over there where you would see this AI agent that’s basically trying to cross this level and makes the same mistakes over and over again. The moral of the story is that AI agents are very, very limited when they’re exploring their environment. Human babies just nine months old have this amazing ability to explore their environment.

Mira Murati: The question is, why are humans so good at understanding the environment around them? Of course, humans … We have this baby running in the playground. Of course, humans are very good at transferring knowledge from one domain to another, but there is also prior knowledge from evolution and also, from your prior life experiences. For example, if you play a lot of board games and I asked you to play a new one that you have never seen before, you’re probably not going to start learning that new game from scratch. You will apply a lot of the heuristics that you have learned from the previous board game and utilize those to solve this new one.

Mira Murati: It’s precisely this ability to abstract, this conceptual knowledge that’s based on or learned from perceptual details of real life that’s actually a key challenge for our field right now and we refer to this as transfer learning.

Mira Murati: What’s the state of things? There’s been a lot of advancements in machine learning and particularly in reinforcement learning. As you heard from the talks earlier, new datasets drive a lot of the advancements in machine learning. Our Reinforcement Learning team built a suite of games, thousands of games, that in itself you think playing video games is not so useful, but actually, they’re a great test bed because you have a lot of problem-solving and also content that’s already there. It comes for free in a way.

Mira Murati: The challenge that our team has been going after is how can we solve a previously unseen game as fast as a human, or even faster, given prior experiences with similar games. The Gym Retro dataset helps us do that. I was going to say that some of the games look like this but the videos are not quite working. But in a way, the Gym Retro dataset, you can check it out on the OpenAI blog, emphasizes the weaknesses of AI which is that of grasping a new task quickly and the ability to generalize knowledge.

Mira Murati: Why do all these advancements matter and what do the trends look like? It’s now just a bit over 100 years after the birth of the visionary mathematician Alan Turing and we’re still trying to figure out how hard it’s going to be to get to general artificial intelligence. Machines have surpassed us at very specific tasks but the human brain sets a high bar for what’s AI.

Mira Murati: In the 1960s and ’70s, this high bar was a game of chess. Chess was long considered the summit of human intelligence. It was visual, tactical, artistic, intelligence, mathematical, and chess masters could remember every single game that they played, not to mention that of their competitors, and so you can see why chess became such a symbol of mastery or a huge achievement of the human brain. It combined insight and forward planning and calculation, imagination, intuition, and this was until 1996, when the Deep Blue machine, chess machine from IBM was able to beat Garry Kasparov. If you had brought someone from the 1960s to that day, they would be completely astonished that this had happened but in 1996, this did not elicit such a reaction because in a way, Deep Blue had cheated by utilizing the power of hardware of Moore’s law. It leveraged the advancements in hardware to beat Garry Kasparov at chess.

Mira Murati: In a way, this didn’t show so much the advancements in AI, but rather that chess was not the pinnacle of human intelligence. Then the human sights were set on the Chinese game of Go, which is much more complex and just with brute force, you’d be quite far from solving Go, the game of Go with brute force and where we stand with hardware today. Then of course, in 2016, we saw the DeepMind’s AlphaGo beat Lee Sedol in Korea and that was followed by advancements in AlphaGo Zero. OpenAI robotics team of course, used some of the algorithms developed in the RL team to manipulate the cube and then we saw very recently, obviously, the Dota 5v5 beat the world champions.

Mira Murati: There’s been a very strong accelerating trend of advancements pushed by reinforcement learning in general. However, there’s still a long way to go. There are a lot of questions with reinforcement learning and in figuring out where the data is coming from and what actions do you take early on that get you the reward later. Also issues of safety, how do you learn in a safe way and also how do you continue to learn once you’ve gotten really good? Think of self-driving cars, for example. We’d love to get more people thinking about this type of challenges and I hope that some of you will join us in doing so. Thank you.

Amanda Askell speaking

Research Scientist Amanda Askell on the Policy team gives a talk on AI policy at OpenAI Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Amanda Askell: Okay, can everyone hear me? Cool. We’ve had like a lot of talks on some of the technical work that’s been happening at OpenAI. This talk is going to be pretty introductory because I guess I’m talking about what is quite a new field, but as Ashley said at the beginning, it’s one of the areas that OpenAI focuses on. This is a talk on AI policy and I’m a member of the policy team here.

Amanda Askell: I realize now that this picture is slightly unfortunate because I’m going to give you some things that look like they’re being produced by a neural net when in fact this is just an image because I thought it looked nice.

Amanda Askell: The core claims behind why we might want something like AI policy to exist in the world are really simple. Basically, AI has the potential to be beneficial. Hopefully, we can agree with this. We’ve had lots of talks showing how excellent AI can be and things that it can be applied to. AI also has the potential to be harmful so I’ll talk a little bit about this in the next slide but you know we hear a lot of stories about systems that just don’t behave the way that they’re creators intended to when they’re deployed in the world, systems that can be taken over by people who want to use them for malicious purposes. Anything that has this ability to do great things in the world can also be either misused or lead to accidents.

Amanda Askell: We can do things that increase the likelihood that AI will be beneficial so hopefully, that’s also fairly agreed-upon. But also that this includes making sure that the environment the AI is developed in is one that incentivizes responsible development. They’re like nontechnical things that we can do to make sure that AI is beneficial.

Amanda Askell: I think these are all like really simple and this leads to this idea that we should be doing some work in known technical fields just to make sure that AI is developed responsibly and well. Just to like kind of reiterate the claims of the previous slide, the potential benefits of AI are obviously kind of huge and I feel like to this audience I don’t really need to sell them but we can go over them. You know language models provide the ability potentially to assist with writing and other day-to-day tasks.

Amanda Askell: We can see that we can apply them to large complex problems like climate change potentially. This is the kind of like hope for things like a large scale ML. We might be able to enable like innovations In healthcare and education so we might be able to use them for things like diagnosis or finding new treatments for diseases. Finally, they might drive the kind of economic growth that would reduce the need to do work that people don’t find fulfilling. I think this is probably controversial. This is one thing that’s highly debated in AI ethics but I will defend it. I’ve done lots of unfulfilling work in my life and if someone could just pay me to not do that, I would have taken that.

Amanda Askell: Potential harms like language models of the same sort could be used to like misinform people by malicious actors. There are concerns about facial recognition as it improves and privacy. People are concerned about automation and unemployment if it’s not dealt with well. Like does this just lead to massive unfairness and inequity? Then people are also worried about things like decision making and bias. We already see in California that there’s ML systems being used for things like decisions about bail being set but also historically, we’ve used a lot of systems for things like whether someone gets credit. I mean so whether your loan’s approved or not given that there’s probably a huge amount of bias in the data and that we don’t know yet how to completely eliminate that, this could be really bad and it could increase systemic inequity in society, so that’s bad.

Amanda Askell: We’re also worried about like AI weapons and global security. Finally, just like a general misalignment of future AI systems. A lot of these are just like very classic examples of things that people are thinking about now, but this should just … We could expect this to be the sort of problems that we just see on an ongoing basis in the future as systems get more powerful.

Amanda Askell: I don’t think AI is like any different from many other technologies in at least some respects here. How do we avoid building things that are harmful? Doing the same kind of worries just apply to like the aviation industry. Planes can also be taken over by terrorists. Planes can be built badly and lead to accidents. The same is true of like cars or pharmaceuticals or like many other technologies with the potential to do good, it can end up … There can be accidents. It can be harmful.

Amanda Askell: In other industries we invest in safety, we invest in reducing accidents, we invest in security, so that’s like reducing misuse potential, and we also invest in social impact. In case of aviation, we know are concerned about things like the impact that flying might have on the climate. This is like the kind of third sort of thing that people invest in a lot.

Amanda Askell: All of this is very costly so this is just a kind of intro to like one way in which we might face problems here. I’m going to use a baking analogy, mainly because I was trying to think of a different one and I had used this one previously and I just couldn’t think of a better one.

Amanda Askell: The idea is, imagine you’ve got a competition and the nice thing about baking competitions, maybe I just have watched too many of them, is like you care both about the quality of what you’re creating and also about how long it takes to create it. Imagine a baking competition where you can just take as much time as you want and you’re just going to be judged on the results. There’s no race, like you don’t need to hurry, you’re just going to focus purely on the quality of the thing that you’re creating.

Amanda Askell: But then you introduce this terrible thing, which is like a time constraint or even worse, you can imagine you make it a race. Like the first person to develop the bake just gets a bunch of extra points. In that case, you’re going to be like well, I’ll trade off some of the quality just to get this thing done faster. You trade off some quality for increased speed.

Amanda Askell: Basically, we can expect something similar to happen with things like investment in areas like the areas that I talked about in the previous slide, where it’s like it might be that I would want to just like continue investing and making sure that my system is secure essentially like forever. I just never want someone to misuse this system so if I was given like 100 years, I would just keep working on it. But ultimately, I need to produce something. I do need to put something out into the world and the concern that we might have is that competition could drive down the incentive to invest that much in security.

Amanda Askell: This, again, happens across lots of other industries. This is like not isolated to AI and so there’s a question of like, what happens here? How do we ensure that companies invest in things like safety? I’m going to argue that there are four things. Some of the literature might not mention this one but I think it’s really important. The first one is ethics. People and companies are surprisingly against being evil. That’s good, that’s important. I think this gets not talked about enough. Sometimes we talk like the people that companies would just be totally happy turning up at like 9:00 a.m. to build something that would cause a bunch of people harm. I just don’t think that people think like that. People are … I have fundamental faith in humanity. I think we’re all deeply good.

Chloe Lin software engineer OpenAI Girl Geek Dinner

Software Engineer Chloe Lin listens to the OpenAI Girl Geek Dinner speakers answer audience questions.  Photo credit: Erica Kawamoto Hsu / Girl Geek X

Amanda Askell: It’s really great to align your incentives with your ethical beliefs and so regulation is obviously one other component that’s there to do that. We create these regulations and industry norms to basically make sure that if you’re like building planes and you’re competing with your competitor, you still just have to make your planes. You have to establish that they reach some of … Tripped over all of those words.

Amanda Askell: You have to establish that they reach some level of safety and that’s what regulation is there for. There’s also liability law and so companies have to compensate who are harmed by failures. This is another thing that’s driving that incentive to make sure your bake is not going to kill the judges. Well, yeah, everyone will be mad at you and also, you’ll have to pay a huge amount of money.

Amanda Askell: Finally, the market. People just want to buy safe products from companies with good reputations. No one is going to buy your bake if they’re like, “Hang on, I just saw you drop it on the floor before you put it into the oven. I will pay nothing for this.” These are four standard mechanisms that I think are used to like ensure that safety is like pretty high even in the cases of competition between companies in other domains like aviation and pharmaceuticals.

Amanda Askell: Where are we with this on AI? I like to be optimistic about the ethics. I think that coming to a technology company and seeing the kind of tech industry, I’ve actually been surprised by the degree to which people are very ethically engaged. Engineers care about what they’re building. They see that it’s important. They generally want it to be good. This is more like a personal kind of judgment on this where I’m like actually, this is a very ethically engaged industry and that’s really great and I hope that continues and increases.

Amanda Askell: With regulation, currently there are not many industry-specific regulations. I missed an s there but speed and complexity make regulation more difficult. The idea is that regulation is very good when there’s not an information asymmetry between the regulator and the entity being regulated. It works much less well when there is a big information asymmetry there. I think in the case of ML, that does exist. It’s very hard to both keep up with like, I think for regulators keeping up with contemporary ML work is really hard and also, the pace is really fast. This makes it actually quite difficult as an area to build very good regulation in.

Amanda Askell: Liability law is another thing where it’s just like a big question mark because like for ML accidents and misuse, in some cases it’s just unclear what existing law would say. If you build a model and it harms someone because it turns out that there was data in the model that was biased and that results in a loan being denied to someone, who is liable for that harm that is generated? You get easier and harder cases of this, but essentially, a lot of the kind of … I think that contemporary AI actually presents a lot of problems with liability law. It will hopefully get sorted out, but in some cases I just think this is unclear.

Amanda Askell: Finally, like market mechanisms. People just need to know how safe things are for market mechanisms to work well. In the case of like a plane, for example, I don’t know how safe my planes are. I don’t go and look up the specs. I don’t have the engineering background that would let me actually evaluate, say, a new plane for how safe it is. I just have to trust that someone who does know this is evaluating how safe those planes are because there’s this big information gap between me and the engineers. This is also why I think we shouldn’t necessarily expect market mechanisms to do all of the work with AI.

Amanda Askell: This is to lead up to this … to show that there’s a broader problem here and I think it also applies in the case of AI. To bring in a contemporary example, like recently in the news, there’s been concern. Vaping is this kind of like new technology that is currently not under the purview of the FDA or at least generally not heavily regulated. Now there’s concern that it might be causing pretty serious illnesses in people across the US.

Amanda Askell: I think this is a part of a more broad pattern that happens a lot in industries and so I want to call this the reactive route to safety. Basically, a company does the thing, the thing harms people. This is what you don’t want on your company motto. Do the thing. The thing harms people. People stop buying it. People sue for damages. Regulators start to regulate it. This would be really uninspiring as your company motto.

Amanda Askell: This is actually a very common route to making things more safe. You start out and there’s just no one who’s there to make sure that this thing goes well and so it’s just up to people buy it, they’re harmed, they sue, regulators get really interested because suddenly your product’s clearly harming people. Is this a good route for AI? Reasons against hope … I like the laugh because I’m like hopefully, that means people agree like no, this would be terrible. I’m just like well, one reason, just to give like the additional things of like obviously that’s kind of a bad way to do things anyway.

Amanda Askell: AI systems can often be quite broadly deployed almost immediately. It’s not like you just have some small number of people who are consuming your product who could be harmed by it in a way that a small bakery might. Instead, you could have a system where you’re like I’ve built the system for determining whether someone should get a loan. In principle, almost every bank in the US could use that the next day and that’s –The potential for widespread deployment makes it quite different from technologies where you just have a really or like any product where you have just like a small base of people.

Amanda Askell: They have the potential for a really high impact. The loan system that I just talked about could, basically, could in principle really damage the lives of a lot of people. Like apply that to things like bail systems as well, which we’re already seeing and even potentially with things like misinformation systems.

Amanda Askell: Finally, in a lot of cases it’s just difficult to attribute the harms and if you have something that’s spreading a huge amount of misinformation, for example, and you can’t directly attribute it to something that was released, this is concerning because it’s not like this route might work. This route actually requires you to be able to see who caused the harm and whenever that’s not visible, you just don’t expect this to actually lead to good regulation.

Amanda Askell: Finally, I just want to say I think there are alternatives to this reactive break things first approach in AI and this is hopefully where a lot of policy work can be useful.

Amanda Askell: Just to give a brief overview of policy work at OpenAI. I think I’m going to start with the policy team goals just to give you the sense of what we do. We want to increase the ability of society to deal with increasingly advanced AI technology, both through information and also through pointing out mechanisms that can make sure that technology is safe and secure and that it does have a good social impact. We conduct research into long-term issues related to AI and AGI so we’re interested in what happens when these systems become more powerful. Not merely reacting to systems that already exist, but trying to anticipate what might happen in the future and what might happen as systems get more powerful and the kind of policy problems and ethical problems that would come up then.

Amanda Askell: Finally, we just help OpenAI to coordinate with other AI developers, civil society, policymakers, et cetera, around this increasingly advanced technology. In some ways trying to break down these information asymmetries that exist and it can cause all of these problems.

Amanda Askell: Just to give a couple of examples of recent work from the teams to the kind of thing that we do. We released a report recently with others on publication norms and release strategies in ML. Some of you will know about like the GPT-2 language release and the decision to do staged release. We discussed this in the recent report. We also discussed other things like the potential for bias in language models and some of the potential social impacts of large language models going forward.

Amanda Askell: We also wrote this piece on cooperation and responsible AI development. This is related to the things I talked about earlier about the potential for competition to push this bar for safety too low and some of the mechanisms that can be used to help make sure that that bar for safety is raised again.

Amanda Askell: Finally, since this is an introduction to this whole field, which is like new and emerging field, here are examples of questions I think are really interesting and broad but can be broke down to these very specific applicable questions. What does it mean for AI systems to be safe, secure, and beneficial and how can we measure this? This includes a lot of traditional AI ethics work, like my background is in ethics. A lot of these questions about like how you make a system fair and what it means for a system to be fair. I would think of that as falling under the what is it for a system to be socially beneficial, and I think that work is really interesting. I do think that there’s just this broad family of things there are like policy and ethics and governance. I don’t think of these as separate enterprises.

Amanda Askell: Hence, this is an example of why. What are ways that AI systems could be developed that could be particularly beneficial or harmful? Again, trying to anticipate future systems and ways that we might just not expect them to be harmful and they are. I think we see this with the existing technology. Maybe it’s like trying to anticipate the impact that technology will have is really hard but like given the huge impact that technology is now having, I think trying to do some of that research in advance is worthwhile.

Amanda Askell: Finally, what can industry policymakers and individuals do to ensure that AI is developed responsibly? This relates to a lot of the things that I talked about earlier, but yeah, what kind of interventions can we have now? Are there ways that we can inform people that would make this stuff all go well?

Amanda Askell: Okay, last slide except the one with my email on it, which is the actual last slide. How can you help? I think that there’s this interesting, this is just like … I think that this industry is very ethically engaged and in many ways, it can feel like people feel like they need to do the work themselves. I know that a lot of people in this room are probably engineers and researchers. I think the thing I would want to emphasize is, you can be really ethically engaged and that doesn’t mean you need to take this whole burden on yourself.

Amanda Askell: One thing you can also do is advocate for this work to be done, either in your company, or just anywhere where people are like … in your company, in academia or just that your company is informed of this stuff. But in general, helping doesn’t necessarily have to mean taking on this massive burden of learning an entire field yourself. It can just mean advocating for this work being done. At the moment, this is a really small field and I would just love to see more people working in it. I think advocacy is really important but I also think another thing is you can technically inform people who are working on this.

Amanda Askell: We have to work closely with a lot of the teams here and I think that’s really useful and I think that policy and ethics work is doing its best, basically, when it’s really technically informed. If you find yourself working in a position where a lot of the things that you’re doing feel like they are important and would benefit from this sort of work, like helping people who are working on it is a really excellent way of helping. It’s not the only thing that you can do is spend half of your time doing the work that I’m doing and the others on the team are doing. You can also get people like us to do it. We love it.

Amanda Askell: If you’re interested in this, so thank you very much.

Brooke Chan, Amanda Askell, Lilian Weng, Christine Payne, Ashley Pilipiszyn

OpenAI girl geeks: Brooke Chan, Amanda Askell, Lilian Weng, Christine Payne and Ashley Pilipiszyn answer questions at OpenAI Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X 

Audience Member:  I have a question.

Amanda Askell: Yes.

Audience Member: For Amanda.

Amanda Askell: Yes.

Audience Member: Drink your water first. No, I think the ethics stuff is super interesting. I don’t know of a lot of companies that have an ethics department focused on AI, and I guess one thing that I’m curious about is, like you pointed out like your papers but like, and I know you talked about educating and all this other stuff but what are you guys…do? Do you know what I mean? Other than write papers.

Amanda Askell: Yeah.

Ashley Pilipiszyn: Oh, Christine.

Amanda Askell: Which one? Yeah, so I think at the moment there’s like a few kind of rules. I can say what we do but also what I think that people in these roles can do. So in some cases it can be like looking at what you’re building internally. I think we have like the charter and so you want to make sure that everything that you’re doing is in line with the charter. Things like GPT-2 and release decisions, I think of as a kind of like ethical issue or ethical/policy issue where I would like to see the ML community build really good norms there. Even if people don’t agree with what OpenAI try to do with its release decisions, it was coming from a place of trying to build good norms and so you can end up thinking about decisions like that.

Amanda Askell: That’s more of an example of something where you’re like it’s not writing a paper, it’s just like thinking through all of the consequences of different publication norms and what might work and what might not. That’s like one aspect, that’s the kind of like internal component. I think of the external component as like, on the one hand it’s just like writing papers so just being like what are the problems here that people could work on and in a lot ways that’s just like outreach, like trying to get people who are interested in working on this to work on it further. For that, there’s a few audiences, so you might be interested in attracting people to the field if you think that there are these like ongoing problems within both companies and maybe with other relevant actors. Like maybe you also want people going into government on this stuff.

Amanda Askell: But also just like the audience can be internal, to make people aware of these issues and they can also be things like policymakers, just inform of the kind of structure of the problem here. I think of it as having this kind of internal plus external component and you can end up dividing your time between the two of them. We spend some time writing these papers and trying to get people interested in these topics and just trying to solve the problems. That’s the nice thing about papers is you can just be like, what’s the problem, I will try and solve it and I’ll put my paper of an archive. Yeah, and so I think there’s both of those.

Amanda Askell: It’s obviously fine for companies to have people doing both, like if you haven’t and I think it’s like great if a company just has a team that’s just designed to look at what they’re doing internally and if anyone has ethical concerns about it, that team can take that on and own it and look at it. I think that’s a really good structure because it means that people don’t feel like … if you’re like just having to raise these concerns and maybe feel kind of isolated, that’d be bad but if you have people that you know are thinking about it, I think that’s a really good thing. Yeah, internal plus external, I can imagine different companies liking different things. I hope that answers the question.

Rose: My question is also for Amanda. So the Google AI Ethics Board was formed and disbanded very quickly kind of famously within like the span of less than a month. How do you kind of think about that like in the context of the work that OpenAI is doing and like how do you think about like what they failed at and like what we can do better?

Amanda Askell: This was a really difficult case so I can give you … I remember personally kind of looking at this and being like I think that one thing that was in it … I don’t know if people know the story about this case but basically, it was that Google formed a board and they were like, “We want this to be intellectually representative,” and it garnered a lot criticism because it had a person who was head of the Heritage Foundation, so a conservative think-tank in the US, as one of its members, and this was controversial.

Amanda Askell: I remember having mixed views on this, Rose. I do think it’s great to … Ultimately, these are systems that are going to affect a huge number of people and that includes a huge number of people who have views on how they should be used and how they should affect them. They’re just very different from me and I want those people to be represented and I want their views on how they do or do not want systems to affect them to be at the table. We talked earlier about the importance of representativeness and I genuinely believe that for people who have vastly different views for myself. If they’re affected by it, ultimately, their voice matters.

Amanda Askell: At the same time, I think I also … there’s a lot of complicating–you’re getting my just deeply mixed emotions here because I was like, there’s a strange sense in which handpicking people to be in the role of a representative of a group where you’re like, I don’t know, we select who the intellectual representatives are also struck me as somewhat odd. It’s a strange kind of … It set off my old political philosophy concerns where I’m like, “Oh, this just doesn’t …” It feels like it’s imitating democracy but isn’t getting there. I had and I was also just like plus the people who come to the table and there are certain norms of respect to lots of groups of people that just have to be upheld if you’re going to have people with different views have an input on a topic.

Amanda Askell: I think some of the criticisms were that people felt those norms had been upheld and this person had been insulting to key groups of people, the trans community and to immigrants. Largely, mixed feelings where I was just like I see this intention and it actually seems to me to be a good one, but I see all of these problems with trying to execute on it.

Amanda Askell: I can’t give an awesome response to this. It’s just like yeah, here it is, I’ve nailed it. It’s just like yeah, these are difficult problems and I think if you came down really strongly on this where it was like this was trivially bad or you were like this was trivially good, it just feels no, they were just like there are ways that I might have done this differently but I see what the goal was and I’m sympathetic to it but I also see what the problems were and I’m sympathetic to those. Yeah, it’s like the worst, the least satisfying answer ever, I guess.

OpenAI Girl Geek Dinner audience women in AI.

OpenAI Girl Geek Dinner audience enjoys candor from women in AI.  Erica Kawamoto Hsu  / Girl Geek X

Audience Member: Hi, I have a question for Brooke. I’m also a fan of Dota and I watched TI for two years. My question is, if your model can already beat the best team in the world, what is your next goal?

Brooke Chan: Currently, we’ve stopped the competitive angle of the Dota project because really what we wanted to achieve was to show that we could get to that level. We could get to superhuman performance on a really complex game. Even at finals, we didn’t necessarily solve the whole game because there were a lot of restrictions, which people brought up. For example, we only used 17 out of the you know 100 and some heroes.

Brooke Chan: From here, we’re just looking to use Dota more as a platform for other things that we want to explore because now we know that it’s something that is trainable and can be reused in other environments, so yeah.

Audience Member: Hi, my question is about what are some of the limitations of training robots in a simulator?

Lilian Weng: Okay, let me repeat. Question is, what’s a limitation of training the robot-controlled models in the simulation? Okay, there are lots of benefits, I would say, because in simulation, you have the ground rules. You know exactly where the fingertips are, you know exactly what’s the joint involved. We can do all kinds of randomization modification of the environment. The main drawback is we’re not sure what’s the difference between our simulated environment and reality. Our eventual goal is to make it work in reality. That’s the biggest problem. That’s also what decides whether our sim2real transfer going to work.

Lilian Weng: I will say one thing that confuse me or puzzles me personally the most is when we are running all kinds of randomizations, I’m not sure whether it’s getting us closer to the reality because we don’t have a good measurement of what the reality looks like. But one thing I didn’t emphasize a lot in the talk is we expect because we design all kinds of environment in the simulation and we asked the policy model to master all of them. There actually emerges some meta learning effect, which we didn’t emphasize but with meta learning, your model can learn how to learn. We expect this meta learning in fact to empower the model to handle something they’d never seen before.

Lilian Weng: That is something we expect with domain randomization that our model can go above what it has seen in the simulation and eventually adapt to the reality. We are working all kinds of technique to make the sim2real thing happen and that’s definitely the most difficult thing for robotics because it’s easy to make things work in simulation. Okay, thanks.

Audience Member: I was just curious as kind of another follow-up question to Brooke’s answer for earlier but for everybody on the panel too. What do you consider to be some of the longer-term visions for some of your work? You did an impressive thing by having Dota beat some real people but where would you like to see that work go or what kinds of problems do you think you could solve with that in the future too, and for some other folks on the panel too?

Brooke Chan: Sure, I would say that pretty honestly when we started the Dota project we didn’t actually know whether or not we would be able to solve it. The theory at the time was that we would need a much more powerful algorithm or a different architecture or something in order to push it kind of all the way. The purpose of the project was really to demonstrate that we could use a relatively straightforward or simple algorithm in order to work on this complex game.

Brooke Chan: I think going out from here, we’re kind of looking into environments in general. We talked about how Dota might be one of our last kind of games because games are still limited. They’re helpful and beneficial in that you can run them in simulation, you can run them faster but we want to kind of also get closer to real-world problems. Dota was one step to getting to real-world problems in the parts that I talked about like the partial information and the large action space and things like that. Going on from there, we want to see what other difficult problems you could also kind of apply this sort of things to. I don’t know if other people …

Christine Payne: Sure. In terms of a music model, I would say kind of two things I find fascinating. One is that I really like the fact that it’s this one transformer architecture which we’re now seeing apply to lots of different domains. The fact that it can both do language and music and it’s really kind of interesting to find these really powerful algorithms that it doesn’t care what it’s learning, it’s just learning. I think that that’s going to be really interesting path going forward.

Christine Payne: Then, also, I think that music is a really interesting test for like we have a lot of sense as humans so we know how we would want the music to go or we know how the music affects us emotionally or there’s all this sort of human interaction that we can explore in the music world. I hear from composers saying well, they want to be able to give the shape of the music or give the sense of it or the emotion of it, and I think there’s a lot of space to explore in terms of it’s the same sort of thing we’ll want to be able to influence the way any program is going to be, how we’ll be interacting with a program in any field but music is a fun area to play with it.

Ashley Pilipiszyn: Actually, as a followup, if you look at all of our panelists and everything everyone presented too, it’s not just human and AI interaction, but human and AI cooperation. Actually, for anyone who followed our Dota finals event as well, not only did we have a huge success but, and for anyone who is a Dota fan in the crowd, I’d be curious if anyone participated in our co-op challenge. Anyone by chance? No, all right. That’s all right.

Ashley Pilipiszyn: But actually, being able to insert yourself as being on a team with OpenAI Five and I think from all of our research here we’re trying to explore the boundaries of, you know what does human AI cooperation look like and I think that’s going to be a really important question going forward so we’re trying to look at that more.

Speaker: And we have time for two more questions.

Audience Member: Thank you. Just right on time. I have a question for you, Christine. I was at a conference earlier this year and I met this person named Ross Goodwin who wrote using a natural language processing model that he trained a screenplay. I think it’s called Sunspring or something like that. It’s a really silly script that doesn’t make any sense but it’s actually pretty fun to watch. But he mentioned that in the media it’s been mostly–the credit was given to an AI wrote this script and his name was actually never mentioned even though he wrote the model, he got the training data. What is your opinion on authorship in these kinds of tools that … also the one you mentioned where you say you’re actually composing? Are you the composer or is the AI the composer? Should it be like a dual authorship?

Christine Payne: That is a great question. It’s a difficult question that I’ve tried to explore a little bit. I’ve actually tried to talk with lawyers about what is copyright going to look like? Who owns pieces like this? Because in addition to who wrote the model and who’s co-composing or co-writing something, there’s also who’s in the dataset. If your model is imitating someone like are they any part of the author in that?

Christine Payne: Yeah, I mean I have my own sort of guesses of where I think it might go but every time … I think I’m a little bit [inaudible 01:37:11] in terms of the more you think about it, the more you’re like this is a hard problem. It’s really, like if you come down hard on one side or the other because clearly, you don’t want to be able to just press go and have the model just generate a ton of pieces and be like I now own all these pieces. You could just own a ridiculous number of pieces, but if you’re the composer who has carefully worked and crafted the model, crafted … you write a little bit of a piece, you write at some of the piece and then the model writes some and you write some. There’s some interaction that way, then sure, that should be your piece. Yeah, I think it’s something that we probably will see in the near future, law trying to struggle with this but it’s an interesting question. Thanks.

Audience Member:  Okay, last question. Oh no.

Ashley Pilipiszyn: We’ll also be around so afterwards you can talk to us.

Audience Member: This is also a followup question and it’s for everyone on the panel. Could you give us some examples of real-life use cases of your research and how that would impact our life?

Ashley Pilipiszyn: An example.

Christine Payne: It’s not an easy one to close on. You want to take it. Go for it.

Lilian Weng: I will say if eventually we can build general purpose robots, just imagine we use the robot to do a lot of dangerous tasks. I mean tasks that might seem danger to humans. That can definitely reduce the risk of human labors or doing repeated work. For example, on assembly line, there are some tasks that involve human hands, but kind of boring. I heard from a friend that there are a lot of churn or there’s a very high churn rate of people who are working on the assembly line, not because it’s low pay or anything, most because it’s very boring and repetitive.

Lilian Weng: It’s not really good for people’s mental health and they have to–like the factory struggle to hire enough people because lots of people will just leave their job after a couple months or half a year. If we can automate all those tasks, we’re definitely going to leave others more interesting and creative position for humans to do and I think that’s going to overall move the productivity of the society. Yeah. That’s still a very far-fetched goal. We’re still working on it.

Amanda Askell: I can also give a faraway thing. I mean I guess my work is,, you know with the direct application, I’m like, “Well, hopefully, ML goes really well.” Ideally, we have a world where all of our institutions are actually both knowledgeable of the work that’s going on in ML and able to react to them really well so a lot of the concerns that people have raised around things like what happens to authorship, what happens to employment, how do you prevent things like the misuse of your model, how can you tell it’s safe? I think if policy work goes really well then ideally, you live in a world where we’ve just made sure that we have all of the kind of right checks in place to make sure that you’re not releasing things that are dangerous or that can be misused or harmful.

Amanda Askell: That just requires a lot of work to ensure that’s the case, both in the ML community, and in law and policy. Ideally, the outcome of great policy work is just all of this goes really smoothly and awesomely and we don’t have any bad things happen. That’s like the really, really modest goal for AI policy work.

Brooke Chan: I had two answers on the short-sighted term, in terms of just AI being applied to video games, AI in video games historically is really awful. It’s either really just bad and scripted and you can beat it easily and you get nothing from it or it’s crazy good because it’s basically cheating at the game and it’s also not really that helpful. Part of what we found out through the Dota project was people actually really did like learning with the AI. When you have an AI that’s at your skill level or slightly above, you have a lot of potential, first of all, to have a really good competitor that you can learn from and work with, but also to be constantly challenged and pushed forward.

Brooke Chan: For a more longer-term perspective, I would leverage off of the robotics work and the stuff that Lilian is doing in terms of the system that we created in order to train our AI is what is more general and can be applied to other sorts of problems. For example, that got utilized a little bit for the robotics project as well and so I feel it’s more open-ended in that sense in terms of the longer-term benefits.

Christine Payne: Okay and I’ll just wrap up saying yeah, I’ve been excited already to see how musicians and composers are using MuseNet. There are a couple examples of performances that have happened now of MuseNet pieces and that’s been really fun to see. The main part that I’m excited about is that I think the model is really good at just coming up with lots and lots of ideas. Even though it’s imitating what the composers might be doing, it opens up possibilities of like, “Oh, I didn’t think that we could actually do this pattern instead.” Moving towards that domain of getting the best of human and the best of models I think is really fun to think about.

Ashley Pilipiszyn: So kind of how I started the event this evening, our three main research areas are really on these capabilities, safety, and policy. You’ve been able to hear that from everyone here. I think the big takeaway and a concrete example I’ll give you is, you think about your own experience going through primary education. You had a teacher and you most likely … you went to science class then you went to math class and then maybe music class and then art class and gym. You had a different teacher and they just assumed, probably for most people, you just assumed you’re all at the same level.

Ashley Pilipiszyn: How I think about it is, we’re working on all these different kind of pieces and components that are able to bring all of these different perspectives together and so a system that you’re able to bring in the math and the music and the gym components of it but also able to understand what level you’re at and personalize that. That’s kind of what I’m really excited about, is this human AI cooperation component and where that’ll take us and help unlock our own capabilities. I think, to quote from Greg Brockman, our CTO, that while all our work is on AI, it’s about the humans. With that, thank you for joining us tonight. We’ll all be around and would love to talk to you more. Thank you.

Speaker: We have a quick update from Christina on our recruiting team.

Ashley Pilipiszyn: Oh, sorry.

Christina Hendrickson: Hey, thanks for coming again tonight. I’m Christina. I work on our recruiting team and just briefly wanted to talk to you about opportunities at OpenAI. If you found the work interesting that you heard about from our amazing speakers tonight and would be interested in exploring the opportunities with us, we are hiring for a number of roles across research, engineering and non-technical positions.

Christina Hendrickson: Quickly going to highlight just a couple of the roles here and then you can check out more on our jobs page. We are hiring a couple roles within software engineering. One of them, or a couple of them are on robotics, so that would be working on the same type of work that Lillian mentioned. We are also hiring on our infrastructure team for software engineers, as well, where you can help us in building some of the world’s largest supercomputing clusters.

Christina Hendrickson: Then the other thing I wanted to highlight is one of our programs. So we are going to have our third class of our scholars program starting in early 2020. We’ll be opening applications for that in a couple weeks so sneak peek on that. What that is, is we’re giving out eight stipends to people who are members of underrepresented groups within engineering so that you can study ML full-time for four months where you’re doing self-study and then you opensource a project.

Christina Hendrickson: Yeah, we’re all super excited to chat with you more. If you’re interested in hearing about that, we have a couple recruiting team members here with us tonight. Can you all stand up, wave? Carson there in the back, Elena here in the front, myself. Carson and I both have iPads if you want to sign up for our mailing list to hear more about opportunities.

Elena Chatziathanasiadou waving

Recruiters Christina Hendrickson and Elena Chatziathanasiadou (waving) make themselves available for conversations after the lightning talks at OpenAI Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Christina Hendrickson: Thank you all again for coming. Thanks to Girl Geek X. We have Gretchen, Eric, and Erica here today. Thank you to our speakers: Brooke, Amanda, Lilian, Christine, Ashley, and thank you to Frances for helping us in organizing and to all of you for attending.

Ashley Pilipiszyn: Thank you, everybody.

Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Bosch Lightning Talks (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Tara Dowlat, Seow Yuen Yee, Yelena Gorlin, LisaMarion Garcia, Panpan Xu, Shabnam Ghaffarzadegan, Sun-Mi Choi

Bosch girl geeks: Tara Dowlat, Seow Yuen Yee, LisaMarion Garcia, Sun-Mi Choi, Yelena Gorlin, Panpan Xu and Shabnam Ghaffarzadegan at Bosch Girl Geek Dinner in Sunnyvale, California.   Erica Kawamoto Hsu / Girl Geek X

Transcript from Bosch Girl Geek Dinner – Lightning Talks:

Angie Chang: We are really excited coming to Bosch to be listening to so many amazing girl geeks tonight.

Dr. Hauke Schmidt: We are very happy to host the Girl Geek dinner as a celebration of gender diversity, and I’m very proud of the team here who has put all this together.

Dr. Uma Krishnamoorthy: How many of you came here looking for headphones, acoustic systems in our demos? We’re not that company. You may have gone outside and you may have seen our car, autonomous car, so I don’t have to speak to our autonomous driving effort.

Dr. Seow Yuen Yee: Have you ever thought of how does the car know when to deploy these airbags? This is thanks to the airbags control unit in the car. It house a tiny little sensors which we call accelerometers.

Tara Dowlat: Did you guys know that at least every single one of you in this room, in your pockets or in your bags, have at least one sensor from Bosch on you? It’s a fun fact.

Dr. Yelena Gorlin: Each new generation of a battery management system looks to increase the charging speed of our device without having an effect on its lifetime.

LisaMarion Garcia: Each of these individual sectors provide us different opportunities to incorporate AI, either as a feature of a product that we sell or as part of the process of producing that product.

Dr. Shabnam Ghaffarzadegan: So our idea is asking human and machine to work together to empower their both abilities with much more perception and knowledge, and also to make a better machine to help us in our everyday life.

Sun-Mi Choi: So how many of you are using ride hailing apps to get from A to B on a regular basis? Mobility is also getting more user centric. The consumer is more and more changing from owned to shared.

Dr. Uma Krishnamoorthy: Big goals here. 2020, the goal is all of our electronic products will be connected. And in 2025, all our products are going to either possess intelligence or AI will have played a key role in their creation.

Angie Chang: Thanks for coming out tonight. I’m Angie Chang, founder of Girl Geek X. We’ve been hosting Girl Geek dinners up and down from San Francisco to San Jose for the last 11 plus years. We are really excited to be coming to Bosch to be listening to so many amazing girl geeks tonight.

Gretchen DeKnikker: I got my own microphone. You guys have no idea what that means. I’m Gretchen. Thank you. How many of you, it’s your first Girl Geek dinner? Good. Okay. So like she said, we do them every week. We also have a podcast, so pull out your phone now and go to your favorite podcast app and then rate it and then write a review or send us a message and say, “This is how it could be better.” Because we’re only doing it so it’ll be awesome for you guys. Right?

Gretchen DeKnikker: Then we also recently opened a little swag store on Zazzle. So there’s all sorts of cute things. I only have one or two cute things tonight. Cute water bottle.

Gretchen DeKnikker: I know. It’s ridiculous. Oh, I kind of had stuff with … There are more designs than this one. Apparently I only just brought things … But it’s a fanny pack. It’s so cute. Okay. Got it. So I’m going to try something new tonight. Who’s found a job through Girl Geek? No one? Okay, get out. Okay, has anyone got a … Oh, you did.

Audience member: No.

Gretchen DeKnikker: Oh. No, definitely not. That’s awesome. Okay, anyone found a job lead? Oh, okay.

Audience Member: I found candidates through Girl Geek.

Gretchen DeKnikker: You found candidates. Okay. So if you guys want to email us, I have these things and you can’t buy them. You can only get it from me. These adorable socks. So if you want to tell us, we would love to feature your story about finding a girl geek, a job through Girl Geek Dinner or something that you built and we want to have little community features and stuff. If you do it, you get those socks and it’s the only way in the world to get the socks.

Gretchen DeKnikker: Okay, so without further ado, how great is this space? This has been so awesome so far. You guys enjoying it? All right, so without further ado, we are bringing this gentleman right here.

Dr. Hauke Schmidt: Thank you very much, and welcome to Bosch. So my name is Hauke Schmidt. I’m the head of corporate technology research for Bosch here in North America. And I’m also the site leader for the innovation center here in Sunnyvale. A few words about the company for those of you who don’t know Bosch all too well. We have our roots in the automotive business, so we’re actually the largest automotive supplier in the world.

Dr. Hauke Schmidt: And very likely, if you open your car, there are a couple of Bosch components inside. You also might know us from household appliances or power tools. We’re also a leading IoT company, as you saw in the videos, here. And we’re driving product and services innovation in the areas of mobility, industrial, and building technologies.

Dr. Hauke Schmidt: One interesting part about Bosch is the ownership structure. We are privately held. We’re a very large multinational out of Germany and privately held. And mostly to the largest part, owned by the Robert Bosch Foundation. And the Foundation then also takes all of the profits and earnings we create and puts them to use in charitable projects. So this gives us an extra motivation to work hard and provide good results.

Dr. Hauke Schmidt: The site here, we’ve been in Silicon Valley for 20 years now. We have our 20th anniversary this year. We moved into this building one and a half years ago so this is now our new home here with a nice Bosch sign outside as well. We have about 200 scientists, engineers, and experts on site, and these experts cover a broad variety of different functions of the company. We have here everything from corporate research, venture capital technology scouting, prototyping, product development, but we also have product sales and engineering services here on site that we offer into the local industry around us.

Dr. Hauke Schmidt: For us diversity is an important thing. We have associates here from a very broad variety of different ethnic backgrounds, also from experts in a large number of different technology fields. So today we are very happy to host the Girl Geek dinner as a celebration of gender diversity and I’m very proud of a team here who’s put all this together since I’m also the executive champion at the Women at Bosch Group here on site as well.

Dr. Hauke Schmidt: Thank you. So with that ,without further ado, I would like to hand over to Uma who has her own microphone to kick off some of the lightning talks that we’ll listen to right now. Thank you.

Dr. Uma Krishnamoorthy: Can you hear me now?

Audience: Yes.

Uma Krishnamoorthy speaking

Director of Research Dr. Uma Krishnamoorthy gives a warm welcome to the crowd at Bosch Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Dr. Uma Krishnamoorthy: Okay. First welcome from my side. My name is Uma Krishnamoorthy and I am a director here at Bosch RTC. We are part of corporate. We, me and my department, are part of corporate research of the bigger Bosch. My particular groups are focused on microsensor systems technologies and multiphysics modeling and simulation areas of research.

Dr. Uma Krishnamoorthy: So today, this works, my role is very easy. It’s going to be a bit longer than the others but my role is relatively easy. I’m going to be giving your introduction to Bosch from a broader scale than what hopefully Hauke did. Then, of course, I’m going to lead into the Internet of Things and how we play a role in there.

Dr. Uma Krishnamoorthy: Hauke unfortunately told you what we do, so I’m going to ask anyway. How many of you already were aware of what Bosch does and what our products are before you came to the dinner today? Oh, that’s quite a few. Okay. The reason I ask, how many of you came here looking for headphones, acoustic systems in our demos or lens solutions? We’re not that company.

Dr. Uma Krishnamoorthy: Yep, we are Bosch. Who are we? First thing, we’re very diverse and the range of products we cover is very broad. I’m going to try to cover some of it today from the perspective of IoT. I’ll start off with this slide here, market figures. Bosch, exactly as Hauke mentioned, is from– originally started by Robert Bosch in 1886. So we’re over 130 years old.

Dr. Uma Krishnamoorthy: Yeah, we’re pretty old. We started in Germany, but as you can see we’re a global company. We have been in the Americas since 1906, I believe, over 100 years old. Very, very long time, very well established manufacturing company. We’ve made a very huge reputation in creating high quality products.

Dr. Uma Krishnamoorthy: We have 268 manufacturing sites across the world. Of course, we have a lot of representation in Asia-Pacific also. I wanted to draw your attention to that number right in the middle, 409,881 associates. That’s a huge number. Just to give you an idea, you take all of the associates at Alphabet, all of the associates at Apple, combine them, multiply it by approximately two. Okay, you’re all Girl Geek so approximately 1.78. And that will be the number of associates at Bosch. This was of course from 2018, so we are huge.

Dr. Uma Krishnamoorthy: To give you an idea of scale. So what do we do? I’m going to try to answer that question with this slide. You may be aware of our products in the consumer goods business. You may have seen our dishwashers, washing machines, maybe some coffee makers, many household appliances, power tools. Very popular there and a leading supplier. We also work in energy and building technology. What is this?

Dr. Uma Krishnamoorthy: Here’s a leading manufacturer of security communication technology. We actually make energy efficient heating products. This is a bigger business in Germany maybe than here, so we’re very well known for that. Or Europe, not Germany. On top of that, Hauke already mentioned mobility solutions.

Dr. Uma Krishnamoorthy: Sixty percent of our sales come from the mobility solutions business. This includes automotive and also consumer electronics. Essentially things like sensors that go in your cell phone, smartwatches, things of that sort. We’re a leading provider of that too.

Dr. Uma Krishnamoorthy: Surprising to me, I’ve been with Bosch for four years so this was a bit of a surprise, industrial technology. We also make a variety of industrial technologies. What does this mean? If you’ve ever been to the Jelly Belly factory, on the way back from Tahoe, you know, it’s a good stop.

Dr. Uma Krishnamoorthy: So if you stop there and look around, take a tour of the factory floor, you will see Bosch equipment, packaging equipment. I believe they might have been sorting the jellybeans, but I can’t remember exactly. So we are pretty broad and you’ll see us in many places, unexpected places. That’s how broad we are.

Dr. Uma Krishnamoorthy: To give you an idea of our culture, Hauke already mentioned our founder, Robert Bosch. We strongly follow the values of our founder Robert Bosch, which comprises of quality and innovation which is what our products are known for. This may not be as well-known over here in the US, but it’s known in Germany for sure, is the aspect of social commitment.

Dr. Uma Krishnamoorthy: Robert Bosch himself gifted the Robert Bosch Hospital to the City of Stuttgart back in 1936, which stands to this day. A lot of very important medical research is done there, including, I believe … I can’t remember all the details but a variety of really good medical research is done there.

Dr. Uma Krishnamoorthy: As Hauke mentioned, we’re privately held. Ninety percent of our shares are held by this Robert Bosch Foundation and this foundation fundamentally finances work that addresses social challenges. So they focus on areas like healthcare, science, society, education, international relations, all about society and life.

Dr. Uma Krishnamoorthy: They have provided, the number’s right there. 153-ish million euros to project grants that are in these areas. So, they really put their money where their values stand. That’s the message there. As I mentioned, one of the one of the cornerstones of Bosch is our innovation. We’re worldwide but we also have a very strong commitment to innovation. We have a, I don’t have the numbers here, a very large number of associates. Believe it was in 65,000 number range of associates who work in R&D across the company.

Dr. Uma Krishnamoorthy: Some of those actually work under a separate division called corporate research, which we’ve alluded to in the past and what you see in the background here is our campus that was recently built in Germany specifically for corporate research that services all of the Bosch groups,, fundamentally, almost all of them.

Dr. Uma Krishnamoorthy: And, what you really … I would like to highlight this one sentence over here our objective. Our motto is invented for life which is pretty much self-explanatory. So everything we do is about the quality of life, enhancing the quality of life through technology. I would like to say one more thing about this. Recently–I’ll have to … Mind me if I refer to my notes. Only because our CEO recently announced that we Bosch were going to be the first carbon-neutral industrial enterprise from 2020. That is a huge statement, and we’re all committed to delivering on that.

Dr. Uma Krishnamoorthy: What we came for, that was the introduction very briefly. I’ll try to go through this pretty fast. IOT at Bosch. This is going to essentially be kicking off a series of tech talks centered around IoT for Bosch. I’m only going to set it up for them. The real speakers will come after me.

Dr. Uma Krishnamoorthy: So what does IoT mean for Bosch? As many of you know, IoT is about creating better customer experiences through connectivity. And Bosch plays a very big role in it because we make a variety of products and we’re connecting them to make our customers get a better experience out of it, fundamentally. That’s the simplest way you can think about it.

Dr. Uma Krishnamoorthy: In the process, though, what we are noticing is industries are transforming, and we are playing a key role in this transformation at Bosch. So how are we playing in this field? Just giving you a sampling over here. You may have gone outside and you may have seen our car, autonomous car, so I don’t have to speak to our autonomous driving effort, our driver assistance efforts. There’s many of those that are ongoing that are widely shared.

Dr. Uma Krishnamoorthy: But on top of our mobility efforts we also work in the smart city area. We have products in all of these areas so connecting them and providing customer experiences goes beyond mobility into smart city, into buildings, industry, industry 4.0. But one of the key things for us, for our connected Bosch systems across these domains is we are creating intelligent user centric solutions without compromising safety or data security. Those are big messages that we carry and we essentially put into all our products.

Dr. Uma Krishnamoorthy: What is Bosch’s IoT vision? Again a borrowed slide. You will see big goals here. 2020, the goal is all of our electronic products will be connected. We’re going to continue working across a variety of domains and in 2025 all our products are going to either possess intelligence or AI will have played a key role in their creation. So AI is closely tied to our IoT.

Dr. Uma Krishnamoorthy: A few examples, I’ll have to go very quick. She just told me I have five minutes left. Quick examples, home appliances. Series 8 oven. It’s an oven, yes, but it’s also a microwave, it’s also a steamer, and it’s connected. So you can bake a cake–if you have the right app–you can bake a cake in it from your phone, and I’ll leave it there.

Dr. Uma Krishnamoorthy: This app is apparently not available everywhere but it is there, the technology is there. Mobility, you already mentioned that powertrains is one of the big areas we contribute in for the automotive business. Electric powertrains is our big area of work now. One thing I’ll show here is we are taking it beyond just electrification of cars, we’re actually moving into other powertrain systems for other vehicles such as two wheelers and trucks.

Dr. Uma Krishnamoorthy: Another aspect here is beyond just building EV vehicles, we’re also looking at connecting these vehicles. So anybody using an EV vehicle cares about charging them. So we actually have an app. Bosch has an app that’ll let you find up to 20,000 charging stations, which is very convenient, in five countries. I believe that will be increasing as this gets used more.

Dr. Uma Krishnamoorthy: Last but not least, the example automated valet parking. This came out recently. I had a beautiful video on this. It took too long so I’ll just tell you in two sentences. Automated valet parking. It’s like a mini autonomous vehicle that you can use in a parking garage.

Dr. Uma Krishnamoorthy: You bring your car to the garage, you walk out of it, hit the park button on your phone, the car will go park itself. When you are done with your dinner or whatever else, you come back to the garage. Say pick up the car. The car will drive itself to you. You can get in it and go home. That’s the idea and it’s actually real and they already rolled it out. So, that’s an example of some of the innovation we contribute to.

Dr. Uma Krishnamoorthy: Now I’ll be talking to you about some of the elements of IoT, not for very long. We have tech talks following me, they’ll go into all the details. So here, I’m going to talk briefly about transformation from the things to IoT. I’ve already mentioned that we make a lot of things here at Bosch across many domains. But one of the fundamental things we do is in the hardware. Sensors is a big area for Bosch, we are one of the enablers–sensors are the enablers for the Internet of Things and we’re one of the leaders in building micro sensors. Bosch Sensor Tech, in fact, is the part of Bosch that builds them, and you’ll be hearing a lot more about that from Tara right after me.

Dr. Uma Krishnamoorthy: Sensors are the data collectors. They are your direct connection to your products, they collect the states of your products, whatever they are. Then, another aspect of it that is kind of hidden, but is very important as batteries. So we need batteries to charge all of our things and our sensors and our phones, everything else. So that’s another aspect that we will be talking about soon. Yelena will be talking about it, I believe.

Dr. Uma Krishnamoorthy: Bosch has a strong background in the hardware aspect of manufacturing and in sensors products. So we understand that, the cause and effect. That’s our core business. So, what else is there to be done in IoT? It’s all about the connectivity. So once you have the data, you have to connect to it. We have the data collectors.

Dr. Uma Krishnamoorthy: So the next thing you need is to analyze the data and to create some–once you acquire the data you want to provide some, I guess models, right, and some plans on essentially understanding the data and to potentially predict what’s going to happen for whatever system you’re working with. So that’s where our AI comes into play right, and LisaMarion will be talking about that. She’s part of our BCA, Bosch Center for Artificial Intelligence.

Dr. Uma Krishnamoorthy: Then, finally, it all comes down to the user and the user interface. So that portion will be handled. It’s an important portion but that portion will be handled by … Panpan and Shabnam will be talking about that. They’re a part of our human machine interface, we used to call the interactions, human machine interaction group.

Dr. Uma Krishnamoorthy: So fundamentally we are integrating our hardware with AI, our IoT products and our sensors and that’s in a very, very high-level picture of what Bosch does in IoT. I’m going to stop there and hand the microphone on to Tara. So Tara and Seow Yuen Yee will be talking about sensors next and they will introduce the next speakers. So thank you very much.

Seow Yuen Yee and Tara Dowlat

Senior Research Engineer Dr. Seow Yuen Yee and Senior Account Manager Tara Dowlat give a talk on sensors for IoT at Bosch Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Tara Dowlat: Hi everyone, my name is Tara Dowlat and I’m part of Bosch Sensor Tech. I’m part of the team that focuses on consumer electronic sensors and I’m an Account Manager, part of sales team.

Dr. Seow Yuen Yee: Hi everyone, my name is Seow Yuen. If it’s hard to pronounce you can call me SY. I’m the senior research engineer here in the corporate research. I’m part of Uma’s team. What I do is I make sensors and these sensors go to your car, your home and your phone. So I’ll tell you more about it later.

Tara Dowlat: So, did you guys know that at least every single one of you in this room in your pockets or in your bags have one sensor and most like the majority of you guys had least one sensor from Bosch on you? It’s a fun fact. Let me tell you that sensors are all around us. We might notice it, we might not, but these tiny, tiny little devices are actually pretty commonly used.

Tara Dowlat: They’re made out of micro electromechanical systems. They go also known as MEMS. These devices are made out of silicon. Silicon is the same exact material we use for semiconductor chips and they are used for really complex circuits or switches that we use in our industry today.

Tara Dowlat: If you look at the picture to the right side over here, this shows the structure of a MEMS and you can see that within a thickness of a hair line how many tiny little springs we’re able to fit in there. That’s a MEMS structure for you and typically these devices are within millimeter square. So we can see that how detailed and small these structures are and I find it personally very impressive.

Dr. Seow Yuen Yee: How are sensors made? The process starts with the silicon ingot that you can see on the left there and then it is later cut into thin slices that we call the silicon wafers. So this is an example of the silicon wafers. By itself it is not useful until we are able to process on it to make intricate features. We are able to do this thanks to our Bosch colleagues Franz Laermer and Andrea because they invented the deep reactive ion etching in 1996.

Dr. Seow Yuen Yee: It is now known as the Bosch Process because it has the ability to create a high aspect ratio profile in the silicon wafers. How high is a high aspect ratio and how tiny is tiny? Here’s an example that is the width of these trenches as five micron wide and the height–the deep is 50 micron deep. So you can imagine how small all these features are.

Dr. Seow Yuen Yee: Accelerometer, we’ll tell you later about it. It’s an example of a type of sensors that we are able to create using this process and Tara will tell you more about the sensors and other sensors, about accelerometers other sensors.

Tara Dowlat: So just as SY mentioned, we have a family of classical sensors known as motion sensors. We have magnetometers, accelerometers, gyroscopes, the combination of two that would be an IMU or you put all of the three together it’s known as nine degree of freedom or absolute orientation.

Tara Dowlat: But why do we care about these sensors in general? What’s the application or how do they improve our lives? Well the most classical approach was the use of sensors and automobiles. You guys might have heard about ABS, ESP or even tire pressure monitoring system on newer cars. These are sensor applications. Without the sensors on your cars, you guys would not have these safety functionalities.

Tara Dowlat: Let me ask you this. If you had the choice between a sports car, a sedan, or SUV for safety of your family which class of car would you guys probably pick?

Audience: SUV.

Tara Dowlat: Okay. Let me tell you. Twenty years ago that was not the concept. SUVs and safety were not two words used in the same sentence. Actually these cars were known to be rolling over on the road and actually not safe at all. So what changed since then? The use of a gyroscope on the car is enabling them to stay stable on the road and not roll over. That makes them safe.

Tara Dowlat: Within 20 years or so the market and perception has changed so much that all of you guys think SUV is the best choice to go with. That’s the use of sensor. But, also the modern applications. Take autonomous driving, everybody in the news is talking about it. Autonomous driving would have not been possible without sensors or even more commonly used applications like Park Assist when you tell your car please park it for me in this tight spot. That’s using your sensors in the car, or when you’re trying to drive on the road and hopefully you guys are paying attention and it’s not dismissing the traffic or texting but more modern cars have this functionality that it actually tells you please slow down there’s an object in front of you. Don’t switch lane there’s an object next to you. These are the functionalities that modern cars have because of use of sensors in them.

Dr. Seow Yuen Yee: Applications that Tara mentioned there’s one more applications that should be familiar to all of you which is the airbags deployment. From 1987 to 201,8 more than 50,000 lives has been saved by airbags according to the US Transportation–Department of Transportation. Have you ever thought of how does the car know when to deploy this airbags?

Dr. Seow Yuen Yee: This is thanks to the airbags control unit in the car and in this control unit it has a tiny little sensors which we call accelerometers. When there’s movement like this impact in your car during the accident this [inaudible 00:29:32] this sudden impact.

Dr. Seow Yuen Yee: So let me show you the video of how it works. The accelerometer chip here contains of two parts, that’s the circuit chip and the MEMS sensors. In the MEMS sensors you can see the blue part is the movable part and the red part is the stationary part.

Dr. Seow Yuen Yee: When there’s movement in your car the blue part will move relative to the red part and from there it caused the relative capacitance change between these two parts. This capacitance change can then be sent to the airbag unit here which will deploy the airbags. For that it will protect you.

Dr. Seow Yuen Yee: The sensing part itself takes around 15 to 30 milliseconds time to sense it and the airbags will deploy from 60 to 80 milliseconds. So that’s how fast it is that can deploy to protect you.

Tara Dowlat: So, a more modern recent application for sensors are consumer electronics, specifically smartphones or tablets. You guys have might noticed over the past few years that actually the cameras have improved quite a bit in terms of picture quality. I hate to take all the credit for the sensors but they did play a part.

Tara Dowlat: You guys have might noticed that when you’re trying to take a picture you’re trying to zoom in and historically I was one of the people that would move the camera back and forth trying to get the best photo and then making sure that my picture’s not blurry. Well today the cameras do that for you and part of it is because of the image stabilization and the sensors that they use with the cameras. That’s one of the applications that uses a sensor.

Tara Dowlat: But another more commonly used one. When you go from horizontal to vertical on your phone when you’re looking at pictures and videos this is something that probably most of us use every day. That’s a use of a sensor on your phone. Or this one I’m a personal huge fan–navigation.

Tara Dowlat: I’m always lost and somehow people trust me to put me in charge of direction. But the reality of it is with my phone, if there is no magnetometer on it I’m looking at the direction and I don’t know if it says right is it really my right or my left.

Tara Dowlat: But a magnetometer on my phone would be able to tell me where is the true north and at what point do I need to truly turn right or left. That’s a really helpful application for most of us that we probably use and don’t commonly notice that it’s a sensor on there.

Dr. Seow Yuen Yee: One other thing is as you all know that GPS hardly works inside the building. In the case of an emergency, especially in tall buildings, it is very critical for the emergency first responder to know exactly where you are and this includes what floor you are in. The GPS do not give you this kind of information but our Bosch pressure sensor comes to rescue.

Dr. Seow Yuen Yee: Because of the as you increase the elevation, the altitude the air pressure decreases and this tiny change of pressure can be sensed by our Bosch pressure sensors. So let me show you another video of how the pressure sensor works. Again in the package it has two chip where there’s a circuit chip and the MEMS sensors.

Dr. Seow Yuen Yee: This time the MEMS sensors consist of a pressure sensitive membrane and on which there is four resistors which are connected in a wisdom bridge formation. As there’s the pressure change the shape of the membrane changes due to the pressure and the resistance is changed due to the change of the membrane.

Dr. Seow Yuen Yee: This resistance change is measured as water changes which ranged from one to five and this water changed correlates to the pressure and this pressure would tell you which elevation you are in. The information from this will be sent to the first responder and they will come to rescue you.

Tara Dowlat: Just as SY mentioned, pressure sensor belongs to another family of sensors that are getting quite commonly adapted nowadays, they belong to environmental sensors. That includes temperature, humidity, gas, or a combination of all those together as one single sensor.

Tara Dowlat: But how did they become so popular nowadays? Well, we are all health aware nowadays. I think most of you guys might be interested, but by show of hands how many of you guys track how many steps you’ve taken or how many stairs have you climbed today? Majority of you. Well, I guess most of us has invested in either a fitness band or a smartwatch or look at it on our phones.

Tara Dowlat: When you go under health application it tells you how many steps you’ve taken. That’s an accelerometer on your phone or on your device. Or if you’re interested in knowing how many climbs of stairs you’ve climbed today. Well, that’s a pressure sensor for you that gives that app information. But it’s not just about humans.

Tara Dowlat: So, I recently heard about a cool application from one of our potential customers that they are trying to put this step tracking option on their chicken. You would wonder why. But, I guess when you go to these stores you notice that there is like advertisement for eggs that are range free and organic, that extra dollar amounts that they are charging is justified because these chickens are taking more steps.

Tara Dowlat: The more steps they take, the healthier your chicken. But today we’re here for IoT and how does the sensor relate to IoT. How does that impact me as an individual? How does it change the quality of my life? I can take the example of a smart home. This belongs to the IoT category. Without the use of all these sensors, smart homes would not be possible. Let’s focus on my case specifically and I think some of you guys might relate.

Tara Dowlat: I’m here with you in the evening or the afternoon today. I will spend some time to drive home and during this drive I would be probably sitting in traffic, it’s hot and I’m thinking I wish when I get home that my Roomba has cleaned the floor. So IoT would be able to enable that.

Tara Dowlat: I wish that the AC has been running for the past 30 minutes because I’m somewhat environmental friendly but not extremely. I still like a cool room. So I’ll take that and I can make sure that a cup of coffee is waiting for me while I watch my last show before I go to bed. That’s a smart home for you.

Tara Dowlat: For IoT to be enabled we need to make sure that all these sensors are effectively and efficiently communicating. But then it becomes a matter of power consumption. That’s why Yelena would introduce battery management, which is a really important topic here at Bosch for us. Thank you.

Yelena Gorlin speaking

Senior Engineer Dr. Yelena Gorlin gives a talk on enabling IoT for batteries at Bosch Girl Geek Dinner.   Erica Kawamoto Hsu / Girl Geek X

Dr. Yelena Gorlin: Hi, my name is Yelena Gorlin and I work in corporate research. As Tara and Seow Yuen just mentioned, we will now switch topics and I will introduce a research topic that we have here at Bosch. It focuses on batteries and specifically battery management systems.

Dr. Yelena Gorlin: Before going into the details of the topic, I wanted to take a moment and quickly introduce to you my home department in order to give you an idea what type of associates are working on the project and also what is our overarching purpose for the everyday work that we do.

Dr. Yelena Gorlin: My home department at Bosch is called energy technologies and we have three areas of research competency and they include electrochemical, modeling, characterization and controls, automatic computation and additive manufacturing. As you can imagine, the associates involved in these areas come from a diverse research background and we actually have research experience from leading academic institutions, both in the US and Germany.

Dr. Yelena Gorlin: We’re specifically strong in the areas of chemical engineering, system controls, material science, and electrochemistry. What unites us all is our interest to work on future energy technologies with the goal of reducing the global carbon footprint.

Dr. Yelena Gorlin: Recently we came up with a new motto for ourselves and it’s putting low-carbon options on the global energy menu. Our department sees the topic of battery management systems, both as a contributor to de-carbonization of our society and also as an enabler to our connected future. But you’re probably now wondering what exactly is a battery management system and how can it be so important to our future.

Dr. Yelena Gorlin: So as the name already gives it away and as I mentioned in the beginning, battery management systems have to do with batteries. Probably all of us in this room have been in a situation that seemed quite dire simply because our phone or maybe our smartwatch, our computer or our car has run out of its battery.

Dr. Yelena Gorlin: In such a situation, we were probably wishing that we could recharge our battery as quickly as possible to bring the device back to life. Well, it turns out it’s not so difficult to recharge a battery very fast once in its life. But what is difficult is to be able to offer consistent fast charging without introducing any aging effects.

Dr. Yelena Gorlin: As you probably have guessed, one of the important functions of the battery management system is to offer precisely this capability at battery management system or as we call it BMS for short controls the operation of the battery. So how fast it charges and discharges and each new generation of a battery management system looks to increase the charging speed of our device without having effect on its lifetime.

Dr. Yelena Gorlin: You can imagine that advances in this area can reduce our anxiety about how long our devices can last and as a result contribute to electrification of our society both in IoT and mobility sector and contribute to its de-carbonization. Now I hope I was able to convince you that battery management systems are very important and very significant to our future and I wanted to take a step back again and bring you to my department and our approach to this future product.

Dr. Yelena Gorlin: At its core, our approach draws on the expertise available within the department, and we rely on the different areas of background, especially in research. As I mentioned, we have chemical engineers, we have control engineers, we have material scientists and electric chemists and we primarily combine three areas and its electrochemical modeling, experimental characterization, and controls.

Dr. Yelena Gorlin: Our typical project workflow starts with the development of an electrochemical model and involves a variety of equations and parameters. We then design and execute experiments to measure these specific parameters and combine them together with a model to form what is known as parametrized model.

Dr. Yelena Gorlin: This parametrized model serves as the basis for the next generation BMS and is used to generate new control algorithms. These control algorithms are what is going to allow us to charge our devices, so our watches, our phones, our computers, and our cars at faster speeds and therefore increase our confidence in all of these IoT components and contribute to the development of our connected future.

Dr. Yelena Gorlin: Thank you very much for your attention. I will now pass the mic to LisaMarion who will tell you about artificial intelligence.

LisaMarion Garcia speaking

Software Engineer LisaMarion Garcia gives a talk on artificial intelligence at Bosch Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

LisaMarion Garcia: Hi, everyone. My name is LisaMarion I work at the Bosch Center for Artificial Intelligence here in Sunnyvale. So, we have a lot of opportunities for AI at Bosch. As my previous colleagues have mentioned, we cover a wide variety of different sectors from mobility, industrial, building, and consumer goods. Each of these individual sectors provide us different opportunities to incorporate AI, either as a feature of a product that we sell or as part of the process of producing that product.

LisaMarion Garcia: As Uma had mentioned before, that is a major goal for Bosch, to by 2025 have all our products either possess some artificial intelligence as part of their features that we provide to the consumers or as we produce them we are using AI.

LisaMarion Garcia: What we need to introduce AI into our products or our processes is–what gets discussed mostly when people are talking about artificial intelligence tends to be focused on the algorithms more. So that’s basically how you actually train a system to be able to learn by itself, how a car can drive itself, for example.

LisaMarion Garcia: We do work on that in-house as well. The Bosch Center for Artificial Intelligence has a pretty sizable research team that is currently working on state-of-the-art research topics. But additionally to actually get it from an idea, from a theoretical idea, into a product we need both compute resources, which we of course have access to, and most importantly, we need data.

LisaMarion Garcia: So, one of the advantages that being such a large company gives us, especially a company that covers so many different sectors is that we have access to a bunch of different types of data. BCAI overview, I guess. Our general mission is to help reach that goal, obviously, of introducing AI into the different areas.

LisaMarion Garcia: I’ve already covered our research team. We also have an enabling team which are–you can kind of think of them as AI evangelists. They go out to the different business units and kind of teach them about what machine learning is, how it can help in their products, what kind of data they need to be collecting if they want to be able to gain relevant insights from it.

LisaMarion Garcia: Then we have the services team which is where I work. We focus more on applied AI. So what we do is we consult with various business units within Bosch who have use cases or interested in introducing machine learning into their products or processes and we basically help them take that from an idea to a reality.

LisaMarion Garcia: We cover these four different areas. I’m going to briefly describe kind of each one. We have a bunch of different projects ongoing right now. But for an example in the manufacturing domain, something that we do is we work with optical inspection, which is where we put a camera in the production line at Bosch’s many plants and we basically collect images of the parts as they come through and try to perform or try to train a model to do automated part inspection. So basically being able to tell if a part is passing or failing by just looking at an image of it.

LisaMarion Garcia: In the engineering space, we do some work around gaining insights from data that is collected as we develop a new sensor, for example, for a new product or if we are trying to add kind of a smart home type of functionality to an existing appliance that Bosch already makes.

LisaMarion Garcia: For supply chain management and controlling we have a financial forecasting platform that basically looks at all of Bosch’s financial data and can make predictions about future sales. Then intelligence services, which I’m going to go into slightly more detail on since that is more of what I have worked on recently.

LisaMarion Garcia: So AI for mobility is obviously a hot topic. We have two main groups at Bosch that are working on that. We have for my friends that work in the autonomous driving space you may be familiar with the L3 to L5 kind of designations.

LisaMarion Garcia: So we have a driver assistance functions which are going to be your L3 and below. Those are things like automated braking when you detect a hazard on the road or lane keeping. Kind of those functionalities that already exist in your car. We also have autonomous driving group, which is the car outside, which would be the car driving itself.

LisaMarion Garcia: Some collaborations that this group has done with BCAI that I’ve been involved with have been lane keeping. So if you see the top image, we basically take a semantic segmentation map of a scene and basically use that to keep the car on the road. We also do hazard detection.

LisaMarion Garcia: So if you look at these two images in the middle, the one on the left is mostly clear windshield, the one on the right the windshield has been obscured with some droplets of water. A human looking at these two images can clearly tell that they’re the same scene. We basically our brains have a really good way of mentally deleting the information that you don’t need.

LisaMarion Garcia: It’s very difficult for a computer to do the same thing. That’s one of the main challenges when we’re training algorithms to be able to see, for example, for driving a cart. So we’ve done some work around helping either make the model itself more robust to these kinds of disturbances or basically just having some kind of a sense so that the car knows when one or more of the cameras has been had its vision obscured.

LisaMarion Garcia: Then the last topic, which I wanted to cover in slightly more detail, is the data privacy compliance topic. So I’m not sure how many of you are aware of the GDPR regulation. Yes, okay, a lot of nodding. So that’s a really important law that was passed by the EU which basically … The general gist of it is that any company that is collecting personally identifiable information from people without their consent basically needs to delete that data every six months or somehow you scrub the personally identifiable information.

LisaMarion Garcia: For our automotive topics, that mainly covers human faces and license plates. So what we did to help our business units and prevent them from throwing away their data every six months is we developed a tool using deep learning to be able to identify, locate the faces and license plates in the data that was generated by the proprietary Bosch sensors and blur those out of the image.

LisaMarion Garcia: So basically what we are doing is helping them generate training data that they can use long term and also store, which will help them basically consistently validate their work over time. So, yeah, just AI for your AI. That’s kind of the overview of what Bosch is doing in regards to AI. I have kind of mostly talked about how we spread AI internally and now I’m going to bring the user back into the conversation and pass off to my colleagues to talk about human machine collaboration. Thank you.

Shabnam Ghaffarzadegan speaking

Research Scientist Shabnam Ghaffarzadegan gives a talk on human machine collaboration research at Bosch Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Dr. Shabnam Ghaffarzadegan: Hi, my name is Shabnam. I’m a research scientist here at Bosch working in human machine interaction group and I’m very excited to be here with my colleague Panpan Xu who is our group too.

Dr. Panpan Xu: Hello everyone. I’m Panpan, I’m also working on the human machine collaboration topic at Bosch Research. So today Shabnam will first give an introduction of what are the topics we have been working on.

Dr. Shabnam Ghaffarzadegan: The topic we are really excited to work here at Bosch is human machine collaboration. If you think about everyday life there’s so many tasks that human is so good at but machine usually has so much trouble doing them. Also there are so many tasks, let’s say repetitive tasks, that machine might be so good at doing them very accurately but human would be having so much trouble to perform them in a short amount of time.

Dr. Shabnam Ghaffarzadegan: So our idea is asking human and machine to work together to empower their both abilities to make a superhuman with much more perception and knowledge and also to make a better machine to help us in our everyday life. Here at Bosch, we do focus on many core technologies such as robotic manipulation, text mining, audio analytics and visualization. We do apply these technologies to so many different use cases such as IoT industry 4.0, smart home, and smart cars.

Dr. Shabnam Ghaffarzadegan: How we do? So here first I’m going to introduce you how AI can help humans. So our goal is empowering human capabilities. What we do in our group is that we take different modalities that we see in the environment such as visual clues, text and audio and speech that we hear around ourselves and we combine this information with domain knowledge, context knowledge and user knowledge and we translate them to some specific applications such as personal assistants, conversational AI, and augmented reality.

Dr. Shabnam Ghaffarzadegan: As I mentioned, our goal is empowering human with domain specific AI. Here our focus on one of the use cases we work that I focus on personally, which is intelligent audio analytics. If you think of course the speech is one of the main … No, it’s okay. We can continue hearing that. It’s fine.

Dr. Shabnam Ghaffarzadegan: Okay, what I wanted to say was that if you think about speech, of course, it’s one of the main input and the way of communicating with outside world as a human, right, but there are so many other sounds that we can hear in the environment such as the sample of sounds you just heard. Right?

Dr. Shabnam Ghaffarzadegan: By these sounds you can guess kind of what kind of environment you were at. Were you at the beach or where you at a restaurant, right, just by listening to the noise in that environment or you can guess what kind of machine are you operating. Is that machine is working in a right mode or is it broken? Right?

Dr. Shabnam Ghaffarzadegan: So here in our group we focus on signal processing and machine learning techniques to discover three kind of sounds. The first one is environmental sounds. As you heard, is it beach, is it in the office, is it in a restaurant? The second one would be machine sounds. Right?

Dr. Shabnam Ghaffarzadegan: We hear, we listen to the different machines in the environment and we try to recognize if they’re malfunctioning or working in the right state. And finally human sound, but non-speech human sound. Imagine you might be coughing or sneezing and that might be a clue that you might have some health issues and you might want to go to a doctor. Right?

Dr. Shabnam Ghaffarzadegan: So the audio analytics field is kind of newer compared to vision or speech technology that already exists so we have so many challenges at this field and the main one would be lack of data as always existing artificial intelligence and also we need to be really robust toward the other different kind of noise and environments that we are at.

Dr. Shabnam Ghaffarzadegan: Here’s some of the use cases we work on. The first one we can focus on physical security and automation. You think that in most places the physical security systems are based on cameras but there might be so many situations cameras might fail. Let’s say, if it’s dark at night or if it’s foggy so the camera might not see what’s happening in environment. But also there are some events that camera is visual clues are not able to capture them.

Dr. Shabnam Ghaffarzadegan: Let’s say gunshot. Right? With a camera if the gunshot is not in the visual field you can’t basically [inaudible 00:54:23]. So, our idea is including microphone to a camera to understand more information about our environments. In this case, such as gunshot, glass break, and a smoke alarm can be sounds that can alarm our physical security system.

Dr. Shabnam Ghaffarzadegan: The next use case is industry 4.0. As I mentioned, we would like to put microphone in our plants and listen to the machines that working on those plants. For this, this is a very easy step to move toward industry 4.0 since the only thing we need to do is basically we put a MEMS microphone on these devices and just listen to them to see if they are operating correctly or not.

Dr. Shabnam Ghaffarzadegan: The third one would be an automotive sensing and diagnosis. Of course, autonomous cars, they are hot topics these days and they are having so many sensor already on them such as radar, camera. But we believe that autonomous cars needs to have the hearing sense as well. One of the important use case would be for example hearing emergency vehicles if there is siren happening for example police car or ambulance so these autonomous cars needs to understand these sounds and act accordingly.

Dr. Shabnam Ghaffarzadegan: Another use case can be listening to your car parts, for example, your car engine. If you go to repair shop so many of the very experienced repair shops they just listen to your engine and they would guess if you have a problem, so this is our idea to do that automatically.

Dr. Shabnam Ghaffarzadegan: Finally to give you some idea how we perform these acts. So basically we do use microphones to get this raw audio input from the environment. This information, we do some signal processing to enhance this signal to remove some environmental noise that we don’t want them and we do use domain knowledge, meaning that we do look into what kind of environment we are performing.

Dr. Shabnam Ghaffarzadegan: Are we in a factory? Are we in a house? Are we in a car? Based on that we extract some features and finally we do machine learning and AI to detect what kind of audio events was in the environment. Next my colleague, Panpan, she will explain now how human can help AI.

PanPan Xu speaking

Lead Research Scientist PanPan Xu gives a talk on human machine collaboration research at Bosch Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Dr. Panpan Xu: So, here comes the other side of story, how can human help make AI more intelligent and more reasonable to the humans. So, our approach is actually very much human in the loop method for big data analysis which we call visual analytics. Visual analytics is actually a technique which combines technologies from many different fields and one of these field is data mining.

Dr. Panpan Xu: With data mining we basically trying to gain insights from data with automatic algorithms and identify the patterns inside it. The other technique is visualization. Basically, we can draw the chart to show different trends and patterns detected by the data mining algorithms and then show or present to the users.

Dr. Panpan Xu: Most important part is user interaction. Actually, in this user centric approach we want to really take in users’ input or users’ knowledge into the data analysis process so it does not appear as a black box choose users. So, one use case that is very much related to this visual analytics topic is expandable AI.

Dr. Panpan Xu: Basically, in most of the cases we use AI as a black box. Basically the machine learning model takes the input and then produce some output to–For example, in autonomous driving we take the video input from the camera and then the steering wheel will take the corresponding directions or in medical diagnostics solutions the AI usually take an image and then tells the doctor or the patient what kind of disease it is.

Dr. Panpan Xu: But this kind of black box approach is usually not much reliable or people do not really want to use the machine learning model as a black box. So, with visual analytics we can present the explanation to the users actually and then the user can provide feedback to the model and continuously improves model until the model becomes transparent or explainable for the users.

Dr. Panpan Xu: Why this is important as I explained, we have these fairness issues because we want to know AI is making its decisions based on some meaningful features instead of other features like gender which can make this model unfair to certain populations and also we want to make this model robust.

Dr. Panpan Xu: On the other hand. There’s also this GDPR regulation which requires every decision made by AI to be explainable to the humans. So the user have the right to assess explanation to the decision made by an algorithm.

Dr. Panpan Xu: So now let’s go in on our deeper technical dive to look at a recent research paper we have published at ACM [inaudible 01:00:04] this year and which is about interpretable and steerable sequence learning. And that has application in many different AI fields like text mining or medical diagnostic sensor.

Voiceover: Recurrent neural networks have shown impressive performance in modeling sequence data. They have been successfully used in a lot of applications, sentiment analysis, machine translation, speech recognition and so on. However, they are considered as black boxes since it is very difficult to explain their predictions. Without explainability it could cause trust and ethics issues.

Voiceover: How can I trust the predictions coming out of a black box? These problems will limit the applications of these deep learning models in various decision-making scenarios. For example, a data scientist has developed a sequence prediction model to predict the risks of future problems of a car based on its historical faults.

Voiceover: However, the mechanics and repair shops may find it difficult to choose the right maintenance strategy with just prediction results. Sometimes they even suspects that the modeling is wrong. The need for explanation is pervasive in such decision-making processes. The predictive model serves as a smart analysis module rather than an automatic end-to-end solution.

Voiceover: Our idea is to explain the predictions by providing similar examples. Such case based reasoning strategy is commonly used in our daily life. For example, why classify a restaurant review, “Pizza is good but service is extremely slow” as negative? This is because it is similar to two prototypical negative sentences, good food but worse service and service is really slow.

Voiceover: We use sequence encoder R which encodes the input sequence into a fixed length embedding vector H. The model learns K prototype vectors that are most representative in the embedding space. We compute these similarities between H and the prototype vectors. The similarity scores are used as a source for prediction. To ensure that the prototypes are readable, we project the prototype vectors to their closest training samples every few epics.

Voiceover: To further improve interpretability, we’ve simplified the prototype sequences using a beam search based algorithm. To utilize expert knowledge, we design an interaction scheme which allows human users to incorporate their domain knowledge into the model. We build interpretable and steerable sequence models for vehicle fault predictions, sentiment analysis, protein classification, and heartbeat classification.

Voiceover: You can get explanations to the accurate predictions on the fly.

Dr. Panpan Xu: I would like to thank [inaudible 01:03:03] for the very nice voiceover of the video. So, if you have any questions about the paper you can search it online. So there’s the title below at the bottom of this slide. So, now let’s move on to the next topic and see how Bosch is enabling a new area of mobility with our presenter Sun-Mi here.

Sun-Mi Choi speaking

Director of Business Development & Strategy Sun-Mi Choi gives a talk on changing mobility with progressive mobility players at Bosch Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Sun-Mi Choi: Hello. Also from my side I guess I’m the last turn. I hope you guys are still with me. That was a little bit too silent. Are you still with me?

Audience: Yes.

Sun-Mi Choi: Okay, good. Thank you. I know it’s late. My name is Sun-Mi Choi. So please just call me Sunny. I’m Sunny from Sunnyvale so it’s easy to remember. I’m responsible for business development strategy within a newly established group. We are probably the youngest group within Bosch. We are eight months old so we were born beginning of this year and probably also the smallest group and we are called progressive mobility players, short PMP.

Sun-Mi Choi: I will tell a little bit more about it later but basically what we do is focus on new mobility startups because we see the mobility world is changing a lot. A lot of new players are entering the market and we are focused on two players which are new electric vehicle manufacturers and at the same time also on mobility service providers.

Sun-Mi Choi: Today we’ve heard a lot about innovative amazing technologies, learning about sensors, learning about battery management solutions, artificial intelligence, and human machine collaboration. I’ve been with Bosch seven years but I didn’t know that we had so much capability in-house. I just moved here beginning of this year so it’s amazing to see how much capabilities we have.

Sun-Mi Choi: I would like to bring in a little bit of a different perspective. Basically bringing a little bit the market perspective customer needs to explain and verify why these capabilities are so important for Bosch and also for the future of mobility.

Sun-Mi Choi: So, before I start, I would like to give a little bit of a bigger picture of why the mobility is changing and what are the driving forces behind.

Voiceover: Our world is changing and this change is visible across the globe. More than 50% of our population now lives in cities. These cities are growing, as is the share of older people in them, while space to live is becoming ever more precious. More and more goods and people need to be transported, pushing the traffic infrastructure to its limits and increasing pollution and noise levels.

Voiceover: But the world is waking up. Regulations are calling for stricter limits and cleaner solutions. A transformation has started, powered by new technologies and services. In a world where everything is connected, mobility is being re-imagined. Solutions like traffic management combined with cleaner and more efficient power trains and the benefits brought by automated driving will make our cities sustainable and livable.

Voiceover: Bosch is driving this change and shaping the future. The future of mobility.

Sun-Mi Choi: Trends they are not new for you. But it’s still very important to understand the fundamental driving forces behind it because this actually has a really big impact on Bosch. Because as we learned from Uma, the mobility part makes 60% of our revenue and all of these changes make a huge change or an impact also our business model if we want to maintain sustainable for the future.

Sun-Mi Choi: So air pollution, congestion, urbanization, and also what we see a changing consumer behavior, all of these factors are really shaping a new focus for us in the mobility area, which we call electrified, automated, connected, and also shared and personalized, which you probably experience and also live every day.

Sun-Mi Choi: At the same time, mobility is also getting more user centric. The consumer is more and more changing from owned to shared. So how many of you are using ride hailing apps to get from A to B on a regular basis? So I see not everyone, but I see a lot of hands raised. So this has become an integral part of how we move from A to B because it brings convenience, especially in congested cities.

Sun-Mi Choi: Also, consumers become more individual and personalized and more importantly, they always want to stay connected. This all relates to mobility and new players, startups see this change and these trends as basically opportunities to come into the mobility market. Because now new capabilities are required and this disrupts the whole mobility value chain also from our Bosch perspective.

Sun-Mi Choi: So what does it mean for us? We also need to understand what these new players are about to develop, what is their thinking. How do they approach innovation? That’s why as mentioned in the beginning we are focusing on new EV based customers.

Sun-Mi Choi: So probably a lot of you know Tesla in this area. So really young companies who are starting vehicles from scratch or the second customer segment is mobility service based customers. So, all companies who provide mobility as a service, the ride hailing apps, car sharing and so on.

Sun-Mi Choi: What we see is that they have quite of a different DNA, they have different requirements. That means also for Bosch, we need to understand the requirements and adjust also the way how we approach customers. Because these young customers, they act differently, they drive innovation differently than the VW or Mercedes driver that we’ve been dealing with for the past hundred years.

Sun-Mi Choi: So it’s time to change and it has also a big transformational impact on us. So, we see in the shared space, for example, the one customer segment we are focusing on is huge change. If you look at an annual number of ride hailing rides you see a tremendous growth over the past four years. It’s been grown more than 60%.

Sun-Mi Choi: From a user perspective, you also see a good reason why they are switching from ownership to shared. One of the reasons is because 96% of the time your asset stands idle. The car is parked, you’re at work, it stands idle for eight, nine, 10 hours while you sleep also. This this is a waste of assets.

Sun-Mi Choi: So people are looking for alternative modes to move, alternative modes how to utilize their assets in a most, more efficient way. So also this is one indication for why people are moving towards shared. Last but not least, from an investor perspective, if you look at how much investments have flown into this area over the past four years only more than 80 billion US dollar have been invested into the ride hailing market.

Sun-Mi Choi: This is humongous. This is likely to grow further. So, this shared mobility will happen. So how do these new customers take, what are the pain points, what are the requirements? These are just some of the requirements or pain points that we identify when speaking to the customer. So operational costs for these ride hailing companies is a sure thing.

Sun-Mi Choi: How can we become profitable? How can I optimize my operations? Second point is how can I ensure safety and security for their passengers, especially when we go towards robo taxis, it will not have a driver anymore being able to control the ride. So we need technology to basically operate and also ensure the safety even without a driver.

Sun-Mi Choi: Third is there are so many players arising, I need to differentiate. If I want to survive in this market I need to have a good differentiation point. So personalization, how to ensure that your ride is individual and a really great experience is one important differentiator that we have identified.

Sun-Mi Choi: For all these pain points, for all these requirements that we see, it kind of makes sense where you bring now the puzzle pieces together of the capabilities that we’ve seen from sensors which connect the cars, can connect the car and the user and a lot of other use cases that we’ve learned today.

Sun-Mi Choi: Battery management solutions is super important because we see a strong push towards electrification pushed by the government. Also end users are looking for environment friendly solutions. Also a lot of these ride hailing companies tend to establish their own EV fleets.

Sun-Mi Choi: So range anxiety and also improving the battery lifetime what we learned today are super, super crucial for the customers in the market. Autonomous driving was something that was mentioned. So a lot of these companies are also going towards robo taxis. So artificial intelligence is also human machine collaboration to really ensure that there is a safe and also unique experience between the human and the machine will be very relevant.

Sun-Mi Choi: When we look at the customer and the market and the customers, we see that these capabilities will be important for the future to come. So I’m very proud to see that we are working on these very future-oriented topics. This is the way how we would like to tackle the new era of mobility.

Sun-Mi Choi: So basically in summary, with these capabilities enable the vision of our mobility customers not only the new ones, of course, also the existing customer base. Second, we want to innovate and co-create with these customers together. Because even though we have the best technology that might be requirements that we may not have seen so we need the customer input to even more improve the technology and also the use case.

Sun-Mi Choi: Last but not least, important point is really to understand and translate what the customers tells it to us into technology. That’s why it’s a good collaboration to have technology and also sales and the market proximity close to each other so that there is always an inter-linkage and a bridge between technology and also market need.

Sun-Mi Choi: So, we’ve talked a lot about AI, about new customers, about innovation, but I think it’s also important to really close with the core, with the tradition to not forget about the core business and also the roots where this company is found on. So two values from Robert Bosch, the founder, since 1886, have been that he says, “I have always acted according to the principle that I would rather lose money than trust.”

Sun-Mi Choi: So the trust to the customers, to the market, providing safety is one really crucial element. Second point for doing business also with our customers is integrity. Integrity of the promises we make to our customers in regards to quality and also in terms of the promises that we make to them. This to the founder and the values still hold today our prioritizing this versus just having a short-term transitory profit.

Sun-Mi Choi: So I would like to remind us all of us when we speak about future topics to think about the core values as well because these are important. This is how I would like to close the presentation. Thank you very much for the one hour attention. So you have been an amazing crowd.

Sun-Mi Choi: I went a little bit over time, so thanks a lot for your patience. I think we had great presentations here today. I would like to thank all of you on behalf of the whole team for coming to our Sunnyvale site, for showing interest in our portfolio, in our technologies. And we would be happy to see you again, also to mingle and network after and to see if we have some collaboration opportunities.

Sun-Mi Choi: Last but not least, of course, I would like to thank all the staff, the presenters, and all the people who have helped to support making this event happen. It was a lot of work. So let’s have a nice evening and please don’t leave too quickly. Thank you very much.

Uma Krishnamoorthy, Hauke Schmidt

Like a Bosch: Tara Dowlat, Seow Yuen Yee, Yelena Gorlin, Panpan Xu, LisaMarion Garcia, Shabnam Ghaffarzadegan, Sun-Mi Choi, Uma Krishnamoorthy and Hauke Schmidt.  Erica Kawamoto Hsu / Girl Geek X

Our mission-aligned Girl Geek X partners are hiring!