Best of 2019 – Elevate Videos

By Angie Chang (Girl Geek X Founder)

Elevate showcased 22 amazing speakers and 7 mission-aligned sponsors at our virtual conference in celebration of International Women’s Day for the past two years. We received rave reviews for the content and accessibility of the online program, and are looking forward to another in 2020!

GIRL GEEK ELEVATE TALKS IN 2019 – TOP RATED VIDEOS

Here are the most popular talks from past Elevate virtual conferences based on attendee ratings of the sessions:

#1 – Always Ask For More (video)Leyla Seka (Salesforce Executive Vice President), Jen Taylor (Cloudflare Head of Product Management)

#2 – Being Unapologetically You (video) Sandra Lopez (Intel Sports Vice President)

#3 – From Office Manager to Chief Product Officer (video)Shawna Wolverton (Zendesk Senior Vice President of Product Management)

#4 – Building High Performance Teams (video)Nupur Srivastava (Grand Rounds Vice President of Product Management), Citlalli Solano (Palo Alto Networks Director of Engineering, Colleen Bashar (Guidewire Software Vice President), Gretchen DeKnikker (Girl Geek X Chief Operating Officer)

#5 – CTO’s Lessons Learned from Software Developer to IPO (video) Cathy Polinsky (Stitch Fix Chief Technology Officer)

#6 – It’s Not Them, It’s You: Self-Awareness & Ego (video) Minji Wong (At Her Best Founder)

#7 – Creating An AI For Social Good Program (video) Anna Bethke (Intel Head of AI for Good)

#8 – Engineering Leadership: From Cat Herder to Air Traffic Controller (video) Laura Thomson (Mozilla Director of Engineering, Rija Javed (MarketInvoice Chief Technology Officer), Miriam Aguirre (Skillz Vice President of Engineering, Vidya Setlur (Tableau Software NLP Manager)

#9 – Using Statistics for Security: Threat Detection at Netflix (video) Nicole Grinstead (Netflix Senior Software Security Engineer)

#10 – The Art of the Interview: How Would Your Candidates Rate You? (video) Aline Lerner (Interviewing.io Founder)

ELEVATE SPEAKERS AND SPONSORS WANTED

We invite the Girl Geek X coommunity from around the world to participate in Elevate to share the latest in tech and leadership with fellow mid-and-senior level professional women.

Sessions may reflect the theme of this year’s conference – “Lift As You Climb” – and content typically covers the following topics:

  • Lightning Tech Talks – Dive deep into an area that’s unique/critical to your business or role (i.e. machine learning, security, usability, UX/UI, ethics in building product, data analysis, etc.)
  • Technical Skills & Tactics – Tutorials, walkthroughs, or deep dives into a skillset or tactical approach to how you solved a real-world challenge.
  • Learning and Development – Topics include negotiation, job search, interviewing tips, being a better leader, self-awareness, career growth, management, etc.
  • Inclusion, Equality, and Allyship – Topics include being a better ally, lifting other women up, and actionable advice for individual contributors or managers.
  • Interesting Life/Career Journeys/Distance-Traveled Stories – Did you overcome socioeconomic challenges (i.e. first in family to go to college, raised in poverty/rural area/etc.) while giving back or contributing to the greater good?
  • Work on a unique technical project or have interesting insights you’d love to share with other other women & allies? We want to hear from you!

Tip: The best proposals include 3-5 key takeaways — what attendees will learn from your talk!

Submit your proposal for a talk and/or panel here by December 24, 2019 11:59PM PDT for Girl Geek Elevate virtual conference.

For conference sponsorship inquiries, please contact sponsors@girlgeek.io


MORE GIRL GEEK DINNERS IN 2020

We would love to have more Girl Geek Dinners at med/health companies, biotech companies, consumer-facing companies… We are interested in partner more with the scientific and ethical-minded companies out there in addition to our slate of tech companies hosting Girl Geek Dinners.

Here’s how to partner with Girl Geek X in 2020. We are currently working with sponsors for 2020 dinner dates, and excited to continue partnering with companies to host Girl Geek Dinners!

For dinner sponsorship inquiries, please contact sponsors@girlgeek.io

“X” IS FOR PODCASTS AND MORE

Girl Geek Dinners, Girl Geek Elevate, Girl Geek Podcasts, and much more!

Here are the best 10 Girl Geek Dinner videos of 2019.

And the most-downloaded Girl Geek Podcast episodes in 2019.

We’ll be releasing the “best of 2019” lists for more content soon, stay tuned!

Girl Geek X + Indeed Lightning Talks and Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang, Allison Dingler

Girl Geek X founder Angie Chang and Indeed Global Diversity & Inclusion Program Manager Allison Dingler from Austin, Texas kick off an Indeed Girl Geek Dinner in San Francisco, California.  Erica Kawamoto Hsu / Girl Geek X

Transcript of Indeed Girl Geek Dinner – Lightning Talks & Panel:

Angie Chang: Hello, and thank you all for coming out to Indeed Girl Geek dinner tonight. My name is Angie Chang and I’m the founder of Girl Geek X. We’ve been doing these Girl Geek Dinners at companies in the San Francisco Bay area now for a very long time, but I’m really glad that you’re here tonight for Indeed’s second Girl Geek Dinner.

Angie Chang: I’m really excited for all the talks that we’re going to hear tonight and please enjoy yourselves and meet someone new. At least one or two, maybe even three new people, get their LinkedIn, exchange LinkedIn information and maybe grab coffee later, ask about jobs, ask about jobs here. There’s always opportunities to level up and that’s why we keep doing this is because we like learning and hearing from other women in tech and other industries about things they learned on the way, and also what are the cool things they’re doing.

Angie Chang: So, please feel free to network afterwards. We also have things like a Girl Geek Podcast, in case you would like to find us on iTunes and all of the different podcasting services, we have a podcast. We also have a conference coming up. It’s a virtual conference we do every year for International Women’s Day. That’s going to be in March, so stay tuned. But I want to turn the microphone over to Alison, but say thank you so much for hosting us, Indeed.

Allison Dingler: Thank you. Thanks, Angie. All right, awesome. Thanks so much, everybody, for coming. This room is so packed, I love it. Yes, everybody excited? Yes. Ooh, energy. I’m about it. I’ve had a lot of caffeine today. I’m going to kick it off and start with our first tech talk of the evening.

Lindsay Brothers

Product Manager Lindsay Brothers gives a talk on “A/B Testing Pitfalls and Lessons Learned” at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Lindsay Brothers: Hello. Beautiful. Hi, how are you doing? Good. It’s Tuesday. Okay, so I’m going to be talking about “A/B Testing Pitfalls and Lessons Learned”. Experimentation. It’s how we learn about the world around us. It’s something we started doing very early on as humans. It’s something we start as babies. How do we learn as babies? Well, we run the original A/B test, which is stick things into our mouths. And we learn, maybe we have a question, a hypothesis, is this edible? I can eat this.

Lindsay Brothers: Unfortunately, sometimes not so successful. It turns out to be dirt, and that test was not a success. Some tests fail. And other times when babies run this experiment, this A/B test, congratulations they’re tacos. Yay, it was a successful test. And we get to celebrate, we’ve learned something new and we ate tacos.

Lindsay Brothers: I’m Lindsay Brothers, I’m a product manager at Indeed. You can follow me on Twitter @LindsayBro. You may know Indeed, number one job search site worldwide. Who here has heard of Indeed? Everyone raise your hands. And who’s gotten a job through Indeed. Yes. That’s awesome. I love it. Yeah.

Lindsay Brothers: Okay, a little bit about Indeed so you have context for this talk. We help people get jobs. This is our mission. This is something we cared deeply about. And some context about just how big we are. We have 250 million unique visitors, 150 million resumes, 600 million salaries, and 25 million jobs. So, a lot of job seekers looking at a lot of jobs, millions of job seekers, millions of jobs.

Lindsay Brothers: We’re a very data driven company. We want to learn about job seekers and how do we do this? Well, A/B testing. What is A/B testing? A/B testing is a randomized experiment in which a new variant is tested against a control A to measure how they perform relative to each other. I just realized I don’t have a timer. Cool. So at any given time, we’re running hundreds of A/B tests to learn about job seekers, which leads to thousands of experiences. We’re running hundreds of A/B tests, which leads to thousands of experiences at any time.

Lindsay Brothers: Which means the person sitting next to you likely sees a very different Indeed than you do. This leads to many different lessons learned. We’re constantly running A/B tests and we’re constantly learning. Now this test, this talk is really about things that have gone not so well, pitfalls. We run a lot of A/B tests and we’ve made some very expensive, painful mistakes that I want to share with you.

Lindsay Brothers: The first pitfall, your metrics don’t matter. Second pitfall, big test, big failure. Pitfall three, most tests fail. And pitfall four, where does vision fit in? As we go through these pitfalls, I’ll share lessons learned. Let’s dive into the first pitfall.

Lindsay Brothers: Your metrics don’t matter. Or in this case my metrics didn’t matter and no one cared. This is a job alert, and you’ll hear more about Job Alerts in the next talk, as well. But this is the most common email we send out. It’s new jobs based on a query that a user has run. In this case, it’s a email developer jobs in San Francisco. Done a search on Indeed, signed up for this Job Alert, and they’re getting new jobs in their inbox, so they can apply to them as soon as they’re posted.

Lindsay Brothers: I was on this team as a product manager on this team and I was highly motivated to get more subscriptions, to get more people to sign up for these Job Alerts. This is an Indeed apply form. If a job is posted on Indeed, you go to apply for that job, you’ll see this form and you’ll apply directly on Indeed. What I wanted to test was adding a simple checkbox. Notify me when similar jobs are available. A job seeker will just check this check box, and the next day begin to receive new jobs in their email inbox. Pretty simple, right?

Lindsay Brothers: A versus B, no checkbox, that’s standard form, versus the test group with that checkbox. Now, it looks simple. It’s just a small change. But this test had a tremendous amount of data. Sure, it’s just one variable in a single test group, but look at all the things that are on this form. You can be logged in, you can be logged out, you can use your Indeed resume, you can attach your resume, you can attach a cover letter, you can add a phone number. So there’s all these different things you could do, just in a single form. So we had a lot of data to analyze.

Lindsay Brothers: And the first metric was really bad. We saw 0.1% decrease in applies. So just adding that single checkbox, less job seekers were going to fill out that form and apply to that job. And this was really scary. Not good. So there’s another team that exists, indeed.com is not just the Job Alerts team. There’s a team called the Indeed Apply team and they own this form, and their metric for success was the completion rate of this form. They wanted job seekers to apply to jobs on Indeed and to finish this form.

Lindsay Brothers: So, 0.1% may not sound like a lot, but we’re talking about millions of job seekers, millions of jobs. That actually can equal hundreds of thousands of applies lost by adding a single checkbox. So this team says to me, “What are you doing? Okay, we’re here to help people get jobs, not email subscriptions.” Yeah, I got some angry emails about that. This was our secondary metric. This was a metric I was keeping an eye on during the analysis of the tests, but it was their primary metric. It was what they cared most about, and they didn’t really care that I was getting more email subscriptions. I was getting a lot. Come on, I can put this on my eval. 50% increase in subscriptions. That looks really good, but they don’t care.

Lindsay Brothers: It felt like a massive win but we were losing applies. So, the question was, are we helping people get jobs or are we just helping them get email subscriptions? So, what is the impact of this test? Now keep in mind, they click that checkbox, they’re getting that job alert, they’re getting that new subscription, and there’s a lot of jobs in here. So, they got to be doing something with those jobs.

Lindsay Brothers: Well, let’s start looking at the standard email engagement metrics. They’re getting this new subscription, how are they engaging with it? Well, okay, they’re less likely to open it. That’s awesome. So, job seekers were checking this check box, maybe less likely to apply and less likely to open this email. And they were less likely to click through by 12%, so open rate was 12% lower, click through rate was 12% lower. Am I tricking people into this email? Is this spam? What am I doing?

Lindsay Brothers: But we had to look further down the funnel and what we found was the apply rate was actually higher. So, job seekers were less likely to open it, they’re less likely to click through, but they were more likely to apply to a job in that email by 0.25%, and this was very, very exciting. This meant that we were actually helping people get jobs. But we had to figure out the total impact. So had to do some math and we had to figure out the total downstream applies, more subscriptions and looking at that higher apply rate, how many additional applies were we getting? Was it making up for those lost applies? And it was, we were getting millions of additional applies from that email subscription. So, it was a success. Yeah, it looked real good on that eval.

Lindsay Brothers: Something to keep in mind is that looking at only short term metrics, that makes [inaudible]. Looking at only short term metrics can mean missing downstream impact. And they really didn’t care about getting additional email subscriptions, which of course they shouldn’t. But we were really aligned on the impact of applies and the power of applies. So, the first lesson I’d like to share is that downstream analysis is a powerful tool, but it does take time. It took time to figure out that we were having a higher apply rate in that email and we were getting those downstream applies. Cool.

Lindsay Brothers: So, second pitfall, big test, big failure. In 2017 this is what indeed.com looked like. And we were due for a facelift. The UX design team really wanted to update this and improve it. This was the vision. This is where we wanted to go. We wanted to modernize Indeed. Again, this is 2017.

Lindsay Brothers: Generally, when you’re doing an A/B test, you have a change and you have a result. You’re changing something on the product, a small variable change, then you see results. Well, okay, now keep in mind this is the entire search results page of Indeed. There’s a lot of things going on, and it was a really, really massive test. We were looking at, we were testing this old version versus this new version, and lots and lots of changes. Okay, lots and lots of changes going on on the site and then more changes we’re starting to see lots of metrics, and then there was lots of results, and more results, and more results and we couldn’t quite figure out what was going on and what changes were causing what metric going up or down. And it got really messy. It looked like that.

Lindsay Brothers: We had changed too many things at once. We wanted to run this massive A/B test where we were updating the old Indeed with this new, beautiful, massive, redesigned, gorgeous. Let’s skip ahead. Let’s go with this big vision. But we had changed too many things at once. And that meant our A/B tests were losing, metrics were going down, but we didn’t know why. And this is really expensive. Now, keep in mind, this is redoing the search on Indeed, that involves a lot of engineering effort.

Lindsay Brothers: We had to go back to the drawing board. 2017, we wanted to test this brand new, beautiful redesign, let’s modernize Indeed. So let’s start out where we started. We had understood when we were doing this test that too many changes at once meant we didn’t really understand what was happening. So we had to start from scratch. We had to really just redo this whole thing. What we had to do was test a single element at a time.

Lindsay Brothers: Now, this is a job card, so you do a search on Indeed, you’re going to see these job cards, which is job title, company location, maybe salary, some additional details. And this is an example of how we could test a single element at a time. You’re like, what’s changing here? It’s the spacing. So this is spacing as a single element. So a single variable in A/B test.

Lindsay Brothers: Another thing we had to do, like I had mentioned, we have this old older design that we wanted to update and originally it was just, let’s test all these elements at once. Something we also had to do was we had to switch to multi-variate tests. And so I say multivariate test, what do I mean? An A/B test in which all possible combinations of variations are tested at the same time.

Lindsay Brothers: Now let’s go back to that job card. We wanted to test a single element at a time. We want to break up all those elements so we can understand their impact. But we also want to A/B… the multivariate testing so we can go through all these different combinations. Now this is a job card. We got job title, company, location. There’s all these different elements to test. Well, let’s dive deep into a single element, which is salary. Let’s look at salary. Salary important. We like to make money at our job.

Lindsay Brothers: This is a single test where we’re just testing the element of salary, but what we’re going to do is a multivariate test to really dive deep into the UI of it. This is control, font size, 13.33 pixels. It’s not bolded and the color is gray. So, one element of the multivariate test is the font size variance, 12 pixels, 13.33, 14, and 16 pixels. Another part of the multivariate test is font weight, regular versus bolded, and finally color: gray, black, orange, and green.

Lindsay Brothers: That’s a lot. So we got four sizes, two weights, four colors. This is 32 groups. And were there spreadsheets? You know there were spreadsheets. I love spreadsheets. So, you’re like, whoa, that’s a lot going on there. Now, if we had not done a multivariate test, if we’ve just done A/B tests, so color as a single A/B test, font weight as a single A/B test, size as single A/B test, it would… like, here’s the control. Okay, that’s control. These would just be the groups.

Lindsay Brothers: So colors, one group, size, one group, you would only have eight different groups. But when you do multivariate testing, you get, you miss 24. So, 32 groups, well that’s a lot. But you get to explore how these elements play against each other. And we would have lost our winners. So if we had only done tests around those single UI elements one at a time, we would have totally missed these. And we would have not picked these. No UX designer would have picked these, because this one looks like Hulk. We call this one Hulk. It’s big. It’s green. It’s bold. No one was going to go with that. But we learned from this and we learned a lot.

Lindsay Brothers: Now, multivariate tests, of course, have some challenges. Well, you need sufficient traffic. I mentioned 32 different groups. Okay, you need enough traffic to learn anything from those groups. Also, significantly more complex analysis. So there’s a really good blog post. Robyn Rap is a data scientist on the Indeed engineering blog. You can look it up. And she actually talks about this specific test, and there’s literally equations in the blog posts about how to analyze it. It’s a lot of math. There’s some really strange combinations that you can get from multivariate tests, like the Hulk. No one was expecting that. We’re like, “All right, what do we do with that?” Okay.

Lindsay Brothers: But it really helps you optimize your UI at a very high velocity, which is really, really cool. We’re learning fast, we’re moving fast. And since August of 2018, we’ve run over 50 tests with over 500 groups, just on that search results page alone, and we’ve learned some really surprising things. Now remember when we started this test, it was old design, new design. All these changes and we couldn’t see what was going on. Metrics going up, metrics going down, not understanding the impact. But we saw really surprising things when we broke it up via by element, and then broke it up into multivariate tests so we could really learn quickly.

Lindsay Brothers: And something we totally missed was that changes were way more impactful on mobile. This data point lost, totally lost in that confusing analysis where everything was going up and down. But we had missed this, so when we broke up the elements, we learned more about the UI changes and their impact.

Lindsay Brothers: So, the lesson here, again, this was a very, very, very expensive mistake to make. We thought we could just skip ahead, come up with the new design. Let’s go there. Oh, we’re so modern. Wrong. Very, very expensive. So when we changed to testing small changes and testing them quickly, we learned a lot more, faster. Which goes directly into the next pitfall, which is most tests fail. And at Indeed, 70% of A/B tests fail. That means there’s not a clear winner. We know we put out this test group, it’s not amazing. It’s not going up. Whatever your metric is, it looks bad.

Lindsay Brothers: It’s really sad. It’s like, what’s the point of this? Why am I here? Why am I running all these tests if they all fail? Why did I make my engineers build this or build this test if it just… It’s just sad. Want to hide under a blanket. There are three different reasons, really, to run A/B tests. We can get faster wins, we can get a better design and we can get a better understanding of the product. So how exactly do we get to these things?

Lindsay Brothers: Well, let’s say we run a test and the test is positive. Your metric goes up. Well, awesome, you have a KPI win. Woo hoo. That’s great when that happens. Or okay, let’s say the test is neutral. It did not change. And this was actually quite common in that test I just discussed, where we were testing all these around elements, all many, many multivariate tests, spreadsheets galore, lots of neutral stuff. That actually means there’s a lot more design flexibility. So if it’s neutral, it means you can play with that element. You can maybe make it a little more prominent or you can do things with it. It’s not a fixed element. It’s more, you can be creative with it. And so that’s actually really exciting.

Lindsay Brothers: And, of course, if something’s negative, what we found is, okay, don’t touch it. Do not touch that element. Like I said, we broke up all these different elements. We’re testing many different variations of these elements, and some things came back consistently negative, and we’re like job seekers like that, we’re not going to do anything with it. And so you’re still winning. Even if a test is neutral, you learn that you can do things with that element. If it’s negative, don’t touch that element. Move on, try something new.

Lindsay Brothers: So, it is successful. Sometimes it’s disappointing when it doesn’t go the way you planned or your hypothesis is totally off, but we do get wins from this. And the lesson here is to change what winning means. So, even when we’re running all these different A/B tests, the metrics aren’t looking great. You’re still learning about your users, are still learning about, for us we’re learning about job seekers and how they interact with Indeed and how they look for jobs.

Lindsay Brothers: So that goes directly into pitfall four, where does vision fit into all of this? We have this big vision. Okay, so let’s go back to that test. This is where we were. So this is again that job card, you do a search on Indeed. You see all these jobs. This was the first iteration after testing. There’s a lot of stuff going on there. And this was the vision. We like to empower design at Indeed. We love to have visions. We love to think about where are we going with things. But when you’re doing a lot of A/B testing and, where does this all fit in? I’m running all these different, I’m testing all these elements. I’m trying to learn. I have these hypotheses. But where does the vision fit into this?

Lindsay Brothers: Well, even failures can inform design vision. Let’s go back to… Now which one is this? Okay, so this is the vision. This is the vision we originally started with, 2017. This is our vision. This is where we wanted to go in 2017. And we started to remember that first big test, not so good. Then we started to break up elements, did multivariate testing, lots of spreadsheets, and we started learning. We got lots of negative and neutral and positive, and we learned things like no blue or underlined needed on the job title. Doesn’t matter if you have that, job seekers are still going to click on the job.

Lindsay Brothers: We can add more spacing. Like I mentioned, spacing was a single A/B test where we’re looking at that element and playing with spacing. We can add more white space. We can do more there. Salary needs to be more prominent. Like I talked about that salary test, we could make it big and green and bold and people love it, they’re clicking through. Do not touch this. If you can apply on Indeed, that Indeed apply little tag, if we changed it, negative. Any change, negative. Do not touch it, keep it there. Font size. We played with font size, of course, the font size was way too small.

Lindsay Brothers: So again, this is where we started after that first iteration, that first iteration of testing, this was our vision, but then we ran all these tests, testing all these different elements, many variations, and we ended up here. Now, you can see salary’s a lot more prominent. There’s more spacing. We don’t need that underline. We played with underline, you don’t need that. So, as we learned about these different elements, tests were negative, tests were positive, tests were a lot were neutral, we were able to take these learnings and put it back into the vision. So we had a vision that more aligned with how job seekers were using our site.

Lindsay Brothers: Sometimes it can feel like a vision is just this beautiful thing you create and it doesn’t always align with how people use your products. And with A/B testing and with learning from many, in fact, failed tests, we were able to take that back and have a vision that aligned with how job seekers use Indeed. So it’s not just a design win, it’s also a business win. We had a lot of KPI wins, many tests not so great, but we did have ones that were delivering KPI wins and we’re able to implement that back into the vision.

Lindsay Brothers: So the lesson here is both, of course, the design vision guides your testing. We had UX designers who were thinking about where Indeed should be, where we could go, and that led us to define some of these tests. But also those A/B tests, as we’re running them, went back into the vision. So, to recap, your metrics don’t matter. Remember we ran this amazing checkbox test and it looked horrible. At first, applies were going down on that page. So, we had to align our metrics, we had to agree on those downstream applies. And downstream impact can take longer. It can take much longer to pan out, but it can prove really valuable.

Lindsay Brothers: Next up, big tests, big failures. That’s when we ran this massive redesign of the Indeed search, and we thought we could just skip ahead from this old design to this new beautiful design. We were wrong. That was big mistake. So, breaking up tests really helps understand impact. So breaking up these tests into different elements, multivariate tests, those multivariate tests really help us understand the different UI elements. And most tests fail. So indeed 70% of tests fail and that meant that we had to redefine winning.

Lindsay Brothers: So, failed test just means there’s lack of flexibility. That element is important to users. They care about it, maybe don’t change it. And a neutral test means that there is flexibility. So, if something’s neutral, maybe we can do more with it, maybe we can play with it. And then finally, where does vision fit in? So, that big redesign test, we had a vision of where we wanted to go. We’d tried testing it all by itself, didn’t work, but we were able to use it to guide those single element tests, the multivariate test, and it helped us define a test plan. But also as we ran those tests, it helped us inform the vision and we adapted our vision to what made more sense for our users and our job seekers.

Lindsay Brothers: So my question for you is, where will your testing failures take you? If you run A/B tests, if you run tests with your users, they will fail and things will go wrong, and you will run tests and will feel like an engineering waste of time. Oh boy. But there’s still learnings to be had, so where will those failures take you? Thank you so much for your time. Please clap. Here’s my information, so feel free to shoot me an email, LindsayB@indeed.com or Twitter, @LindsayBro. If you tweet, I get metrics. Awesome. Thank you so much. So Allie is going to give a little intro.

Allison Dingler: Thanks, Lindsey. Everyone give it up one more time for Lindsay, A/B testing. Yes, I’m here for it. Awesome. Everyone having a good time so far. Getting some good food in your belly, some good drinks, some good friends, yes? Awesome. So I’m going to kick it off. We have our next tech talk that’s about to get started. We have Janie and Rohan here, so I’ll have y’all come on up and get you going. Do you want another microphone?

Janie Clarke: Yeah.

Allison Dingler: Microphone.

Janie Clarke: Thank you.

Rohan Kapoor: [inaudible].

Janie Clarke: Can you get a timer?

Allison Dingler: Timer.

Janie Clarke speaking

Senior Product Manager Janie Clarke gives a talk on “AMP for Email” at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Janie Clarke: Hi everyone. My name is Janie Clark and I’m a product manager here at Indeed. I’m here today with Rohan Kapoor who’s a software engineer on my team, and we’re here to talk about our adventures with AMP for Email. Did you click? All right, there’s a little bit of content here that might be repeated from Lindsay’s talk. As you probably just learned, we are very data driven company here. We test a lot of things. And Rohan and I work on Job Alerts, which Lindsay also used to work on.

Janie Clarke: It’s an important product for Indeed. I assure you it’s not the only product, even though we’re talking about a lot tonight. Job Alerts is a big product. We have over 250 million active Job Alerts subscriptions. We send it to over 60 countries in 28 different languages. And we do a lot of A/B testing within this email. At any given time we have dozens of A/B tests running.

Janie Clarke: You’ve also seen this before. This is a Job Alert email. Basically it’s a way for job seekers to get the newest jobs emailed to them. We’re here today to talk about our adventures in AMP. So first, I’ll give a quick background about traditional email development and some of the challenges that we had faced with it, and we were hoping AMP would solve these challenges for us. And then Rohan will go into some technical considerations and obstacles that we faced and how we solved them. And finally, we’ll share some of the results of our testing so far.

Janie Clarke: So first up, traditional email development, or why email development sucks. Do we have any email developers in the room or anyone who has worked on it before? Woo! Email geeks. So you may have heard, email development can be difficult. Because emails are sent out as static HTML content, it has to be able to support whatever old arcane email client that your users may be using to read their email. Outlook 2007, being a good example of this.

Janie Clarke: Many different email clients have their own special rules about what kind of content and markup they require and can render. And if your inspect the source of an email, that in your inbox, you may notice that it’s a mess. Lots of nested tables, inline CSS with special rules for different Outlook versions. Again, some of you may have noticed this before. But Outlook aside, even modern email clients like Gmail have issues of their own.

Janie Clarke: So remember, at Indeed our mission is to help people get jobs, and on our team we do that by sending Job Alert emails to millions of job seekers every day, so that they can get the latest results for their search. We know from extensive experimentation on the site, as you know, and also within our email, that job seekers get a lot of value from personalized contextual information about the jobs that they’re looking at. Providing them with this information helps them make better decisions about which job they want to click on. It helps them apply to the right jobs and it helps people get jobs.

Janie Clarke: But whenever we do a test in the Job Alert email to add this information, we run into a problem. And this happens. Have you seen this before in Gmail? The email clips, since traditional email cannot load any external content, no JavaScript, no external CSS files, all the content has to be static and contained within the HTML. And along with all those nested tables that you need for Outlook and custom inline CSS, for the older email clients, it can be a real challenge to squeeze in all of the information that you want.

Janie Clarke: So it’s something we’re constantly trying to find the right balance on our team. I’m including as much information and as many jobs in the email as we can, while also not running into clipping. So there’s a certain size limit that Gmail hits, when the email will clip if the HTML is over that limit. So we’ve tested adding more jobs to the Job Alert, more content always leads to more clicks, more engagement, more applies. But the more jobs we add, the more it clips. Right now, about 5% of our emails clip.

Janie Clarke: Another problem we’ve run into with Job Alerts is how to include the most Up-to-date information. I’m going to go into a little bit about sponsored jobs. A sponsored job is one where the employer is paying to promote that job and get it in front of job seekers. When we’re sending Job Alerts, we send out both sponsored and organic jobs. We will not send the job alert if there are no new organic jobs, but we do allow the sponsor jobs to be a little bit older.

Janie Clarke: And the thinking there is that, if the employer is paying to get this job promoted to more job seekers, they’re actively trying to fill that position. So, we want to help them do that by including the older job. But it’s also a challenge because job seekers can open their email hours or even days after we send it. And if a job is older, it’s more likely to be closed by the time they see it. If you look closely at this screenshot, you might notice a difference between the job at the top, which is sponsored, and the organic job below it.

Janie Clarke: In our Job Alerts, we worked around this problem in a pretty clever way. When the user opens their Job Alert, a request will be sent to our server to fetch the latest sponsor job for that slot and that alert, and a screenshot would be taken and we would render it into an image. So we call this image ads. Image ads were really clever workaround for the problem of how to show the latest sponsored jobs, but they came with their very own problems. So, every email renders HTML a little bit differently. Many of them render poorly, and different clients have handled different images in very strange ways.

Janie Clarke: So some problems that we’ve faced with this are giant sponsor jobs or tiny sponsor jobs, grainy images, you name it, we’ve run into it and fixed it. So getting image ads right was something we had struggled with on our team for a long time. And because of this, when Google first announced AMP for Email, using it to replace image ads was the first thought that we had. We are also very excited about how AMP for Email wouldn’t necessarily need to include all of the older markup that’s required by older clients because it’s only supported by a certain newer clients. Now I’ll give a little bit of background about AMP.

Janie Clarke: AMP is a web component framework that is used to help create interactive websites, stories, emails and ads. AMP is designed to create a user first experience, which they define as being mobile first and loading fast. And it’s especially helpful for users on poor quality connections because they try to load the most important content first. AMP also does not allow for content to change positions once it’s loaded. Meaning that all the page elements have to have a fixed width and height.

Janie Clarke: The reason for this is they want to avoid the page jumping around as it loads, which you’ve probably noticed a lot on the internet is the thing that happens, especially on slower connections. And AMP achieves these goals by providing a set of predefined components that you can use to build web pages. AMP for Email is a way to offer email users an interactive experience within email, by allowing certain AMP components to be used in email. It brings a lot of modern app functionality directly into emails that has been impossible before.

Janie Clarke: If you use Google Docs, you may have seen this email, which allows you to reply to a comment directly from your inbox without having to leave. It’s pretty amazing. I use it every day. It’s a killer use of the AMP functionality. Some of the benefits of AMP for Email are, it can dynamically load content from a remote server, not just images, but also text. Users can also submit forms and information to a remote server. So users can submit feedback and content to you. And lastly, the layout can change as the user interacts with it.

Janie Clarke: So in some ways this allows email developers to build an entire web application inside of an email. AMP for Email allows for a level of customization that has literally never been possible before in email. It’s very pretty exciting if you’re an email person like me. So back to our use case. Google opened up the AMP for Email preview in April of 2018, and we jumped on the opportunity to participate in the developer preview. The timing was really good because we had an intern starting for the summer in May, and at the time Google expected their launch to be around September. So it would be perfect for an intern project. He’d be able to see his functionality go live, sound really great. So we assigned him the project of creating an AMP version of the Job Alert that loads the sponsor jobs using AMP.

Janie Clarke: Now the reason we assigned it to an intern, it was hard to justify putting full time resources from the team onto this, due to the large time frame before launch and the general unpredictability. And this has been consistently a challenge when working with AMP, which we’ll go into more later. So now Rohan is going to talk about what it was like to actually work with AMP.

Rohan Kapoor: Before jumping into AMP, I wanted to take a second to talk about emails traditional development. Traditionally emails have had two MIME types, the text MIME type and an HTML MIME type. So a modern email client, such as a Gmail or Outlook on the web, that can support HTML will read HTML, and text only clients that run into terminal, something like Mutt, will read only the text part.

Rohan Kapoor: So then came AMP, which was implemented as a new third MIME type. Clients that support the AMP MIME type will read the AMP part and display that, while other clients will fall back naturally to HTML and text. So, there’s full backwards compatibility, there’s no risk that an email client will display incorrect content or garbage just because you start sending AMP.

Rohan Kapoor: As Janie mentioned, one of the biggest considerations that Google had when they built AMP was mobile first experiences. In a mobile world, it’s quite common for the user’s device to lose their connection as they’re moving around from place to place. And on the web, AMP works around this by using AMP caching and caching some components on the user’s device. This reduces the network traffic required to load pages. But, we’re talking about email. And in email, developers have to send fallback content like above, which can be used if the network request fails to return data. So in this case, this email failed to load this data, and so this data that was preloaded as fallback content shows up so that there’s not a giant white space where it should be.

Rohan Kapoor: With AMP’s emphasis on user first design and development, there’re some imperatives that we found particularly tricky to work with. For example, AMP requires that all content has an integer valued width and height. For an email system, that means that at the time you’re sending it, the system needs to know what the width and height of all of your dynamically generated content will be, even though that content doesn’t exist yet.

Rohan Kapoor: So, in our case, we fetch text like the job snippet dynamically. And it can lead to situations where the content is too long and gets truncated. See the text sponsored on the image on the slide. Or too short with extra padding on the ends. And on this slide we can see that the difference in height is pretty remarkable between the job in the middle and the one above and below it.

Rohan Kapoor: Now AMP also suffers from the clipping problem that Janie was mentioning, but it behaves very differently than the traditional HTML email. If the AMP MIME type is larger than 100 kilobytes, the email client silently drops it and falls back to the HTML MIME type. There’s also additional limitations on the size of the entire MIME tree that can cause the whole email to just disappear into the void. It’s also important to keep in mind that this is still an email. Everything is still happening inside an email client, inside the user’s browser. And UI elements such as lightboxes may not work exactly the way you expect them to.

Rohan Kapoor: So we had created mock ups for replicating an Indeed view job page inside our Job Alert email. This was a little bit of a slimmed down version, just because you know it’s running inside an email. And the idea would be that the user clicks on the job and instead of leaving the email, inside the email themselves, inside the email itself, they can view the job description. However, it didn’t quite work the way that we expected it to.

Rohan Kapoor: So the video loads, so you can see, when you click on the job at the top, you get a normal job description. But as you scroll down and then click in, it never scrolled up. The job description is up there. And if you go all the way down to the bottom and then click in, the job description is gone. But actually it’s all the way up there. Because of the way AMP content is rendered, for security reasons it all runs inside an iframe. It has no idea what the viewport is and so lightboxes don’t quite work the way we would think they would.

Rohan Kapoor: So we emailed the team at Google, filed a ticket, and a little bit of back and forth happened, and then they closed the ticket, and said that they’re going to remove AMP lightbox from the list of email approved components because they couldn’t find a way to make it work. So, at Indeed, we’ve built our own email service provider or ESP. And so our journey with AMP begun by adding support for sending the AMP MIME type through that platform. As I said earlier, the AMP MIME type, oops. The AMP MIME type is backwards compatible, so any Indeed application that doesn’t support sending AMP would have no change in behavior. But any application that is sending AMP is received by a client that supports AMP, everything will work fine.

Rohan Kapoor: We also ran into a bunch of specific challenges while working with AMP, and there’s some interesting workarounds that we wanted to share. As Janie mentioned, we started working with AMP during a developer preview period, and at the time AMP for Email was considered bleeding edge. As many of you may know, when you’re working with bleeding edge software, sometimes you have to be ready to bleed. One of the biggest challenges that we ran into was that AMP specifications changed a lot during this time, and the documentation frequently lagged behind. And here’s one such story.

Rohan Kapoor: One day a QA in our team reached out to me and he was telling me that none of our AMP emails were working anymore. And keep in mind that these were emails that were in his inbox from yesterday and worked fine yesterday, but it’s dynamic. So what worked yesterday may not work today. What happened was that where the sponsor jobs were supposed to be, there was just a large white boxes. So we opened the developer tools, looked in the error console, and it’s full of incomprehensible red error text. And all of the error text is minified, so no idea what any of it means.

Rohan Kapoor: So we reached out again to our developer contacts at Google, and they told us that, “Oh yeah, they have now enforcing Gmail-specific CORS headers. The documentation doesn’t come out yet, but it’ll be there in like a week.” And they sent us a quick and dirty version, but basically we had to now add the Gmail-specific CORS headers. That’s an example that we use, and without those all the Ajax requests to fetch the content failed.

Rohan Kapoor: So AMP initially required that the domain that it sends requests to matches the email sender domain, and at Indeed we use alertatindeed.com when we’re sending Job Alerts. However, jobs can live on a variety of different domains depending on what country they’re for. So jobs in the UK live at www.indeed.co.uk, for example. You’ll notice that indeed.com, where the email is coming from, and indeed.co.UK where the job is hosted don’t match.

Rohan Kapoor: So, one possibility that we had was to change the sender email address to match the job domain, but this raised another potential issue for us. It could dilute the sender reputation of the indeed.com domain. So, for those of you that send a lot of email, you’re probably familiar with the concept of sender reputation. Basically, all of the incoming email providers like Gmail, Outlook, Yahoo, have a score that they assign to emails coming from your domain. And switching to a new domain like indeed.co.uk would have no sender reputation and we’d lose all of the reputation we had from indeed.com. So this was not really something we wanted to do.

Rohan Kapoor: What did we end up doing? Well, the most straightforward solution that we could come up with was creating a proxy web app. This web app lived at an indeed.com sub domain. In our case it was called ampxy.indeed.com, and it accepted a base 64 encoded URL that told it which domain to actually go fetch the job from. And it handled all of the AMP security validation, handled the CORS headers, and then did the request to the real domain to get the data and passed it back. We called this solution AMPXY for AMP Proxy, and it effectively allowed us to perform cross-site origin requests while masking them, so that as far as AMP and the Gmail client were concerned, we were still hitting indeed.com.

Rohan Kapoor: Ironically, a few weeks after we built this solution, the team at Google reached out to us and told us that based on our feedback, they were removing the same domain requirement. Again, bleeding edge software developer preview all of that. Well, fortunately there was some other benefits that AMPXY gave us, so it wasn’t all just wasted effort. AMP for Email doesn’t allow for any redirects, and if redirects are present, requests fail automatically with no error. AMPXY allows us to proxy the redirect on the server side and once again hide it from Google and make it look like everything’s fine.

Rohan Kapoor: Second, QA testing. So, since AMPXY exposes a single endpoint, we can put that through the QA firewall and have a very simple, straightforward way to test in QA rather than having to open up a bunch of different end points for various functionality. But there’s one significant downside: URL lengths. Because we’re using a base 64 encoding, your URLs ended up almost twice as long as before they started, which makes email clipping much, much worse. In the future we’re planning on building a URL shortening system, which we will integrate into AMPXY, which will hopefully allow us to have shortened URLs and use them in AMP and everything will hopefully work, but it hasn’t been built yet.

Rohan Kapoor: Unlike regular HTML, email AMP does have strict validation requirements. If AMP content validates, it shows up. If it doesn’t validate, you get HTML content instead. And this doesn’t actually show up in any way that you can see. So, one such scenario here is that the AMP spec doesn’t allow you to import functions and then not use them. We use AMP list components for sponsored jobs, and every time we have a sponsor job, it goes in its own unique AMP list.

Rohan Kapoor: So if we have no sponsor jobs at the time, we’re rendering an email, then we don’t use any of our AMP list components, but the import was still there. We learned the hard way that we had to remove the import, and so now it’s wrapped in a nice if-statement to make sure that we don’t fail validation that way.

Rohan Kapoor: As I mentioned, when emails fail validation, there’s no reporting back from the Gmail system as to why it failed or the quantity that failed or anything like that. There is however, this very convenient developer sandbox where you can copy and paste in your entire email, and it will tell you line by line what you did wrong. So, in this example, the tag image is disallowed, because AMP doesn’t allow you to use images, you have to use AMP images instead. Some dynamic caching magic stuff, I’m sure.

Rohan Kapoor: So as a result we are adding AMP validation to our internal ESP because as you’ve probably heard, Indeed is a data driven company. We love our metrics. We want to know how many of our emails that are leaving fail validation and why and hopefully correct them.

Rohan Kapoor: One last thing with AMP is that there’s additional security requirements than traditional email. It’s a trend here. You take traditional email and then add a bunch more requirements and then you get AMP. So to pass AMP validation, you must always pass DKIM, SPF, and DMARC, and email is must always be encrypted in transit using TLS. If not, they magically disappear into the ether. We learned that one the hard way, too.

Rohan Kapoor: So, rounding up the list of important considerations with AMP is that users probably aren’t reading your email exactly at the time you’re sending them. So Gmail supports rendering AMP up to 30 days after the email has been sent, at which point it will permanently switch over to the HTML content forever. So this means that as a developer, when you’re sending out your emails, all your URL endpoints, all of your paths, all of that has to be valid for 30 days, otherwise the email fails. Another use case for fallback content, which can be displayed if those requests do fail.

Rohan Kapoor: So, few technical takeaways. Working with bleeding edge software is hard. Specifications change frequently and you have to be willing to adapt at all times. You have to plan for fallback content with AMP. You are in a mobile world, network connections change all the time and you don’t want large holes in your email where the dynamic content was supposed to be. And you have to find a way to work around the fixed width and height limitations. Basically, you want to make sure that parts of your email don’t clip internally. There’s no giant white spaces, so find some sort of an easy medium. Make sure all of your content that’s coming dynamically is capped at that limit.

Rohan Kapoor: And now I’m going to hand it back to Janie to talk about some of our tests results and conclusion so far.

Janie Clarke: Thank you. How is our test going so far? You learned all about A/B tests before. We test everything, and AMP is no different. We are running AMP in an A/B test right now, that’s targeted at gmail.com users only. So the control group does not send the AMP MIME type at all and the test group does send the AMP MIME type. When we are also testing a few other little functionalities, but the main thing we’re testing right now is the sponsor jobs. So far when we look at our test at the aggregate level, we’re not seeing much change in our test group, and there’s a reason for that.

Janie Clarke: When we compare emails that were opened as AMP to emails that were not opened as AMP, they were just opened as regular HTML, we do see two times more clicks in the AMP opened emails, which is a really great promising early results. So, the reason for this is mobile. Google has not started to roll out AMP for mobile gmail.com yet–for the Gmail app. And in Job Alerts, 82% of our opens are on mobile. So that means most of the users that are getting that AMP MIME type aren’t seeing it. So since AMP is currently not supported on mobile, our test results are pretty limited.

Janie Clarke: It’s hard to spot many behavior changes when we look at the test at the aggregate level. And so we’re waiting right now for Google to start rolling out the AMP functionality for the Gmail app, so that we can really see how it does at full scale. We’re in a little bit of a holding pattern right now, just watching and waiting. We do have some future ideas for features that take advantage of AMP that we’re really looking forward to testing and we’re working on them right now. One of them is an interactive NPS survey as shown here, so we can show NPS right at the bottom of the email and users can answer the question, even type in some feedback for us without even leaving Gmail.

Janie Clarke: We also working on an interactive unsubscribed surveys, similar idea, someone can unsubscribe and tell us why they’re unsubscribing right there. So, it’s a great way to capture some user feedback from your email users. So here are a few things to consider about AMP from a product manager’s point of view. Firstly, be aware of the current limitations when you’re planning. As I mentioned before, there are some challenges with mobile support and with how certain components behave.

Janie Clarke: And secondly, you have to plan out how you’re going to measure your test. We didn’t go into much detail about that, but since AMP adds interactivity to your emails, you need to know what actions you might want to measure and make sure that you’re logging those, tracking them and whatever system they use, so that you can see what’s happening inside the email. And lastly, designing for AMP brings new challenges. So it’s different from designing for a traditional email and it’s also different from designing for the web or mobile. Like the lightbox case that we mentioned, the design that you have in mind when you first do it might not be how it actually works in real life. So you have to be aware of that and be ready to adapt.

Janie Clarke: So we’re very excited about AMP but cautiously. So as I mentioned, it totally changes email. It lets email do some things that have never been possible before. It brings the whole conversion funnel directly inside the email. In short, we think it’s awesome. If you’re excited about AMP as we are, we recommend giving it a try. You do need to have some development bandwidth to work on it. But there may be dragons. So there are some challenges when it comes to working with AMP, so just make sure you’re prepared. Thank you.

Allison Dingler: Everyone hyped? Everyone ready? Yeah? Can I get woo hoo? Y’all can do better than that. Here we go, awesome. I’m going to pass it over to Galina.

Galina Merzheritaskaya, Nitya Malhotra, Erin McGowan, Alison Yu

Indeed girl geeks: Galina Merzheritaskaya, Nitya Malhotra, Erin McGowan, and Alison Yu speaking on women in leadership at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Galina Merzheritskaya: Hi, everyone. My name is Galina. I’m a QA Engineer on the data science platform at Indeed. I have a few question we got from panelists tonight, and once I finish them, I would like to move to the audience and hear your questions. Before I start with my questions I would like all panelists to introduce themselves.

Nitya Malhotra: Hi everyone. My name is Nitya. I’ve been at Indeed for about five years now. I’m an Engineering Manager. I transitioned from an IC to engineering manager about two years ago. Before that, I was a product manager with Merrill Lynch, Bank of America, and then Indeed. And yes, that’s where I’ve been ever since.

Erin McGowan: I’m Erin McGowan. I’ve been at Indeed for three and a half years. I’m our Associate Site Lead in Seattle. That’s part of our chief of staff organization, so we’re looking at overall site health and ensuring engineering growth across the site.

Alison Yu: Hi everyone. I am Alison. I am the Open Source Community Manager here at Indeed. I’m part of the Open Source Program office and I report directly into the Engineering Capabilities Organization. I’ve been here a year and a half. I think I said that. I have a little bit of a cold, so if I start to cough, I’ll exit to stage left.

Galina Merzheritskaya: Thank you. I think all of us know that tech is a male dominated industry statistically. You have this number. Does anyone know the ratio between men and women in tech for the past five years? Okay.

Audience Member: 10:1.

Galina Merzheritskaya: 10:1. Close. So, Catalyst is a nonprofit organization collects a lot of data to help women at workplace, and they provide the data that 30% of women in tech industry, only 20% in a leadership position, and Forbes also did their own research showing that you have some growth coming from 3% to 6%, from average 15 to 17. So there’s some work done, there are some changes, probably it’s why you’re here.

Galina Merzheritskaya: So I want to ask panelists, what do you think can be done to change the situation? To make it better and have more female, the women in a male dominated industry?

Erin McGowan: [inaudible].

Nitya Malhotra: Hang on for a second. I think something that in my mind would really make a difference is A, seeing more women in leadership positions. This is something that I personally find motivating, or demotivating sometimes, to not see. But how do we do that? How do we actually get to that place? Now, I know that from Indeed, I’m again speaking from Indeed’s perspective, I know that support, having either great mentors or great support within your organization has also been really helpful.

Nitya Malhotra: Another thing that Indeed has been doing is we’ve actually been partnering with a few programs. I’m actually more familiar with the program in the Seattle office where we’ve been partnering with the Ada Developers Academy. The Ada Developers Academy is a training program for women who have not necessarily been through the traditional CS program in school, and it’s basically a bootcamp after which they go through an internship and then end up joining a bunch of tech companies.

Nitya Malhotra: So the Ada program in Seattle has actually been pretty amazing. I actually ended up working with a lot of ADs as we call them in Seattle, and it’s been great to see so many women in the Seattle office, thanks to the Ada program, and they’ve all been doing such an amazing job. So that in itself has been pretty encouraging to see more women in general in a lot of the offices. So, I think just it’s a numbers game out there and seeing more women in tech is I think what is going to solve the problem in the end.

Galina Merzheritskaya: And [inaudible]. Is it a major program for anyone who works at Indeed or anyone from outside [inaudible].

Nitya Malhotra: It’s actually outside Indeed but they end up partnering with Indeed and a bunch of other tech companies as well, and do internships at Indeed and end up getting them full-time offers with tech companies.

Galina Merzheritskaya: Thank you.

Erin McGowan: And in San Francisco, Indeed is also partnering with Techtonica. They are hosting a cohort of 15 on site now. It’s the second cohort. The first cohort has four interns that are currently interning here at Indeed. The other from that cohort are in interning at other tech offices. So San Francisco is also helping to increase the funnel to help grow women in these engineering roles.

Alison Yu: And I can expand on Techtonica. I am based on the San Francisco office so I work closely with them. One of Techtonica’s missions is to make sure that women and non-binary individuals who have non-traditional tech backgrounds have a way to join the tech industry. So they put them through a bootcamp, essentially, and then help place them with jobs. At Indeed, we not only host them but we, let’s last quarter, also help get them to Grace Hopper. So we did sponsor travel and passes for Tectonicans who went and hosted a project at Grace Hopper’s open source day, which the Open Source Program office did sponsor.

Alison Yu: So, we’re trying to make sure that their name is getting out there, that there is more ways for people to get involved with tech. And then just being a part of the Open Source industry as well is, one thing that we like to stress is that technology isn’t all about codes. So there’s so many different jobs within tech that don’t require you to be a hard engineer or a coder, even. So there’s so many different facets of tech that have roles open. So marketing, legal, et cetera. So, I think there’s a lot of other ways that people can evolve in tech, which aren’t traditionally thought of.

Galina Merzheritskaya: That’s right. Thank you, Alison. You all [inaudible]. Great. Based on the value you see right now, have you, having mentioned of leadership roles, how did you land there? How do you get to this current destination where you’re right now?

Erin McGowan: Sure. I’ll take this one first. I didn’t always start in tech. I actually started in hard science as a lab tech, and it was awful. If you think tech has low men to women ratios, hard sciences has nothing on that. I was the only woman in both labs that I worked in. And looking at it and as we’re using computers, running instruments, it was interesting and fascinating way more than the hard science. So at that point I transitioned into tech largely in quality assurance. I learned how to become an [S debt 01:09:45] at Microsoft way back when. And I always said, “I don’t want to be a manager. I don’t want to be a manager. That’s awful.”

Erin McGowan: And then I started mentoring people and seeing them grow and I was like, “Oh, this is what management is about. I can help drive careers.” At one of my jobs prior to Indeed, I took the plunge and I accepted a leadership role. I had an awesome, awesome female advocate who was big on ensuring that women had access to those leadership roles, and I loved it. When I joined Indeed, I came in as a quality assurance manager, actually in Austin, and I grew my team from four, and before I transitioned I had a team, teams in Tokyo, Austin, and Seattle, and tried to help them grow.

Erin McGowan: And also growing the next generation of leaders really helped fill my bucket. So even if you think you don’t want to be a manager right now, you never know, don’t close that door, leave it open.

Galina Merzheritskaya: You’d like to add anything?

Nitya Malhotra: Yeah, I can go. I actually started as a product manager in, I was working with Merrill Lynch. So I ended up being a product manager. Now after school, despite good grades, et cetera, I had a computer science degree, I wasn’t really sure if I was maybe cut out for software engineering. And I know a lot of women who have gone through similar experiences, it’s amazing how many women I know who have basically had similar experiences. Anyways, so at that point, it was when I missed getting my hands dirty and actually coding. And that’s when I realized that, “Oh, this is maybe something that I do want to get into.” And at Indeed is when I would say I completely transitioned back to a software engineer.

Nitya Malhotra: And again, I think my journey, even at Indeed was, it took a lot of work, a lot of, I would say a lot of coaching and a lot of mentoring from really great managers, really good mentors, that really helped. Because there was always this, [inaudible] I did this, but is it really that great? Everyone could have done this. And it required a lot of my manager be like, hey, you know what, this is actually something good that you’ve accomplished, et cetera.

Nitya Malhotra: So, I would say it was a lot of that. It was a lot of help from my managers and mentors to actually get me to a state where I felt that I was confident enough to go ahead and then ultimately make a switch from IC to tech lead to then engineering manager. And I think that transition off of that has happened really smoothly.

Alison Yu: Yeah. I started out in clean tech in solar, right, if anyone remembers when [inaudible] happened. That was a fun time. I really got thrown into the tech bubble in a sink or swim situation. From there I transitioned from clean tech into tech. I was really lucky. I had a great manager who I actually followed from one company to the next, and she really encouraged growth. And here’s some different ways you can help expand what you’re doing.

Alison Yu: Manage some vendors, manage different contractors, figure out how your style is, and that’s how I’ve gotten to where I am. I don’t currently manage any people, but I do manage many different relationships cross departmentally and within my own program. So, external and internal. That’s how I’ve navigated the waters.

Galina Merzheritskaya: It sounds like you can come from individual contribution to a manager. Maybe you can advise something, what steps look like to come from IC to manager.

Nitya Malhotra: I can take this one one first. When I was thinking about moving, becoming an engineering manager, I think my biggest question was, am I going to lose technical focus? My goal was to be an engineering manager who was also really technically strong. And I was really worried about either plunging into management a little too early and losing the technical focus.

Nitya Malhotra: Fortunately that has not happened. What I have learned over time is that being an engineering manager has given me, I might not be directly making a lot of technical contributions, but it’s given me the chance to make those contributions by influencing others. And that has been the big switch in my thinking. So I can still be involved in really big, really technical changes.

Nitya Malhotra: The only difference is I’m influencing those changes rather than actually executing on them. And that has also been pretty, I still feel technically involved, at the same point in time, I’ve still felt that I could influence others careers. Coaching has also been really, really rewarding and I have no regrets.

Erin McGowan: I think what I would tell someone who is interested in going from being an IC into management, is to try mentoring. See if there’s an opportunity to mentor an intern or a new hire, and see how you like that. As that is successful, working with your manager to say, you know what, I want to do this full time. Where do you see my skill gaps? What do you see that I need? Is there any training that you can attend, books that you would suggest? And making sure that your manager is aware.

Erin McGowan: If your manager is not receptive, finding someone who has seen some of that leadership, your mentorship to get as an ally. Unfortunately, sometimes we need to have that external ally who’s not our direct manager. But the biggest thing is doing it, showing leadership, stepping up in meetings, stepping up to volunteer to take on some of that extra unofficial leadership. As people see you in that role, it is a lot easier for them to see you in a full time management role.

Alison Yu: Yeah, and I would expand on that as well. Not only just finding one-on-one mentorship, but if you can try to step into a leadership position in what we would call here a ERG, like Lena is leading the women in tech group at in the San Francisco office for Indeed. If you can find different programs where you can help lead and bring people together, it makes a big impact. People see you. It’s a highly visible role and it’s really easy way to also find other people who can help mentor you.

Alison Yu: Because otherwise, unless you’re asking for mentorship or asking for help, if you don’t raise your hand and look for those opportunities, people don’t know. So I think the first step is really asking and looking for those opportunities. Prior to joining Indeed, for example, I led the philanthropy efforts at my last company for about three and a half years prior to joining Indeed, but I didn’t have a formal role there. But that gave me the tools to actually be able to step into more of a leadership role where I had the experience behind me.

Alison Yu: And it was something that, even though it wasn’t official, I could put on my resume, I could talk about my experience. So, I think as Erin said, getting your hands dirty and actually doing it is really 80% of the work.

Galina Merzheritskaya: Yeah, I can probably [inaudible]. You try and you like it but we have [inaudible]. You probably had a lot of advice in your career path. Can you guys discuss for some areas, some industries that at least have [inaudible] of sharing? can you share maybe one of the advice with us?

Alison Yu: Sure. I’ll start. One of the things, one of the pieces of advice that I’m still grappling with is, don’t focus so much on perfection and don’t burn yourself out. I am probably a Type A type of person. If you know me, I very much focus on perfection. But one thing I’ve learned over the years is, if you’re burned out, you’re not putting out your best work. You might be running for perfection but you’re not going to get there if you’re completely burned out. So, take time for yourself and rest. I know it seems counterintuitive, but recharging is really one of the most important things that you can do.

Galina Merzheritskaya: That’s great

Erin McGowan: Sure. I think the piece of advice that has stuck with me the most is, if you’re not uncomfortable, you’re not growing. It can be easy to get into a role or into a position where you’re comfortable, you know what you’re doing, you know you have this. But when you’re in that steady state, you’re not pushing yourself and you’re not growing. So it’s okay to get comfortable for a little bit, but then bump yourself to that next level and look for that harder challenge, because that is when you’re going to grow and take your career to the next step.

Nitya Malhotra: Completely second that, by the way, completely. My end, I would say the biggest career advice that I have received is–something that I’ve also really followed, is I’ve really looked out for mentors outside my manager as well. Whether it’s peers or managers on another team, basically looked around the office for people that I know I can go to with questions about various different things. I have a mentor that I go to for technical questions. I have a mentor that I go to if I have questions around being a manager.

Nitya Malhotra: There’re so many different ways. There’s so many different people that I know to reach out to if I have questions on so many different things, and having that these are often the people who will end up supporting you and end up giving you that additional visibility as well within the office. That’s something.

Nitya Malhotra: Another thing that this applies specifically to mentoring, and this advice that I received while mentoring which I’ve always moved–paid forward is, when mentoring, always think about the why rather than, focus on the why rather than the what. No matter what it is that you’re teaching someone, focusing on why something is important rather than what, actually really helps a concept stick. I think this is just in general, good advice that I’ve always passed on.

Galina Merzheritskaya: Okay. So really great advice. I don’t [inaudible]. This is a good time to take questions. If you have a question, they have a mic over there [inaudible] so we can [inaudible].

Vanessa: Hi [inaudible]. Is this working?

Galina Merzheritskaya: No.

Vanessa: Hi, can you hear me?

Galina Merzheritskaya: Yeah.

Vanessa: Cool. My name is Vanessa and I was wondering if you could give me some advice for a situation I find myself in pretty often, which is I oftentimes find that when giving my opinion on a situation, it’s either implied or explicit that I back my decision up with data. Whereas I find a lot of male colleagues can just give their opinion and not back it up with data, and be considered credible. When I back up my decision with data, sometimes it’s met amicably and sometimes it’s met with skepticism, even if it’s based in reality. So I’m wondering if you have any advice for those kinds of situations.

Galina Merzheritskaya: Tough one [inaudible].

Erin McGowan: That’s a tough one. I am really, really happy that Indeed does not have a culture like that. I think the biggest thing is to be consistent. And I would also, with those men being questioned, ask. Be the one to stand up and ask them, what are you basing this on? What did you see that led you to this result? And be consistent with that and they can come to expect the same level of questioning as yourself.

Nitya Malhotra: I would ask the other questions, are there others in the room who probably feel similarly? Are you the only one in the room who is feeling that way or there may be others who are feeling similarly. Maybe this is a wider problem as well and needs to be addressed in a wider way, but that’s the other avenue that I would go down, and the me–

Alison Yu: Yeah. I would also say, if you feel like this is the only… you’re the only one on the team for example, bring it up to your manager, talk about it. If they don’t realize that it’s an un-bias–or an unconscious bias, then how can they address it? So, I think that’s something that we should be talking about more in the workplace, anyways, I am very happy that we don’t have that culture here at Indeed. But I think that until you raise that issue with others and even talking to your teammates one-on-one, pull them aside, say, “Hey, why are you questioning me about this?” Until you have those open conversations with them, I don’t think the situation can change. But I think that’s your first step.

Nitya Malhotra: And I think it’s fair to ask for data, but I think it should be applied consistently. I think it’s absolutely fair to ask for data when you’re making a statement. But the inconsistency is the issue. And the other thing that I would say is don’t let that stop you from actually providing your input. Even if you have to back it by data, go ahead, keep going strong with providing your opinions, even if you have to back it with data. But don’t stop doing that.

Vanessa: Thanks.

Galina Merzheritskaya: Anyone else? Everyone seem to ask big questions, tough questions. Okay it’s 8 o’clock already.

Audience Member: I’m not sure if anyone, any of you, y’all said this already, but can you tell me how long it took you to get into leadership and are there any other steps that could take to be there?

Erin McGowan: Sure. I’ll go first. From the time that I really decided I wanted to move into leadership, was probably about 12 months, and that included doing an extensive six month leadership program that had some very intensive training sessions, talking with other leaders in the organization. This was prior to Indeed. It’s really going to vary, depending on your organization. I’ve had people on my team that I was able to get from an IC into a management, that was anywhere between say six months and 18 months, depending on where they were at on their career when they started addressing interest and wanting to go to management.

Nitya Malhotra: And I was actually similar situation once I figured out that I wanted to be in a leadership position, was about asking, “Hey, what are the next steps that I can take, move into a tech lead role?” And tested that out for a while, then moved on to engineering management. It was a similar 12 month period for me. The advice would be if that’s something that you’re interested in, I would say ask for it and test it out, see how it goes.

Galina Merzheritskaya: One time I got advice that if you’re solving problems, you lead something. It can be project, a team, a team has issues and so you have to handle them, overcome them, failure. Then it’s just next step if you want to make them official or if you want to just [inaudible]. If you want to make official you speak with the manager and [inaudible] talk with what you want and how fast you want it.

Nitya Malhotra: Yes.

Galina Merzheritskaya: What should I do to make manager? Steps just what can [inaudible] and what can make a [inaudible]. If it’s two months, three months [inaudible] you solve the basic problem [inaudible]. For [inaudible] company, tomorrow you [inaudible] [inaudible]. Yeah.

Alison Yu: Yeah. It seems like it really does vary depending on where you start in your career path. If you start very early on, knowing that you want to manage someone, it’ll take you much longer than if you’ve been in your career for five, 10 years. So I think it’s just knowing what you want and going after it. When I manage people, I actually did a job switch and looked for particular roles that had managing positions and where I would hire on a team. I know that’s not always the most ideal way to do it, but sometimes the company that you’re currently at lacks resources. Here at Indeed, we thankfully don’t lack resources. I just decided that that wasn’t the path that I wanted to be on at the moment. So, many different ways.

Galina Merzheritskaya: [inaudible]. Anyone else?

Audience Member: Hi. I just wanted to ask, what advice do you have for somebody who is starting off in their career but wants to have influence within the team? What kind of strategies or what did you do when you were in that position to influence the decisions that your team makes or just to have some influence, because as a leader, obviously you have to make decisions and have influence over your team. So do you have any advice for people and they’re starting their careers?

Nitya Malhotra: So, actually this is a good question back to you. I would ask yourself what is the thing–What is the… there’s always, no matter what the team, what the company, there is always some room and some scope for improvement. And finding things that either you are passionate about getting… about the team getting better about team’s process improving, about how the development processes. Or if you want to let’s say, if you’re really passionate about this particular, it could even be at this particular class in your service that you think is not tested for example. Or you see a bunch of errors and no one’s caring, but there’s so many.

Nitya Malhotra: I would say pick something that, let’s say, that you are passionate about that, let’s say, bothers you and go ahead and fix it. Things like these, the tiny, tiny things that you spot and as you keep improving these, people are going to notice that you’re taking the initiative to go ahead, find something that you don’t like, and improve it. I think that is a great leadership quality in itself. And once you start doing that in the small level, you will start doing that on a larger scale, as well.

Erin McGowan: I would say one of the mistakes I made early in my career, is I was afraid to speak up. I would be sitting in a meeting room and I would be afraid to actually voice my opinion. I would have the thoughts and I would just be afraid to actually put them out there. Put yourself out there. Don’t be afraid to say, what about this? Have we considered this? You know what, I tried this and it really didn’t work. They’re not going to bite. At least if you’re in a good workplace, they’re not going to bite. So I would say just go ahead and speak up. They want to hear your voice. You’re in that room for a reason, so don’t be afraid to use it.

Nitya Malhotra: Also to be fair, that never gets easy. Today there have been times where I am like, “Oh, should I see this? Everyone else seems to know.” But that never gets easy and that’s always a struggle. That’ll continue to be a struggle, times you just have to push past it.

Alison Yu: I would say try to find gaps and try to become a subject matter expert in one of those areas. I was on a marketing and communications team across multiple different companies. I specialized in social media for a while, then I became more broad. And because of my expertise in one area, people came to me from many different departments, from different business units. Even though I was in the marketing team and I sat in marketing, people from different engineering teams would come to me and ask, “How do I market my product better?”

Alison Yu: So once you get your name out there and you’re proven that you’ve done the research, you’re doing a good job, people will seek you out anyway. So focused on something that you’re passionate about, that really fires you up because that’s something that will be recognized that you’re doing a good job and then they will naturally follow.

Galina Merzheritskaya: I can add the two pieces. One as jealous [inaudible]. If you think I don’t know team, I was like, “Oh my God, she loves this team, loves the product, what can I improve? Like everything’s great.” And then, “Oh is it [inaudible] feels great.” And he’s [inaudible] today [inaudible] every day since 5:00 PM, do you guys use [inaudible] and then he’s trying to give [inaudible] aspect. Not [inaudible] but [inaudible]. Whereas where I can help to improve this. And as it little by little you have this guy like, “Hey, this is actually not a big problem.” And [inaudible] somewhere you can show the need to do such.

Galina Merzheritskaya: You’re also independently and [inaudible] are like this is my research and definitely how can you grow. And I’ve learned [inaudible] goals, that sometimes will follow when you lose everyone else, you just have your own goals, but for [inaudible] before breaking them, bring on some rules that exist. And knowing how everything works. So like okay then you can figure out different ways that basically can prosper in that area.

Alison Yu: I’m just going to add to that I think different perspectives and the way that different people will look at a problem or a situation, can only make a product or a team stronger. So even if you think, “Hey, my opinion, what is it really valid?” Your opinion is valid so never question that. But also know that the different ways that you look at a problem is a different way that someone might have never seen it, and you can be revealing something about a weakness that maybe no one else on the team had thought about and that can be a really great thing for you and for the team.

Galina Merzheritskaya: That’s true. Okay.

Audience Member: Hi there. I have a question about how do you grow on the job? I know that you mentioned that there is a women in leadership program that you went to to get a lot of training before you became a manager. What if there’s no training like that or you’ve thought about becoming a manager maybe down the road, how do… what kind of things have you done, maybe in the past, to prepare yourself to get to this road? Do you read books? Do you listen to podcast or do you go to meet up to meet mentors? Yeah.

Erin McGowan: Yes. All of the above. I think the biggest thing, if you’re looking for some leadership training and your organization doesn’t offer it, look external and ask. Often time your organization will pay for external training. Most of the time there are budgets for that and people don’t even know to ask. So ask for it. Perhaps they can bring some in. You’re probably not the only one. There are a number of books that you can help read. I [inaudble] in the management before podcasts were a big thing, but I know that a lot of people now are using podcasts to help learn and having a mentor. Having a manager mentor that is not within your organization can be huge.

Erin McGowan: You’re able to really have good conversations with them without any fear of bias or… I asked this question, is my manager going to think I’m not ready? So having someone who’s outside, whether they’re outside just your group or your company, is invaluable to really help understand where you as a person need to go.

Nitya Malhotra: I’m not great with podcasts or it reading outside work. For some reason after work, I just switch off and I… it’s really hard. So a lot of my growing I feel happens, majority of it for me has happened at work. Something that I found useful is putting myself in, let’s say my manager’s shoes, and thinking about A, how would I have done this differently? How would I have done this better, or do I like what they’re doing? As time has gone by, this has created a mental model of how it is that I want to be as a manager, and that has really helped me model my experiences with my reports as well.

Alison Yu: Yeah, I am also the same way. I do not want to read or listen to a podcast about work after work. I work enough, so I think one of the things that I try to do, is at work or when I travel for work, so speaking at conferences, et cetera, one of the things that I try to do is make sure I go to the networking events at those things. Talking to my peers in the same industry, seeing where they are in their careers, how they’ve been able to progress, and then finding other people who are already in other higher management roles and seeing and talking to them about what was their career path, if they’d be open to mentoring.

Alison Yu: I think that’s really important that you want to see what other people in your same industry are doing, and not just only stick to what’s the norm in your company. Because sometimes when you’re so isolated and so siloed in your own company, you might not notice that the path that your management runs, it’s not the same as the rest of the industry. So I think that’s also very important to keep just a tab on.

Galina Merzheritskaya: Yeah. Sometime it’s called shadowing, if you think you would like to try some role and be like a measurable, you speak with your and you’re like, “Hey, I want to see what you do so I can know what you do from seeing what you’re doing.” And then you will have questions and then you will have a pass while you actually [inaudible]. And then you always say, “Oh, I did [inaudible].” And if you have an opportunity to speak with the manager be clear what you want, or how will the manager help me.

Galina Merzheritskaya: Or like if it’s not in your company, there are so many programs outside. We have so many mentorship right now. There are many [inaudible] to step up higher and you learn. It’s a great time to do it now because, if you can do more then you can do tomorrow [inaudible]. It makes it so that managers who are [inaudible] can draw a time just to help you to grow as well. Yeah. I think that’s fine. And we have time for a few more questions if you have any. Yes, okay.

Audience Member: Hi. Just wanted to firstly thank you guys for sharing your experience. My question was around conflict and some strategies that you use when you experience conflict in the workplace. I know sometimes women can be more of the supportive or the accommodating role, where men can be more dominant in that conflict situation, so just wondering what tips and tricks you have for dealing conflict.

Erin McGowan: I think the first tip is to make sure that both parties are leaving emotion at the door. If either party starts to escalate with emotions, asking for a timeout and table, and have that conversation at a later time. You’re not going to have any kind of a productive conversation if someone is angry, yelling, or on the flip side, if someone is upset, perhaps crying. You’re not going to be able to have a good healthy conversation, table it and schedule time. If you need to involve a neutral third party, either your manager or that person’s manager, or both, to help, if you are afraid of emotions escalating again.

Erin McGowan: I think the other thing is that some conflict is healthy. It’s okay to disagree, but you want to make sure that you are disagreeing respectfully, and making sure that you’re not crossing any of those professional lines disagreeing. Having data can really help. This is where I am on this because one, two, three, can you help me understand how you came to this?

Erin McGowan: So, really understanding their point of view can help bridge. You might be closer than you thought, but you’re talking sideways. You’re just in a different place, but you might still actually be really close to each other. So taking the time to understand where they’re coming from and how they got there. And helping them to do the same on your side.

Nitya Malhotra: I might have slightly different opinion, especially around the emotions. I know emotions in general are turned on as this bad term. I actually don’t think they’re bad. I think that a lot of times when there are strong emotions associated with how someone’s feeling, there’s generally a reason behind it, and it’s always, I think it’s important to figure out what that reason is. So even if there is emotion around it, obviously we don’t want things to escalate, but that emotion is stemming from something and it’s really important to figure out where that’s coming from.

Nitya Malhotra: So stating, let’s say in a conflict, stating exactly why you’re saying something, you’re stating where you’re coming from, what your intentions are. My intentions are not to disagree with you, but I really think that this is the right thing that we should bring. I’m just giving a stupid example. But stating your intentions, where you’re coming from, I think that really helps clarify and make someone understand that you’re not actually attacking them and that you’re coming from a place of logical reason.

Nitya Malhotra: Crucial conversations was this class that actually I took at, it was one of the trainings that was offered at Indeed. I thought it to be super helpful, not just in my work life, but also in my personal life. I would recommend, I would look into it as well.

Alison Yu: I think for me it’s making sure that everyone’s on a neutral playing field. Making sure that someone doesn’t feel ganged up on. For example, taking any of those external situational feelings that could happen if you were to say, “Hey, I want to address this.” But you’re doing it, for example, in the cafeteria or in a meeting room full of other people. I think there’s a time and place for everything, so if you are feeling those emotions, I don’t think they’re necessarily bad, but I do think that you do need that time to have everyone simmer down a little bit, so you have the root of the issue that you’re actually trying to get to, versus the feelings that are really getting there.

Alison Yu: I do agree that when you have those charged feelings, there is something that is sparking that and you need to get to the issue and resolve that, but you need to also do it in a cool, calm, collected manner, because you going in there guns blazing is not going to help with anything, neither you or the other party. But making sure to have it on a neutral ground, I would say before you really escalate it to managers, try to see if, hey, can you work this out with your peer? Is there a way you can take this out? Try going for a coffee and just going and talking it out. Usually that, I think, helps it.

Alison Yu: It’s not necessarily in the office or in front of others, but it gives you a neutral ground and it’s time away where other people can’t really hear what’s going on, and I think that’s very important when you’re trying to hash out the details when there’s a serious conflict.

Galina Merzheritskaya: And I’ll just maybe say super [inaudible] personal. If you personally got offended or someone’s [inaudible]. All my [inaudible] and game matters. I’ve been thinking about it and [inaudible] up there. You guys [inaudible]. And I [inaudible] solve problems, do it once I think. I’ve been told if you have to do [inaudible] five times breathe in, and breathe out five times [inaudible], you will get more oxygen to your brain and you will be feeling a little more rational. You cool down and then you can move on to the next step. You can also agree to [inaudible] partner [inaudible]. Thank you.

Audience Member: Thank you.

Galina Merzheritskaya: And I think one more question then we’ll close. Okay. One, two, three. Okay, I [inaudible]. Thank you so much for your time.

Allison Dingler: Awesome. I won’t take up any more time. Another round of applause for our amazing panelists up here, yes.

Alison Yu, Erin McGowan, Lindsay Brothers, Janie Clarke, Rohan Kapoor, Galina Merzheritskaya

Thank you to Indeed’s Alison Yu, Erin McGowan, Lindsay Brothers, Janie Clarke, Rohan Kapoor and Galina Merzheritskaya for speaking at Indeed Girl Geek Dinner.  Erica Kawamoto Hsu / Girl Geek X

Indeed Girl Geek Dinner attendees

Thanks to all the girl geeks who came out to Indeed for dinner, networking, talks, panel discussion and more networking!  Erica Kawamoto Hsu / Girl Geek X

Our mission-aligned Girl Geek X partners are hiring!

Win FREE tickets to 12 Girl Geek Dinners in our 12 Days of Girl Geek Giveaway! UPDATE: Winners announced!

Girl Geek X - 12 Days of Giveaways Holiday Sweepstakes

What’s better than a free Girl Geek Dinner? How about 12 free Girl Geek Dinners!? That’s exactly what’s in store for ONE lucky girl geek this month! But we’re not stopping there — 11 more girl geeks & allies will be winning too!

We’re super excited to announce that we’re giving away a whopping total of 78 totally FREE Girl Geek Dinner passes in our December contest — which we’re launching TODAY!

Day 1 - 12 Days of Girl Geek X Giveaways - Pack of Playing Cards and a Dinner Ticket

The grand prize (12 tickets to Girl Geek Dinners plus a Girl Geek X hoodie or jacket) will go to the entrant who has referred the most friends to enter the contest & sign up for the Girl Geek X weekly newsletter during the contest period, which ends on December 20, 2019 at 11:59am PST.

The other 11 winners will be selected at random — 1 each day for 12 days, beginning Monday, December 9, 2019 at noon PST. The first drawing is just a few days away, and we’ll kick off the fun by giving away 1 free Girl Geek X Dinner ticket and a pair of Girl Geek X socks!

The number of tickets we give away will increase each day, until we award the grand prize. The Girl Geek X swag item we’re giving away will get a little better every day as well — we’re starting with a super cute pair of Girl Geek X socks, and working our way up through coffee mugs and t-shirts! (See the full prize list and official rules in the widget below.)

To improve your odds of winning, you can earn extra entries for every person you refer, for liking or following us on social media, listening to the Girl Geek X podcast, or for sharing the contest with your Bay Area friends & colleagues via the widget below!

Stitch Fix Girl Geek Dinner

Girl Geek Dinners are hosted throughout the Bay Area year-round, and winners can choose to use their tickets to attend multiple dinners individually, bring their bestie to a few, or use all their tickets at once by bringing friends and team members to a single dinner. Your call!

Good luck, and we hope to see you at a dinner soon!

12 Days of Girl Geek X Giveaway

Update: Congratulations to our 12 Winners!

Prize Name
12 Alicia Livingston
11 Gabriela Mujal
10 Patti Mangan
9 Jenh Vo
8 Michelle Jin
7 Herra lee
6 Shubhi Asthana
5 Sarita Agrawal
4 Nancy Cao
3 Alagu Valliappan
2 Linda Wang
1 Esmi De Anda

See your name on the list? Keep an eye on your inbox! We’ll be in touch to confirm your mailing address (and size if applicable) in the next few days!

Thanks for entering, and for being a member of the Girl Geek X community!

Speaker submissions for Elevate 2020 are now open!

We are seeking session proposals for the 3rd Annual Girl Geek X ELEVATE 2020 Virtual Conference to be held March 6th, 2020.

We’re inviting women technologists, innovators and tech leaders from around the world to participate in Girl Geek Elevate to share the latest in tech and leadership with fellow mid-and-senior level professional women.

This virtual conference is FREE for attendees – last year, over 2,500 women signed up to attend – tuning in from 31 countries around the world – to be inspired by speakers on the latest in tech trends and leadership.

Sessions may reflect the theme of this year’s conference – “Lift As You Climb” – and content typically covers the following topics:

  • Lightning Tech Talks – Dive deep into an area that’s unique/critical to your business or role (i.e. machine learning, security, usability, UX/UI, ethics in building product, data analysis, etc.)
  • Technical Skills & Tactics – Tutorials, walkthroughs, or deep dives into a skillset or tactical approach to how you solved a real-world challenge.
  • Learning and Development – Topics include negotiation, job search, interviewing tips, being a better leader, self-awareness, career growth, management, etc.
  • Inclusion, Equality, and Allyship – Topics include being a better ally, lifting other women up, and actionable advice for individual contributors or managers.
  • Interesting Life/Career Journeys/Distance-Traveled Stories – Did you overcome socioeconomic challenges (i.e. first in family to go to college, raised in poverty/rural area/etc.) while giving back or contributing to the greater good?

Work on a unique technical project or have interesting insights you’d love to share with other other women & allies? We want to hear from you!

Tip: The best proposals include 3-5 key takeaways — things attendees can expect to learn from your talk!

Submit your proposal for a talk and/or panel here by December 24, 2019 11:59PM PDT for Girl Geek Elevate virtual conference.

We’re looking for speakers with unique perspectives to share their successes, failures, insights, advice, personal journeys and learnings with the community! Come share your story and elevate fellow Girl Geeks as they navigate the choppy waters of their own tech careers.

Both first-time and experienced speakers are welcome to apply. All nominations will be considered, and all selected speakers will participate in a speaker prep session with the Girl Geek team and your fellow panelists and moderators.

Why Speak at Elevate Virtual Conference?

  • Share what you’ve learned the hard way so that other women can more easily navigate their own careers — your talk will reach thousands of viewers!
  • Share the technology you’re working on and talk about the tough problems you’re solving
  • Increase your visibility within your own organization and position yourself as a subject-matter expert in your field
  • Open yourself to more career opportunities
  • Highlight issues unique to women in technology/leadership, and issues you’ve experienced or are passionate about
  • Connect with other great women leaders, peers and mentors
  • Elevating other women is a fun & rewarding experience

We’re open to presentations, one-on-one interviews, and panels… choose the format you’re comfortable with!

Submit your proposal for a talk and/or panel here by December 24, 2019 11:59PM PDT for Girl Geek Elevate virtual conference.

For sponsorship inquiries, please contact sponsors@girlgeek.io

Toyota Research Institute Girl Geek Lightning Talks (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Kelly Kay, Rita Yau, Suzanne Basalla, Jen Cohen, Carrie Bobier-Tiu, Ha-Kyung Kwon, Steffi Paepcke, Fatima Alloo

Toyota Research Institute (TRI) girl geeks: Kelly Kay, Rita Yau, Suzanne Basalla, Jen Cohen, Carrie Bobier-Tiu, Ha-Kyung Kwon, Steffi Paepcke, and Fatima Alloo, at TRI Girl Geek Dinner in Los Altos, California.  Erica Kawamoto Hsu / Girl Geek X

Transcript of TRI Girl Geek Dinner – Lightning Talks:

Angie Chang: Okay. Hi. Thank you all for coming out tonight. My name is Angie Chang and I’m the founder of Girl Geek X. If this is your first time, welcome. I’ve heard several stories of women who have got tickets for Girl Geek dinners over the last decade and missed one, two, three and four and they’re here tonight. So thank you so much for coming after a long day of work and having fun with us at Toyota Research Institute. I’m really excited to be here and learn all about the cool things. I wanted to point out at these actually really fun stickers. They have the robot arm and the car. If you want to pick them up, they’re not business cards. You can pick them up and take them home with you.

Gretchen DeKnikker: Hi everybody. I’m Gretchen also with Girl Geek. How is that food? Can we just all applaud for how amazing that was? You guys, Toyota has just killed it tonight unlike every level. It’s [inaudible] and I’m not jinxing anything. I swear. And if this looks fun, you can do it at your company too. We do these almost every week and a different company hosts. And so grab someone in a red t-shirt who can put you in touch with someone who actually did the planning, who can tell you how much work it takes, that it’s really fun, that you can bond with your colleagues, that you’re elevating women in your company. You have a cool video to submit to Grace Hopper next year. All these awesome things that can come out of it. So with that, how many of you it’s your first time? Okay, so we want to see you. You can come all the time now. You can do this all the time with all these awesome women. Okay, perfect. I would like to welcome Rita from team Toyota.

Rita Yau: Thank you, Angie and Gretchen. Hi, everybody. Thank you for coming tonight. Hopefully you guys are full with all the food and have a beverage in hand. A few shout outs. We have a few of our executives here this evening. We have our SVP of our autonomous driving organization, Ryan Eustice, as well as his two VPs, Wolfram Burgard and Steve Winston here with us as well. And then we have Max Bajracharya, who is head of our robotics organization. And so we’re really excited to have you guys and have them supporting us. And with that I just wanted to give the platform to our Executive Vice President and CFO, Kelly Kay, who’s going to go over our mission statement.

Kelly Kay speaking

EVP and CFO Kelly Kay gives a talk on “TRI’s Mission: Improving the Quality of Life” at Toyota Research Institute Girl Geek Dinner.

Kelly Kay: Welcome everybody. I’m super excited to have you all here today, and it’s been so much fun to wander about and hear everyone talking and learning more about what we do here at TRI. It feels like we’ve been living in the shadow and now that so many wonderful people have come to learn more about what we do, I’m hoping we could do more events like that here at TRI. I need the clicker. Okay. So I want to talk a little bit about improving the quality of life and TRI’s mission, but I’m going to actually start with myself and my journey that actually brought me here to TRI.

Kelly Kay: I actually went to Ohio State, go Buckeyes IO. So really excited, especially since we have partnerships with the University of Michigan and when I go to Michigan, I have to wear my Ohio gear. Sorry Ryan. I also went to law school, so I’m actually a lawyer. I practiced law for over 20 years before I made it to TRI and I did so kind of in a unique way. My life was actually really focused around taking traditional products and turning them into online products. So I started in the banking industry, so and I don’t want to date myself, but back in the day before you could actually log into a website and look at your bank balance or pay your bills online or apply for a credit card, I was working at a bank trying to make those products possible.

Kelly Kay: So it kind of progressed through my life of kind of taking these really old fashioned, old school regulated industries and turning them into something that was cool and new and innovative. And it kind of ended most recently at Lyft, where I was working as the VP of Operations and helping them actually take their product around America and working in negotiating with different regulators on how you actually take what’s traditionally a taxi industry and turning into a ride sharing industry. So it was really fun to help Lyft grow. And I came to TRI, actually, to be the Chief Operating Officer almost three years ago, and it hit me at just the right time.

Kelly Kay: So Lyft is an amazing company. I was doing amazing things, in my opinion, changing the world as it came to transportation. And when I was approached by TRI, they came to be in the first conversation was more about the mission at TRI. And I was in a really unique place in my life where my stepmother had just had two strokes. My father had fallen trying to help her get up. They ended up on the floor or calling me and calling 911 and they’re like, “Wow,” I’m like, “What’s going to happen to my parents when I live so far away and I have to find a way to actually make sure that they can take care of themselves at home because we can’t always be there and it’s really hard and expensive to afford a home to put them in.” At the same time, my dad could no longer really drive well because he’s disabled and I was thinking, what am I going to do?

Kelly Kay: And when I was talking to the recruiter, they were telling me about the vision and the mission at TRI and what it was all about. You guys have learned quite a bit about it tonight and I was thinking, wow, this company, I could go work at a company that’s actually going to change the future and enable people like my father and my stepmother to actually be able to stay home and age in place and not have to worry about caregivers coming in and being embarrassed by caregivers coming in and taking care of them all the time. And my dad’s not going to have to worry about how is he going to get to his doctor’s appointment or how is he going to get to the grocery store because he can’t drive anymore, because we’re going to have autonomous cars and I’m going to work for a company that can help solve these problems for people in my life that I care about, and ultimately for myself.

Kelly Kay: This probably isn’t going to be around when my dad is still alive, but it’s going to be around when I need it most. I’m an only child. What happens when I’m by myself at my house and I need to get to the doctor? So our vision and mission is really about that. We are actually envisioning a future where Toyota products dramatically improve the quality of life for everyone. And our mission is to develop automated driving robotics and other human amplification technology for Toyota in this space that will enable us to actually allow people like my father and your parents to age in place gracefully, to be able to still move around the home and have robots help them move around the home, and help us transport ourselves from point A to point B in a safe way through autonomous vehicles. So, our leadership team was another thing that really inspired me to come to TRI.

Kelly Kay: We’ve got some of the most amazing minds when it comes to autonomous driving and robotics here at TRI. And the best part about them, is I was very scared, I’m a lawyer, I’m not an engineer. What am I going to do? How am I going to sit at the table with these people because they are so smart and they don’t have the egos that you think these people would have. There’s a lot of doctors that you’re seeing. So most of these folks are PhDs and they really bring a lot to the table without the ego that usually comes with it. And I found that to be one of the other amazing things when I was thinking about coming to TRI, is the people I’m going to be working with really matter.

Kelly Kay: And we just spent three days together, everyone on this screen, actually, learning and thinking together about how to design the future of TRI and what should we be thinking about? How do we even be more innovative than we’re being today? It’s a constant question that we’re asking here everyday at TRI is how do we do more? How do we think about the next great thing that we’re going to do to help transform Toyota and the world today to be more mobile. And our values are another reason why I wanted to come to TRI. When I came here there were… Toyota has principles and it’s got these 10 principles that they bring from Japan and they exist and it’s really, how do you think about working at Toyota? But I wanted TRI to have its own values.

Kelly Kay: And what I did with the team was really think through what we want them to be. And I wanted to put a lot of myself into this because at that point in a lot of what my role is, is really about enabling the company as a whole to be more effective, to helping them design the future, to work with the HR team on the type of people in the culture we want here, to work with the engineering team to get things done, which should the processes be that we have to get things done, building all of that within the company. And the values really are the contract that we have. And when we think about how we work together at TRI, the be yourself value is the one that is the most important to me. And I’m kind of the sponsor of that value, if you could possibly sponsor a value.

Kelly Kay: And we spent some time during our retreat with a famous actor, a Japanese actor, and if anyone watched the TV show Heroes? Yes. So, the Japanese character that reads the comic book and goes back in time. So he came to our retreat with us and worked with us on actually being yourself even more and kind of almost being a child. And we were doing improv and he’s like, “Make a blue doctor machine.” And we’re like, “What?” So we all had to act out a blue doctor machine. It was really like anything you could think of to really be yourself. And he’s like, “Everyone is a genius.” And he really got us to think of our internal genius. So even though we all bring different skills to the table and we all have different levels of education, we actually are in each individual a genius in and amongst themselves.

Kelly Kay: So what my genius is may not be the same as somebody else’s, but we each have individual characteristics and at TRI, it’s really important that everyone has a seat at the table and a voice at the table to bring their unique characteristics to what we’re doing here. The next is respect one another. And that one is just as important as being yourself. And I think they balance each other out really well because if you’re a jerk, you still have to respect somebody else. So we think about these of kind of making sure that we’re really thinking about all of these as a whole and respecting one another is another one. It’s super important. Again, we might not all have the same opinion, but we want to hear everyone’s opinion.

Kelly Kay: We want to think about it, we want to debate, we want to make informed decisions. So we have to sit back and respect everyone’s opinion here at TRI. And the next is assume best intentions. And this picture is mine. And I will explain the picture to you. I foster kittens throughout the kitten season here in Silicon Valley. We do about 2,400 foster kittens every season here. And I have a German shepherd and he loves to play with the kittens. He’s always licking the kittens and things like that. And the question is, is this dog eating the kitten or is he saving the kitten? He’s in fact saving the kitten. And even dogs are basically good. So we’d like to assume here at TRI that people are basically good. We need to assume best intentions.

Kelly Kay: We’re working in an interesting environment. We have offices all over the world. A lot of email and email can always lead to miscommunication and misunderstanding. So we’d like to make sure that when we’re like, when we get upset, when we read that flaming email, it could be a cultural issue because people are speaking many different languages at Toyota. It could be an issue someone’s just grumpy because they worked late and they just sent something out they shouldn’t send. So we like to think about things of always step back, don’t get upset, assume best intentions, and that really allows us to interact in a different way here at TRI than you would find in other companies where you’re like, people come in very aggressively into meetings like, “Well, why did you send that?” We are like, “Hey, now I assume you didn’t really mean this, but this is how I took it. Let’s talk.”

Kelly Kay: So we just take a different approach to things here. And then thinking globally, again, we’re a global company. TRI’s a small part of a really large company. There’s 360,000 people at Toyota. We are 350, 340 people. So we need to make sure we’re thinking globally on everything we’re doing when we’re designing the car, when we’re designing a robot, we’re actually looking at it from a broader perspective than most companies would. And then finally, make it happen. We’re here, we’re in Silicon Valley, we need to move quick. Japanese companies are historically slow. TRI I was created to actually move faster than a traditional Japanese company. So we’d like to think of ourselves as kind of this company of making things happen.

Kelly Kay: That means taking risks, doesn’t mean jeopardizing safety, but it means taking risks and making decisions and things like that to allow us to move as quickly as possible. I probably completely failed to talk about what I was supposed to talk about, which is what I do here, but these are the things that I put together that are a main part of what I do here at TRI. I have CFO in the title. It’s about this much of what I do. I spend a lot of my time in meetings, just making sure things actually get done. Sometimes it’s coaching people, sometimes it’s helping with reorganizations, sometimes it’s figuring out what should we do strategically? How should we design a program? Why should we do things in a certain way?

Kelly Kay: So I spend a lot of time with the executive team, with anyone who wants to come and talk to me. I believe in an open door policy, and I work really closely with the CEO as well. So, if you want to know anything about TRI, I probably know it. If you wanted something technical you should probably talk to somebody else. But I am a work in progress. So a lot of the professors have taken me under their wing and have been teaching me a little bit more about the technology behind what we do. But again, I think we have a great lineup of some amazing technical people who are going to come up and talk to you and some of our really good leaders here at TRI. So thank you for all coming out. And again, if you have any questions, feel free to ask me or anyone on the staff. We’re super excited to have you here tonight and hopefully you learn a lot about TRI. Thanks. And I didn’t do everything and that’s what really matters.

Rita: Thank you, Kelly.

Kelly Kay: Do you want this?

Rita: All right. So I also just wanted to add to that, one of the reasons why I joined TRI was because of the amazing, awesome things that we do here. And everybody here at TRI really does have the same mission and goal to better the world. And so I’m super privileged to be working with a bunch of amazing, awesome people. And with that we are going to kick off our next round of lightning speakers and we’re going to have the next couple of speakers come up. And Carrie Bobier-Tu, who is our manager of the… Or one of our managers in our autonomous driving organization is going to come up and give us her overview of why she’s here.

Carrie Bobier-Tiu speaking

Manager of Control, Planning and Control Driving Team Carrie Bobier-Tiu gives a talk on “Building the Uncrashable Car” at TRI Girl Geek Dinner.

Carrie Bobier-Tiu: Thanks everybody for coming. I’m really glad to see so many people here. My name is Carrie Bobier-Tu, and I’m the manager of our control team, which is part of our autonomous driving team. And I’ll talk a little bit more about what that means in a minute. But first I’m also going to tell you kind of how I got to TRI. So, the first kind of engineering project, hands on project that I worked on was, I was a member of the Solar Car team at Stanford University when I was an undergrad. So it’s the picture of upper left for you guys.

Carrie Bobier-Tiu: But I worked, as you can see, on a team of me and a bunch of guys, pretty much, and we built this solar car and raced it from Texas to Canada–cutting out–over a week one summer. And that was kind of my entry into cars and my love for cars. A lot of hands on experience there in engineering and also met who would become my advisor for my PhD and kind of build the next many years of my life at Stanford. So Chris Gerdes is a professor at Stanford in vehicle dynamics and control, and I started talking to him about suspension design for the solar car, really got along with him and his students [inaudible] lab and ended up staying in the lab for the next eight years to do my masters and PhD there.

Carrie Bobier-Tiu: The two cars that you see that kind of looked like black dune buggies are the vehicles that I worked on there. The top one I used for my research, which was based around safety systems and advanced stability control and how we can kind of take the stability control that we have and vehicles on the road to the next level by having advanced sensing capabilities. And I also worked on building a new platform, which is the car on the bottom, two students and many after us, kind of, we worked on this car together and I worked on designing a suspension system that can enhance friction estimation capabilities. So I really got deep into vehicle dynamics and engineering here. I was really hands on with it.

Carrie Bobier-Tiu: I liked building cars in our test beds and working with all the sensors and computers, but when I graduated not that long ago, even, there weren’t really any autonomous driving or vehicle control jobs in the Bay Area. So I went to work at HGST, which is a hard drive company. It’s now part of Western Digital. But I am really glad that I went and worked there because I got the experience of designing a control system for a product that you had to have this controller that worked on millions and millions of devices that were going out to customers. So I think having that product experience early in my career was really helpful to me in seeing how to design a robust product and a robust controller. I also got into CrossFit there and got to work with my husband, who’s the second from the left in that picture with all of the red shirts.

Carrie Bobier-Tiu: They had a CrossFit company, or CrossFit gym, at the company, which was really cool and kind of has spurred my interest in health since then, which I’m really appreciative. But I really missed working on cars. So after a couple of years, the autonomous driving business was kind of starting to pop up and I went to work with a few of my old lab mates at Renault, which is the sister company of Nissan for those of us in the United States who aren’t familiar with Renault. But they have a small outlet here in Sunnyvale. So we built this a test vehicle, Callie, the white car with the stripes, and it was just a team of three of us. We were working on controls research. It was basically an extension of my PhD. We were all kind of coming from that same background.

Carrie Bobier-Tiu: So I had kind of the research experience there that I was really enjoying, but it wasn’t going anywhere into a product or out into the company. And we didn’t have a lot of support from the Renault or Nissan itself to do that. So I was starting to look for something different. And what drew me to TRI, it was a couple of things. One was that continued ability to do the research that I was really engaged with, and continuing on for my PhD and bringing that experience with me. And the other piece of it was the ability to work on a safety system that would go into real vehicles. So around the time that I was at Renault, I had a son. So it became very important to me.

Carrie Bobier-Tiu: It’s kind of like figuring out how to make cars safer for my family and for everyone that I know and since I have that background and expertise, I was really excited to come here and work on the guardian system that we have, which I’ll get into in a minute. The other thing that’s really great about TRI is our ability to do research with universities. That’s been mentioned a few times before, but this is the bottom of the last picture is me with some of the students at Stanford. We went out and got to drivers around with one of Toyota’s Drift drivers.

Carrie Bobier-Tiu: So, TRI, we have kind of two approaches to automated driving. So we’ve talked, think of it as one system, but two modes that are built on the same technologies. The first one is Guardian, which is a parallel autonomy system where the driver is still in control of the car. So we try to follow the driver’s intended commands but with minimal and intuitive interventions to maintain safety. And then the second half of our automated driving stack is the one that more people are familiar with, in general in the industry, which is the fully autonomous system where the autonomy system is determining some kind of policy for the vehicle to drive.

Carrie Bobier-Tiu: And we calculate commands that can maintain the safety of the vehicle, like for steering, acceleration and braking. So how does my team’s work fit into this? In the autonomous driving stack, as we call it, which is kind of the full set of software that runs the autonomous vehicle, we have some large groups of kind of algorithmic expertise or design. So there’s a perception system, which is taking in all the information from the sensors of the vehicle and figuring out what does the road look like, who’s on the road around us? And then on top of that comes the prediction system, which is, we know what’s happening now, but what’s going to happen in the future?

Carrie Bobier-Tiu: And finally, planning. So what should the car do, given what the environment looks like or in the Guardian case, what do we think the driver’s going to do in the near future? And control is a part of this planning problem. So what my team does, is something that we call Envelope Control. It’s something that I started developing, as I mentioned when I was doing my PhD. Envelope Control is a holistic control scheme that keeps a given system inside a safe operating regime or envelope. So we have a few things that we have to do. One, is stay on the road and don’t hit anything.

Carrie Bobier-Tiu: Two, is to maintain stability of the car. So, for example, if you’re driving on an icy road, you can lose control of the car and spin out. So we’re trying to prevent situations like that or situations where like a kid runs out in front of the car and you might not have time to stop, so you have to swerve around them. We also don’t want to ask the car to do something it can’t. So if it can’t swerve around for some reason, there’s something in the way, don’t ask it to do that.

Carrie Bobier-Tiu: If we don’t have enough steering capability or enough or the ability to brake fast enough, we can’t ask the car to do that. So we need to know what the limits are. And finally, for the Guardian system, we want to give the driver as much control of the car as possible, but help them maintain the safety of that vehicle. So the technology that we build kind of incorporates all of these things together. And that’s how we, here at TRI on the control team, are trying to create an uncrashable car. Thanks.

Ha-Kyung Kwon speaking

Research Scientist Ha-Kyung Kwon gives a talk on “Accelerating Materials Discovery by Helping You Fail Faster” at TRI Girl Geek Dinner.

Ha-Kyung Kwon: Hi, my name is Ha-Kyung Kwon, research scientist on the Accelerated Materials Design and Discovery or AMDD team here at TRI. So I’m also going to talk about my path to TRI, but I’m going to start from the very beginning. So I was born in Seoul, South Korea. Spent most of my childhood and adolescent years in Manila, in the Philippines, where my dad’s job took us. And it was fantastic growing up in the Philippines. There’s warm tropical weather, great food, white sandy beaches, which was on the picture there. But my favorite part was meeting friends from all over the world.

Ha-Kyung Kwon: And in fact, when I’m not here talking about science on a Thursday night, I’m out there playing flag football with my high school friends, their college friends, and their friends. So anyway, when I graduated from high school, I went to Princeton to study chemical and biological engineering. And my first year at Princeton I started doing research in an organic solar cells lab, and I loved it so much that I continued to do research in the same lab for the next three years. When I graduated from Princeton, I decided that I wanted to do even more polymer science research.

Ha-Kyung Kwon: So I went to Northwestern to get a PhD in material science and engineering. My PhD work was on the face behavior of ion containing polymers. These are polymers that are great candidates for polymer or plastic batteries. Plastic batteries, you might ask. But think about all the plastics or polymers that you know, and their rich properties, like styrofoam, which is light but rigid, nylons and polyesters, which you can wear, and Kevlar, which is extremely tough. It’s Bulletproof. Think about this wide array of properties that polymers have and imagine the possibilities. A polymer battery that is flexible, lightweight, safe, and even recyclable is not out of the question, but it’s going to take many, many years before we can get there and that’s because the scientific research process is incredibly slow.

Ha-Kyung Kwon: First, you have to understand the problem. What are the technical challenges of making a polymer battery? What’s been tried, what hasn’t been tried? Then you have to formulate a hypothesis. Maybe this material has a mechanical strength in the ionic conductivity that’s relevant for a polymer battery. Then you have to figure out how to make that material. Then you test it to see if it has the properties that you want. Once you have the results, you analyze them. Did your hypothesis work? Yes, no, maybe you don’t know. So you have to repeat the entire process. And this process is slow, not only because each step can take a long time, but because many scientific hypotheses end in failures, and that’s part of the process.

Ha-Kyung Kwon: It’s what enables us to learn, to refine our understanding of science and to take a step in the right direction. But this necessary process is unnecessarily slow. In a traditional industrial lab, the R&D cycle can take anywhere between five and 25 years, even more. And we don’t have this kind of time to solve the challenges that we face today. As of 2017, transportation accounted for more than 29% of greenhouse gas emissions in the United States, and more than 82% of that came from driving. The technologies that we have in our cars and trucks today simply aren’t cutting it. For a sustainable future we need new materials and new technologies, and we can’t wait tens of years. So how do we accelerate this materials discovery process while enhancing scientists’ ability to learn, to discover, and to advance scientific knowledge?

Ha-Kyung Kwon: And that’s where my team comes in. Using big data, machine learning methods, and high throughput automated experiments that are driven by these methods, we develop tools to accelerate the design of advanced materials for zero emission technology, such as batteries and fuel cells. Our tools accelerate materials discovery by helping researchers fail faster. And what does that mean? Here are some of the projects that our team’s been working on. Matscholar uses natural language processing that contains over… Sorry. It contains information from over four million scientific abstracts. In the matter of minutes, it can help you discover whether materials that are similar to yours in composition, property, or application have already been studied.

Ha-Kyung Kwon: Something that without this tool could have taken you years, if not decades, to achieve. Using machine learning methods, we’ve also created beep, which can predict the lifetime of a battery from just the first hundred cycles. And this saves a ton of time on the battery development cycle because currently in order to test a battery’s lifetime, you have to cycle it until it dies, which can take more than thousands of cycles. In addition, using optimal experiment design, we can recommend the next set of experiments to run, given the last set that you ran and its results. We can even teach a scientific tool the scientific method in CAMD.

Ha-Kyung Kwon: Given an objective, such as discover a new stable material, it can formulate its own hypothesis, launch simulations according to that hypothesis, refine its hypothesis according to failed simulations, and keep running them until it discovers a new stable material, and it does the leg work so that you as a researcher don’t have to start from ground zero. As you can see, our work in AMDD really spans many materials, many applications, and many scientific disciplines. And this brings me to my favorite part, the people. Our team really brings together scientists and engineers from a diverse set of backgrounds anywhere from computational physics, applied math, to software engineering, who are passionate about discovering materials and accelerating materials discovery for zero emission technology.

Ha-Kyung Kwon: Our tools are interdisciplinary because they’ve been developed, tested, and used firsthand by researchers with diverse backgrounds: us. We also maintain deep connections to fundamental science. Our tools really draw from and build upon the intuition and knowledge of scientists and seek to empower scientists in their learning. And to do this, we work very closely with our consortium of more than 10 university partners and 125 researchers from those academic institutions. Our goal is to accelerate materials discovery, not just for the autonomous vehicle industry, but for the scientific community as a whole. By building tools, we’re building connections and communities because we believe that working together and failing together is the fastest way to sustainable solutions. Thank you.

Rita Yau: Thank you, Ha-Kyung. All right. Okay. To kick off the next section, we are going to be having Jen Cohen, our VP of Operations to tell us about her journey.

Jen Cohen speaking

VP of Operations Jen Cohens gives a talk on “I am the IT Guy” at TRI Girl Geek Dinner.

Jen Cohen: Hi, everyone. I’m so glad you’re here tonight. Thank you, Rita, for the introduction. My name is Jen Cohen, I’m VP of Operations for TRI. You’ll notice my presentation is I am the IT Guy. I promise I’ll explain that. How many people here are in IT? All right, I guess I kind I am. Ian, did you raise your hand back there?

Ian: Halfway.

Jen Cohen: Okay. Halfway. How many of you have had to say, “I am the IT guy?” Okay, so it’s not just me. I feel better. So I’m going to talk to you a little bit about my journey, a little bit about operations, and some of my hacks for high performance teams. So over the years I’ve had to say I am the IT guy. And recently I had to do a presentation talking about my career, and I realized that not only have I had to say it a lot, but I’ve actually learned to love it. I love that surprise when they realize that it’s actually me who’s going to be giving them the answer. And, you know, Facebook’s great about reminding us of what’s happened in the past. So I found this memory recently from 2011.

Jen Cohen: So apparently I’ve been saying the same thing for a while, “Wow, just had another vendor say, ‘Your IT guy needs to…’ It’s funny and sad that they never assume it could be a woman.” But at the end of the day, I love it now. I’ll share that way back when those were popular phones, I started my career as a sys admin at Cisco and I grew my career to do IT management at companies like Smith and Hawkin, which is gone now, but Birkenstock, anybody rocking the Birkenstocks tonight? The big question is do you have socks on? And then I grew from there to do technology development and platform management.

Jen Cohen: So platforms in the convention and real estate industry and the gift about doing technology, the gift about being in IT is that you get to see problems across the entire organization. And I really love to problem solve, but I realized technology wasn’t going to be enough. And so recently, a few years back, got into operations management at Line2, and then here at TRI. I will also share with you that I’m a mom. I have a daughter, Sabrina, who is a junior in college. She’s a computer science major, but she thinks she’s totally different than me. And I have my son Logan, he is a senior in high school and he’s an artist. And based on these slides we know he didn’t get it from me.

Jen Cohen: And then I’d like to talk to you a little bit about operations at TRI. So the teams that I am either responsible for or support some of them, IT of course, because I am the IT guy. But there’s more than me and the amazing team that does that work. CyberSecurity. I co-sponsor the Infrastructure Engineering team. So, that’s essentially a fancy way of saying dev ops. Facilities, Internal Comp, I’m not going to go through the list, but a good group of people who support a lot of fun things here. We have three sites in California, in Michigan, and where am I missing? Massachusetts. I can’t believe I forgot that. I was born there. I should remember.

Jen Cohen: We manage over a hundred key systems, and we support over 300 employees, contractors, and interns. And I decided to show you a picture of some of the amazing folks we have in our ops team because while I say I’m the IT guy, these are the really, the folks who make it happen. And the challenges that we have as Operations at TRI are speed. And while I’d like to have that car, that’s not actually the speed I’m talking about. We move really fast. Our researchers, they need things. We need to make sure that they have what they need at the right time, and we need them to be able to move quickly. We need to be, as has been mentioned, able to fail fast, and we need to get it done at my absolute favorite deadline, which is yesterday.

Jen Cohen: Ask anyone on the team. But we also have to have balance. Part of our job is to protect the company, to protect ourselves. And so we need to make sure we have things like cybersecurity. At the same time we need to enable people to use the technology and get out of their way. We need to have freedom within constraints. And we have these amazing, really smart researchers and software developers. And so we want to make sure that they have the time to work on the things that they’re there for and don’t have to build the technology. But at the same time, some of them know far more about the tech at their desk than we do. So we want to make sure that they have the self-service.

Jen Cohen: So we’re not blocking them. And finally we want to make sure that we’re flexible because there are things that are failing fast, and we need to make sure that we can pivot when the direction of research changes. And we need to build platforms that don’t pin us into a corner. So flexibility. So that’s a little bit about TRI, and the fun things that we do in our Operations team. And then I will briefly share with you three of my hacks for building high performance teams. So how many people here have heard of the concept tank in relation to support teams? Lauren, you don’t count. You know what that is. Anyone else?

Jen Cohen: So learned about tank, a few companies back at a PagerDuty summit, and the idea comes from video games, and I hate to say it, but it’s the person who takes all the hits in a video game, right? So from a support perspective, we have tier one and we have tier two and we want to make sure that our tier two teams who are handling escalations also have time to do the projects that they need to do to help our researchers be successful. And so if they’re handling escalations all day long, they’re not getting the chance to do their project work. So the idea of tank is that one day a week, each sys admin takes on the escalation, they get all the interruptions, they have to deal with it, they don’t get to do their project work, but they get four days the rest of the week relatively uninterrupted.

Jen Cohen: The nice thing about this is it absolutely forces cross training. So if Ian is the only one who knows how to fix the mics and Ian is on vacation, somebody else knows how because they’ve had to handle it on their escalation day. So that’s probably the most powerful thing I think we’ve brought to IT and Infrastructure Engineering here at TRI. My next hack is to celebrate wins. Now I know this can sound a little bit Pollyanna, but I will say, how many people here are problem solvers, and how often do you think about the ones behind you that you finished? Are you mostly looking at the ones coming forward? Yeah, so the problem with that is it’s easy to burn out. So I think it’s really important that we celebrate our wins, not just look forward to the next problem.

Jen Cohen: So in our weekly meetings, our teams list their wins first, so we get a chance to memorialize them and then we list our challenges, so that we have that moment of really acknowledging the work that we’ve done. The other nice thing about this as we put it into a deck that we can go back and look at, because I don’t know about you, but I have no idea what problems I was working on in January, February, March. Anybody here remember theirs? So the nice thing is because we memorialize this, we can go back at the end of the year and look back at what we’ve done and have that moment to remember. And I think that’s really important for support teams, especially, to keep moving forward.

Jen Cohen: And my final hack, and I am the IT guy and I love my technology, but is get off your keyboard. How many of you have been on the email, the GChat, the Slack that has turned into a book? Yeah, not just me. One of the things I found a few years back, I was working with these two developers who were, I think, on GChat and they were going at each other, but they were saying the same thing. They just didn’t realize it. We got them on the phone and within five minutes, they realized they were saying the same thing. The argument was over and they were coding the solution. And I realized at that point how about some part of my job is making sure that people connect.

Jen Cohen: So how many of you have seen the keyboard warrior at work? How many of you have been the keyboard warrior? I’ll admit it, I’ve been there, I’ve sent a flame email I probably shouldn’t have. So the reason I put this up here, and whether you’re a manager or you’re an individual contributor, doesn’t matter. Get off your keyboard. If that’s starting to happen, get on a call. I hate the phone too, but get on a call, get on a VTC, go to somebody’s desk. Because that really will help to work out those problems. And I will also say use Kelly’s tip. Assume best intentions when you do. All right, that’s it for me. Thank you guys so much.

Fatima Alloo speaking

Legal Counsel Fatima Alloo gives a talk on “Navigating the Intersection of Law and Technology” at TRI Girl Geek Dinner.

Fatima Alloo: Hello? Looks like it’s working. Thank you. Hi, everyone. My name is Fatima Alloo and I’m part of the legal team here at TRI. Thank you for coming. So many of you. So, I actually will be talking about navigating. It’s actually a less daunting presentation that’s, than my title might indicate. But what I really want to do is share a little bit about what it is that I do here and some of the awesome issues that we get engage on here as part of the legal team at TRI. And, first I’ll go ahead and start with my career background. So for me, it all started a long time ago when I graduated from law school. And there I am with my parents who are super proud and excited at the time.

Fatima Alloo: After law school, I went into patent litigation, and essentially, I was defending clients in patent infringement lawsuits. So that meant that I had to get quickly up to speed on the mechanics of various technologies, including fun topics like semiconductor fabrication and audio and visual signal processing. And while I loved it, the more I interacted with various tech companies, I started realizing that I was more interested in how the technologies that I was defending were actually developed. So I knew someone at an augmented reality startup called Meta, and it turned out that they needed some legal support. So, I convinced my law firm to second me there, part-time.

Fatima Alloo: And in short, I absolutely loved it. And since that time, I was just so eager to find a way to work full time for a cutting edge tech company with a heart. And that’s how I ended up here at TRI. Now, from all in working with these clients and companies on various, on existing and new technologies, what I realized is, I actually discovered something about the law. And what I realized is that while it’s obviously really important as a lawyer to know what the existing laws are, the law is actually a pretty dynamic and adaptable and can actually be shaped by individuals in this space. In short, the law can actually be fun. Surprise, surprise.

Fatima Alloo: So let me take a quick poll. Who gets excited when you hear the words, “legal’s involved”? Wow, thank you. Wow. It’s more than I expected. Most of you probably think something closer to this. And don’t worry, I’m not going to take any offense. Sometimes these feelings or thoughts are justified, but at TRI, as part of the legal team at TRI, we like to see ourselves a little bit differently instead of trying to attack or pounce on your project, we’re here to support it. And while I can’t help you with the technical side of things, what I can do is amplify your voice in how the next generation of tech is received. I can enable you to partner with other players in this space and I can help ensure that all of your hard work is properly secured.

Fatima Alloo: And ultimately for me, this is what makes it super rewarding to be part of legal team here at TRI. Now, I’m sure all of you are just itching to know, what does my day to day role look like as a lawyer at TRI? Fear not. I have put together three words to describe what I do here, and my team does here. Pioneer, partner and protect. First as a lawyer at TRI, we get to pioneer and all of you have heard Carrie’s presentation, all of the awesome work that we’re doing in the automated vehicle space. And we all have some sense in terms of how automated vehicles are going to disrupt the automotive industry. But the big question on the legal side of the equation is what should the laws and regulations that govern automated driving look like?

Fatima Alloo: What standards should manufacturers that are making automated vehicles adhere to? So, imagine with me for a moment that you are in an automated vehicle and it’s taking you to your destination, but for some reason you need to stop abruptly. Where is the stop button? What does it look like? What color is it? What shape is it, where is it located? And is it located in the same place across vehicles made by different car manufacturers? These all might seem like trivial questions, but it’s important to build a consensus with commercial players in the space for the industry to flourish. Now, we’re super lucky here at TRI because we’re part of Toyota and one of the biggest automobile makers in the world.

Fatima Alloo: And because Toyota is also part of the automated vehicle space, we actually get a seat at the table in determining how these laws are developed. And as a lawyer, consensus building, negotiation, drafting laws and regulations and standards, those are right in the wheelhouse of my skillset and our skillset. But what we need to do is hear and understand from our engineers on what they think the solutions to be to issues like this. And once we do hear from them, we can actually advocate on their behalf. The second thing that we get to do is partner kind of like C3PO and R2D2. Anyway, so as you’re on your way to bringing, onto building groundbreaking technologies, you’re going to need some support. And while we have many brilliant minds here at TRI, many of whom you’ve heard from and will hear from, no company can do this alone.

Fatima Alloo: So you might need to find support outside of our company. Maybe you want to partner with a university or a consultant or a startup that’s developing a component that you just need to have to make your solution come to life. Our job then becomes to make that partnership happen, support the development of your tech, and then to think through about whether this project is really in the business’s best interest. So let’s say, for example, you’ve decided to partner with one of our universities and, as Carrie mentioned, we partner with so many universities, and Ha-Kyung. And so let’s say one thing you might want to think through is, what does each player want to get out of that deal?

Fatima Alloo: The university might want to ensure that they own the IP that’s generated in a joint collaboration. TRI might then want to ensure that we have licenses to that jointly developed technology in case we want to commercialize the tech down the line. The point is, that as lawyers, we often have to think through these situations and then memorialize these agreements in writing. The last thing that we do is protect. Now while you’re engaging with different partners, one big question for TRI is how do we make sure that we’re protected in the process? So I often, am asking several questions, basically. For example, does the partner have access to our systems, data, or code?

Fatima Alloo: Has a partner agreed to be liable if they fail to protect our systems, data, or code? And, sometimes it’s more along the lines of are their cybersecurity standards strong enough to actually guard our code? Occasionally I ask the question of–sometimes the questions are very different in, might be something closer to, as a TRI employee, you think copyrighted images that they don’t have a license for when giving a presentation before hundreds of people. The point is that because we’re in a very hot space, that being self-driving car research and robotics for mobility, we’re subject to a lot of both malicious and inadvertent threats that could cause a company like ours to lose their competitive edge.

Fatima Alloo: And for you Black Panther fans out there, I like to think of our role as protecting the secrets of Wakanda. So, now that you’d have a better idea of what it is that I do here at TRI, and what a lawyer does more generally at a tech company in terms of pioneering, partnering, and protecting, I hope that the next time you have a project you might be the one to get legal involved, and see how we can help you get to wherever you’re going. Thank you.

Rita: Thank you, Fatima. All right. With that, we’re going to welcome Steffi Paepcke who is a Senior UX Designer on our robotics team to the stage.

Steffi Paepcke: All right. Am I on? Can you hear me?

Audience Member: Yes.

Steffi Paepcke speaking

Senior UX Designer of Robotics Steffi Paepcke gives a talk on “Designing Robots to Serve an Aging Population” at TRI Girl Geek Dinner.

Steffi Paepcke: Yes. Okay, great. Cool. Hi everyone. My name is Steffi Paepcke. I’m a Senior UX Designer here. I work on the robotic side of the world and I don’t want to leave you hanging. I’m going to tell you how I got here as well. I started by studying psychology at UC Santa Cruz and after that I kind of didn’t really know what to do. I thought about being a therapist. That had been sort of my goal for a long time. And I wound up at Willow Garage as a research assistant.

Audience Member: Oh.

Steffi Paepcke: Oh, someone’s heard of Willow?

Audience Member: Yes.

Steffi Paepcke: Yeah, cool. Willow Garage, for those of you who don’t know, it was a now defunct privately funded research company. We did all kinds of really exciting work. The PR2 robot, we made turtle bots. We made what now is beam telepresence robots and we did a lot of the maintenance, the primary maintenance on ROS robot operating system and it was at Willow Garage, it was really a pivotal position for me is where I realized that I can combine my interest in humans and how they think and feel and interact with other people and objects. I could combine that with technology and in this case robotics.

Steffi Paepcke: And that was a really big sort of turning point for me where I kind of found robotics is the field that I wanted to work in. I realized also that I needed more training. So I went to Carnegie Mellon and received a Masters in Human-Computer Interaction. And then after that I came back to the Bay Area where I grew up and co-founded Open Source Robotics Foundation, which is now just called Open Robotics. And they are now the primary maintainers of ROS and Gazebo, which is a physics-based robot simulator. One of the biggest projects OSRF worked on was the DARPA Robotics Challenge.

Steffi Paepcke: Which was a little while ago now, but it was a very impressive program put on by DARPA where teams competed in search and rescue tasks with an Atlas humanoid robot or with a robot that they had built themselves. And the program manager of that project was Gill Pratt, who is the TRI CEO now. So I ended up here and have been working on really exciting robots since arriving about three years ago. So I’m part of the UX team. We have user experience researchers, designers and industrial designers. And our main goal is to help TRI figure out what sorts of robotic capabilities to make to improve the quality of life for an aging population.

Steffi Paepcke: So you’ve probably heard that the population is aging relatively quickly in the world right now, approximately 8% of the population is 65 and older. By 2050, that’s supposed to double to 16%. And in Japan, this problem is the most pressing. That’s where the population is aging the fastest. Currently about a quarter of the population in Japan is 65 and older. And by 2050, that number is supposed to be one third. If you think about it, that is staggering, it’s one out of three people will be 65 and older in Japan in 2050. So it’s critical that we find solutions also to the shortage, the caregiver shortage.

Steffi Paepcke: The goal is to make robotic capabilities that can support older adults aging in place longer, taking care of tasks they don’t want to do anymore or can no longer perform. And also alleviating some of the sort of day to day tasks that caregivers need to take care of. So alleviating the chore-like tasks so that they can focus on the human to human interactions that really make caregiving what it is. So that’s our main goal as a UX team here and user experience as a field has become a lot more prevalent in tech companies over the years. It took a little while for companies to really understand that UX was a critical part of creating a successful product.

Steffi Paepcke: And it’s been similarly slow now with robotics as more and more robotics companies crop up. Some of them have user experience teams, a lot of them don’t. I think hardware is obviously very challenging and takes longer than software in terms of development process, but it’s really critical that we have UX in the workflow from the very beginning because you can spend a whole lot of time creating a hardware solution and then you get to the point where you realize you were solving the wrong problem or a problem that doesn’t even exist and then you’re really sort of in trouble.

Steffi Paepcke: So I’m going to walk you through some of the methodologies we use to combine user experience with robotics. What you probably have heard of is just interviews and focus groups. So we do those. Those are pretty standard in UX and we also do participatory design sessions, which is when you work with your target population, in our case, older adults, to come up with solutions together. So you’re not just doing the research and then going back to your office and coming up with the solution. You’re actually sitting down with an older adult and designing something together and co-creating it. Another really valuable methodology we use is called contextual inquiry, which is when you follow someone around and observe them doing a task that you want to learn more about without really interrupting, just sort of asking questions so that you understand the process.

Steffi Paepcke: And in our case we wanted to understand the grocery shopping process for older adults. So we followed them, we met them at their house, followed them in their car to the grocery store, did the whole loop around the store, came home, watched them bring the groceries in, put them away. And it was very illuminating. You can see on the bottom there’s one kitchen that we saw, which has pretty much no mess in it. It’s pretty sparse, a lot of cabinetry, really spacious. And then compared to the kitchen above, it was a very small little apartment with items stacked on the walls and the cabinets were very full.

Steffi Paepcke: So it’s important for us as designers to understand the workspace that our robots will be functioning in, but also very important for the engineers to see what sort of dynamic environment their robot needs to be successful in. You can also see the white cabinetry is pretty reflective, which can cause problems for certain sensors on robots. This is all really important information to bring back to the engineering teams and we try to bring at least one engineer with us when we do these visits so that it’s not just us sort of regurgitating what we saw, but really bringing them along for the ride. Another valuable insight we got was the image with the fridge.

Steffi Paepcke: So we opened the fridge and notice that the woman we were chatting with had kept all of her items at the very edge of the shelves. And when we asked her why that was, she said, “Oh yeah, I just can’t bend over and reach in.” And it’s not something that I would have asked about, “How far back do you put your items on the shelf?” But by being there and really observing it firsthand, we’re able to understand that, that’s one of the problems that comes up a lot. And that turned out to be a really big trend. Being able to stoop safely and without pain is something that is challenging for a lot of older adults, which is sort of common sense, but it helps to see this in the context of people’s lives over and over again. It really drives it home.

Steffi Paepcke: Finally, we do a lot of home walkthroughs, which are probably my favorite. We find people who are pretty open to sharing their lives, which not everyone is, but we meet them at their home and we go into every single room in their home and talk about the tasks that they do in their home, the challenges that they face and the goal there is to figure out if there are any things that we can create solutions for to help them. This again, is really good for context setting for us. There’s one, I apologize that it’s so small, but there’s a person, she wheeled an office chair onto her little patio and she weatherproofed it with plastic bags and whenever she needs to reach the hose that’s on the ground to water the plants, she sits on the chair, lowers herself down, reaches for the hose, raises herself back up and does it that way.

Steffi Paepcke: And that’s not ever something I would have thought to ask about. How do you reach the hose on the floor? But we, by being there with her, we got to witness the trouble she goes through, right? To do something as what I would consider simple as picking up a hose. It can be a real challenge for some people. She also is the owner of the closet next to her and she said that anything above about shoulder height she just pretty much consider as lost and she doesn’t ever expect to get to it again. Yeah. So you really learn about the challenges very viscerally that people face. And then finally, I really like the dishwasher down below.

Steffi Paepcke: This was another participant who doesn’t generate enough dirty dishes to need to run the dishwasher. So she stores her plastic bags and her plastic containers up there. And again, she doesn’t use the lower rack because that’s too far down. So we learned a lot about the importance of designing robots that can reach areas that older adults are not able to, or people with different physical abilities. And yeah, so these are some of the methodologies that I think are really critical in getting on the right track to making a robot that actually solves real problems. You can get pretty far with the interviews, but the data is just much richer when you can actually follow people around. And whenever I have a captive audience, I like to make a plug for getting more diverse folks into robotics. Robots are going to be everywhere in our homes, all over.

Steffi Paepcke: We’re going to be riding around in them. And if robots are not designed by a very diverse group of people, they’re not going to serve people equally and fairly. And we’re at the point now in robotics where it’s really starting to pick up. And if we don’t have diverse designers working on these challenges now, it’s going to take a really long time to catch up in the future. So there’s a niche for everyone really in robotics. You can come at it from the law perspective, mechanical software, electrical engineering, design, psychology. There’s so many ways that you can contribute to the robotics field. And if you’re thinking about making a change, I really encourage you to consider the robotics field and just getting involved. It’s a really exciting time to be a part of this industry, and that’s all I got.

Suzanne Basalla speaking

Chief of Staff Suzanne Basalla gives a talk on “2020 Olympics Showcases Mobility and Inclusion” at TRI Girl Geek Dinner.

Suzanne Basalla: Good evening everybody. I’m Suzanne Basalla. I’m Chief of Staff here at TRI and I’m going to talk to you about mobility and inclusion and really talk about inclusion from two different lenses. Like the speakers before me, I want to tell you a little bit about my path to TRI, which is a little bit different. I did actually take a fair number of STEM courses when I was in college, but I majored in Asian studies and the reason I had STEM courses is because I joined the Navy right out of college and spent 13 years as an intelligence officer in the Navy. And so I took the engineering and physics classes you need for that.

Suzanne Basalla: And the Navy is what gave me the opportunity to go to Japan. And what you’ll see about my career and what brought me to TRI is I’m very passionate about working with US and Japan, both countries, and bringing the best from both countries to solve problems and the issues that are super important to both of our countries, whether it’s our economy or our national security or issues like that. So the Navy, I was with the Navy in Japan for four years and I really fell in love with Japan. And more importantly, I found my passion, which was to really work at that intersection between the United States and Japan, and committed my career to alliance management, which is really focusing more on the national security side first of the relationship.

Suzanne Basalla: And so through my career I’ve had a chance to work between Japan–Tokyo and Washington, DC, mostly on the relationships. So I had a chance to work for brief time at the White House, worked at the department of defense where I was the Japan director working on our defense relationship. And then I also had a chance to serve as the Senior Advisor to our ambassador in Tokyo. And that was a really pivotal time in my life because I was in Japan on March 11th, 2011, which if you may remember, was the triple disaster of the earthquake, tsunami, and the nuclear disaster and was part of the US government’s response on that.

Suzanne Basalla: But I actually then really realized that I wanted to get involved in the nonprofit side of… I’d gotten [inaudible] nonprofit in order to really help the people because fundamentally it’s the people of our two countries that make the relationship strong. So I spent five years as the COO, EVP of a nonprofit, the US-Japan Council. And the bulk of my work that I was doing was helping the next generation for US-Japan relations, particularly a lot of work on women and girls empowerment, which was really exciting for me. Now working in US-Japan relations, Toyota… I, of course, got to know Toyota. Toyota is a global brand.

Suzanne Basalla: If you’re in Japan, Toyota is such a dominant company. There’s even the headquarters is in Toyota City, which gives you a sense of how important Toyota isto Japan. It’s also really important the United States. We have plants in 10 states in this country and many, many jobs are created through Toyota. So when I was invited to come and work at Toyota, at TRI, for me it was a huge opportunity to continue to my work on really solving the most important problems before our countries working in an exciting space of AI and thinking about the important economic and social issues from robotics and automated driving that you heard about earlier tonight.

Suzanne Basalla: So I am Chief of Staff and a lot of people ask me, “What is Chief of Staff?” And I usually tell them that means I’m a Jack of all trades. But today I’m excited to say I’m a Jill of all trades. And so I want to talk to you about two areas that I get to focus on, what my job is to really follow the priorities and strategic issues that are important to my CEO. And the two I want to talk to you about, one is the Olympics. So I hope you all know that the Olympics going to be Tokyo next summer, Tokyo 2020, and Tokyo plans for that to be the most innovative Olympics in the history of the Olympics. You may or may not know that Toyota is the most… The largest sponsor of the Olympics and Paralympics and Special Olympics in history.

Suzanne Basalla: And we’re going to be sponsoring the Olympics all the ways to at least 2024, the Paris Olympics. Here at TRI we’re really privileged because we’re working on a lot of the technology that’s going to be shown in Toyota’s demonstrations at the Olympics. So the bulk of the driving team and much of the robotics team and others in the ops and PR part of TRI are focused on getting ready for Toyota’s presence at the Olympics next year, which is really exciting. My job as Chief of Staff includes kind of helping the leadership team organize across the company so that we are prepared and doing what Toyota needs. And then kind of my sweet spot is engaging with the external stakeholders, especially in Toyota.

Suzanne Basalla: But I also get to work with the Olympic committee. I get to work with other companies that are sponsors, such as Visa or Intel. I also get to work with the broadcasters, and really cool, as occasionally I get to interact with some of the athletes and the Paralympians and they’re super inspiring. Toyota is sponsoring the Olympics because it wants to transform ourselves as a company into a mobility company. And what I’ve been really proud of is, through my work, seeing how much Toyota is using its sponsorship to lift the Paralympics and really focusing on the topic of mobility for all, which aligns with the things you’ve already heard tonight, and accessibility and inclusion.

Suzanne Basalla: The second area that I get to work on is be a champion for diversity and inclusion in the company, which is very important to our CEO. And you heard from Kelly who… Our Executive Vice President. Diversity and inclusion is important throughout the company. It’s not my job to do it. I’m the champion for it within the company, but it’s the responsibility of all the leaders. And you saw that all of our leaders from… Who are here in the headquarters today are here showing their support and that is very typical of the company. But I have the privilege of being the champion for diversity and inclusion in the company, which means I get to be in a lot of different conversations and continue to help us to think about how we can do better and do more.

Suzanne Basalla: Because we want to create as inclusive environment as possible here to attract diversity and get the most out of diversity that’s here. People call employee resource groups different things at different companies. But we at TRI have started three employee resource groups and these are initiated by employees. We’re still a pretty new company. I don’t know if that actually came across in the speeches yet, but we’re pretty new company. We’ve only started our ERG program just a little over a year ago and we’re really excited that we have three employee resource groups already up and running. One is Women and Allies. Yeah. One is LGBTQ, sorry. Plus.

Suzanne Basalla: And then one is Parents, which is our newest ERG and were just started and it’s focusing right now mostly on new parents or parents, young children, but bringing together people who are facing issues as working parents. And so we call these resource groups for a couple of reasons. First of all, they’re a resource for the members of the affinity group that belonged to it. It’s a place for them to get together and share their concerns and work together to find creative ways to address those concerns. They’re also a resource to the TRI leadership, to the CEO and the leadership team because it gives us a way to hear about what are the issues that are important to those communities and they are a resource for the company.

Suzanne Basalla: Because they advance a caring community in a respectful environment in the company, which aligned with the values that Kelly talked about earlier. So hopefully you’ve had a chance to learn about some of the ERGs. If not, ask us more questions. We love to talk about the activities in the ERGs but of course, part of the emphasis we place on diversity is why we’re so pleased to be hosting Girl Geek X tonight. We are so grateful that you’re here tonight. So I want to thank you. I’m going to turn it over, I think to Rita for some final remarks, but thank you very much.

Rita Yau: Thank you. Did all of you guys have a good time tonight? Yeah. Awesome. We again, we are so thankful for Girl Geek. Thank you guys for coming tonight.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Podcast Highlights: 15 Helpful Insights on Managing Up

In continuing our Podcast Highlights mini series, this week, we’re sharing 19 helpful insights from the Girl Geek X podcast that will help you manage your career by “managing up!”

If you haven’t already subscribed to the Girl Geek X podcast, head on over to iTunesSpotifyStitcher, or Google Play and get ready to start binge listening! 

This week, Girl Geek X Co-Founder & CTO Sukrutha Bhadouria is breaking out quotes and insights from her favorite release on the Girl Geek X Podcast — Episode 15: Managing Up & Working with Your Manager.

Why this topic matters, and why it’s her favorite episode:

Sukrutha Bhadouria, CTO & Co-Founder of Girl Geek X
Sukrutha Bhadouria, Co-Founder & CTO at Girl Geek X and Sr. Manager, Engineering at Salesforce

“I think managing up is so hard that no one really taught me, or it didn’t even come to my attention that I needed to focus on it until at least five or six years into my career. It’s really hard to know what’s expected of you and how you’re being evaluated if you don’t know how to manage up. I think the best way is to get on the same page and understand a bit more about what your manager wants to see, what their goals are and how you can help them reach their goals. We don’t talk about this enough, and learning how to manage up earlier could help so many of us move ahead faster.”

15 Helpful Insights on How to “Manage Up”

15. Make your manager look good.

“If you want to be a better employee, think about ‘What is my manager measured on, what would make them look good to their boss?’ What are the metrics, what are the things that they really care about? Then when you’re prioritizing your time, think about your decisions in the context of ‘How can I get my manager promoted?’ Because if you can get your manager promoted, you make everybody look good.” —Gretchen DeKnikker, COO at Girl Geek X

14. Ask questions.

“Have direct conversations. Do not guess. Do not try to guess. For example, I was hired for a job and two weeks in, my boss was fired. My brand new boss was fired. He and a few other people who were brand new to the company and brand new to the team – we were opening a San Francisco office – they were all I had to turn to.

We had a trip planned to go to New York, and my colleague who’d only been there a few months longer, was like, ‘Well, I don’t know if we should go.’ And I was like, ‘I’m going, and I’m going to sit down with the CEO while I’m in New York and I’m going to ask him, ‘What were your expectations for my role? What were the goals? What are the things I could do in the first 90 days?’ Because I don’t have a manager anymore and I need to know.’ It was a brand new role.

If I hadn’t asked questions, I would have worked on the wrong things. I wouldn’t have prioritized my time in a way that would have allowed me to meet expectations.

And my colleague’s first response was this idea that you shouldn’t go meet, you shouldn’t go ask, and you should just sort of sit back and wait to see what happens. I’m so glad that wasn’t my first instinct and that I went in and had the conversation.” —Gretchen DeKnikker, COO at Girl Geek X

“I’ve asked questions like, ‘What is the thing that’s worrying you the most, work-wise? Or what is your biggest goal? What do you want your org to be known for?’ And through that, I get a sense of where I can insert myself and make my manager successful. That is the main thing. When you’re managing up, you want to make your manager a success in their job by basically managing them.” —Sukrutha Bhadouria, Co-Founder & CTO at Girl Geek X and Sr. Manager, Engineering at Salesforce

13. Be vulnerable.

Sandhya Hegde speaking
Sandhya Hegde, EVP of Marketing at Amplitude

“You have to be vulnerable. You have to say stuff like, “Hey, I care about how you feel about me,” which is a vulnerable place to be. When I worked up the courage to say it, it made a huge difference. Because you are vulnerable, the other person starts being more vulnerable. So yeah, if you feel like you’re working with someone who’s not opening up, honestly the best thing to do is just be vulnerable with them and create that space for them to reciprocate.” —Sandhya Hegde, EVP of Marketing at Amplitude

12. Learn your manager’s working style.

“Try to understand your manager’s style. Do they like going for walks in their one-on-ones or do they prefer it to be a coffee? Or do they prefer to be in a conference room? Trying to understand more about their working style will help you get on the same page. It will break the ice, and then you can get to the real stuff.” —Sukrutha Bhadouria, Co-Founder & CTO at Girl Geek X and Sr. Manager, Engineering at Salesforce

Angie Chang, CEO & Co-Founder of Girl Geek X
Angie Chang, CEO & Co-Founder of Girl Geek X

“Managing up is about the willingness to do a little bit of work and ask more questions instead of just being like, ‘Well, my manager is not giving me what I want and I’m just going to be resentful.’ Ask more questions to figure out what the working relationship is going to be with this type of person. I’m sure there are professional tests that will then name this personality and give you hints on how to best interact with this type of person that you can investigate.” —Angie Chang, Co-Founder & CEO at Girl Geek X

11. There’s an app for that!

“There’s a piece of software if you’re using Gmail for work called Crystal Knows – it’ll tell you how to best communicate with anyone through email. If you were to read mine, it would be like, ‘Use short, concise sentences. Make your point quickly. Don’t use a bunch of flowery language.’ That sort of thing. And I thought it was pretty accurate, but it’s super interesting. I think you can get an initial thing for free and then you have to pay, but it’s pretty amazing. Even if you just run it on your own inbox to see, ‘Oh yeah, that is how I like to get emails.’” —Gretchen DeKnikker, COO at Girl Geek X

10. Talk to your manager’s past reports.

“When I get a new manager or assigned to someone new or move to a new org, I talk to people who reported to them for a long time. I try to get a sense of what it’s like to report to them and what their managing style is. Just so that I’m better prepared. It’s helped me so much to know what kind of things they focus on from someone else’s perspective.

I personally don’t think anyone is a perfect manager. A lot of how good of a manager someone is to you is within your control. I have had some good managers in the recent past, but I’ve also seen other people struggle to report to them. Taking things into my own hands and really, really focusing on the relationship and managing up has helped tremendously. I did my homework to get a sense of what it’s like to report to them, what they like and what they don’t like. And I figured out how to work around their dislikes. I haven’t had a situation in a really long time where things just aren’t working, because I invest a lot very early in the relationship.” —Sukrutha Bhadouria, Co-Founder & CTO at Girl Geek X and Sr. Manager, Engineering at Salesforce

“If you’re reaching out to a new manager’s former team members, you just have to approach it from a positive angle. Like, Hey, I’m just trying to do really great. If you could give me three pieces of advice on how to be successful in working with him, what would you say?” —Gretchen DeKnikker, COO at Girl Geek X

9. Know when to move on.

Gretchen DeKnikker, COO at Girl Geek X
Gretchen DeKnikker, COO at Girl Geek X

“There are people that are just not people that you enjoy working with, and that’s managers or colleagues or subordinates and at some point, there’s only so much you can do to try to smooth that over. Then you just either take that person at face value and accept that there are just times where things aren’t gonna work, or you go somewhere else.

Sometimes you’ll have a manager and you just know that they are never going to lift you up. They are never going to put you center stage. They are always going to keep you in their shadow. I’ve had those, and you have to move on. You absolutely have to move on. You cannot let someone steal your spotlight. Not on your career path.” —Gretchen DeKnikker, COO at Girl Geek X

8. Ask for a performance review.

Sandy Liao speaking
Sandy Liao, Head of Talent, Culture & People Operations at HomeLight.

“Incorporating performance data is crucial to the business, as well as your own career growth. If your manager has not spoken with you for the past quarter or past six months about how you’re doing from a performance standpoint, it’s super important to make that calendar invite and make them have that conversation.

Especially working in a startup, these things kind of get out of hand when we’re trying to do like 100 things at once. But before any of us start analyzing a new opportunity, it is just necessary to have conversations with people that are mentoring you and that are working with you directly.” Sandy Liao, Head of Talent, Culture & People Operations at HomeLight

7. Be objective & use data when navigating a challenging relationship.

“Using data is a great way to ask for help and make progress with your manager. It’s like, ‘Okay, we set these goals and I didn’t meet two of them, so here’s what I need to meet the rest.’ If you’re able to kind of frame these conversations with your manager objectively, then that’s one way to get help.

Take it back to an objective place of like, ‘We’re here to do this job. These are the goals along those lines and can you just tell me the extent to whether or not I’m fulfilling that?’ I think being able to bring the conversation back to that is an effective way to manage a more challenging relationship.” —Rachel Jones, Podcaster at Girl Geek X

6. Be specific when asking for help.

“Probably the hardest part of this when you’re earlier in your career is that you may not know exactly what you need to hit your goals. It’s hard to articulate to your manager, this is exactly why. ‘If I had XYZ, then I feel like ABC would…’ Right?

I think it can be dangerous if you’re like, ‘Oh well if I had this one piece of software, I could do this better. Or if I had an extra person, I could do this better.’ Those are hard cases to make to your manager, particularly if there’s an impression that you’re not hitting your goals already.

You want to be very specific on what it is that you’re asking for and what you think the ROI will be. Because a fuzzy ROI is a hard argument to make to a manager to get additional resources.” —Gretchen DeKnikker, COO at Girl Geek X

5. Take responsibility for your career growth.

Don’t think that your career growth is just fully your manager’s responsibility. It is just as much yours. And so if you don’t see those conversations coming up, you need to be bringing it up. 

As a manager, I’m super excited and motivated to help people who seem like they want to be helped and who are motivated as well. It’s really difficult to grow someone’s career when they’re just not as motivated to do it. And that’s fine too. Sometimes people want to just stay at their level. That’s totally cool. But if you really want to grow, you want to be bringing it up a lot with your manager. —Sukrutha Bhadouria, Co-Founder & CTO at Girl Geek X and Sr. Manager, Engineering at Salesforce

4. Know who your manager is investing in.

“Your manager controls your advancement and your visibility within the company. So if it seems like your manager is investing in other people and not investing in you, rather than just being a manager who doesn’t really invest in anyone in their team, definitely consider whether this is the right place for you. Because a manager can have a huge impact on your career, and you don’t want to be begging for attention from someone who’s just never going to give it to you.” —Gretchen DeKnikker, COO at Girl Geek X

3. Find a mentor.

Vidya Setlur
Vidya Setlur, Staff Research Scientist & Manager at Tableau Software

“Some of the best mentors that I’ve come across have been people who were my managers in the past, maybe at a different company or in a different line whom I have respected and trusted, but because they are not my manager anymore there is a different type of relationship where it can be more mentoring as opposed to managing.

There’s a lovely inflection there that happens. So kind of seek out into your network and find those people that you’ve worked closely with, or that managed you — directly or indirectly. See if they can help mentor you in your next path or next effort.” —Vidya Setlur, Staff Research Scientist & Manager at Tableau Software

2. Maintain relationships with past managers.

“I keep really strong relationships with managers… and they’re people that I go back to when I’m looking for a new job. Not necessarily for them to hire me, but they know me so well, and when I’m trying to figure out what I’m good at, what I like doing, and what direction might I go in, their input is helpful.

A past manager is someone who knows you really well to be able to kind of give their two cents, even if they haven’t been working with you recently.

I mean, not all of your managers are people that you want to necessarily keep taking advice from, but I think I’ve been really fortunate that most of my previous managers are people that I want to reach out to. I still go back and can be like, ‘I don’t know what I want to be when I grow up, help me!’ And they do.” —Gretchen DeKnikker, COO at Girl Geek X

1. Remember that your manager is human.

Rachel Jones, Podcaster at Girl Geek X
Rachel Jones, Podcaster at Girl Geek X

“Knowing how awkward the transition into management can be for people is definitely something to keep in mind when you’re working with your manager. A lot of people are put into this role without getting any kind of specific training or support on what it means to be a manager. Keeping that in mind when you’re interacting with your managers or setting expectations for what that relationship should look like. Definitely focus on the work that you have to do to maintain that relationship and drive your career forward, and involve your manager in that.” —Rachel Jones, Podcaster at Girl Geek X

Check out the full episode or podcast transcript for more great insights on managing up and managing your career, or subscribe to our YouTube channel for even more insightful content on topics that matter to women and allies.


About the Author

Amy Weicker - Head of Marketing at Girl Geek X

Amy Weicker is the Head of Marketing at Girl Geek X, and she has been helping launch & grow tech companies as a marketing leader and demand generation consultant for nearly 20 years. Amy previously ran marketing at SaaStr, where she helped scale the world’s largest community & conference for B2B SaaS Founders, Execs and VCs from $0 to $10M and over 200,000 global community members. She was also the first head of marketing at Sales Hacker, Inc. (acquired by Outreach) which helps connect B2B sales professionals with the tools, technology and education they need to excel in their careers.