“A/B Testing Cheap & Easy with Open Source”: Dena Metili Mwangi with Sentry (Video + Transcript)

Speakers:
Dena Metili Mwangi / Software Engineer / Sentry
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X

Transcript:

Sukrutha Bhadouria: Hi, Dena, how are you? You’re muted, so you need to unmute. So, hi everyone. I’m Sukrutha. A couple of housekeeping notes as we’ve been doing through the day. We’re recording all these sessions and we’re going to have the videos ready for you in a week. I saw some of you asking questions about the previous sessions. We will also have the slides for you to be able to view. Please share the fun that you’re having, whether it’s through the content, or selfies of your viewing party, or you watching at your desk on social media using the hashtag GGXelevate.

Sukrutha Bhadouria: We’re going to do Q and A at the end if we have the time. So, please post your questions. At the bottom, there is a button for Q and A. If we don’t have time for it, we’ll do it over social media, and I’m sure Dena would be willing to do that for us. Also, we have a job board on our website, GirlGeek.io/opportunities. So, please check it out.

Sukrutha Bhadouria: Now our next speaker is Dena. I’m super excited. She’s a software engineer at Sentry where she works on the growth team. Fun fact, she graduated from Hackbright where she learnt–did a ten week program studying Python. Before that she was a graduate from Duke University and was working as a research analyst at World Bank. Her talk today is about A/B Testing: Cheap and Easy With Open Source. And I’m sure everyone is excited to learn more about this.

Sukrutha Bhadouria: So, thank you so much, Dena, for taking the time.

Dena Mwangi: Hey, thank you so much. I’m going to go ahead and bring up my slides. Can you hear me okay?

Sukrutha Bhadouria: Yes, we can hear you.

Dena Mwangi: Excellent. Okay. Hi, guys. It’s so nice to be here with all of you. Happy International Women’s Day. Today we’re going be chatting a little bit about A/B testing and specifically how to do it cheap and easy with Open Source. You can find me on Twitter as Dena Mwangi, or on Linkedin. Feel free to connect.

Dena Mwangi: So, before I jump in, just a little bit about me. As was mentioned, I’m a software engineer at Sentry.io on the growth engineering team. I did go to boot camp, and that’s how I got into tech. I went to Hackbright and I think there are few Hackbright grads in the audience today. So, hi to all of you. I’m also a data enthusiast. I am into quantified training. So, I really like thinking about data and-

Sukrutha Bhadouria: Dena, sorry to interrupt you, they are soft on your volume. Can you speak up or move the mic over.

Dena Mwangi: Yes.

Sukrutha Bhadouria: You may need to start again.

Dena Mwangi: Okay.

Sukrutha Bhadouria: Yeah, this is better.

Dena Mwangi: This is better? Okay.

Sukrutha Bhadouria: Yeah.

Dena Mwangi: Thank you so much for the heads up.

Sukrutha Bhadouria: Thank you.

Dena Mwangi: Awesome.

Dena Mwangi: Yeah, so software engineer, bootcamp grad, studied economics. So, I really like thinking about the world in terms of data, which is how I ended up in the role that I’m in now. I also really like thinking about diversity, and inclusion, and how to do tech for good. So, I really liked the talk that we just had about AI. So, if you want to talk about any of those things, feel free to connect as well.

Dena Mwangi: Our agenda for today, we’re going to go through what and where is A/B testing? We’ll talk through the general MVP requirements, if you want to build your own. And then we’ll talk a little bit about PlanOut, which is an Open Source framework that you can use to help you out.

Dena Mwangi: So what is A/B testing? Simply put, it’s just a way of comparing two or more versions of a thing to determine which performs better. And the magic sauce that lets us do that is we are able to randomly assign samples of people to each variation and use statistical analysis to evaluate how legit our results are. If we do this correctly, we’re able to take the insights that we get from our small samples and say something meaningful about our larger population, which is what we’re really interested in. You’ll also hear this called split testing or bucket testing.

Dena Mwangi: Now, where is A/B testing? And the answer might freak you out. It’s everywhere. So, as you are using your applications, as you’re surfing the web, tons and tons of organizations are running A/B tests on us all the time. But, for the most part, it’s because they want to make sure that we’re getting the best out of their products that we can possibly get.

Dena Mwangi: So, one example of this is Netflix. So, while you’re Netflix and chilling, Netflix is running tons of experiments. One of these is what image they show when you’re surfing and trying to figure out what show you want to watch. So, they’ve played around with the title they show you, the image that they show you, and they run experiments to see which one gets the most clicks and which one ends up with more people watching it. The quick example of that is with a show that I love called Sense 8. If you haven’t seen it, you should.

Dena Mwangi: So, they ran this when they first had this show out. And this is just three of quite a few variations and buckets of this that they experimented with. So, you’ll notice that they’re playing around with the text, they’re playing around with the image that they’re showing, and they set this through all their markets. So, if you look at this, try and think about which one of these appeals to you the most. And, in the U.S., if you chose the middle one then you’re in the majority.

Dena Mwangi: So, most people in the U.S. ended up picking the middle one. So, most people who saw this ended up clicking on it and actually watching the show, which is what Netflix cares about. But as with A/B testing, you’ll find that, once you start digging into the data, there is often quite surprising insights to be found. So, while the middle one did the best in the U.S., all of these were winners in different markets. So, the last one won in Germany, the right one won in Brazil, and this actually tends to make a big difference.

Dena Mwangi: So, they saw, once they started running these experiments, a 20 to 30% lift in engagement with people clicking on these titles and actually watching the shows. So, you can make a difference.

Dena Mwangi: One more example for that, quick note, this stuff is hard. Computers are really hard. They do this with tons and tons of different shows and this is one where it kind of went awry. If you’ve seen Tidying Up with Marie Kondo, maybe this is the vibe you get, probably not. But this was a case where they kind of mismatched the image that they were showing in their tests.

Dena Mwangi: So, one other quick example of where A/B testing is is an example that’s kind of famous with Google where they weren’t quite sure which shade of blue they were going to use. And I think things like this are why A/B testing actually has a bad rap, because people think, really? Are we going to spend our time thinking about shades of blue? And actually, yeah, we are. Because this actually translated, by figuring out which one worked the best for their users, it translated to an increase of 200 million dollars in ad revenue. So, A/B testing can end up being quite profitable.

Dena Mwangi: So, if I’ve convinced you that perhaps A/B testing is something that could be useful for your organization, what do you do next? How do you even begin? So, let’s talk through some of the MVP requirements. Really, it boils down to two things. You want to think about how you’re going to bucket people and how you’re going to do it correctly. And the second thing you’re going to want to think about is your data, the data that you’re getting out, because you need to know which of your variations performs the best.

Dena Mwangi: So, for the first bit, you want to think about randomization. You’re going to be randomly assigning your users as they come through, but they’re going to come through your website multiple times, hopefully. And so, you want these randomization, these assignments to be deterministic and counting is hard. So, this is a nontrivial task. As you scale out your experiments, you’re also going to want to account for parallel or iterative experiments. So, if you have a user that is going to be exposed to multiple parts of your site, you want to be very intentional about what you’re showing them.

Dena Mwangi: As far as the data, you want to think about how you’re getting the data out for analysis so you can actually decide who wins. You want to think about how you’re linking it to your internal metrics. So, like with the Netflix example that we saw, they really care about people actually watching the show, and they really care about the people who are paying them, how much they’re paying them. So, you want to have a way of linking the success of your experiments to your internal metrics like activation and paid users. And you want to think about how are you going to be seeing this? What does your analysis look like? Do you need dashboards to make that easier for you and your team?

Dena Mwangi: When we thought about this, we had to make a decision between whether we were going to build something or whether we were going to buy something. And there’s pros and cons to both of these situations. So, with buying, of course, it costs money. That’s a downside. These can be pretty pricey. They run up to 40 to 60K sometimes. But, on the plus side, they’re almost ready out of the box. Bit of a negative is you have to do a little bit of extra work to link them to those internal metrics, and you have to also think about do you want to send all your sensitive information about your users out to a third party? If not, which you probably don’t want to do, how do you get that information from the experiments back and connected into your internal metrics?

Dena Mwangi: As far as building it, the downside is, well, you have to build it. So, you have to customize it to your exact use case, which is great, but that takes engineering resources, building and maintaining it. And again, counting is hard, so you have to think about how you’re going to be implementing that correctly and validating the results that you’re getting.

Dena Mwangi: So, when we thought about that at our institution, we decided to use Open Source for the first section, for the first problem of how to bucket people correctly. We don’t want to think about that. We figured if there was someone who has already done the work of implementing that, why reinvent the wheel? So, for the first part we used Open Source, but for the second part, we kind of had our data pipeline already in place. And so, we were able to leverage our existing infrastructure and just hook that into place.

Dena Mwangi: So, what did we use? We ended up using an Open Source framework called PlanOut. It’s Python-based and it’s from Facebook. It’s been around for a few years. So, it’s had various ports from other teams. For example, HubSpot built one for JavaScript. But the best thing about PlanOut, and the thing that really sold us is that it’s low entry but high ceiling. So, you get the bare minimum to get you started running experiments very quickly, but it’s extensible. So, you’re able to scale it out to lots of users, and you’re also able to have lots and lots of add ons.

Dena Mwangi: So, things that you get, you get random operators, you get deterministic assignments for your hashing, you get name spacing. And to do all this, it’s really simple. If you’re familiar with Python, you’re able to create new experiments simply by inheriting from a base experiment class and modifying the assignment logic.

Dena Mwangi: But, what you don’t get is you don’t get a GUI. So, everything is code based and every time you want to create a new experiment you have to write it out in code and write it out in Python. You also don’t get any post experiment analysis assets. So, the nice dashboards to help make your analysis life easier, those don’t really exist, and that’s something that you have to implement on your own.

Dena Mwangi: So, I find it best to learn about a new tool by walking through an experiment. So, we’re to walk through a really quick one with a pet adoption profile. So, suppose you had an app that was trying to get a pet adopted. Suppose it’s this guy. And you think that, if you play around with the image that you’re showing, we’ll be able to have more interest and more clicks on this lovely cat’s profile. You also want to have a blurb with it because why not? So, we’re going to have these two images and these two blurbs, which gives us four options that we’re experiment and randomly showing to our users as they flow through.

Dena Mwangi: If we wanted to run this with PlanOut and actually have an experiment up and running, this is pretty much all that it would take. Put some code, it’s always scary when you see code on your screen, but don’t fear. We’ll walk through it really quick. So, basically what this is is it just pulls from a simple experiment class from PlanOut, and it gives you all your random operators all in this one thing.

Dena Mwangi: What you have to do on your end is tell it the required rules of engagement. So, tell it what you’re trying to do, who you’re trying to experiment on. In this case it would be a user ID. Tell it what your varying. In this case we’re varying an image and a blurb. Tell it also how you want to vary this. And, in this case, we’re going to be using uniform choice. We don’t really care, 50/50 split with each one. And that’s all it really needs to know.

Dena Mwangi: But where does this actually go in your code? So, if you played around with Flask, for example, wherever it is that you’d be using this image and this blurb, regardless of what language you’re using, that’s where this would go. So, in this case, if you have a route, then you just throw in your assignments and you’re able to pull directly from them and put them into your template.

Dena Mwangi: But okay, so you did the thing, but where’s your data? For this, all you have to do is tell it how to do the logging in your setup. So, you tell it where you want to log all the things, what file you want to send it in, whether or not you have a data pipeline or not, you have this option of just throwing into JSON. So, as people are flowing through your website and seeing all the options that you’re showing them as you’re randomly assigning them into particular variation, all of this is getting logged and put into a JSON that looks like this that will make it easier for you to pull from it later on.

Dena Mwangi: And the important bit here is that you’re able to see what the image was that they were assigned and what the blurb was that they were setting. Also who they are and what time it was, but really these are the two main things that you care about. And that really is it. That’s your first experiment, and you’re ready to go forward and A/B test all the things.

Dena Mwangi: But, before you do, I will leave you with a few A/B testing sanity tips that we’ve learned on my team that have made our lives a lot easier. The first being, you really want to have well defined metrics of success before you start running your experiment. I think a lot of teams get really excited and they think, obviously, this is going to be great. I’m sure it will be a success, but they’re not very clear on what success looks like. So, before you run any experiment, be very clear to write this down and know what your metrics of success are.

Dena Mwangi: The second thing I would advise is to make sure that you’re doing all your experiments in small, measurable iterations instead of doing large sweeping changes. Sometimes this isn’t always possible. For example, if you have an experiment that’s being run that requires a lot of design or it’s very greenfield, then you might have to do a lot of front end work, a lot of front end cost work. But, for the most part, you really want to be doing this in small measurable iterations. That way you’re able to attribute what exactly changed to give you the lift that you might be seeing in your data. Otherwise, it gets very confusing. Was it the button color that you changed? Was it the language that you changed? It’s unclear. So, do small, measurable iterations.

Dena Mwangi: The last thing is, A/B testing is not a silver bullet. Data is one thing in your toolbox. It’s not the entire tool box. So, this really should inform your decisions. It shouldn’t be the one guiding light. So, if you see a lift in an experiment, for example, you really want to think about it and look at it in context of the whole picture of your application and what you’re trying to answer. So, with the Netflix one, for example, they could have said, oh okay. This one particular one won in the U.S. Let’s do this everywhere. But instead, they dug deep and they were able to desegregate and see that, actually, they had different winners in different markets. And they were able to leverage that information and go forth with that and be a bit more successful.

Dena Mwangi: Thank you so much for your time. It’s been so great chatting with you. Happy International Women’s Day. If you have any questions, I’m happy to answer them.

Sukrutha Bhadouria: Thank you, Dena. This was amazing. What a fun image at the end. All right. So, there are a few questions. So, let’s roll through them. Susan asks, do the logs include the end action that is the click event of user wanting to adopt the cat?

Dena Mwangi: That’s a great question, and no it doesn’t. So, this really logs the exposure. So, you would have to do the extra step, which is sometimes non trivial, of having to connect the exposure with the actual action of interest. So, what we do is we have lots of different analytics events. So, in addition to the exposure one, we think about what we want them to do, and what success looks like, and we log that as well separately. And, when we run the analysis, then we combine the two.

Sukrutha Bhadouria: Got it. How do you know what is a good sample size of data to test with?

Dena Mwangi: So, this actually, there’s equations that we run. It’s pretty standard like statistical modeling that you can just like put number in, figure out what power you want, figure out what level of statistical significance you want that you’re comfortable with, and play around with that. And that will spit out what sample size you should be going with, at the bare minimum.

Sukrutha Bhadouria: All right. And is there any project too small for A/B testing to be useful? For example, a small app in private data with only 20 users?

Dena Mwangi: Yes, unfortunately. So, with the statistical significance to be able to say something that’s truly meaningful, you would have to run the numbers and see like what the minimum number would be. But I think 20 would definitely be too small. You want something in the hundreds and up.

Sukrutha Bhadouria: Yeah, that makes sense. And finally, what did you find most challenging when you transitioned from bootcamp grad to working full time as an engineer?

Dena Mwangi: That’s a great question. That should be a talk all on its own. I think for me it was still like a steep learning curve, but it was getting really comfortable asking questions and asking one more question than you feel comfortable asking, and getting over that fear of being seen as not knowing enough or all the imposter syndrome things that come with being a bootcamp grad and being in your first tech role. I think, honestly, that was the biggest thing is just getting over that and saying, it’s fine. I just need to learn the things. So, I’m going to ask the questions.

Sukrutha Bhadouria: Thank you so much, Dena.

Episode 12: Intersectionality

Intersectional feminism

Resources mentioned in this podcast:

Kimberlé Crenshaw’s TED Talk “The Urgency of Intersectionality”

Spring Reading: 20 Books to Help You Become a Better, More Self-Aware Ally

Transcript:

Angie Chang: Welcome to Girl Geek X Podcast connecting you with insights from women in tech. This is Angie.

Sukrutha Bhadouria: This is Sukrutha.

Gretchen DeKnikker: This is Gretchen.

Rachel Jones: And this is Rachel, the producer of this podcast, and we’re the team behind Girl Geek X. This podcast brings you the best of Girl Geek X events, dinners, and conferences, where we’ve been elevating women in tech for over ten years.

Angie Chang: And today we’ll be discussing intersectionality.

Rachel Jones: Is this something that comes up a lot at the dinners?

Gretchen DeKnikker: I think so, and you see it a lot in the press also, and I think there seem to be a lot of different interpretations. There’re the people who are deep in doing the work, that are using intersectionality in the way that it’s intended, and then you see on the other end of the spectrum where it’s become like a buzzword, and people hear it and they start saying it without actually fully understanding what it means.

Rachel Jones: What does it mean?

Gretchen DeKnikker: Well, there’s a great TED Talk by Kimberlé Crenshaw, where she outlines everything. For years and years it existed and it was called a bunch of different things–double jeopardy–it was something that black feminists talked about a lot, and then there was a case, I think in the seventies, with General Motors where there was a woman who was suing them and the court wouldn’t let her combine race and gender. So because they hired white women in the secretarial pool and because they hired black men to work in the factory, she could not prove a case that she was being discriminated against as a black woman because they wouldn’t let her join the two intersecting–where the word comes from–parts of her identity that were preventing her from getting a job. So that’s kind of where it started.

Angie Chang: That’s wild.

Gretchen DeKnikker: Yeah.

Rachel Jones: Silly pop culture example that I always think of, there’s an episode of Scrubs where Elliot, who’s a white female doctor, and Turk, who’s a black man, are having this debate about who has it harder, black doctors or female doctors, and then thankfully a black woman doctor walks by and they’re both like ooh. Wow. This argument is dumb for the two of us to be having. And how have you seen this definition play out in tech and how people understand it in that context?

Gretchen DeKnikker: I think I see it a lot, where you’ll hear diversity and inclusion and intersectionality and they’re not the same thing. You know, if you wanna back up and really try to understand intersectionality, then you first need to understand privilege. But then once you acknowledge privilege, then you can really understand how these identities intersect, because each time you add one, right you’ve got “I’m a woman”, and “I’m a black woman”, and “I’m a lesbian” or even a trans woman, right, and you start adding and then you can look at, as you add those identities, how marginalized those people actually are. And I think that’s my definition, what do you think?

Angie Chang: A group called Project Include, which includes a woman by the name of Ellen Pao and in case her name doesn’t ring a bell, she’s a woman who sued the venture-capital firm Kleiner Perkins for gender discrimination and–

Gretchen DeKnikker: Thanks, Ellen.

Angie Chang: –and lost, and I remember feeling so sad and everyone feeling so sad when that happened because we all felt in our bones that we were being let down in the workplaces around the world and we can’t prove it in court. Anyways, she is now part of this amazing Project Include and they have so many resources on the web, I read some on Medium that were very illuminating in terms of the stories that we are not living ourselves, we can listen to in podcasts and documentaries, talk to people on the street.

Angie Chang: So the blog post that really struck me was one called, it was actually titled #FFFFFF Diversity, and if you don’t read HTML, that’s the code for white diversity, and it was written by a engineer, now engineering manager called Erica Joy, and she talks about how her experience at Google–when she worked there, she was confused when the company started spending a lot of money on very outward facing initiatives about how they’re inclusive to women, and women of color, and yet she was feeling like they were not really spending any of that money internally on their black women at Google, who were a very underrepresented group, and she said that if she was a white woman in tech she’d probably have been delighted that they were investing in a diversity initiative but she’s a black woman in tech so things didn’t feel so good to her. And whether by design or by inertia, the favor seems to land on white women.

Sukrutha Bhadouria: Every time I read any news that’s celebrating how many more CEOs that are of Indian origin and you notice that none of them are women. Microsoft and Google and so many other big companies that have visible Indian men running the company, and maybe, I don’t know if people are starting to notice now this obvious ceiling for Indian women and Asian-American women in general, for rising in the ranks, and so there’s this Harvard Business Review article titled “Asian-Americans are the least likely group in the US to be promoted to management,” where they say Asian-Americans are the forgotten minority in the glass ceiling conversation. We have, like I said, you know, not just for Indian men but when you see the Asian women and Indian women at the top very rarely.

Gretchen DeKnikker: When you’re talking about privilege and talking about intersectionality and diversity and inclusion, I think sometimes they all get swirled together and you can lose sight of what those things are individually and what they mean individually and that they are very unique distinctive things.

Rachel Jones: Yeah. Language, it gets tricky, there are so many words and some of them mean similar things, and there’s new ones, which never travels well. I remember even when people were getting really fed up with diversity as a word because it was losing its meaning and they’re like, oh we’re not gonna do that anymore, now it’s about inclusion, and now inclusion has lost all of its saltiness so it’s interesting just what happens with language.

Angie Chang: And now we have the letter X which has… has the letter X, as a variable it’s supposed to be more inclusive, I think the word come around the corner as the next big thing.

Gretchen DeKnikker: Well, it’s a good thing we got on that train early then.

Angie Chang: So, Sarah Staley leads communications and culture at Realtor.com and she spent a portion of her talk at the Realtor.com Girl Geek dinner touching on intersectionality. Here’s what she said.

Sarah Staley: You know, intersectionality–we often talk a lot about–you may not know what this means, but intersectionality is beyond inclusion. It’s beyond diversity. It’s when you really take time to get to know a person’s story, and that’s when the real color comes in.

Rachel Jones: What would you add to Sarah’s definition of intersectionality?

Gretchen DeKnikker: What’s not in the quote is just before that she’d had people think of something unique about themselves, then turn and share with the person sitting next to them, that unique thing about them. So I think, she’s very much trying to bring out a full personality, find a way to bring someone’s full self into a conversation. It’s not quite the traditional sense of how intersectionality would be used but you can sort of see where she’s headed and she’s really close, but I think we see that a lot, right, like people are understanding part of what it means, but not all of what it means. Or we see it where it’s very close to inclusion, which as Rachel was saying earlier, has also lost its shine and luster, given that the word itself is trying to say that it’s an assimilationist theory that the group needs to be brought in and assimilated into this environment, they need to be included in the existing environment as opposed to the environment adapting to all of the people.

Sukrutha Bhadouria: I think it’s important when you’re trying to create an equal environment that there’s no one definition of equal, right? That’s the whole problem when you assume there’s one definition of what it means to feel marginalized or to feel like a minority, so you have to identify where you stand and what the differences in experiences are for other people and perhaps that’s what she was driving to. Like, learn more about other people’s experiences, especially when they’re different from yours so you can be more informed when you’re trying to create a more equal environment. But you have to have a good understanding of what it means and not have this blanket oversimplified definition of what intersectionality is in the first place. What do you think, Angie?

Angie Chang: I don’t like to define what intersectionality is, I just like to talk about the different ways it shows up.

Rachel Jones: I have definitely been in a space where I’ve had to challenge people bringing weak versions of diversity. So when I was working in my last job in Chicago, I became a member of our “culture committee”. You can’t see this on the podcast but I’m doing air quotes.

Gretchen DeKnikker: Oh yeah, they’re very cute air quotes too.

Rachel Jones: Because, so we were a nonprofit organization serving mostly black and brown youth, and a lot of what was happening in the organization was just plain racist. The way that student stories were being used and the ways that they were being tokenized and positioned. As we were trying to hold the organization accountable for that ,they were like Oh! Let’s have a culture committee help figure this out. And for a while, the only assignments that we had were planning parties and that was very confusing, like this is not why we exist. And then we brought in a consultant to help do a race and justice training, that ended up getting us in trouble with our executive director and HR, and so it was like they were asking for diversity and inclusion and asking for our expertise and outside expertise, and then when actually confronted with, “Hey here are the ways that you’re falling short and here’s exactly what you can do to make a change,” they kind of faltered. So even just by the show of creating a culture committee they thought the work was done, when we were trying to do the actual work.

Gretchen DeKnikker: No, no, they wanted you to do the work–

Rachel Jones: –But it’s just like–

Gretchen DeKnikker: –right oh we’re gonna have a committee we’re gonna put people of color on it, they’re gonna solve this problem. We, the white people, the executives, the whatever, we don’t have to stop and look and think about our role in this.

Rachel Jones: And, yeah, I think that’s the danger when these words kind of lose their meaning, it’s something that you can just pass off to someone to figure out, and that’s something that takes actual investment and work and thought and a complicated lens that’s looking at these issues in an intersectional way.

Gretchen DeKnikker: So, this is a great lead in to a quote from Audrey Blanche who is the Global Head of Diversity and Belonging at Atlassian. She did a talk called Thank You, Next, at our recent Atlassian dinner and she is amazing, and she purposely doesn’t have inclusion in her title for the reasons that we were kind of talking about a little bit earlier, and in this quote talks about how you build a greater sense of belonging within your culture.

Aubrey Blanche: We need to move away from ‘diversity,’ which has a limited meaning and actually is not aligned with the goals that we’re trying to build. We need to build balance in our organizations. We also need to move away from ‘inclusion’. Inclusion assumes that I can fit like an add-on into a power structure that was built for straight white men, and I have no interest in doing that. I’m not any of those things and I don’t know how to show up that way. I wanna actually build belonging, I wanna show up in a space where I was considered and where I was thought of.

Aubrey Blanche: And that doesn’t mean… it can be the littlest things that show me that. You’ll see here, research shows that women feel like they belong when there’s more plants in an office. You’ll see that our bathrooms, even the ones that because of building codes have to have gendered words on them, do not actually contain pictures of what a man or a woman looks like. That might not matter to a lot of you. But to folks who are gender-nonconforming or non-binary or transgender, that has huge meaning. That little subtle clue actually tells their brain that they belong in that space, and that’s what we’re trying to build at Atlassian and I think we can all resonate with wanting to feel like we belong.

Aubrey Blanche: We need to stop thinking that women = diversity, and embrace an intersectionality-first strategy. So how does this look like? It’s pretty simple. If I think of someone who has intersectionally marginalized identities, let’s say myself. If I as a queer Latina woman can succeed in the organization, any changes that are made are definitely gonna benefit straight white women, too. But when we start with diversity = women, we only build programs, processes, and structures that help straight white economically privileged women succeed. Who certainly face barriers compared to their male counterparts, but we end up further marginalizing anyone who doesn’t fit that bucket. And I genuinely, genuinely believe that we can all win together, this does not have to be a competition.

Gretchen DeKnikker: I love her. I’m gonna get like, I love Aubrey t-shirts printed, stickers, I might just stand outside Atlassian with a sign.

Gretchen DeKnikker: Ooh, maybe I could do the Say Anything where he’s holding up the boombox, play her a song. I love her that much.

Rachel Jones: Everything that she said is every critique of white feminism, ’cause there are so many approaches like white feminists organizing and trying to push for women in workplaces and just leaving so many people out, but what was harsh about that was that so many times when people would try and give that feedback and hold these women accountable, they’re saying oh we’re trying to find solutions that work for all women, and you’re proposing things that are just specific to you, not realizing that the things that work for white women are specific to white women and not just the general universal solution for everyone. It’s actually solving the easiest problem, and leaving most of the struggle still there and saying like, “We did a good job.” So I like that she pointed that out.

Gretchen DeKnikker: And white women, if you’re out there trying to figure out how you play a role in this: Pull up a chair. When you get a seat at the table, pull up a chair for someone of a different race, or sexual orientation, or whatever.

Angie Chang: It is a little frustrating to me whenever I hear about ‘a woman has cracked this big tech board!’ And I’m like, “White woman. It is another white woman.” And it’s gonna be, if we’re lucky, the second white woman on an all white male board, so definitely, if you are a person of privilege, pull up a chair for someone that doesn’t look like you.

Sukrutha Bhadouria: Yeah and [inaudible 00:15:55] other women with different experiences will start to see someone that they can identify with, because not everybody will identify with you, they’ve got that amazing ability that we really need.

Gretchen DeKnikker: That’s what I love about the dinners, ’cause it’ll be all different roles and all different levels of experience, and the more diversity we can get there from even just a cognitive level, that means that there’s one more person in the audience who, at least one more, who’s like I wanna be her someday or like I finally feel this connection with this person and I see it being possible, kind of like you were describing earlier, Sukrutha, that panel where you were like, Oh I’m out of excuses, all of these women have all of the things and now I feel empowered to go do it.

Sukrutha Bhadouria: A lot of times–speaking of intersectionality–I’ll go to a panel or at a Girl Geek dinner, I will find something that I can connect with each speaker because we have so many different identities, and I also notice that the meter of how diverse the panel is or the speaker roster is defines how diverse the attendee group is, so we have to be really mindful of, look beyond just gender or [inaudible 00:17:25].

Gretchen DeKnikker: Yeah, as we’re in the fourth wave of feminism and I feel like we can get it right this time, and not end up in a bunch of fragmented… with different goals.

Rachel Jones: How do you actually use an intersectional lens to create inclusion or belonging, in your…?

Gretchen DeKnikker: I feel like as a manager you have to, particularly once you start managing a team, you have to start thinking about how each individual person is approaching this, and at that point they’re individuals and they come with their own little unique snowflake selves that you have to start thinking about how those things impact. Right, and someone who’s more soft spoken, right, and at that point you’re not really thinking about what the race of the person is who’s more soft spoken but like how do I help this person get their ideas out there, stuff we’ve talked about on previous podcasts.

Gretchen DeKnikker: But I think from a day to day management level it’s really thinking about little things, the way that Aubrey talked about like simple little things, the not using the male and female icons on the bathroom, like at tech conferences for a long time and you still see it sometimes they wouldn’t have women’s t-shirts, and it’s like, oh well, you know, there just weren’t very many of you so they’re unisex shirts. And the conferences that I’ve planned, it’s like you have to have women’s shirts and they have to be nice women’s shirts, cut for a woman without a huge logo right in the middle of her breasts, right, a smaller logo moved up a little bit so she’s not some weird warped billboard with your logo on it. But it’s like those little things that make people feel welcome, it’s those little things of like, oh you know I was gonna be here and you knew there weren’t gonna be a bunch of me here, but you thought about it, and there’s something for me. And so I think, it’s not in the big things, it’s in the teeny tiny little details.

Angie Chang: You talk about little details and in Aubrey’s talk she mentions putting plants and magazines in work spaces so women can feel more comfortable within their workplace and studies have shown that women and girls react better to a workplace that has plants and magazines, so just little things to make women or other underrepresented groups feel like they’re part of this culture.

Gretchen DeKnikker: Yeah, but I mean that’s also not, if there’s someone listening to this and they’re like oh if I get plants and magazines women will feel more included, it has to be unique to your company, you have to be thinking about what are these things, because it can work in the opposite way too, right. You walk in one day and they’re like we heard women like plants and magazines like, oh perfect, well I must be like all other women so I’ll be thrilled.

Angie Chang: I think the study was probably meant to say if you were, I don’t know, very technology heavy environment and there were servers everywhere, maybe do some things to make it more hospitable.

Rachel Jones: There are big and small changes that you can make that communicate to a person. Yeah, you do belong here, or it’s something that’s actually accommodating needs that are specific to different groups, like providing childcare, providing transportation or other things that specific groups have a harder time accessing or could be more of a barrier for them for participating. It really should be led by what are people’s needs, what are people’s barriers and how do we respond to those?

Angie Chang: Yeah, I think it’s also understanding and helping people who are fasting for Ramadan through their work day that month, and not making them… helping them feel not so awful when they have to remind people this is why I’m not drinking water right now and stuff like that.

Sukrutha Bhadouria: And even when you’re pregnant, or you have someone on your team who is pregnant, being mindful of all their unique things and calling for the appropriate number of breaks, and when someone’s just had a baby, they shouldn’t feel stressed out or pressured to dial in while pumping.

Sukrutha Bhadouria: That can be very stressful so just, I mean…

Sukrutha Bhadouria: Thinking about barriers, scenarios and barriers. Situations that everyone else might be in, and building a team is like building a puzzle that when you take one piece out you have a totally different piece that you might want to add to it just based on who else is left on the team, so constantly thinking about who your next hire should be based on the team that you have. In terms of filling the blank, not extending the blank that might be there. Heidi Williams offered her own advice during our recent Elevate conference. We met her when she was the VP of Engineering at Box, and now Heidi Williams is changing the world for women as the CTO and co-founder of tEQuitable. At her Elevate conference talk, which was hugely popular by the way, she recommended using an engineering approach to debug the workplace culture and make systemic change that improves the lives of working women everywhere.

Heidi Williams: Psychological safety more than anything else is critical to making a team work. And so what is psychological safety? It’s the shared belief held by members of the team that the team is safe for interpersonal risk taking. Psychological safety may sound like it’s all about the emotions or about the mental aspect of the game, but really it’s the way that you encourage and promote behaviors that reinforce trust and respect and mutual empathy and authenticity, and discourage behaviors that tear those down. So the key here is don’t measure how people feel, measure how people behave and the impact it has on others.

Heidi Williams: So let me give an example of one of the things that we’ve seen and how we identified behaviors and talked about the possible systems that might have been causing those behaviors, and of course then what actions you can take to improve those systems. So the example we saw at one company was that interactions between teams were being reported as aggressive and bullying. So you could imagine that those people are just mean. But that’s probably not really the case. There’s some reason why they’re being aggressive and bullying.

Heidi Williams: There’s a bunch of different systemic issues that could be the root cause here. Maybe the two different teams don’t have aligned goals. They have goals that they are totally different from each other, they’re not reliant on each other, and they’re both measured on the success of achieving their own goals. And maybe the success metrics don’t include that you have to collaborate and help others achieve their goals, that you’re only measured on whether your team achieves their own goal. Or maybe the peer feedback system is not part of performance reviews, so there’s actually no way to even report this behavior, so maybe the team that’s being aggressive and bullying has no idea the impact of their behavior on others. So only if you look at the data, and then ask the five whys and debug it like an engineer will you get at the possible root causes, the systemic issues that could be causing that behavior.

Heidi Williams: So to talk through that, if you’re gonna make systemic change about your supportive culture, and create a supportive culture, start by examining behaviors to understand your current culture, then ask the five whys to find the root cause behind the behavior, and then create systems that encourage psychological safety so that you can promote all these good things around trust and understanding and empathy and communication. And don’t tolerate destructive behaviors.

Rachel Jones: I really like that she says don’t measure how people feel, measure how people behave, because I feel so often in these conversations, they get lost in this place of feelings and people don’t feel included and they don’t feel like they belong and then it makes other people feel like they don’t actually know what to do in response to that. But thinking about behavior, that’s something that you can observe and easily change and that doesn’t let you get away with, oh maybe people are always gonna feel some type of way, like no. We can actually change these behaviors. It also takes the pressure off of the person who is marginalized, so they’re not having to explain why they feel the way that they feel, we’re focusing on the behaviors that are damaging and working to change that.

Sukrutha Bhadouria: Yeah and you also don’t wanna wait till someone tells you how they’re feeling. You need to be observing their actions, and whether they’re showing up to events, those events aren’t inclusive enough, and when I say events I mean we do a lot of, in tech companies we do have a lot of events that are supposed to be team building and team bonding events, happy hours, ping-pong championships, and if you’re not seeing people participate, people of various groups participating, don’t wait till they tell you why they don’t.

Gretchen DeKnikker: You know when you were talking about don’t wait to have someone tell you how they feel, right, but if someone does, there’s certainly better ways to handle it than if you… a common thing that you’ll hear is giving feedback to a white person then at first they’re defensive, then eventually they feel victimized in some way, so then the person who is just trying to express their feelings is not only not getting their needs met but now they’re trying to meet the needs of the person who’s upset at having their behavior called out and called into question, which is sort of the thing that you really wanna stop doing and start avoiding, and listening. I don’t know, do you guys have tips on how people could hear feedback differently? Giving feedback and…

Sukrutha Bhadouria: I wish that when people–when I give feedback and I try to do this when I receive feedback–I wish the listener or the person who’s supposed to receive the feedback asked questions instead of responding to the feedback right away. I think the worst thing you can do is start to get defensive, is try to explain your actions when you’re getting feedback about something specific that you did or said. So if they ask the questions to get the context, to understand why someone’s feeling the way that they’re feeling. I’ve found that I’m more likely to give feedback if the person that I’m giving it to is listening and asking questions as opposed to responding to it straight away.

Rachel Jones: I think another thing you can think of, kind of flowing from what Heidi said, is really to focus everything on behavior, because it makes sense to be defensive if your whole identity is being called into question, and a lot of times when feedback is given people don’t hear it as, “Oh you did this one thing that made me feel this way,” they hear it as you are homophobic or you are a sexist or you are a racist. So it’s saying I’m not calling your whole identity into question or saying that you are one type of thing. We’re talking about the specific behavior that you exhibited and the impact that that had and the causes behind that so lets focus on that. I’m not talking about who you are as a person, so don’t take it personally in that way.

Gretchen DeKnikker: I think that’s perfect. And I think white people in this country, me included, need to get very comfortable with the fact that we are all racist on some level and that everyone has racist behaviors and that while the word is really powerful, we need to actually think about the definition of it and what that means, and how to correct those behaviors because even a racist will tell you they’re not racist, they’re just separatists. They just want you live over there. I don’t not like you, I just don’t wanna live with you kind of a thing, so just understanding there are racist behaviors. We all have them every single day and being open to hearing that feedback.

Angie Chang: I think intersectionality just reminds us how much further we have to go and be open to continuing to learn about each other, this evolving conversation and just keep trying to be curious about other people.

Sukrutha Bhadouria: Yeah, my final thoughts are just a reminder that must be [inaudible 00:29:46], more than just your gender or more than just your race, and there’s all kinds of diversity that you have to pay attention to in the workplace or in your social environment. So always pay attention to the space that you fit into and the space that you don’t fit into that others might fit into.

Gretchen DeKnikker: Right, I know for this wave of feminism to be successful it has to be intersectional, and if you find yourself in a place where you don’t totally understand what that means and the ways to approach it, then there’s only like–you know we just did a post on 20 books that you can read–you don’t have to read all of them–that can definitely, on the girlgeek.io site, a book on building allyship, but if it’s not intersectional, it will fail, the same way that the past three movements have failed on some level to elevate all women. And so it’s really, really important, and what’s that quote? My feminism will be intersectional, or it will be full of shit.

Rachel Jones: It’s definitely true, you’re setting yourself up to fail if you’re using definitions of these things that are empty or you’re using solutions that only work for the group that’s struggling the least out of everyone, and we’re at a point where people can very easily see through these things when they are bullshit, like people aren’t just gonna say like we have a culture committee and take that to mean like the work is done, people actually wanna see tangible results so… yeah, I think–

Gretchen DeKnikker: We have an ERG! Clearly, everything’s fine!

Rachel Jones: Yeah, mm-hmm (affirmative). Like as much as you can, hold people accountable to do the actual work and not just fly a diversity flag and say the work is done.

Gretchen DeKnikker: Or they think that the people in the marginalized group are the ones who are going to do the work. Ask them what needs to be done and then do it.

Angie Chang: Then give them money to do it.

Gretchen DeKnikker: Oh, money!

Angie Chang: Thanks for listening to this episode of the Girl Geek X podcast, we’ll be back for more advice from women in tech.

Rachel Jones: This podcast is produced by me, Rachel Jones, with recording help from Eric Brown. To learn more about Girl Geek X, or buy tickets to our next dinner, visit girlgeek.io. You can also find full videos and transcripts from our events. If you’re interested in hosting a Girl Geek dinner, email sponsors@girlgeek.io.

Angie Chang: This podcast was sponsored by Realtor.com, offering the most comprehensive source of for sale MLS-listed properties. For years, millions of home shoppers have turned to Realtor.com to find their dream home, and this podcast was also sponsored by Atlassian, a leading provider of collaboration development and issue tracking software for teams.

“Office Manager to CPO – Keynote”: Shawna Wolverton with Zendesk (Video + Transcript)

Speakers:
Shawna Wolverton / SVP, Product Management / Zendesk
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X

Transcript:

Sukrutha Bhadouria: Hey, Shawna.

Shawna Wolverton: Hello.

Sukrutha Bhadouria: All right, so gentle reminder of a few things, so I’m Sukrutha. I’m CTO of Girl Geek X. We’re recording these videos. They’ll be available for you in a week. Post your viewing party, selfies of you watching this, and any other learnings that you have on social media with the hashtag GGXElevate. We’re going to do a Q and A at the end, if we have the time. So, use the Q and A button at the bottom to ask questions. If we don’t have time, we’ll answer the questions later tomorrow or later this week. So, please check out our job board on GirlGeek.io/opportunities. Yeah, that’s it.

Sukrutha Bhadouria: So, let’s enjoy Shawna’s talk. Shawna, Senior Vice President of Product Management at Zendesk. Before which she was the chief product officer at Planet. Prior to that, she spent 14 years at Salesforce, going from the first localization manager to growing into being the Senior Vice President of Platform Product. So, that was a great growth, and Shawna is going to be talking to us today about her growth from office manager to CPO in over a thousand steps. Thank you, Shawna, for making time for us. I’m super excited.

Shawna Wolverton: Thank you. Awesome. I will just jump in, since we’re running a little late on time.

Shawna Wolverton: Welcome, everyone. I hope you’re having a great day here. Thank you so much to the Girl Geek X crew for hosting today. I’m really honored to be here. And, as Sukrutha mentioned, I have had quite a career, not always the one I planned. It’s been an interesting odyssey, and I thought I would share a little of that with you.

Shawna Wolverton: And really, this is the only good place to start, because so much of my career really does come down to this. I think there is some myth out here that we can all– we’re self-made, we’ve worked hard, every accomplishment we have came–just sprung forth from our amazing intellect and crazy persistence. And I don’t want to discount that, but the universe is large and we are all very, very lucky to have been born in this place and time. And so many people have helped me along the way, and I’m incredibly grateful to them.

Shawna Wolverton: But really, when I think about career plans, we just heard a little from the good crew of Grand Rounds about planning. And so much of life is really a fantastic stochastic kind of adventure. And we can’t always get all of the steps right for how we want to get there. But, at the end, there’s some really great goals and great milestones that we get there.

Shawna Wolverton: So, in my lifetime, we had an entire industry come and go. We had entire things– dotcom dissolution, number one. We had a grand financial crisis. Entire industries are gone. So, we really have to think about agility. We do it a lot, in terms of how we do our work. But I think we sometimes kind of get a little locked in and forget that we wouldn’t make a solid waterfall five year plan to do anything else in our lives. And being agile in our careers is really critical.

Shawna Wolverton: And there’s no one right way to go, and sometimes things change. I spent the first 20 years of my life assuming I was going to be a physician. It turns out my university transcript and I had very, very different ideas about that future. And there was no product management Barbie set when I was a kid. And coming out of school with my fantastic degree in Russian studies and political science didn’t set me up for anything really obvious. And it took quite a bit of experimentation and curiosity. And I think that early curiosity is what has also kind of driven a whole bunch of my career. A strong desire to learn new things, and an absolute hatred of being bored has been probably the two biggest things that have driven my career to date.

Shawna Wolverton: And I think we think a lot about driving our careers. We hear about this all the time, right? What are you doing to drive your career? What are the activities that you’re doing? I think we get a little lost sometimes and lose the journey. I like car racing, maybe you don’t, but I thought this was a really amazing analogy, right? There is a fantastic race that goes from Paris to Dakar in Africa, and you have this amazing adventure. And you have a whole crew that comes with you, because you assume your car will break down, and you will go the wrong way, and it will take much longer than you anticipated. And it’s glorious. And the sort of alternative is this ring of never ending struggle.

Shawna Wolverton: And I think, when you think about your career and how you progress through it, kind of an adventure attitude and a fantastic kind of see what will happen is a great way to approach things and make sure that you’re not missing out on the amazing experiences that come sort of between those promotions. I think we sometimes put milestones in the ground about where we’re supposed to be at certain points in our lives. And, when we don’t get there, we can be really disappointed. And I always think that’s really unfortunate, because there is so much to learn along the way on these journeys.

Shawna Wolverton: My career was clearly not a straight line. I did start out as a localization project manager. You can see I did that job three times in my career. Sort of moving on from it, finding myself in a position where it was skills I needed to rely on to kind of go back into the job market when things had changed. I certainly didn’t expect to learn much that would help me in my career, taking that nine month apprenticeship as a handbag manufacturer with an Hermes trained designer. But, my goodness, did I learn a tremendous amount about human nature, about satisfying the wants and needs of customers in a way that I don’t think any other technology job would have given me.

Shawna Wolverton: And it was a tremendously long time between that first localization manager job and that SVP of Product manager job. I spent 14 years in the same job–or not in the same job, but with the same company, at Salesforce. And I think it’s another thing we think a lot about is, have I stayed too long? Have I stayed long enough? And I think that a lot of those thoughts are silly. If, like the previous speakers were saying, if you find the thing that matters and that gives meaning for you, then you keep going, and that’s what gets you up in the morning.

Shawna Wolverton: I like to tell people that product management is a job that by all measures is pretty terrible, right? If everything has gone swimmingly well, you’ve reflected that back out on your developers, your sales engineers, your sales team, your marketing team. And, if it goes absolutely sideways and the organization is at a loss, you stand up in front of the room and it’s all you. But what I found is that there was this fire in my gut that was about helping customers help their customers. And I couldn’t imagine doing anything else. And I think that love of my customers and that sort of obsession with helping them is the other sort of huge part that grew my success.

Shawna Wolverton: And then, I think this is a big thing that we don’t talk a lot about and that’s the–well, we do. We talk about it a lot. Who am I kidding? But having it all is a lot, right? You can have a family, and children, and a job and it’s not easy. But I think a lot of the things that we tell ourselves and a lot of the things that the media tells us about what it’s like to be a working mother are really bananas. There is this myth that we’re going to go out on maternity leave and we’re going to come back and we’re going to be distracted and we won’t be as good as we were. And there is this amazingly strong body of scientific research that’s happening about what happens when, not just women, but men and women come back from parental leave or after the birth of their children.

Shawna Wolverton: Amy Henderson, who is the founder of a company called Tendlab that really focuses a lot on sort of parenting in the workplace, has done this amazing research. And the women primarily that she spoke with, the senior women, all with hindsight, were able to look back and find an acceleration in their careers post childbearing. And I think, whether you have children or not is a totally personal choice. But I want everyone to know that it’s not, I think, the horrible, awful, career ending thing that we’ve thought it was for the longest time. And there is a fantastic thing that can happen. That little J curve was definitively, after my daughter was born. And, at 12, we’re having a fantastic time together in a way that allows me to be here and also there, and to be a fantastic role model for her about what it looks like to be a woman who is in the workforce.

Shawna Wolverton: And this one is important to me. It’s, success is not a pie, right? I think oftentimes we think for us to win or to get a promotion or to get ahead, someone else has to lose. And it’s not like there is this finite amount of success where we get our pieces of the pie, and we eat it, and then they’re all gone, right? It is really more of this sort of random, infiniteness that is more pi than pie. And so much of I got to where I am is about the people who put a hand out to me and who supported me in my career.

Shawna Wolverton: You heard from Leyla and Jen earlier. A huge part of my network of support in my career. And it’s incredibly important, I think, for us to think about how we turn around and put our hands up too for the next group of women coming up in the world.

Shawna Wolverton: So, yeah. That’s sort of my journey through my career. I think I went a little fast. I woke up this morning with a cold and I’m on a little cold medicine. So, maybe I’m going a little fast today, but it might leave us with a little more time for Q and A.

Sukrutha Bhadouria: All right. Shawna, that was amazing. My video feed is taking a bit of time. Hi.

Shawna Wolverton: Hi.

Sukrutha Bhadouria: You’re so awesome. I love the fact that you said that success does need to be shared, for sure. Sometimes we’re a little bit hard on each other, right? And we get a little competitive and we don’t help each other out enough. What has been, I guess, for you–I have a question. What has been, for you, the most fun role that you had while you transitioned over the years?

Shawna Wolverton: I think almost every job I’ve had I’ve found the fun. But I think I have a kind of warped sense of fun. Things that are really, really hard are fun. So, at Salesforce, we had a giant project to rewrite the entire front end of a 17-year-old software product. And it was an entire company motion, took me way out of my comfort zone, and it was hard. But, at the end, it was just so fantastically rewarding. And the fun part was really so much about the people and conscripting a sort of unwilling team onto team lightning, and then going out in the world and talking to customers about it.

Sukrutha Bhadouria: Yeah.

Sukrutha Bhadouria: There’s another question. What is a mistake you made in your journey that you could share with us?

Shawna Wolverton: I think one of the mistakes I made was thinking that I could go fast. Like there were times in my career I got a little ahead of my skis. Where I saw other people getting promoted, I let my ego get in the way, and, when I didn’t get a promotion I asked for, I took it really personally. I was devastated. And, in hindsight, I’ve realized that extra year spent in the job before I promoted was important, and I learned a lot. And I wasn’t ready when those things had happened. So, I think–yeah. Getting a little too attached to my own ego and letting some of that go.

Sukrutha Bhadouria: Yeah. It happens to all of us, I bet.

Sukrutha Bhadouria: Okay. We have another question that says, how did you get the opportunity to be a handbag apprentice for Hermes? How does one find additional opportunities like that, Shawna?

Shawna Wolverton: Strange things happen when you take City College classes for fun, and you meet interesting people, and when you’ve been laid off, and you have a little bit of severance, you can say yes to some things that you normally wouldn’t say yes to. I think I probably would have stayed longer, but my husband started sort of like, you know, it might be time, Shawna, this little adventure you’re on.

Sukrutha Bhadouria: So, what do you think kept you going when things were not going the way you meant for them to go? Like how do you keep yourself positive?

Shawna Wolverton: I mean, I think, for me, it’s about what’s always been important to me. And what I found when I became a product manager, and why I’ve never wanted to do any other jobs since I started is that connection to customers. And knowing that, even when things are really hard or things aren’t going great in the office, that the things that I do have impact on real human beings. And I was lucky enough, over the course of my career, to really get to know–I count a number of former customers as friends today. And getting to know how the things I did impacted their lives, and that really is the thing that keeps me going.

Sukrutha Bhadouria: All right.

Sukrutha Bhadouria: So, some more questions pouring in while I’m reading them out. So, what is the biggest leap mentally and leadership wise you think you experienced in moving from director, to VP, to SVP, to C-level?

Shawna Wolverton: Yeah. I mean, I think one of the craziest things is about–I mean, it’s funny, I talk about this a lot–and I noticed it the most when I became SVP, and that’s moving from what I call the person who was sort of, I had this brand as being the person who fought the man, right? I was speaking up for the customer, and I was really adamant, and I was pounding the table, and sort of advocating really hard, often against management. And then, I became the man. And it was a really interesting adjustment for me to understand that, often, there is a toll in this sort of, I disagree but I’m going to commit and go forward that is really required at an executive level. And that adjustment was probably the most interesting of my career.

Sukrutha Bhadouria: Wow. All right.

Sukrutha Bhadouria: Maybe last few questions. I’m curious because a little birdie told me you were best known for getting the customers to fall in love with you. What’s your secret?

Shawna Wolverton: Oh, listening. I think a lot of it was really active listening. And then, here’s the strange thing. I told them no. And I think sometimes we try to please customers, or bosses, or colleagues, and we say, oh yes, all the time. And then, we either can’t deliver, or can’t deliver in a timeline, or people make plans based on that yes. And sometimes a really, really clear, I’m not going to be able to do that is so much more powerful than our sometimes, like, extreme anxiety about wanting to be able to say yes.

Sukrutha Bhadouria: That’s amazing. I didn’t think of that at all.

Sukrutha Bhadouria: Just one last question, and then we should wrap. What was the pivotal moment that helped propel you into senior management roles?

Shawna Wolverton: Yeah. I mean, I think a lot of what I did to get propelled into senior management was taking on projects that no one else wanted to do. I’ve had the good fortune, I don’t know, to work in really fast growing companies where there was always more work on the ground than there were people to do it. And finding out which one of those things was actually really important, and then taking ownership of it, and showing management that I was the kind of person who could take on those hard things was a huge part of my career growth.

Sukrutha Bhadouria: Thank you, Shawna. This brings us to the end of your talk. Thank you everyone who posted questions and the amazing comments. By the way, you got a lot of love for your glasses.

Shawna Wolverton: Thanks.

Sukrutha Bhadouria: Thank you again.

Shawna Wolverton: Thanks, everyone. Have a good rest of your day.

Sukrutha Bhadouria: Thank you too. Bye.

Shawna Wolverton: Bye.

Girl Geek X SurveyMonkey Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Gretchen DeKnikker, Robin Ducot

COO Gretchen DeKnikker of Girl Geek X introduces CTO Robin Ducot at SurveyMonkey Girl Geek Dinner in San Mateo, California.

Speakers:
Robin Ducot / CTO / SurveyMonkey
Sarah Cho / Director of Research / SurveyMonkey
Sarah Goldschmidt / Product Design Manager / SurveyMonkey
Mala Neti / Software Engineer / SurveyMonkey
Shilpa Apte / Engineering Manager / SurveyMonkey
Jing Huang / Director of Engineering, Machine Learning / SurveyMonkey
Erica Weiss Tjader / VP, Product Design / SurveyMonkey
Gretchen DeKnikker / COO / Girl Geek X

Transcript of SurveyMonkey Girl Geek Dinner – Lightning Talks & Panel:

Gretchen DeKnikker: Hi everyone. I’m Gretchen. I’m with Girl Geek X. Thank you so much for coming tonight. Welcome to SurveyMonkey. How awesome is this space?

Gretchen DeKnikker: It’s so good. I love it. I love it. Also, I’m going to introduce a unicorn in one second after I do my little spiel.

Robin Ducot: Spiel?

Gretchen DeKnikker: Spiel.

Robin Ducot: Spiel.

Gretchen DeKnikker: How many first Girl Geek dinner?

Gretchen DeKnikker: A lot. Welcome. We do these every week. We’ve been doing them for ten years. We’ve done like 250 of them so far. We also record them. They’re on YouTube. We do a podcast. It’s in the podcast places, so definitely check those out. Something we were talking about a little earlier, I was thinking the reason that we do these is to give women the opportunity to get up on stage and talk about being awesome rather than talking about being women. One thing I’ll challenge you guys with is find yourself a woke male and bring him next time, because there’s nothing gendered about this content, and building ally-ship is a really good thing.

Gretchen DeKnikker: Without further ado, I know I’m going to get comments on this, but I can’t wait to hear what the feedback is. Send it my way. I’m ready.

Gretchen DeKnikker: Now–

Robin Ducot: Hello.

Gretchen DeKnikker: Hi. I don’t know if you’re ever met a female CTO live and in person, but you’re about to. She’s amazing. In the like three minutes, we’ve just decided we were separated at birth and are going to be best friends now. Right?

Robin Ducot: We’re going to get caught saying something inappropriate at least once. We know this.

Gretchen DeKnikker: Yeah. We were really happy our mics were muted is basically what happened. Without further ado, please welcome the CTO of SurveyMonkey, Robin.

Robin Ducot: Hi. Nice to see everybody here. I’m so glad y’all could come. I’m Robin Ducot. I’m the CTO of SurveyMonkey, and we’ve got a great set of talks tonight. I guess we’re just, without further ado, we’re going to get started. Please, at the end we’re going to be taking questions right after the panel. We hope that you guys will have some interesting questions for us because we are excited to talk to you then and afterwards.

Robin Ducot: Why don’t we get started? Yeah? You ready? Why don’t I bring on stage Sarah. She is from our survey research team, and she is going to talk about surveys.

Sarah Cho speaking

Director of Research Sarah Cho gives a talk on “Surveys: Why Do They Matter?” at SurveyMonkey Girl Geek Dinner.

Sarah Cho: Hello. Thanks. Hi. I’m Sarah Cho. I’m Director of Research here at SurveyMonkey. You’re probably wondering what a director of research at SurveyMonkey actually does, but what we do is we consult with all the different areas of the business from product to engineering to marketing to sales to give them advice on what is actually good survey best practices. You can think of our team as basically being the biggest survey nerds, or I guess for this crowd I should call them survey geeks, at this company. Of course, we can’t invite you to SurveyMonkey and not talk about surveys, so that’s what I’m going to be talking to you about today is first, let’s level set a little bit about what I mean about at survey. There’s a lot of different things that the word survey can mean. While this photo looks amazing and awesome, like I bet a lot of us would like to be up high on this mountain top, and we’re not talking about land surveying here. We’re really talking about survey in questionnaire form.

Sarah Cho: What I like to kind of describe surveys as is you’re having a conversation with a very specific purpose at massive scale. First, I’m going to walk through a couple of different ways that you can do surveys, talk about the different types of surveys that you can conduct, and then hopefully you’ve been inspired that you can go home and do some surveys on your own. I’m going to give you a couple of tips on how to make sure that your surveys are going to yield the highest quality data because it’s a little bit of art and it’s a little bit of science.

Sarah Cho: Surveys. They come in many, many different shapes and sizes. Has everyone taken a survey before? Yes. It could be on the phone. These are like the surveys that call you when you’re sitting down at dinner and you’re right about to take your first bite of food and they’re like, “Hey, will you take a survey?” It could come in person. Maybe it’s someone who’s knocking on your door or maybe you’re at the doctor’s office and they’re asking you a little bit about your medical history or increasingly, obviously what we deal with is surveys that are online. The great thing about online surveys is that they can be provided in a variety of different avenues. One of the more interesting things is surveying kind of where people are working, so utilizing Slack if you guys utilize Slack, and surveying people in Slack or maybe surveying people in Facebook Messenger. Really going to where the people are rather than trying to bother them in the middle of dinner or maybe in their face knocking on their door.

Sarah Cho: The beauty of surveys is that they can really be on any kind of topic. The most famous survey is probably the US Census, or the decennial census. Actually next year in 2020 is going to be the next census, so for everyone regardless of whether you are a citizen or not, that is kind of one of those myths they tell you about the census. Everyone needs to respond. If you don’t respond, technically you could go to jail, although I wouldn’t worry too much about it because they actually haven’t prosecuted anyone since 1970. It’s very highly unlikely, but the census is a really important survey. Things like congressional seats, which eventually turn into who is going to be in the White House, which we’ll leave politics out of it, but anyways, obviously very important decisions are made from the census.

Sarah Cho: The census is a really big government–Surveys aren’t necessarily meant for just only for big government or big business. You really can use them in many, many different contexts. If you know me in my social life, I tend to spam you with surveys a lot, all for various things like organizing camping trips. For this screenshot, which is a little bit blurry, I’m trying to organize friend’s birthday dinner at Beretta in San Francisco if any of you guys have been and just trying to figure out what to offer on the prix fixe menu. Very practical. Very helpful.

Sarah Cho: Another good way that people have utilized surveys in sort of a personal context that I haven’t really shown you here in this deck, but one of my co-workers have actually utilized surveys to ask people about tell me three words about myself on what you think about me. There was a lot of words that came up, things like thoughtful, really considerate, but what was actually more revealing in terms of thinking about her own professional development and growth is the words that weren’t represented. You can also utilize surveys for your own professional growth and development.

Sarah Cho: I talked a little bit about the government context, but another interesting government context is actually how you can utilize surveys to form opinions, sorry, to gauge opinions and make changes, whether it be in the government context with this where they were asking people who were serving in the military about what they thought about serving with a gay or lesbian service member. What they overwhelmingly said was that actually we think that regardless of someone’s sexual orientation, they can effectively do their job. That eventually was evidence that congress used to repeal Don’t Ask Don’t Tell. You see this in the workplace a lot. A lot of organizations are asking employees about what they feel about their workplace. Employers are actually taking that information and making meaningful changes.

Sarah Cho: We do that, for example, at SurveyMonkey. Obviously we like to eat our own dog food or drink our own champagne, whatever the better phrase is for you guys. For example, we did a survey about do people feel like they are included within our SurveyMonkey community? Do they feel like they belong? Actually, because of that, we found that a lot of people didn’t feel like there was a good path to growth within the company, so we made a lot of different changes like having a career ladder, changing from a yearly review cycle to a quarterly what we call growth impact and goals cycle. We’re able to make a lot of these changes and a lot of organizations are doing similar things.

Sarah Cho: Then finally, a lot of people use them for market research. I like to use this example because it’s a silly example. Maybe they should have used some market research here. This is a product called a UroClub. In case you’re ever on the golf course–I don’t golf myself so maybe it’s a bad example, but in case you’re ever on the golf course and you need to go to a restroom and there is none available, you can utilize this golf club to relieve yourself. Clearly they didn’t use any market research for that, so obviously it’s good to use surveys to make sure that you are doing well in that area.

Sarah Cho: Now going onto a few tips about how to create better surveys for you. The first thing is questionnaire design. You can see here, if I were to read this entire question, that would take up the rest of my 10 minute lightning talk, so I’m not going to, but you can see already visually without even reading any of the words, it’s a bad question. There’s a lot of words, TLDR. Remember when you’re writing survey questions, surveys are super visual. If you have a huge block of text, whether it be in the questions or a lot of answer options, people don’t like to read through that, not because they’re being lazy, but you really should be designing with your respondents in mind.

Sarah Cho: The next thing that I like to point out is jargon or industry terms. A lot of times we are so specialized in our industry, so in the survey industry there is this thing called acquiescence bias. Has anyone heard of it? No.

Sarah Cho: If I asked you guys about acquiescence bias, one person maybe, yeah. Most people would be like, huh, what. That’s the whole point. You shouldn’t use terms that only people in your industry are aware of, or the worst is acronyms. Here it’s how would you rate our POS system? Has anyone worked in retail or restaurants? You probably know what a POS system stands for. It stands for point of sale. If you don’t work in restaurant or retail, POS might mean something very different, like are they asking me about the toilet? Instead, you should be very clear. You really want to know how was the checkout process? Instead of using any acronyms or jargon, really stick to something that’s really clear that can be understood by everyone. A good rule of thumb is things are at an eighth grade reading level. That’s generally a good rule of thumb for surveys.

Sarah Cho: You can spend a lot of time on your questions and a lot of time looking at your answers, but then, sorry, you already got the joke. Sometimes you also need to pay attention to your response options, not just the questions themselves. This is a good example. It’s probably someone who maybe was asking about ethnicity and accidentally included Chinese in sexual orientation, but clearly that’s not right. This is also a key for you guys just to remember if you are doing surveys, always have someone else preview your survey for you because it’s like writing a term paper when you’re in college. You don’t realize there’s a typo in the first sentence of your paper that’s like 20 pages long and you’ve read it 15 times. A lot of the times, if someone else takes a look at your survey, they’ll be able to point out things like this that you may have missed.

Sarah Cho: You could write the perfect survey with the perfect response options and then kind of bungle the data analysis. In this particular example, does anyone know what might be wrong here?

Sarah Cho: Yes. It adds up to more than 100. It adds up to 120 and the last time I checked, a closed ended question like this typically adds up to 100. Just make sure that you’re double checking what we like to call number checking or fact checking. That is a crucial step. Another, again, I’m sorry I’m picking on Fox News, but their graphics department is pretty horrible. You can see here that they’re showing a difference between, it looks like a really big difference, right, like between now and January 2013. It looks really big, right, but actually it’s only 4.6 percentage points. It’s 35 to 39.6. That is actually where someone is playing with the axis and making a difference much more magnified than it actually is. This is a good example of not only things you should avoid, but other things you can kind of see when you’re interpreting other people’s data to see if they’ve kind of messed around with the interpretation of it.

Sarah Cho: Just to wrap up because I only have around a minute left, or actually I’m over a minute, sorry. I don’t know how to use this timer. Here are a couple use cases that you can think about. We’ve talked a little bit about them, but remember you can always survey your customers. Even if you don’t have customers, say you’re a teacher, you have students. Survey your students. You can always gain product feedback. Say you are thinking of starting your own company, but you’re probably going to get very biased opinions if you ask your friends and family member. They’re either going to say that’s a really awesome idea, because they don’t want to offend you, or they’re going to say maybe you should stay away from that because they think you shouldn’t take that risk. Always ask for feedback from people you don’t know.

Sarah Cho: We talked a little bit about employee engagement, but there’s a lot of different ways to make sure your team is happy. We talked about inclusion, but there’s always from the minute they step in your door as a candidate to the minute they leave as someone who is no longer an employee of your campus. You can always, we like to say we want to empower the curious, so satisfy curiosities whether it’s thinking about what markets you can potentially expand to or maybe if you want to go back into academia or just put your academic hat on and just think a little esoterically about one specific problem, you can always think about what is the survey component to that.

Sarah Cho: Finally, I talked a little bit about the example about utilizing it for personal and professional growth, but just to wrap up, this is just saying that even if you don’t want to create your own survey, you can still help us survey nerds out there by being a respondent. Make sure if someone sends you a survey request, just take a few minutes out of your time, respond to that survey because that’s really going to help the people out who are sending it out. I’ll be here around in the networking hour and for Q&A. Thanks for listening.

Sarah Cho: I’m also going to now turn it over to Sarah on our product design team. We’ll keep it easy with the same names.

Sarah Goldschmidt: We good? Hello? Hi everyone. Good evening. Thanks for coming. I have a slide, I promise.

Sarah Goldschmidt: It’s not? I’m holding the clicker. Duh. Anyway, user experience. Thanks all.

Sarah Goldschmidt speaking

Product Design Manager Sarah Goldschmidt gives a talk on “Human Readable: Designing Data for Carbon-Based Lifeforms” at SurveyMonkey Girl Geek Dinner.

Sarah Goldschmidt: I’m a product design manager here at SurveyMonkey, and I’m here today to talk about designing data for carbon-based lifeforms. What does that mean? I know, it’s funny. You’re laughing. It’s designed to laugh. That’s great. We’re going to talk about designing for carbon-based lifeforms, which is us, people.

Sarah Goldschmidt: I want you to take a moment and just think about in 2019 the way we think about data has really evolved from the output of a product to kind of the product itself. I want you to hold that in your mind for the next ten minutes because it’s really the crux of the conversation that we’re going to have today. We as makers of products, whether we’re designers or engineers or product managers or marketers, all play a role in defining the relationship between the product that we make and the data that it collects, creates and disseminates into the world. What I’d like you to walk away with is kind of a call to be more curious and more creative and more expansive in your thinking about how we define this relationship.

Sarah Goldschmidt: This is a chart. Everybody has probably seen a chart like this at some time in their life. You’ve probably made one. You maybe made one in elementary school. Charts and data visualizations are a powerful way to make data, particularly numeric data, visible to the human eye. They can also be a crutch. We can over index on them. I say this as a designer who has probably been guilty of this. We receive data that we’re told is important for a product. Users must use this somehow. We spend a lot of time designing the container for that data. We visualize it. We make really beautiful lines and then we kind of leave it there. We don’t ask more questions. We don’t curate the palate of data that we might be working with.

Sarah Goldschmidt: Charts are great for making data visible, but not human readable. That requires meaning. What’s this idea of human readability? I will confess, I made it up. I will define it for you. Data becomes human readable when through the power of design we wrap that data in meaning. Meaning is how we get from 1, 2, 3, to aha. That aha being how people understand how to feel about the data, how they react to the data, and how they can act with it.

Sarah Goldschmidt: I want to talk a little bit about two products here that are really wonderful examples of thinking more expansively about data as a product and about making that human readable. One of my favorite examples is Spotify. Spotify is not typically what you think of when you think of a data company, but Spotify is sitting on mountains of data, millions of users and all the data associated with how they listen to music. They don’t surface this in the app. You probably don’t want to know how many times you’ve replayed the Justin Timberlake song over and over and over. It probably wouldn’t be good for their business model, but somebody at Spotify decided to sift through that data and thought maybe it could become a product itself. Thus, the Wrapped report was born.

Sarah Goldschmidt: I don’t know how many people here get a Wrapped report. I get a Wrapped report. You get a Wrapped report? Awesome. For those of you who haven’t gotten one or don’t know about it, what happens is at the end of the year, in this case, 2018, Spotify is going to send you out this beautifully designed, curated mini-site that essentially tells you the story of your year through the way you interacted with music. There’s really lovely insights. Sometimes it’s funny. Sometimes it’s cringe worthy, but it’s really designed to share. Of course, this exploded into a phenomenon. We’re posting it all over buildings. We’re talking about it on the news, and of course, we’re sharing it with our friends. One of the remarkable things about this is that it wasn’t successful because the data was solving a really huge problem or being used in some kind of scientific theory. It was making people laugh. It was making people reflect, and it was helping them connect with each other. That’s human readable data turned into a product.

Sarah Goldschmidt: This is one of my favorite examples. They went further, right? They were farther and somebody sifted through all that data that Spotify has to come up with these advertisements that went a step further and told the story of the cultural year of 2018. Some of you might have had the great pleasure of seeing shark do-do-do-do-do, a viral kids video that’s actually quite annoying. This is just one example of this advertising campaign that they’ve rolled out. I encourage you to look it up. They are really funny, but again, taking what is actually just a pile of data somewhere in a database and telling a story with it, making it meaningful and having it touch people in various ways. Making them laugh.

Sarah Goldschmidt: Another example that might be more relevant to us in the room, those current product managers, designers, and engineers or aspiring ones, this is SurveyMonkey Engage. Engage is a stand alone product that we make here at SurveyMonkey. It’s designed to help employers connect with their employees to better understand what’s going on and improve their workplace culture. The data that Engage collects and disseminates is almost completely numerical, so we’re stuck in that position of saying we’re going to have a lot of charts because we have to display that data. We have to display the relationships between that data, allow people to filter it, et cetera, et cetera. What you also see in this interface are a lot of words. That was the secret to making this data more human readable.

Sarah Goldschmidt: Where we got to this is we decided people are changing human culture. What’s important to them? It’s probably not sitting around and looking at a bunch of numbers. Our users want to spend more time connecting with their employees and creating change, and numbers don’t create culture change. People do. Conversations do. Relationships do. We thought how do we apply that to numeric data? And the secret was in what we call the core factors of engagement. We did a bunch of research, looked at all the data points we were collecting, and we saw that they grouped into conceptual chunks. What that allowed us to do is instead of giving a bunch of data points related to how people interact in your workplace assigned to a number of questions in a table, we wrapped it in words. That’s team dynamics. Then we named a bunch of areas of engagement, purpose alignment, visible future. We created language. The focus for our users, instead of nitpicking at the data itself, which is available when you want it, you have shared language with your employees to be able to talk about what’s going on. The data, the numeric data displays that kind of supporting role to be able to talk about direction at a later date.

Sarah Goldschmidt: Something that these two really have in common, you’re probably hearing me say the word story. It’s not just meaning that we need to make data human readable, it’s story. When you string meaning together, you really get a story. That’s what a story is. Stories have been used for thousands of years for humans to be able to understand their world, their place in it, and how they can act. It provides that extra oomph. When design transforms data into story, that’s when the magic happens. The magic is that connection and action.

Sarah Goldschmidt: What I think is really important for the take away today is that anybody can make the magic happen. It’s all about starting a conversation and providing perspective. I’m going to give you four steps that anybody in this room can take to start the conversation in your organization or your school or your product company to get data from being kind of just a set of numbers in a table, maybe, to human readable data.

Sarah Goldschmidt: Step one, find what’s important to your people. I use the word people very specifically here, not users. Users are users when they happen to be using your product, but they’re always people. Ask yourself what’s important to them when they’re using the product. Sure, it might be in a business context, but what’s important to them as a human being? Maybe they’re particularly concerned about what their boss thinks of them. That’s nerves. Maybe they’re stressed out. Maybe they want to buy a really cool shirt and your data is going to help them do that. I don’t know, but that’s the questions. When you find out what’s important to your people, it tells you what kind of meaning you need to wrap your data in.

Sarah Goldschmidt: Step two, prepare your palette. Your data palette. As a painter, I want to be able to paint with as many colors as I possibly can to open up more and more types of stories that I can tell. Don’t cut yourself off early by assuming you know what kind of data you need to display and what you can do with it. Take it all, but remember to do it ethically and responsibly.

Sarah Goldschmidt: Step three. Design a story, not a chart. Those charts are going to come in and really help you tell a story visually, but think about the story first. What kind of meaning do you want to string together? What do you want somebody to be able to do, and then build your charts after that to help support the story.

Sarah Goldschmidt: Step three. Validate the aha. All I know for designing with data for many years is that every time I think I’ve really got it down and I’ve got a universal story that everyone is going to understand, it’s absolutely not true. Users find out a new thing to do with it. I get to learn from that, so always validate your aha and watch your story evolve. It’s a participatory sport.

Sarah Goldschmidt: With those four steps, you’ve really got what it takes to start working with human readable data like a champ. I’m going to encourage you today to be curious, have fun, and make something meaningful. Thanks so much for having me today and for listening. I will be around for questions if you want to talk about product design or data or this weird concept of human readable data. Thank you so much.

Sarah Goldschmidt: I’d like to call my colleague Mala up here to talk about engineering.

Mala Neti speaking

Software Engineer Mala Neti gives a talk on “Transforming SurveyMonkey’s Front-end Platform with GraphQL” at SurveyMonkey Girl Geek Dinner.

Mala Neti: Hi everyone. I am Mala. I’m a software engineer here at SurveyMonkey. Today I’m very excited to share with all of you our journey of transforming SurveyMonkey’s front end with GraphQL. Let’s start by talking about 2019. 2019 was a pretty fun year for us here at SurveyMonkey in part because we were able to completely reimagine our front end architecture. What did this mean for us?

Mala Neti: To start, we were able to work towards consolidating our numerous individually deployable and siloed micro webs into just a handful of apps built on React and Node.js. We were able to introduce a very slim aggregation layer with GraphQL that sits between our front end and our back end collection of rest APIs where all of our business logic and application logic are hosted.

Mala Neti: Today I wanted to focus on the aggregation layer portion using GraphQL, but before I do that, I would love to get a gauge from the audience. How many of you have heard of GraphQL?

Mala Neti: Nice. How many of you have worked with GraphQL in any capacity? Not bad. More than I was expecting.

Mala Neti: For those of you who haven’t, GraphQL is simply a query language for APIs. You can think of it as making it easier to build out your front end by providing a declarative way of fetching data without having to have knowledge of the entire system. In that way it kind of separates the back end–innovation of the back end, from the front end and kind of makes them completely independent. In addition, graph APIs are organized in terms of types and fields rather than endpoints, so the client can get as many resources as possible in one single request to one single endpoint or as I like to call it one endpoint to rule them all. I did not come up with this even though I think it’s pretty genius.

Mala Neti: I think it’s easy to see why GraphQL or the performance benefits that probably come from using GraphQL with this declarative and efficient type of data fetching. Spoiler alert, it worked. For the pages that we have today powered with GraphQL, we saw a huge reduction of payload and a huge reduction of data sent to the client, and a huge improvement in our time to first interaction. This is a really big win for us and for all of our users of SurveyMonkey product.

Mala Neti: Now that we know that it works, let’s talk a little bit about how it works. We have our GraphQL client and our GraphQL server. At the center of our GraphQL server is our schema. Our schema is basically just the contract between our client and our server telling the client how it can access the data. It does this via API operations and data types. The API operations can be read and write operations like queries and mutations, which are basically sort of top level entry points into the graph, and data types like [inaudible], scalers, objects, and the list goes on.

Mala Neti: Facades are then R functions, sorry, resolvers are then R functions that map our API operations to our back end services code. Once we have our schema and our GraphQL server sort of built out, we can easily then build out our GraphQL client knowing what data is available to us and how it’s available to us. As we talk about building out our GraphQL client, it’s also very important to talk about the amazing tooling ecosystem that GraphQL provides us. At SurveyMonkey, we use a GraphQL playground which is basically an in browser IDE and it allows me to hit my GraphQL server and build my queries directly in here in this tool and see my responses. If I didn’t have any sort of involvement in actually designing my schema, I have all of my schema details documented right here. I have my queries and mutations that are available to me, my type details, what fields are nullable and non-nullable and so much more.

Mala Neti: Not only do we have an amazing performance benefit that we talked about earlier from GraphQL, but we also have an increased developer discoverability and productivity, especially from a front end perspective, which is pretty exciting. Now that we have a little bit of context as to what the GraphQL server entails and some of the tools that we can use to build on top of it, let’s dig a little bit deeper into building out our GraphQL client.

Mala Neti: Say I was building out this My Surveys page and I wanted to fetch data that told me what surveys to render. Looking at my tooling and my schema code, I know sort of that I have some options and what’s available to me. I can start by building out my query. This is just a simple query. I have named my query for the purposes of debugging and discoverability. As you can see, I pass in an argument which is language ID, which we have set to be a non-nullable integer, and I pass that in as variables as well as in pagination properties to my survey categories nested field. Under the hood, this basically dictates how my data is going to be resolved and what data I get back.

Mala Neti: Then, as you can see, all I asked for, the only subfield I asked for on items is ID and name. Nothing more and nothing less. I’m truly just asking for all I need. As you can see from my response, the structure of this response mimics the structure of my query. This is really great because I ask for something and I get predictable results back. One of the things to keep in mind as you’re building out your GraphQL client is that at the end of the day, GraphQL can be driven by the data requirements of your products. The people who are building, or the developers who are building your UI can have a little bit of that responsibility as to how they get the data that is building their UI.

Mala Neti: Now that I have my query, how do I actually, or I’ve built out my query, how do I actually incorporate it into my front end? How do I incorporate it into my React application? At SurveyMonkey we use the Apollo client platform, which is basically just a GraphQL implementation. It exposes me to a lot of cool things, one of which is my query component. I can basically pass in the query invariables that I wrote and built earlier as props to my query. I can also pass in this render prop function, which as you can see, exposes me to my various different query states, whether that’s loading, error, and so on. Based on my query states, I then can intelligently render my UI, which is pretty awesome.

Mala Neti: One sort of best practice that we like to follow at SurveyMonkey when we’re building our components and their respective queries is modularity, the idea of modularity and reusability. GraphQL works very well with that because it allows me to co-locate my data with my components. The data requirements of my components can live right beside it. That’s important because if I needed to add fields, delete fields, delete a component, really make any sort of change, GraphQL, doing it this way allows me to do that. I don’t have to worry about going up the component tree and having a query that basically powers multiple different components. I can just deal with the component at hand and the data that belongs with it.

Mala Neti: Another sort of application of this concept is component boundaries. If I have a network error or error fetching my GraphQL data, rather than having my entire page fail or multiple components fail that are dependent on a particular query, all I have to worry about is gracefully degrading my components that are dependent on that query, which is great because this is a much better user experience.

Mala Neti: All of this is awesome, but this means that by default every component will have to make their own network request, and we did just talk about how one of the selling points of GraphQL is minimal round trips. Because of this, GraphQL has actually come up with various ways to deal with this problem. What we are currently using today is query batching. That essentially allows all of my queries to be combined into one request. This has advantages and disadvantages. There’s alternatives to this, but we found as of now this works for us because it allows us to think about modularity while still reducing the number of requests and kind of gaining that performance benefits from GraphQL.

Mala Neti: From this example, I hope that it’s easy to see why we at SurveyMonkey love GraphQL. I think, A, our developers are happy because we have exposure to an amazing tooling ecosystem. We have a predictable developer experience. Our back end engineers are also happy because we have, they can sort of iterate on their code independently of sort of the new requirements that are coming in on the front end. My customers are happy because I have a performant robust product. If my developers are happy and my customers are happy, we have a really happy SurveyMonkey.

Mala Neti: With that, I wanted to say thank you so much. We’re really excited about what we’re working on here at SurveyMonkey. If any of this sounds exciting to you, please come hang out with me after and I’d love to chat. Thank you.

Robin Ducot: Now we’re going to have a little chat with some of our leaders here at SurveyMonkey about career topics. I’d like to welcome to the stage Mala. Not Mala. Mala just left. Shilpa, Jing, and Erica. Thanks, guys.

Robin Ducot, Shipla Apte, Jing Huang, Erica Weiss Tjader

SurveyMonkey girl geeks: Robin Ducot, Shipla Apte, Jing Huang and Erica Weiss Tjader share advice and stories at SurveyMonkey Girl Geek Dinner.

Robin Ducot: I have a few questions, but let’s get started with introductions. I’m the CTO here at SurveyMonkey. I have been, let’s see, 30 years doing technology and about 15 of those years, last 15 years as an executive. I’ve managed everything from product, program, engineering, QA, ops, I don’t know, security, IT, all different types of areas from companies as big as Adobe to companies that you’ve never heard of that are gone now. I love engineering, and I’ve been here at SurveyMonkey 18 months. It’s just been a really exciting journey. Fun fact. Fun fact, I used to play in punk rock bands. People will tell you this and laugh. Also, 25. This number is the metric I use to measure the number of engineering leaders I’ve created in Silicon Valley. I love coaching and mentoring leadership skills for technologists and helping people grow into leaders of technologists is one of my passions, so 25 in Silicon Valley.

Robin Ducot: With that, why don’t we get started with some introductions of the other folks. Shilpa?

Shilpa Apte: Sure. Hello? Can you hear me?

Robin Ducot: Yeah.

Shilpa Apte: Sorry. Hi. I’m Shilpa. I’m an engineering manager here at SurveyMonkey for the respondent experience team. We’re the team that manages and innovates on the survey taking page. It’s the page you see when you take a survey. I’ve been at SurveyMonkey since I was an intern in 2012, so just about seven years. Let’s see. Highlight of my career has been getting to move to Dublin for two years to set up the engineering team there. That’s Dublin, Ireland, not Dublin, East Bay. My fun fact is that while I lived abroad, I traveled to–I did 17 weekend trips to countries I’d never been to before.

Jing Huang: Hi everyone. My name is Jing Huang. I’m a director of engineering focused on [machine learning here at SurveyMonkey. I joined SurveyMonkey for three years. Fun fact, I did a hiking trip to Everest base camp with a 14 day hike. If you’re ever hiking in a high altitude you’ll know like if you get a cold, it’s devastating. I got a cold in day five, but luckily I’m here, so I survived. I was able to complete the hike. The highest point I was able to reach was more than 18,000 feet, which is something I get to blab a lot. Some highlights of my career, I studied robotics and artificial intelligence. After graduation, I didn’t find a job in ML that was like more than a decade ago within a prime time for ML, but I was able to work on different applications from network security appliances to cloud management to big data. Finally I got an opportunity to work on ML here in SurveyMonkey, so that’s where I am.

Erica Weiss Tjader: Hello. Good evening. I’m Erica Tjader and I’m the VP of Product Design.

Erica Weiss Tjader: We had a joke in rehearsal that I wasn’t even going to need a mic I’m so loud, and they turn mine off. That’s a funny joke.

Erica Weiss Tjader: I’m Erica Tjader. I’m the VP of Product Design here at SurveyMonkey. I have been here two years. The baby is due August 1st. I thought I’d just get that out of the way so we can be done with the awkward like is she, is she not. Anyway, highlight of my career was landing this job at SurveyMonkey. I know that sounds really cliché. I promise they’re not paying me to say that. Tom, my boss, isn’t here, but I think the reason why it stands out as such a highlight to me is not just because of the great role that it is, but actually what it represents in terms of me doing some things that were really out of character for my personality. One was risk taking. When I first kind of, well I don’t know if I applied for the role, but when I first pursued the role, it was a really big step up for me in my career. I definitely did not meet all of the requirements in the job description, so it was a big risk to kind of throw my hat in for it. I think that was one big piece.

Erica Weiss Tjader: The second was intentionality. When I was starting my job search, I made a list of two, just two, criteria that I absolutely must have in my next role. I was really specific about looking for that and not settling for a role that had less. Then I think the third is patience, not my strong suit. Over the course of a couple of years, the first time I passed on SurveyMonkey, the second time they passed on me, the third time was a charm. Just the highlight of my career just I think because of how I’ve reflected on what it says about me.

Robin Ducot: Thanks, Erica. All right. Let’s start with some questions. How about we talk about your current work life. One of the things I think is really, really important is that for you to be successful, if you don’t, aren’t really engaged in what you’re doing, if you don’t really like it, it’s really, really hard to be successful because you’re just not going to be excited about putting the effort in that it really takes to make a success of yourself. What I’m curious about is what is one thing you do in your job function that gets you really, really excited to come to work? Why don’t we start with you, Shilpa.

Shilpa Apte: I love coaching and mentoring people. I love getting to know people and figuring out what their aspirations are, what skills they want to learn, where they want to take their careers and kind of seeing that progress being made over time. I see every single day as a new opportunity to learn something new about someone on my team mainly because people are complex and they change. I just think it’s an interesting problem.

Robin Ducot: Jing, what about you?

Jing Huang: I mentioned I studied robotics and I was a SciFi fan ever since I was little. I was always curious about how AI will shape our future. Getting to work on machine learning here is just dream come true. We have more than 40 billion people [inaudible] data collector here at SurveyMonkey, so this is really a dream job.

Robin Ducot: Erica?

Erica Weiss Tjader: For me, I think one of the things that I really enjoy about this role is the strength it plays to. Anybody familiar with the StandOut strengths assessment? It’s a survey template you can use in our library, but anyway, they also didn’t pay me to say that. Imagine that. I’m a connector. That’s one of the strengths that always stands out for me. I think it’s something that I get to do a lot of in my role. Whether it’s the day-to-day of connecting designers on my team with new challenges or mentors or growth opportunities, whether it’s working with teams to find ways to connect teams with other teams that have similar products or project challenges, and of course even in the nature of our product itself which is about connecting our customers with their customers for feedback. I think it’s just a role that I really get to leverage the strength a lot and that’s really energizing for me.

Robin Ducot: Thanks, Erica. Why don’t we talk a little bit about your career journey. One of the things that’s really, really important is being able to advocate for yourself. How have you guys advocated for yourselves throughout your career and what tactics have really been helpful in making you be heard? We’ll start with you, Jing.

Jing Huang: Sure. To be honest, I’m an introvert, so self advocating wasn’t natural to me. My pivotal point came when I realized self advocating is not only about self, it’s actually about advocating for your team. It’s also mutually beneficial for you and for your manager. There are two tactics I want to share today that will be helpful. I think it was helpful. Number one is to do regular updates of your work progress. Doesn’t matter if your manager asks or not, but do it. You can do it through email. You can do it through your one-on-ones. Do keep those cadence. That’s just going to be mutually beneficial because your manager wanting to know your work. That’s the number one thing.

Jing Huang: Second thing is share your knowledge with your peer, with your cross functional partners. Do tech talks. Do lunch and learn. When you go [inaudible] try to talk in conferences. This is also a mutual beneficial thing. It’s self advocating in one way, but it’s also sharing your knowledge. Everyone else is going to learn from you. Those are two things I wanted to share.

Robin Ducot: What about you, Shilpa?

Shilpa Apte: Yeah, for me self advocacy, for me it was a pivot when I realized that if you do it a little bit over time, it’s much easier to have important conversations when you really need to rather than trying to do a big one all at once. The way in which I do that is, one, to just believe in myself. I think if you believe in yourself, build confidence in your abilities, it trickles into everything you do. Two, understanding what your manager expects of you and meeting those expectations, it’s just easy points. That’s not to say that you should feel restricted by those expectations, but I think that’s kind of a baseline. Beyond that, you should kind of like, what Jing said, be giving regular updates about what you’re working on. The third is really being able to articulate what value your work is creating for the business and for your team and making sure that your manager understands that as well.

Robin Ducot: Those are great suggestions. It’s funny, for me, one of the things I’ve noticed is that developing a very strong sense of entitlement, which sounds negative, but I swear it’s positive, developing a sense of entitlement that you are allowed to be in the room, especially for female technologists I’ve noticed this is an issue. I was fortunate in that I’m third generation technology leader in my family, so I didn’t realize women weren’t supposed to have a voice around technology. I’ve noticed that it’s really, really important that you feel entitled to speak up. Your voice is just as important as anybody else’s. If somebody interrupts you, interrupt them back. You know?

Erica Weiss Tjader: Well said. Well said.

Robin Ducot: All right. Moving on. Mentorship. People talk a lot about mentorship, but what’s interesting is some of the most influential relationships that you end up having because they’re more common are the allies that you develop in the workplace. I’m a little bit curious about, for you, Erica, tell me a little bit about some of the experiences you’ve had with allies.

Robin Ducot, Erica Weiss Tjader

VP of Product Design Erica Weiss Tjader shares tips for making allies in the workplace, for your professional success, at SurveyMonkey Girl Geek Dinner.

Erica Weiss Tjader: It’s an honor to have Robin Ducot ask me this question because she is one of my greatest allies here at SurveyMonkey, but that’s not what I’m going to talk about. I was actually trying to think of a really good example, and I found the list was so long I couldn’t even remember the last names of half of the people that were coming to mind, which I think is either indicative of my memory or actually the nature of allies. Unlike mentors, allies are not big investments in relationships over time. They are episodic. They are based on a specific purpose at a specific place and time. As a result, they can have a really much bigger impact on something you’re trying to achieve at the time.

Erica Weiss Tjader: A good example that I thought of is in a previous role I was a design leader of a smaller team. One of my biggest challenges that I was facing was making inroads with our engineering leadership around the notion of, the importance of front end development, design systems, some of the topics that design and leaders and engineering leaders often talk about. I was having a hard time getting traction. It was one of those tough problems because it was probably the most important thing to my team and yet the thing I had the least direct control over. This was an example. I have to influence because I don’t own the answer to the problem. This particular ally was a new engineering manager that joined the organization. In my initial just meet and greet with him, I learned that he had some expertise around developing front end teams and design systems and sort of an interest. Perhaps most importantly, I learned that he had a personal relationship with our CTO who was the person I was having the hardest time making inroads with.

Robin Ducot: Wasn’t me.

Erica Weiss Tjader: They were personal friends. No, it was a different story. I’ve got a lot of stories.

Erica Weiss Tjader: What I did is I really just started out by befriending this guy. I’m like I’m going to make your transition into this company really easy. I’m going to introduce you to people. I’m going to tell you all the secrets. We had lunches. We had coffees. We started to build a relationship, and in a very short period of time we were able to transition that relationship into finding a mutually beneficial place where he was able to leverage his expertise and his influence in the engineering organization to start a front end team and I was able to give him disproportionately more resources and support from the design team to really improve the value and success of that.

Erica Weiss Tjader: I think it’s just a great example of an alliance that was very intentional, but looked very different than a mentorship relationship because it was really about a place and a time and a need and a relationship right in that moment.

Robin Ducot: That’s a wonderful story. I mean I think that’s something that’s really, really important. I know as an engineering leader, one of the allies that I always develop when I first join a company is the person who runs the customer support team, making sure that they understand that I am there to help them when we screw up because obviously the site does have issues on occasion. Making sure that you have that relationship so that they support you when you inevitably screw up and then you can support them. I’ve noticed over the years that this is a pattern that really, really works.

Robin Ducot: I do have a question. We are in sort of the center of the feedback economy, SurveyMonkey is. I feel like feedback is–one of the reasons I love working here is because I love data, I love feedback, I love learning new things and driving insights from data. One of the things, though, that I’m curious about is feedback is so important in developing our career. What is some feedback that you’ve gotten? We’ll start with you, Shilpa, that you’ve gotten that really made a material impact on you?

Shilpa Apte: It really can be summed up in two words. Be reliable. That’s not to say I was unreliable, but you commit to something and it starts slipping and you don’t tell the person that you told you would do it in time that it’s slipping. I don’t know. These little habits that build over time I think I kind of realized that you can be the smartest person and the most talented person in the room, but if people can’t rely on you long term, they’re not going to want you to be on their team. I think once I realized that, that kind of just staying on top of things and making sure, even if I’m not going to complete something that I said I was going to, just communicating that out made my relationships and trust between colleagues a lot stronger. That was kind of a career shift for me.

Robin Ducot: That’s great. For me probably, if I think of some of the best feedback I’ve ever gotten, I have a lot of imagination and I like organization structures and redefining things. As a leader, I got the feedback from an executive coach some time ago that, you’ve got to stop reorging every week. You’ve got to stop all that shenanigans with the reorgs because people, you may be able to surf over the top of all this change and this chaos and you like change, but people, a lot of people like stability. Stop. It was really, really helpful to me because it helped me see the world slightly differently than the way I’d been seeing it. I was like but it’s so exciting all this change. They were like actually you’re killing people. I think that getting feedback, soliciting feedback and listening to it is a really, really important part of developing your career.

Robin Ducot: Advice for others. What skills and experience have been the most valuable to you in developing your career growth? What advice would you give to somebody if they wanted to turn to a career in tech? Jing, we’ll ask you this question.

Jing Huang: Sure. When I think of this question, I actually think of it from a different angle because tech, or the industry we’re in, is fast growing, ever changing. It is not any particular skill or experience that will determine our successful in this industry. It is what I believe is the growth mindset that we all should have and the willingness to learn new things and the courage to actually take on new challenge. I think if you have that, you will be able to learn. Just be sure that you’re actually enjoying doing tech in the first place, right? If you’re sure with that and learn, you will be successful. It’s just time.

Jing Huang: One thing I liked about what Steve Jobs said, long pause, this is an old sentence, but I think it always inspired me is really you cannot look on how dots connected when you look forward. You only see the connections backwards. When you see a new challenge, when you see a new opportunity, take it. Do it. You will find how that’s connected after some years. Robin would know better.

Robin Ducot: Because I’m old, yeah. Lots of connections.

Robin Ducot: I think one thing that’s really important, if you’re thinking about a job in tech is that you have to like solving problems. As an engineer, that is your job. Your job is to come up with interesting and creative ways to solve problems. If you don’t like solving problems, if you think problems are a pain, then you’re not going to really enjoy being an engineer. To Jing’s point about start first to see whether basically this looks like an interesting career to you because if you don’t enjoy it, if you don’t enjoy solving problems, you’re not going to enjoy being an engineer.

Robin Ducot: With that, our final question we’ll take to Erica about finding the right company, the right place to work. What advice would you give the women in the audience who are looking to change companies?

Erica Weiss Tjader: First, I’d say the recruiting booth is over there. No, in all seriousness, I think the first question I’d ask is why are you looking to change companies? Are you running away from something or to something? I think it’s really important to be true to ourselves and be really specific with ourselves when we start to think about making an important change like that and start with the foundational assumption that there’s no perfect job, there’s no perfect company, there’s only the right job and the right company for you right now. I think how do you get to that assessment? I always encourage people to start with a list of what are the top three most important criteria for your next role. You only get three. There’s somebody in the audience I know I’ve interviewed recently, so she’s familiar with the question. Don’t worry. She’s joining.

Erica Weiss Tjader: Anyway, I digress. Start with your three criteria and then be really intentional about how you go out and look for a role that meets that criteria. As women in technology in the Bay Area, we all have the luxury of recruiters hitting us up all the time, of going to events like this where hiring is a big focus. Talk to those people who are seeking you out, but also seek out companies and people who you think are interesting or who you think might meet your criteria well. Have coffees. Have lunches and casual conversations. Find that role that really speaks to what you’re looking for. I often say that the best reason to look for a new job, not the only reason, but probably the best reason is because you’re looking for a very specific type of growth or opportunity that your current company can’t give you at this time. Usually it’s no fault of the company. It has to do with the size or the stage or the priority.

Erica Weiss Tjader: Have those specific criteria, be deliberate, and then go out and look for a job and do not accept a job that meets any less than two of your three criteria. I think just be really true to yourself there. I think the last thing I would say in terms of advice here is to really be open minded and flexible because after you’ve gone through that process of thinking through your criteria, of talking to people, you actually might find that you look at your current company through a different lens or see opportunities there in a different way. It might be that right now that company is the place that meets your criteria the best. Really just be open minded in that search and clear about what you’re looking for and what you’re running away from or running towards.

Robin Ducot: That’s really, really great advice. Another thing I would also think about is that you are joining a group of people and if you don’t like them or if you don’t like the person you’re going to work for, doesn’t matter how great the company is, your job is going to suck. They really are not going to be good advocate for you. If you don’t connect with them, if you don’t feel like they are somebody who is going to really understand your deal and what’s good and strong about you in particular, they don’t appreciate it, then find somebody else to work for. That’s my small piece of advice on finding companies to work for.

Robin Ducot: That is our last formal question. What we’d love to do is get questions from the audience. We are, I’m just wondering, do we have a mic we can hand around? All right.

Erica Weiss Tjader: Bernice is over there with a mic.

Robin Ducot: We have a question here.

Natalia: Hi. I’m Natalia.

Robin Ducot: Hi, Natalia.

Natalia: Should I stand up so everybody can see? I’m really curious about how you approach difficulty to make people feel in the survey. We’ve already heard about best practices, how to run great survey, but how to encourage people to look into it? Maybe let me also explain the question where it’s coming from. Five years ago whenever I received a survey, I just filled it in because it was new and it was so cool. Now I receive surveys like in my career and then private ones and then some place for my husband and so on and so forth. There’s a lot of them. How do you make people start the survey?

Robin Ducot: I think I’m going to have Sarah, if she’s okay with that, come up here and maybe answer that question just as our survey research expert.

Erica Weiss Tjader: Sarah, I’ve got a mic for you. You thought you were getting away.

Sarah Cho: It’s a little hard to hear over there. Do you mind repeating what you had to say?

Natalia: Can you hear me?

Sarah Cho: Oh, how to increase response rates. Was that the question?

Natalia: Yeah, so how do you make people actually start the survey because [inaudible] so many of them from various sources.

Sarah Cho: That’s a really good question. A lot of people worry about response rates, so there’s a couple of things that you can think about. Number one, how are you inviting them? How many of you guys, every workplace has this, has this type of person. We utilize Slack here, so I’m sure you guys might utilize this. I hate it when someone Slacks me and says, “Hey.”

Sarah Cho: Then you’re forced to say, “Hey, how can I help you?” I find that, that’s like one of my pet peeves. It’s the same thing with survey invitations. If you just say, “Hey, take my survey” what’s the incentive for anyone to actually take your survey? That’s not engaging. That’s not personalized. That’s nothing, right? One of the things is to really think clearly about your invitation. It’s best if you can customize it. If you know the person’s name, utilize the person’s name. If you can disclose what the survey is about, give a quick one sentence, not too long again, description of what the survey is about. The other thing to remember is the first impression is the biggest impression that you can make. Again, thinking about the invitation, you don’t want to use too much text because people are going to get overwhelmed and not read it. Same thing goes for the first question in your survey.

Sarah Cho: If you start with a really hard question, we actually see this because we collect so many responses. We can actually go through our database and see what decreases response rates and what increases it. The first question is actually one of the most crucial. If you start with an open-ended question, which is really hard for people because you have to type out your answer, your completion rate automatically drops on average around 15 percentage points. That’s a lot. If you just start with an easy question, even if it’s like a soft opener that you kind of need to throw away, that’s always best. Start with a multiple choice easy question.

Sarah Cho: Those are two small things that you can do. Number one, personalize how you’re inviting people in, and number two, make your first question easy. There’s a lot of other things that you can consider, like incentives, but that’s a whole ‘nother talk. I’ll save that for later.

Shilpa Apte: I would also just add on that within the product itself when you’re creating a survey, there’s a tool called Survey Genius that actually tells you, you’re welcome, that tells you, kind of scores your survey and gives you feedback on how you can improve response rates based on question ordering and length and all that.

Robin Ducot: Who’s got the mic? Bernice.

Audience Member: Can you hear me? My question is if we’re not necessarily sure our three, oh sorry. I won’t yell. All right. If we’re not necessarily sure what our three criteria are, what we want to do, what we want our next step to be, how can we go about finding that next step and just kind of a general what do you want to be when you grow up more?

Erica Weiss Tjader: A couple of us might have thoughts here. My first thought is don’t try to make one of your criteria like this is what I want to be 20 years from now when I grow up. That’s like this well thought through career that probably won’t even exist 20 years from now, right? I think when I talk about criteria, I’m actually talking about really specific attributes of your role. For example, one of my criteria when I was joining SurveyMonkey was I had a 10 month old at home. I knew that eventually I was going to want to have another child. I knew that I was going to work a lot of hours no matter where I went, so I knew short commute was super important to me. One of my two was short commute.

Erica Weiss Tjader: I think it’s about what is important to you in your life right now. If you’re earlier in your career, you might say what I really like is I’m a really outgoing person and I really get a lot of energy from talking to people. If I don’t know what function I want to be in, I know that it’s super important to me to have a role where I interact with people for more than half of my day. Then you can start having the conversations and you’ll find out which roles actually fit that criteria and which don’t, but don’t start out with like I must be in sales or I must be a front end engineer, like trying to get too specific about the career path. I think that will emerge for you as you think about what kind of work or environment gives you energy and you’re passionate about.

Jing Huang: One thing I think I can add on is when you really don’t know what you really like, I think when I was in school this was a question. Look at what other opportunities are out there. Try to do them. You can always think about you like or not, but that’s just thinking. You have to do it and actually test if you actually like it or not. Don’t pass an opportunity thinking you may not like it. If there’s an opportunity, actually catch it and try to do something about it.

Robin Ducot: Experiment.

Audience Member My question is how do you solicit constructive feedback? I find, personally I find asking directly is not the most effective way. Actually, I’ve been thinking about sending out survey. I’m wondering if you have any insights.

Robin Ducot: We’re running a feedback survey right now in engineering. Anonymous is good. If you really want to find out, for an organization or for something that you’re doing, anonymous is good. If it’s personal, if you’re trying to get feedback from people personally, you just have to show that–it takes time to get people to trust you, that you won’t freak out. Ask for feedback and it’s still too sugar coated, poke a little. Have it be a little less sugar coated. Poke a little, less sugar coated. Not reacting. Just keep asking questions. Then people will start being willing to give you the kind of feedback that you really need because you need people to believe that you can handle it, right? That’s why people sugar coat things because they don’t want to deal with people’s emotions when they upset them.

Robin Ducot: Present yourself as somebody who is just interested in the facts and it really, really helps.

Erica Weiss Tjader: I think another trick I’ve called on for some of the folks on my team, I’ve had people reporting to me who are the type of people who always get positive feedback. Every time we send the anonymous peer feedback 360 survey, everybody says they love working with me. By the way, that’s true. That means that you’re great and you should feel really good about that, but obviously it’s not as constructive. I think there’s a couple tricks. One is ask a different level of person for feedback. Perhaps you’re asking people who are your current peers, not the people who you would like to be your peers in your next role, right, in terms of thinking about asking the next level.

Erica Weiss Tjader: The other thing is asking the question in a different way. Rather than saying, let’s just say your ambition is to be a more influential leader. Rather than asking somebody what room for improvement do I have or some sort of generic question, ask I’m really focused on trying to have more influence as a leader. What are some areas where you think I could be more influential? What could I do differently to have more influence? Really ask about the thing that you have identified as the thing you want feedback on specifically as opposed to just general feedback questions.

Robin Ducot: Yeah. I’ve noticed also real time feedback. You’re in a meeting and the meeting’s over. Asking the person do you think that thing I said in the meeting was okay, where you’re right there in the moment and they might be more inclined to give you feedback because you just had this experience. It’s hard also for people to remember if you’re asking things that are too general. It might be another thing that could help you.

Robin Ducot: We’ve got a question back there in the orange.

Audience Member Can you hear me?

Robin Ducot: Yes.

Audience Member Thank you again for organizing this. I have used SurveyMonkey for many years for my own events. I used to be in marketing, so thanks for this opportunity to see you in person. I used to be an engineer who moved into marketing, product management, sales ops, kind of seen all the 360 of the business. One of the comments Robin made around problem solving, that one aspect that I learned in engineering has stayed with me throughout my career, so I just wanted to say that to the audience and also to anybody who is in tech, if you don’t like problem solving, then your career, you basically are lost.

Audience Member: The one question I have is in engineering especially, given that I’ve done everything, I’m realizing now that if I wanted to come back, what sort of strategic leadership roles could one aspire to? Is that even possible?

Robin Ducot: You mean you want to move laterally from where you are, the level that you’re at now, back into engineering?

Audience Member: Yes.

Robin Ducot: Probably you’d have to go join engineering at the level you left, not the level that you are, just because engineering leadership, usually you need to have the respect of the engineers that are working for you, and that means having been sort of through the ranks of leadership and engineering. At least that’s been my observation. If you’re willing to go back and if you left as an engineering manager, joining as an engineering manager again and working your way up, at least that’s what I would guess unless you go to a start up. A start up, everything’s possible. You can be a CTO at a start up with an accounting degree. It’s totally fine. Just write some code.

Robin Ducot: Sometimes I’ll tell people that. You get somebody who is like I want to be a CTO now. I’m like go join a start up. You can do that. Knock yourself out. Anyway, I digress. I think if you really want the leadership role early, start ups are great for that.

Audience Member: Hi. I have a question. What if you like what you’re doing, but you just don’t like the people you’re working with?

Robin Ducot: Get a new job. Get a new job.

Robin Ducot: We got recruiting over there.

Robin Ducot: Life is too short. If you hate the people you work with, get a new job. You’re not going to be able to fix them. I’m just saying. I don’t know if somebody has a more sophisticated answer than that.

Audience Member: Hi there. My name’s Vidia. It’s fantastic to be here and see all these women at the panel here. It’s been great. Robin, my question is for you. It’s great to see a female CTO. My world is full of the other gender, so it’s very nice seeing this. Can you talk about your journey? You said you’ve been in tech for 30 years? What were your pivotal moments when you look back now and said okay this was a game changer? I’m sure you have a few.

Robin Ducot: Oh boy. I was really lucky. Like I said, my mother ran a huge research team at MIT, so I didn’t know. I didn’t know. I didn’t know women weren’t supposed to be engineers. Nobody told me that. I also liked being an outsider. I was a punk rocker. I didn’t mind being the only female engineer in the room, and again didn’t realize I wasn’t supposed to be there because nobody told me and I didn’t mind being an outsider. I was really, really lucky. I think that along the way things that I had to learn that were hard to me, when I realized that after getting into fights with more than one boss over multiple companies realizing that it was me, that it wasn’t them, that it was me, and that I had to work for people that I liked and who understood my special snowflake self.

Robin Ducot: I think that everybody’s got a certain set of strengths and weaknesses. Over the years, developing a career around my strengths and then hiring from my weakness. It’s not one specific story. It’s sort of a constellation of experiences that have led me to believe that if you focus on your strengths and hire for your weaknesses, you could actually be really, really successful. I don’t know. It’s kind of a long, windy answer to that because I don’t really have specific things that come to mind as oh my God that time when. There have been moments in time of pieces of advice I’ve gotten. Because I will have a street fight about things, I got the advice once of, Robin, sometimes you just got to keep your head down. Keep your head down and just keep moving. All those people that you want to kill, they’ll be gone and you’ll still be there. Then you get to run the show.

Robin Ducot: The guy who gave me that advice is my boss here. I worked with him 20 years, I mean we’ve been working together on and off 20 years. That was a long time ago. It was great advice. Anyway, there’s been lots and lots of little experiences that have brought me where I am, but I do think focusing on your strengths and being resilient. You’re going to get your ass kicked every day. Just keep showing up. That’s been my experience. Again, like I said, I liked being an outsider so I didn’t really mind. I always thought this is my fight to have, but what I’m realizing and what I’ve come to realize as trying to help other women move through, not everybody likes being an outsider. Not everybody wants to have that fight and they shouldn’t have to. You shouldn’t have to have that fight. It shouldn’t be part of the deal. You should just be able to be a woman in tech and not have it be a big deal.

Robin Ducot: It kind of annoys me actually sometimes. I feel like I’m a prized poodle that’s being trotted out. Female technology leader. I want that to be the norm. I want that to be everyone can be a CTO and be female. It’s like that’s not a thing, but we’re not there yet. I recognize that. That’s just my long winded answer to your question.

Audience Member: Hi. My question is SurveyMonkey consider yourself like Uber in survey industry? If not or yes, what is your major competition?

Shilpa Apte: Do we consider ourselves to be Uber in surveys and who is our major competition?

Robin Ducot: We are the Uber of surveys. Oh yes. Oh yes.

Robin Ducot: It depends on what market, actually who our biggest competitors are. It depends on whether it’s the consumer market, sort of low end surveys, basic surveys, and then you have Google forms and things like, I don’t know, Type Form. That’s who I was thinking of. Type Form. See how much I worry about them? Then at the high end in market research you have Qualtrics. One of the things I love about SurveyMonkey is that our product works for all of these markets, for the big and the small, and that’s actually what makes us unusual. Type Form doesn’t work for market research and Qualtrics doesn’t work for the basic user.

Robin Ducot: Yes, we are amazing. I totally think that.

Angelica: I had a question. I actually had the reverse question of someone here who asked a question. My name is Angelica and I’ve been in the tech industry for about ten years now. I’ve been fortunate to have a very supportive boss and team. My situation is I don’t really like what I’m doing anymore. It’s a short glass ceiling where I’m at and there’s not really a lot of room for growth or advancement. Just asking for anyone’s opinions about being in that kind of situation, having a supportive team and a supportive boss, but not really being happy with what you’re doing.

Erica Weiss Tjader: I think we call that the curse of being too comfortable or something like that. Again, I started with the there is no perfect job, there is no perfect company. I think it depends where you are in your life. I think you might be at a point in your life where you’re like these are the years where I want to work hard and I want to charge ahead and I’m willing to pay the price to do that. Therefore, staying in a role where you know that there’s no growth for you is probably not a good idea. You may be at a stage in your life where you’re like you know what I need? I need to feel supported. I need to feel happy going to work every day because the people I’m working with. At this stage right now maybe actually climbing that ladder is not my top priority.

Erica Weiss Tjader: I think it really depends where you are, but I think if you are at a place where growth is important in terms of your–personally and you recognize that is not what you have at this company, I guess the first question is are there other roles within the company where you could find growth and other avenues to explore there? If not, that’s probably the right time to start looking at what else is out there.

Robin Ducot: Absolutely.

Audience Member: Hi. Thank you very much for sharing your advice today. I came from a finance marketing background and being in the Bay area, obviously tech is where it is. Want to know what’s your perspective on boot campers as well as if you have any advice for reframing other industry experience for transitioning to tech. Thank you.

Robin Ducot: I have some thoughts about this, but I don’t know if you guys want to share. I think that boot camps are typically really, really great feeders for small startups and tend to be less effective for getting jobs at larger companies who have big internship programs, but startups usually don’t. They are very excited about taking boot camp folks. I’ve worked at all different size companies and that’s been the pattern is that typically small companies eventually grow out of the boot camp feeder just because there is a lot more risk. As companies get bigger, they’ll take the CS degrees from their intern programs as the sort of feeder into the low level engineers.

Robin Ducot: I love startups. I think the experience you get at startups is amazing. I think that going through a boot camp and ending up at a startup is an amazing way to start your technology journey.

Jing Huang: One thing I think of, like your background in finance, right? Cross functional. If you think about tech today, having a knowledge of a different industry is actually very valuable. Taking tech applied for finance. Think about, for example, data science. There’s a big application on data science for finance. Those are areas where you actually have a unique strength to segue into tech if that’s what you want to do.

Audience Member: Everybody heard about the bro culture here in Silicon Valley. Is there a thing in SurveyMonkey or any other job you had before and how did you deal with it? How did you handle situations like mansplaining or competing for role, climbing the ladder?

Robin Ducot: Mansplaining? Sarcasm. Really? Really?

Audience Member: What about the career ladder?

Robin Ducot: For career ladder, I mean SurveyMonkey does not have a bro culture. It’s actually one of the wonderful things about SurveyMonkey. Do you guys agree with that? Yeah. I think that’s one of the things that actually was a little bit interesting of a transition for me when I joined was that I didn’t have to have that kind of, didn’t have to whip out the sarcasm quite as frequently. I think from the career ladder, I think it’s really just important to advocate for yourself. Men ask. I always asked. Always. Every single promotion I’ve ever gotten, I asked for. Every single one. It’s not because they didn’t think I was good. It’s just that people pay attention to people who are asking. That would be the thing that I would say is that the bro culture thing, I don’t know what to do about that.

Robin Ducot: I just tell them to knock it off and move on. I don’t know if you guys–

Shilpa Apte: Getting more women into tech.

Robin Ducot: I think that’s actually the best way to solve for it is if you’re a leader is to bring more women in. Then it gets diluted and, I don’t know, slows down. As an individual that doesn’t have the power to change it, if the company is really obnoxious and doesn’t fit your cultural values, then maybe look for some other place. Also, just tell people to knock it off. That’s my favorite thing to do. Whether you feel comfortable doing that is really sort of your thing. As for career growth, you really have to just keep, I mean one of the things that Jing mentioned, which is so important, is making people aware of what you’re doing. Just making sure that your competency is visible and whether it’s tech talks, speaking, things like that so that people become aware of your value to the organization.

Robin Ducot: I don’t know if that’s helpful. I don’t know what to say about the actual bro culture, I mean to fix it. It’s mostly just bringing more women in and not tolerating it when people are obnoxious.

Olga: Hello everyone. My name’s Olga. My question is about technical excellence. Whenever you come as a technical lead or an architect to the company, to the new company, you always have to prove yourself. Unfortunately, I am doing my engineering for 15 years. I built system that always run whenever I leave the companies, but still, like five years ago when I joined Salesforce, I had to prove myself by designing this complex system and building it myself. Then people realized what I really worth and was very surprised that I did this. Unfortunately, I do the same at Google right now. I still have to prove myself and building this complex system and designing this myself because I find that people, maybe they trust me, but they don’t trust as much as I would like. I see they trust some other people. They’re just not the same kind of expertise and the same bar that they expect from the technical lead.

Olga: I was wondering, is it going to happen at all the companies, just like when I join any new this is what happens? You try to stay at the same company for longer or there’s some other skills you can do so you can prove without building this complex system, like writing the code yourself?

Robin Ducot: I think storytelling is a gift that it takes, it’s useful to develop. I’m not sure if you guys agree with this, but I think that one of the challenges, and I don’t think this is a male/female thing actually. I think this is maybe an introvert/extrovert thing. You’ll get people who are just really good at telling their story and selling themselves. These are skills you can develop. Storytelling, selling skills, being able to tell the story of the things you’ve done in the past and being confident about it, feeling a sense of being entitled to be in the room and telling your story is really, really an important skill that will shortcut some of that. I think in engineering, though, engineering tends to be a show me kind of culture. It’s this intersection of being able to talk about technology and learning how to tell the story of all the things you’ve done in the past so that people will listen and telling it in a way that they will listen, not the way that you want to tell the story, but the way they will hear it.

Robin Ducot: You can definitely get coaching in that specific area if it’s something that you’re finding frustrating. I mean some of it is just when you start a new company you do have to tend to prove yourself a little bit. That’s just sort of part of it. I don’t know if you guys have some thoughts about that.

Jing Huang: Totally agree. I was an engineer developer myself. Like I said, self advocating was not natural. Same thing for you. It just feels like we need to do the work to prove ourselves, but in a lot of cases, we have peers or male peers that came to them more naturally where they’d just be able to tell the story about the work they have done instead of having to build something again and again doing the same job to prove what we already been able to do. Learn the skill to really be able to sell your story, self advocating. If that doesn’t come naturally, like Robin mentioned, get some coaching. Really just improve that set. Be confident.

Robin Ducot: Strategically placed solving other people’s problems also becomes a story that people will tell about you. One of the ways when I was still writing a lot of code, I would come in and help somebody solve a problem that they were having. You do that enough and people will trust you without you having to be designing something from scratch and building a whole up system, but being able to troubleshoot their systems because if you’re good at writing a system, if you can really build such a system from scratch, then you might be able to help somebody else fix theirs. Helping people fix their problems makes you very, very popular. Very popular.

Robin Ducot: What’s going on back there?

Audience Member: First of all, thank you for sharing your experiences as inspiring technology leaders. Looking back at your career journey, what was the best decision you made and the worst decision you made and why?

Jing Huang: It’s such a hard question.

Robin Ducot: Don’t try to get your boss fired. Don’t try to get your boss fired. It does not work. Even if they suck and they deserve to be fired, don’t try to get them fired because…. Complex, yeah. How about that?

Robin Ducot: Jing, you got something more?

Jing Huang: I couldn’t think of any like really worse decision you could make. I think every decision, there is different perspective. There is different outcome, but there is always a way out. I think that’s an opportunity. Nothing really bad could happen. Either you stay with a job or you leave for another company. It’s a choice. It’s an opportunity that you will make because of your decision. There’s nothing really bad that’s going to happen.

Robin Ducot: Yeah. It’s actually interesting. The most important thing is how you get back up if you, so take risks. You’re going to fail. It’s okay. You learn. I mean in my experience you learn from failure, not from success. Take everything as a learning experience and get back up and do it again. I don’t know. I’ve had so many learning experiences that have helped me learn.

Robin Ducot: Okay, we are at 8:30. Holy mackerel.

Robin Ducot: One more question right here I guess, and then we’ll-

Audience Member: I’m actually looking for advice. Imagine a manager kind of experienced outsider looking for a new position. This manager, let’s say me, I find a position in a company that I really admire. I look through the qualifications and I look through this job responsibilities and I know that I’m going to excel at this job, but all of this preferred qualification may not match what you actually have on resume. How do you grab this attention? How do you break through? How do you be noticed by either a recruiter or a hiring manager when you apply to this job?

Robin Ducot: I mean the question is, so you have the qualifications, but your resume doesn’t illustrate that you have the qualifications?

Audience Member: [inaudible] some specifics. For example–

Robin Ducot: Yeah, what’s an example of a specific? A specific programming language or something?

Audience Member: Yeah. It’s like a particular experience or, let’s say, experience in network or experience of working with healthcare for example.

Robin Ducot: You know, it’s funny, men will just apply. You just apply. This is one of the things we talk about. Actually when we create job descriptions, we’re really careful to make them so that they’re inclusive, that women are not going to automatically exclude themselves because there’s too much specificity in them. If you reduce the amount of specificity, then you’re going to get a wider range of people because the reality is that you don’t really know what you want exactly. You write a job description and then people show up and you’re like, hey, I kind of like you. You really don’t match the job description exactly, but you have something that’s special that will really resonate with the team. Apply anyway.

Audience Member: [inaudible].

Robin Ducot: Sorry, I can’t hear you.

Erica Weiss Tjader: I think, how to stand out. One thing, I’ll take on that, the reality of, depending where you’re applying, most companies the resumes that are coming through the application system are being reviewed by the recruiters and the recruiters, while great partners to the hiring managers, they know what they’re supposed to look for based on the resume. If it doesn’t match, you’re most likely to get passed on. Doesn’t mean you shouldn’t apply and you shouldn’t go through the recruiter. I think it’s a yes and figure out how to get in to a company, how to talk to real people, how to network. Again, I think it’s a lot harder. Let’s say you were interested in a design manager role on my team, which hopefully you’re not because then this will become awkward, but let’s just say you reach out to me and you’re like I really want to be considered for the role. Can we talk?

Erica Weiss Tjader: I might look at your resume and go I’m not sure, but if you reach out to me and say your experience looks really interesting, I’d love to pick your brain. Can I buy you a coffee? I might be like, oh yeah, okay, maybe, yeah. Then while we’re having coffee, you might somehow slip in the fact that actually I’m looking for a job. Would you ever consider somebody like me? Again, hopefully I didn’t just play out some future scenario that we’re going to have because that would be really weird, but I think it’s like if you come at it from I’m a job seeker, I’m a job seeker, I’m a job seeker, you might find that’s not the best way to build a relationship. Talking to people in the organization and finding your way through is how you’re going to stand out either because your resume looks exactly the same as everybody else’s or not quite the same as everybody else’s.

Robin Ducot: See if you can find out who the hiring manager is and then use LinkedIn to see if you’re connected to them. I mean that’s probably the most straightforward way to sort of do an around the process. Yeah. LinkedIn is your friend.

Robin Ducot: We are being kicked off the stage. Thank you so much and please enjoy the rest of the evening.

SurveyMonkey girl geeks and Robin Ducot

Thanks to all the SurveyMonkey girl geeks who helped make the SurveyMonkey Girl Geek Dinner possible! We had a fantastic time.


Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X HomeLight Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Sandy Liao speaking

Head of Talent, Culture & People Operations Sandy Liao kicks off a HomeLight Girl Geek Dinner in San Francisco, California. HomeLight girl geeks share how they use data and human emotion to empower decision making.

Speakers:
Sandy Liao / Head of Talent, Culture & People Operations / HomeLight
Mary Remillard / Talent Operations & Culture Specialist / HomeLight
Molly Laufer / Director of Offline Marketing / HomeLight
Sam Ryan / Product Manager / HomeLight
Ames Monko / Product Designer / HomeLight
Jenn Luna / Senior Software Engineer / HomeLight
Tina Sellards / Facilities & Administration Manager / HomeLight
Vanessa Brockway / Senior Manager of Business Development & Strategy / HomeLight
Gretchen DeKnikker / COO / Girl Geek X
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X

Transcript of HomeLight Girl Geek Dinner – Lightning Talks:

Gretchen DeKnikker:  From Girl Geek, thank you guys so much for coming tonight. If you guys want to come in and have a seat, we’re going to get started. So we’ve been doing these for about 10 years and this is over 200 that we’ve done so far, so we do them every week. How many people is this their first one? Cool. So stay on the mailing list. Come, you can do these every week up and down the peninsula into the South Bay. And this is Sukrutha.

Sukrutha Bhadouoria

Girl Geek X CTO Sukrutha Bhadouria encourages people to sponsor and rally their teams to plan a Girl Geek Dinner to increase your visibility in the organization and boost your career.

Sukrutha Bhadouria: Hi. I’m glad that there’s no more feedback. It was driving me crazy. Welcome, everyone. Like Gretchen said, this has been going on for 10 years. We branched into virtual conferences and podcasts, so you can listen to our voices some more if you go to whatever your favorite podcasting service is. A little history; we started off in 2008 and it was just a way to get women from various companies together. You got a sneak peak into the company and what they’re working on and the companies had access to these amazing women and these amazing women had access to each other. You don’t often get to hear about these wonderful accomplished women like those who are sitting behind me as easily if you don’t attend an event like ours.

Sukrutha Bhadouria:  So we’re hoping that you’re going to be building your network tonight and making connections. The other thing is that we’re always looking for ideas and content that you might want, so do send us your recommendations and your requests, but please do get your companies to sponsor because that just increases your visibility within the company. I can tell you I got a lot of visibility and access to my CTO in my company when I got Salesforce to sponsor. It was great. You don’t get to get access to them as easily. As it is out of it, it just made my career trajectory improve drastically. So you want to do that.

Sukrutha Bhadouria:  More than that, you want to, like I said, make connections tonight and please, there’s a lot of people who wanted to be here and couldn’t make it because we filled up and also because it’s a weeknight, so please tweet and share on social media. Our hashtag is GirlGeekXHomeLight, and I hope to see you at more events. We have one every week, like we probably already said. But we’re filling up really quickly this year, so hope to see you soon. All right, thank you. Sandy.

Sandy Liao:  Hi, everyone. We’re going to … it’s always funny at these events where there’s no one ever sits in the first row or the first seat next to the speaker. But make yourself comfortable. Move up if you have to for anyone else in the back. So quickly, I just want to introduce myself. My name is Sandy. I’m actually the head of Talent and People Operations here at HomeLight. This is actually super exciting personally. I’ve actually been a fan of Girl Geek for almost two years now. Thanks, Jenn here, who introduced me to the group. I’ve had the opportunity of attending multiple events. These things are really … people ask me, is this for recruiting, is this for anything? To me, it really is just for yourself. It’s all of you guys taking the extra time and after a long day of work, after a long week, you guys are taking the extra time to come out, to network with other women in different positions also working in the technology world. I just want to thank you guys all for coming. This is our very first women’s sponsored event at HomeLight. I’ve envisioned this for a long time and I couldn’t have expected a better outcome. So thank you all for coming and welcome to our office here.

Sandy Liao:  Great. So I want to kind of start off … we actually have a really awesome panel tonight. The way how we structure our panel is that we want everyone here to get a different flavor of the people that we have and all of our team, so from facilities to operations to marketing to product and engineering, we want everyone here to be able to understand a little bit about everyone’s role here at HomeLight and how we’ve all kind of came through all of our stories to become in the role that we are today. So we have a ton of buzz words in this valley talking about the word “diversity.” To me, I truly believe outside of diversity, it really means more than that. It means a balanced culture, it means a culture where you’re learning a little bit about everyone’s experience and background. It shouldn’t just be about your ethnicity or your race or anything else. It really should be about how everyone came about to become the person that they are today, and that’s how I define diversity and what it means to have a balanced culture. Very proud; I’ve been here for about three years. I’m very excited, I’m proud to have the team that we have today and I hope all of you guys are going to have a chance to meet some of our team member, and have a good time. So, thank you.

Sandy Liao:  A little story before we begin to pass it on to the panel, I kind of want to quickly introduce myself as well as tell you guys a little bit about HomeLight. So who here has actually heard of HomeLight before the event today? Okay, nobody. No worries. That exactly was my response three and a half years ago. Who here has actually gone through a home-buying or selling experience in the past? Awesome. There’s a few of you.

Sandy Liao:  So, the story of how I came about HomeLight was that three and a half years ago, I was actually in the process of buying my first home in San Francisco. I grew up in the city and I thought that I knew a lot about San Francisco, I know what neighborhood’s great, and I also consider myself fairly tech savvy. You know, I know how to use the Internet, I can go on and search for a realtor and do all that fun things. My mom was actually telling me, oh, use my friend. She’s great. She’s been a real estate agent for 25 years, but she’s never sold a home in San Francisco. And that’s the interesting part about this real estate world is that all of us really envision an idea of what the process would look like to one of the most important buying decisions or selling decision your life, but the truth is is a very complicated and emotional process.

Sandy Liao: So while I was going through it myself, I went online and I searched a couple of different real estate company online, and to be tech savvy that I am, I put in my contact information online and I was like, all right, I’m going to get some help here. And I did. I got 27 missed calls from random numbers within the same day. No idea who these people were. I got a whole lot of voicemails. Everyone’s introducing themselves, they know my name, they were telling me how wonderful they can be to help me, but the truth is I can’t … I don’t know who to call back. I can’t decide who’s actually going to be great. So I didn’t end up responding to anybody.

Sandy Liao:  So that was my story initially while I was going through the process. Through a very similar networking event like this, I actually met a woman who was actually in a viewing at HomeLight and I was telling her that I’m looking to buy a house and I’m in the process of looking, she end up referring me to HomeLight.com, and she said, “Hey, maybe you should check out this company. I just interviewed with them. I don’t know what I’m going to …” She didn’t end up working here, but she was the reason that I’m here today.

Sandy Liao:  HomeLight’s mission is that we empower people to make smarter decisions during one of life’s most important moments: buying and selling their home. Our goal is we analyze millions of home selling datas to be able to find you the best performing real estate agent in your area, so that you can make a great decision when you’re looking to buy or sell. Our CEO, Drew, who founded the company eight years ago was going through a very similar process with his wife, also looking to buy a house in San Francisco, and just find out how challenging and difficult it is to go through that process. With everyone here, we all believe there is a better way of doing this, and buying and selling a home should be a very, very exciting process in everyone’s lives. So we’re here to make that better.

Sandy Liao:  Before I pass it down, today our topic here is to talk about how we utilize our data as well as our human emotions to empower all of our decision making. I’m going to hand it off here to Mary on the talent team who’s going to share with you guys a recent experience of hers utilizing HomeLight, our platform. So, thank you.

Mary Remillard speaking

Talent Operations & Culture Specialist Mary Remillard gives a talk on “How the Marriage of Data & Human Connection Resulted in my Successful Home Purchase” at HomeLight Girl Geek Dinner.

Mary Remillard:  Thanks, Sandy. Hi, everyone. Thanks for being here today. I’m actually on Sandy’s team. I’m the Talent Operations and Culture Specialist, which essentially means that I spend most of my days focused on recruiting on our higher needs across the board. I also do have a hand in our HR and administrative tasks, as well as our cultural and employee engagement initiatives, so a lot going on there. But I truly have been passionate about HomeLight since Sandy reached out to me, two plus years ago on LinkedIn and introduced me to the company and the people and the culture and our mission, which is to empower folks to make a smart decision as they’re going through one of life’s most important moments, which is buying or selling their home.

Mary Remillard:  I have, in my two plus years, been pitching HomeLight as a recruiter literally thousands of times. I did the math, which is crazy. It’s been thousands of times. I have always felt like I genuinely appreciated the service that we offer, and I understood why we have this purpose of giving this service to folks. However, it was only until recently when I went through the buying process myself with my husband that I was able to fully comprehend the emotions and the weight of such a big moment in your life of buying your very first home.

Mary Remillard:  So my husband and I, we’re super frustrated. We have always rented and we were probably touring our maybe 12th or 13th complex, and we were just never able to find that perfect situation for us. It was either too far from work, too expensive, not clean, the folks at the front desk were too grumpy for us. Whatever it was, we just figured out, you know what, maybe it’s time we stop buying … or renting, rather. So we’re very proud upstate New Yorkers, never thought we would leave home, but we found ourselves in Scottsdale, Arizona, and decided that, you know what, we are not getting any sort of return on investment as we’re continuing to rent. Why not put down some roots in the desert here and embark on this buying process.

Mary Remillard:  So luckily, having been an employee for two plus years, I knew exactly where to turn. So within an hour of my husband, Will, and I making that decision, we were on our couch and we were engaging with Kimmy, who’s one of our home consultants in the Arizona office. She just asked us some quick questions around the home that we envision ourselves in. She asked us price point, location, timeline we were working with, those things of that nature. And basically she’s gathering that information to then plug into our own algorithm that analyzes this really robust database that we’ve been building since 2012. That’s how we’re able to determine who are going to be these best agents for Will and me as we go through this crazy task of buying our first home. But by the end of the day, we had engaged with two phenomenal agents and then we had this whole other issue where we had to decide between two great people. That’s a great problem to have. Unfortunately, a lot of folks don’t get to have the experience of that being a problem.

Mary Remillard:  So ultimately, Will and I went with a gentleman by the name of Chris Benson, who’s an agent in Arizona. He’s lived there for many, many years, knows the area like the back of his hand, has closed 425 transactions, has nearly two decades of experience and he specializes in single family homes. So Will and I, we went with Chris because it was clear to us that he was going to be someone who would hold our hand and answer our seemingly endless list of stupid questions and just not make us feel bad about it, and make himself available to us.

Mary Remillard:  So within 30 days, and mind you, Will and I decided pretty quickly that we’re sick of touring, we don’t want to rent, let’s buy a home, but our lease was coming up in less than a month. So we did not want to have to figure out a situation between different homes. We wanted to be able to move in immediately. So I don’t know if many of you can relate to that, but when I told people I wanted to buy a home and move in within a month, I usually got a pitying laughter, like good luck, lady. But we were able to do it, and the thing is why we were able to accomplish that is because HomeLight’s algorithm did its job.

Mary Remillard:  Chris Benson was able to help Will and I get into this beautiful townhome that we’re so excited about because he’s closed homes with our needs, we wanted to buy a home around the 200,000 mark, which believe it or not in Arizona that’s doable. And he’s closed the majority of his homes averaging at 214,000. So the reason why he was so good for us is because he’s done this 425 times. He could do it with his eyes closed. So not only are we having a great experience, but so is Chris working with HomeLight because we’re basically teeing him up to work with people whose areas of need are right in his areas of expertise.

Mary Remillard:  So we obviously had a really great time, but what also made this so wonderful of an experience is because Chris is a human. Chris has raised two daughters who he absolutely glows when he gets to talk about them, he knew the highways that I just wouldn’t tolerate for traffic and a commute, I live a mile away from work now. He speeds on the 101, he’s proudly proclaimed that, and he’s a Cubs fan, so him and my husband were able to go back and forth while I was just like, okay, baseball, woohoo.

Mary Remillard:  But long story short, Will and I now are so excited about our future in Arizona because HomeLight did its job, its algorithm worked, it matched us with Chris, and because Chris is a phenomenal agent who’s experienced, and again, he’s done this 425 times. So that’s really the HomeLight difference right there, and I’m so proud to work for a company that now I know firsthand is truly making a huge difference.

Molly Laufer speaking

Director of Offline Marketing Molly Laufer gives a talk on “Offline Performance Marketing: Using Art and Science to Drive Response and Revenue” at HomeLight Girl Geek Dinner.

Molly Laufer:  Apparently I need a new headshot. That photo was like, eight years old, one toddler and many gray hairs later. I’ll work on that. So hi everyone, my name is Molly Laufer. I’m the Director of Offline Marketing here at HomeLight. I’ve been here for about seven and a half months, so I think on the panel I’m probably relatively newer to the team from everyone else. Here at HomeLight, I’m responsible for channels like TV, radio, podcast advertising, out of home, direct mail, and some of our large scale brand sponsorships. Now here at HomeLight, we utilize these channels not just to drive top funnel awareness and sort of general brand awareness and market share, but also to actually drive immediate performance. We utilize these channels as performance channels to drive leads and revenue, and I’ll get into that in a little bit more detail in a minute.

Molly Laufer:  All right, so we were prompted here to talk about some of our passions and our hobbies that we have as well outside of work. When I thought about it and realized that I have a gregarious 17 month old, I realized that most of my current interests outside of work really are just around keeping my toddler alive and trying to main some semblance of balance, which to be honest, I don’t know how that works. So don’t ask me for any advice on that. But professionally outside of offline marketing and customer acquisition, the things that are really important to me specifically are around supporting veterans and their transition to the tech world from the military through networking and storytelling, as well as finding community and support for myself and other working-out-of-the-home moms, especially when they’re making their transition back into the tech world.

Molly Laufer:  I started my professional career in 2007 as a surface warfare officer in the US Navy. I spent four years deployed as part of Operation Enduring Freedom, Operation Iraqi Freedom, as well as a handful of counter-narco terrorism missions as well. I was the ordinance officer and force protection officer onboard the USS Samuel B. Roberts where I was one of three women on a ship of about 280 men. So being in this room with this many women in the inverse is freaking awesome. This is a really cool balance. And then I was also a training and readiness officer onboard the USS Nimitz, which is the photo here. This was probably back in 2010.

Molly Laufer:  So anyways, that’s how I started my career. In 2011, I made what I think, looking back, was a very clumsy transition to the civilian world in technology, specifically in Silicon Valley. I joined a pre-revenue, pre-funding, pre-product, pre everything startup. I was the first employee at the direct to consumer e-commerce net company called NatureBox where I worked on community, I did a lot of our social media, our initial paid social media marketing, as well as influencer marketing, which then, for me, really pivoted into focusing on offline, like podcast and TV and radio. I was at Nature Box for about four years, and then I made a kind of non-traditional transition to the agency world. I worked at an offline performance marketing agency for a handful of years, and it was a great experience. I got to work with a lot of really interesting businesses and a lot of really interesting business models, but ultimately I was really eager to get back to be really hands on internally. I found HomeLight about seven months ago, and I couldn’t be any happier.

Molly Laufer:  So I like to make this joke in some of my interviews which was in the Navy, the closest point of approach is when two ships are passing in the night and you want that CPA to be as high as possible, otherwise you have collisions at sea. When I came to the tech world and realized that CPA actually meant Cost Per Acquisition, and if you were doing your job right, you wanted it to be lower, that was a huge surprise for me, but that’s a whole ‘nother TED talk for another day.

Molly Laufer:  So I’m going to talk a little bit specifically about how we approach, or how at least I approach, offline media planning and it’s really interesting that our topic today is around sort of using data and emotion to make decisions because I’ve always said offline marketing is this real mix of art and science. There’s a lot of data, there’s a lot of those really concrete, quantitative information that can go into making a media plan. But at the end of the day, like every decision that we make, whether it’s buying a home or even just looking at your grocery budget, you have to make some sacrifices, you have to make some decisions in your media plan because most of us work in areas of business where you can’t … money doesn’t grow on trees, you can’t afford to do anything.

Molly Laufer:  So, you know, at HomeLight, I’m going to speak to a couple of these things. This is certainly not exhaustive. I listed a couple of different attributes that go into building an offline media plan. But some of the things that we do here at HomeLight that I think are really unique, we’ve been in business for about seven years, so we have really robust consumer data, and we can take that lead data and we can input that into traditional media planning tools like Nielsen and like MRI, and we can actually get really great personas about who our customers are, how old they are, generally where they live, what type of media habits they consume, are they more likely to watch TV on Roku or an AppleTV or Hulu versus a traditional linear buy, what types of stores do they shop at, and these are all the different types of inputs that we can start to input into a media plan.

Molly Laufer:  Because when you think about buying cable TV, there’s hundreds of channels, there’s many different approaches. You have to start to narrow down the way that you think about what are going to be the right buys to attract the customer that’s right for HomeLight. Things like seasonality is also really important when it comes to doing an offline media buy. You know, there’s … every business has its own unique seasonality that they tend to see better efficiency for their business, but the thing that’s really challenging is that the media landscape also has its own unique seasonality. So for example, things like political campaigns, Black Friday, the end of quarter, and all of the local markets, you’ve got all the guys that are selling mattresses and trucks and they’ve got to get them off the lot at the end of the month. All of these things where you think yeah, I’m really jazzed up because I’m going run this amazing campaign at the last week of May, well guess what? Every single car dealership in America is trying to sell cars on Memorial Day weekend, and so you might end up being kind of SOL if you’re really banking on certain weekends like that.

Molly Laufer:  So there’s all these factors that are kind of outside of your control that you need to have a really good grasp on before it comes to planning a media campaign. The other one that I’m going to touch on here before I move on is specifically around the competitive landscape, and I’ll talk a little bit more about this later on, but what’s really interesting is that in certain offline marketing channels, the competitive landscape either can work to your advantage so you can see where your competitor’s advertising, and you can take the move to maybe follow them. In absence of data where you haven’t advertised before, you could look at competitors or like-minded companies, see where they’re advertising, and choose to do the same thing. However, this approach really doesn’t work in other media channels. For example, podcast advertising or radio endorsements where you have an actual human, a person who’s standing up and saying all right, now onto a word from our sponsors. Those types of ad placements, they can really only have room for one type of product at a time. It would be very, what’s the word I’m looking for? I don’t know. It would be very inauthentic if a person were to endorse, say, one mattress company and then the next week turn around and advertise for another mattress in a box company. So things like competitive landscape and this sort of winner-take-all in the space can be really important.

Molly Laufer:  These are just a couple of the facets that go into planning an offline campaign. The output that you see here, which I realized just looks like a bunch of dots and bar charts, because everyone’s impressed by dots and bar charts. No, but in all seriousness, what this tells us is this gives us an output of who our customer is, what types of media are they watching, and where are we going to be more likely to not only reach a higher percentage of our audience, but as you can imagine those are the placements that tend to be really expensive. It’s no surprise that most of our customers and probably all of yours are watching ABC and NBC and CNBC because guess what? That’s what all of America is watching. And so you get a lot of really interesting data down here on the other end when you look at well, what are some of the smaller networks that the audience is also watching? Can I add frequency and can I add additional touch points for our brand using lower reach, but very low-cost and high efficiency media.

Molly Laufer:  So again, those are some of the factors that go into when you’re actually looking at a media plan. When you are using these tools and you get an output, at the end of the day, you can’t buy everything on a media buyer. You have to use some sort of prioritization and rankers. It’s different for every business. It’s different if you’re a national company versus a geo-based company. But those are some of the factors that I use, at least, here at HomeLight.

Molly Laufer:  Another chart with lots of dots and bars. But this is really interesting. So I thought a lot and hard about how can I talk about offline media measurement. I could take an hour, and I think I have seven and a half minutes and I’ve probably already burned through five of them right now telling you about crazy stuff I did before I joined HomeLight. So I wanted to use this specific example because I like to be specific when possible. So the team knows this. I was in the Navy, so I use really nerdy, nautical analogies that no one really understands. But what’s really interesting is when you’re in the Navy, there’s actually two places that you can drive a ship from. The first is you can be in the bridge, right up there with a little wheel. It’s not really big like you see on the Titanic. It’s actually a little wheel that’s this big. It’s super anti-climatic.

Molly Laufer:  So you can either be up there on the bridge looking out, seeing, hey, I see a ship over here off the port side, hey, I see a ship over here off the starboard side, and you can use your eyes and drive the ship, right? You can also, this is crazy, you could not have anyone on the bridge of the ship. You could all be farther down in the ship in the combat information center and using your radars to drive a ship as well.

Molly Laufer:  Now I wouldn’t necessarily recommend that because you lose that eye contact to actually see what’s out there, but the analogy that I always like to make in offline marketing, and I promise there’s a good analogy here, is that when it comes to offline marketing, we have a really tangible way to get sort of directional signal-based indication of what type of media is working better than others. I would really equate that to sort of being downstairs in the combat information center, being able to just look at what the radar is telling me and using that to make navigation decisions. You’re certainly not going to get the full picture, and there’s no substitute for actually going above deck and putting your eyes out and saying, does that ship actually look like it’s pointing in the direction that the radar says it is?

Molly Laufer:  But for us, specifically on … I use this example for TV because I think it’s really visual, but what we do here is, and this is … I wouldn’t say this is necessarily unique to HomeLight, I think this is pretty common in offline marketing, but a lot of people don’t know. We’re all sitting at home, we’re all watching TV. If you’re like me, you’ve probably seen a million e-commerce, direct to consumer companies pop up on TV over the last couple of years. And what’s really cool, and I had a stock image of it but it was kind of cheesy but I took it off, it was basically a couple sitting on the couch watching TV while also scrolling on their iPhones. Because let’s be honest, who does that with their spouse or their friend every night?

Molly Laufer:  Yes. I love that. That is an offline marketer’s dream, right? Because when they’re watching a commercial and they see something really interesting, they just start Googling it. So this is the type of signal that we get when our TV commercials air. We can start to see directionally, well what type of response do we see when we air a spot on CNN at noon? How does that compare to a spot that we air on HGTV Property Brothers at 8 p.m.?

Molly Laufer:  That’s certainly not going to give you the full picture of the impact of your media buy and I would probably need another two hours to go into that, but this is giving you some really good signal-based direction that we can use to make media optimizations. If you have a background in digital marketing, I would say this is the equivalent … this is about the closest thing that you would get to a direct click. In digital media as well as sort of some older types of advertising where you’ll see phone numbers on TV. You still see that today if you’re looking at a lot of lawyers and there’s a lot of local businesses that will really utilize phone numbers, and that’s what we use here at HomeLight.

Molly Laufer:  All right. So I’m going to pivot and just sort of close with that’s all great, but if you don’t do offline marketing, how is this actually going to be interesting for you? So I kind of took a step back and thought, all right, what do I do when I have a decision to make? I like to use all of the data. We all do. But guess what? As we’ve talked about here and as you’re going to continue to hear, the data only goes so far when it comes to making a decision. So I thought I’d kind of leave with four pieces of advice that I try to follow myself when I have a decision to make, and I don’t necessarily have all of the information that I need.

Molly Laufer:  So the first … maybe I should have put this last, but this is my favorite is building a professional or a personal board of advisors in your general role or industry that you can turn to to help when you’re facing a tough decision and you just need a little bit of outside perspective. So you certainly wouldn’t want to go to a competitor, you certainly wouldn’t want to ask a agency who’s working on a competitive product. You’re not going to maybe give them all the answers, but this has been really helpful for me and it’s events like Girl Geek, it’s events like even just talking to some of the partners that you work with. For example, at HomeLight, we do a couple of key large national sponsorships.

Molly Laufer:  And so even just reaching out to those folks and saying hey, I noticed that you also have Wayfair sponsoring. Can I talk to the person who runs offline marketing at Wayfair? Hey, I noticed that Visa’s a sponsor, too. Would it be possible for you to be put me in touch with the person who has my role at that company? More often than not, people tend to want to be helpful and give advice in areas that they have experience in, especially if it’s not competitive. So trying to build up that board of advisors wherever you go in your career, it’s always been really helpful for me to get that outside perspective from someone outside of HomeLight.

Molly Laufer:  This is interesting. So evaluating the risks and having worst case scenario planning. I’m a very positive person, but when it comes to making a decision, my mind first thing goes to what if this is the wrong decision and it completely fails? Not everyone’s like that and if you’re not, teach me your ways. But if you are and you tend to go to the worst case scenario, I like to think well, could I handle that? What would be the worst case? What would be the worst case scenario? And then what would I do about it? So when I made the decision to leave a very fast growing startup to go to an ad agency, I was really, really worried because I thought, God, this could be a career killer for me. Everyone says don’t go to agency side, you don’t get the hands-on experience, you’re going to be working crazy hours, it’s going to be crazy. They were right. They weren’t lying.

Molly Laufer:  But I said, okay, what if they were right and I’m absolutely miserable in this role? It was just the worst decision I’ve ever made. I said well, I would leave and I would find another job. I said, huh, you probably can’t do that every career move, right? If you start to do that over and over again, you just become a career hopper or a serial hopper, but I thought if that’s the worst case scenario, I could handle that. Now what I didn’t do is evaluate what would be the best case scenario and the best case scenario, to be honest, was I think what ended up happening which was I got great experience, I touched different business models, I touched different products, I got my hands on media channels that I would have never otherwise had the opportunity to work on. And now, to be honest, that’s a strategy that I use in general, personal and professional.

Molly Laufer:  This one I think is really interesting. I’m going to use an example from HomeLight which is when in doubt, let your values, whether it’s a personal decision or your company’s decision, guide what you do. If you’re ever at a turning point and it’s yes or no, you say what do my values tell me? And the example that I would use for this most recently was we recently announced a sponsorship as a title sponsor of the US ski and snowboard team, which we’re super excited about as you’ve seen probably from our conference rooms, all of our conference rooms are named for different ski resorts because one of our values here at HomeLight is work hard, ski hard. In my case, it’s work hard, mom hard. I don’t do a lot of skiing right now. The point is when we evaluated this proposal from the US ski team, we used a lot of data, we looked at what were overall CPMs, what type of response rates do we think we could get from the live and broadcast opportunities. But there was all this unknown that we weren’t quite sure about how we were going to measure or was it going to work.

Molly Laufer:  So after evaluating the worst case scenario planning and saying if we were wrong, how will this impact our bottom line, we said, let’s let our values really guide us. One of our mottos here at HomeLight is work hard, ski hard. Let’s do this. And so that has been an area where I think whether it’s in your personal life or in your career specifically, taking a minute to think about the company values, which we all have on our walls and we talk about in all hands, but when you ever need to make a decision, let the values guide you because that’s technically what values should be for is for guiding decision making, not just for putting on a wall and using it in recruitment.

Molly Laufer:  And the last thing I’ll say is trust your gut. I know I’m running over so I don’t think I’m going to give a specific example of this. Only just to say that personally, since becoming a mother in the last year and a half, I’ve realized that out of all the … you can go as evidence-based as you want on everything, but at the end of the day, there’s no one right way to be a parent, just like there’s no one right way to do your job. At the end of the day, trust your gut because it’s probably a lot better than you think it is, and have confidence in what your gut tells you. So, thanks for letting me chat.

Sandy Liao:  If anyone wants a refill on wine, feel free to do so. I’m going to do that myself, so help yourself.

Sam Ryan speaking

Product Manager Sam Ryan talks about her career journey and product management at HomeLight Girl Geek Dinner.

Sam Ryan:  Hey guys. Sorry. Apologies in advance. I’m suffering from a little bit of a cold right now. But, Molly, thank you for that inspiring talk. Hard to follow up on that. But hi, I’m Sam Ryan. I’m a Product Manager at HomeLight. Thank you all for coming here. It’s actually quite unbelievable that we’re actually hosting this event in this office. So today I’m really excited to talk to you about a little bit of my journey at HomeLight into product, and a little bit about what products and engineering looks like at HomeLight.

Sam Ryan:  So I was hired at HomeLight in 2016 by Sandy and I think I was employee number 38 at HomeLight generally and employee number four or five, I believe, in our Phoenix office. So not only was I hired at HomeLight, but this cemented my move from New York City to Scottsdale, Arizona, which I never expected. I had a very memorable first day in which I was tasked with building my desk, my IKEA desk, and hooking up my computer. But you know what? It really inspired, I think, my journey up until today.

Sam Ryan:  So I was actually hired as what they called I think at the time, Sandy, you can correct me, but I think an experimental account executive/sales person. But really my job was I talk to agents for 40 plus hours a week because those are our users, and talk to them about what they like, what they dislike, what type of problems are they facing not only within the HomeLight platform, but generally, and what we could do better to support them. I was trying to solve a problem. We would introduce highly motivated buyers and sellers to top performing real estate agents across the country, but agents did not really like the HomeLight platform, and therefore we were having a lot of trouble getting updates from them in terms of the progress that they were making with the clients that we were introducing to them. So this caused inefficiencies not only within the sales organization, but obviously company wide.

Sam Ryan: So after a few months of this, I was obviously overflowing with feedback. I would turn to all of my teammates any time we would have someone from our San Francisco office at the time visit and I would just be like, guys, we have a problem and we need to improve the product or we can’t solve the problem at hand. So I think it was a few months into my career at HomeLight as an unofficial product manager, because we didn’t have a product team at the time, I had my first release. So I worked with one of our awesome UX designers, Wally, and one of our killer engineers, Charlie, who both are still on my team today almost three years later, which is quite amazing, in redesigning the referral manager or the CRM type product that our agents use to update us on the current status and progression of the clients that we introduce to them at HomeLight. And the users loved it, which was pretty crazy.

Sam Ryan:  So I think it was really maybe a week after my year anniversary at HomeLight where I was officially moved to the product team, and I think I was employee number two. And the one thing I carried with me, or team member number two, the product team, and I think the one thing that I carried with me from working this experimental role where I talked to a lot of agents to being actual product manager was there is nothing more important to being close to your customer. However, HomeLight moves quickly. We release, and I think rapid cycles is putting it lightly, we release at a pace that’s unreal sometimes. Our team is just here for it and work so hard to do it. But, it’s really hard to use the traditional surveys and interviews, though we still do, to get rapid customer feedback.

Sam Ryan:  So I think it’s so important. I spend many hours on the Internet scouring relevant news articles, forums, threads, reading the comments on these news articles, digging into Reddit. I have spent way too many hours on the real estate subReddit, just understanding, querying every type of HomeLight query that is possibly out there and just trying to dig in to what people are talking about, not only for HomeLight but the industry in general. Trying to embody my user but I don’t have time to be a real estate agent, though I worked in the industry before in New York City. To try to understand and this is an industry where we’re constantly innovating and the tech industry’s constantly innovating and real estate agents have a thought of fear and a thought of let’s embrace this, and what are we doing. HomeLight’s here to empower them and that’s super exciting. So staying close to the user.

Sam Ryan:  Things that we do at HomeLight to kind of embrace the user and embrace their feedback and experience, but also release rapidly. I kind of gave a brief strategy and I kind of compare those and it’s funny that Molly talked about working on a ship and kind of some things there because I compared it to a rocket ignition system, which I had to caption because I kind of just Google image searched a launch button and this came up and I’m like, oh, there’s a lot of switches here and that makes sense. Because what we try to do because you have to release rapidly and we’re trying to gather all this feedback and all of these plans and just go, we have to have a lot of checks and balances in place to ensure that we’re not going to burn down the house, and we haven’t yet, which is super exciting. It’s been two years, so maybe I should knock on some wood somewhere.

Sam Ryan:  But I wanted to give you guys, I don’t know, some of the strategy that my team uses to just ensure that we can release quickly, efficiently, but also keep the user at top of mind and ensure the success of the user and ensure the success of our product. So some of the things that we’ve done; we find users who love our product generally, and users that are … this might not be the best advice, but users who might be a little bit more tech savvy than the average user, and are willing to use things until they break and are willing to struggle a little bit to provide us feedback. We have small focus beta groups that we can roll out to. We really utilize blind, human testers.

Sam Ryan: Actually, before every single deploy at HomeLight, we employ our global app testers, GAT. It’s a great product and they employ actual human testers all across the country that will go through and follow step by step directions and use your product and provide you very, very granular feedback and the test usually fail because of silly things like copy or you accidentally said next and it says, “submit.” But it does point out, I don’t know, very interesting insights about the product and things that you can improve upon, fix, or fix your instructions.

Sam Ryan:  Also, our support and sales team, so for every major release, I really encourage, generally, not only in support and sales, actually company wide, who wants to be involved in this testing process? I got this spreadsheet going, let’s go bug bash. So I really try to widely encourage company-wide involvement in those types of things, not only in forums that people who are talking about these things, but it helps me be informed about my product.

Sam Ryan:  We utilize roll outs often. Any type of risky, major feature, we really like to utilize roll-out flags so that we have that after-launch protection. So worst case scenario, again, hasn’t happened yet, but we can roll back.

Sam Ryan:  I also … In the planning stage, we implement tracking so that there are those flags that pop any time that something could go wonky even a little bit. So I really like utilizing user events, maybe a little bit too much, but I always have a dashboard the day before release that’s ready to go the second that we release and can trigger anything that could go awry at that time. Sometimes it’s triggering nothing. That’s best case scenario.

Sam Ryan:  Also, we use Sentry for error tracking, which we use for all of our staging environments as well as production and consistently monitoring that around these major releases as well. Again, around release time, I stay very close to our sales and support team. They’re generally headquartered in our Arizona office, so we just made a major app release last week, so I was there in the Arizona office literally sitting at the desk next to them. It’s like, okay guys, let’s go. What’s happening? You guys are talking on the phone. What’d they say? Have they used it? Have they used it? They’re like, we’re making them download it now. I’m like, good. Let’s call them back tomorrow. But I think that’s some of the most valuable feedback that I get, of course.

Sam Ryan:  Really encouraging direct channels of feedback, and making it not weird. You’re not bothering me. If I don’t answer you, it’s not you, it’s me. I’ll answer you eventually or maybe I won’t, but again, I really value all of it. I personally like utilizing Slack for these things, but of course not everyone loves Slack, so email, whatever. Just get in touch with me and get in touch with me two or three times if you need to. I don’t know, just encouraging that direct feedback loop. One thing my team embraces and the ping pong emoji is something that we use back and forth on Slack often. The ball is in your court. HomeLight generally really encourages ownership, not only over the products that we manage, but over the release and the success of those products. It’s not only me personally, it’s the team. So I currently oversee what we call the pro’s team, and that’s the professional experience at HomeLight that not only spans real estate agents, but other real estate professionals generally within the industry.

Sam Ryan:  So we kind of pass this emoji back and forth because if the product fails, it’s not only I failed or you failed or he failed, it’s all of us. So we really like saying the ball is in your court. Like hey, I wrote this back, but … this name tag keeps falling off. But hey, you perform again, Sam, we’re going to test together and if I miss something, it’s all of our fault. But I don’t know, we like passing the ping pong emoji back and forth. But anyway, it’s been great having you guys in our office and it’s amazing that HomeLight has grown to the size that we are at today to be able to host this type of event. And Sandy, thank you again for introducing me to the HomeLight family. It’s been a great almost three years. Thanks.

Ames Monko

Product Designer Ames Monko gives a talk on “Using Design and Empathy to Create Joyful Product Experiences” at HomeLight Girl Geek Dinner.

Ames Monko:  Hello. My name is Ames and I’m a product designer. Tonight I’m here to talk to you a little bit about how I use design and empathy to create joyful product experiences. So I was going to ask this question, but Sandy stole my thunder, about if any of you have had gone through the process of buying a home, or know a friend who has gone through it. It sucks. It’s like the worst thing. So … you can go to the next slide. And I also need to go to the next slide. Manually. It’s mine. Ugh, this is the worst. Okay.

Ames Monko:  So we all know, or if you don’t know, mortgages are a time consuming, confusing, and overall stupid daunting, bureaucratic process. Fun fact; traditional lenders don’t actually care about you or the experience that you have. Mortgages are technically, I would say, a very small percentage of their actual revenue making, they make money in tons of different ways. So they just kind of choose not to fix the process and then in turn, will just make you go through their very inundated, crappy process. Essentially they attempt to try to innovate, but by innovating, they kind of put a shiny UI on the top of the funnel, like put in your name in this and it’s so sleek and it’s modern because it’s the Internet. And then once they get you, you’re kind of just thrown back into their very clunky, not … just very cold process.

Ames Monko:  Since the subprime mortgage crisis, home buyers know they deserve a higher quality of experience and making one of the biggest financial decisions of their lives.

Ames Monko:  Oh yeah, I should put that there. So the already anxiety-provoking experience of borrowing money is currently made worse by multi-step process riddled with mortgage jargon, AKA anything you’ve ever seen from Rocket Mortgage, it’s not a rocket. It’s not. Sorry if anybody … you work at Quicken. HomeLight’s hiring. Just saying. My approach is like, what if in addition to streamlining the process … next slide. We could approach our design from a place of compassion and empathy. Throughout the past six years, I’ve worked in the mortgage tech industry with the goal of demystifying the process. I spent four years prior to being at HomeLight at Better Mortgage. I was one of the first initial employees of that whole project. It was myself, one front-end engineer, one quote unquote mortgage professional, he was just sent to go learn about mortgages, and one back-end engineer. We sat in a very small conference room in New York. In three months, we basically built what is the backbone of Better Mortgage.

Ames Monko:  So that’s when I started … my Aries brain was like, oh, this is a pretty tough problem to solve and it’s kind of holding my attention and I would never say that in a million years, like I’m really passionate about mortgages. Because talk to me eight years ago, I’ll be like, what? I don’t care about mortgages. It’s not a thing. But as an empathetic person myself, having seen people, friends of mine, family members, go through this process, I’m like, oh, maybe I can use my expertise in design and also my empathy as a human being to try to start fixing this process. Next slide.

Ames Monko: From a design perspective, to remedy any potential pitfalls and offer support when needed. My approach is you want hands on? You can have hands on. You’re tech savvy like Sandy? You don’t even have to talk to anybody. If you can figure it out? Cool. You can do it all by yourself.

Ames Monko:  These are millennials. Apparently I’m considered … I was born in 1980, which I’m apparently a millennial, but I didn’t really get the Internet until, I don’t know, I was graduating from high school, which is in 1999. I didn’t get my first cell phone until 2004, which was a cool flip phone. So the fact that … I was like, I need to find a picture of millennials and I just put these together. They look like millennials, I think. But more importantly, more than any other group, they are relying on financing for their home purchases. Many have already been confused and sort of let down by the student loan process, but nonetheless are still willing to borrow. But they expect the former archaic, home financing process to be simplified, transparent, and pleasurable? Which I don’t think we’re there yet. Next slide.

Ames Monko:  So there’s this really great article. I put a link in here and I was like, oh wait, but these people are probably not going to get the link. But Adam Grant and Erin Henkel wrote a piece for the Harvard Business Review. In it they say that the first step in empathizing with your customer is to gather insights and ask what is broken, frustrating, surprising, or uncomfortable for your customer. The second that you can train employees to put a customer first, it will dictate how you build and design a product. From a design’s perspective, fixing these problems in a visual way that makes people laugh, feel reassured, or feel like their needs are being met or anticipated is a solution that builds trust in my work, or our work.

Ames Monko:  When we take something tedious and scary and turn it into a pleasurable experience, we make the applicant feel valued. If you can give a customer the tools, they feel empowered by that. This is their most important decision and if you can help them get there by just simply adding a more confetti button, do it.

Ames Monko:  My goal has always been rooted in keeping joy at every step. People should feel excited about buying a home, not dreading it. I want them to look back and think about how great it was to buy a home, not the horror of the experience. It will definitely make meeting up with your friends less interesting because they don’t have anything to complain about and talk like, ugh, that was just like the worst thing in the whole world. So that will go away. Unfortunately, you’ll have to talk about more positive things. Next slide.

Ames Monko:  The future direction of HomeLight at the helm is one where curiosity about customers’ experience gives us a unique perspective to stay connected to them. And with all the technical difficulties, I am now done.

Sandy Liao:  Thanks, Ames. They flew all the way here from New York just to join us for the evening, so thank you so much for being here.

Jenn Luna speaking

Senior Software Engineer Jenn Luna gives a talk on “Engineering’s Software Stack and How We Power our Matching Algorithm” at HomeLight Girl Geek Dinner.

Jenn Luna: Hey, everybody. Software Engineer, introvert, so I’m going to do my best. I was really nervous to go after Ames because her slides were so beautiful and mine basically look like a 10-year-old’s book report, and not a gifted 10 year old, just a regular 10 year old. So bear with me. Okay, so these are the things that make me who I am. Software Engineer for, I think, almost 10 years now, which is absolutely crazy. I am also a real estate agent on the side. I only do it for friends and family because I don’t have time. I’m a new-ish mom. New-ish because she’s almost nine months old and time flies. I’m all these things; I’m a teammate, an employee. In my free time I like to snowboard. I have dance lessons on Wednesdays and travel is pretty much the most important thing to me sometimes, besides my daughter, of course.

Jenn Luna:  So here’s the most embarrassing picture of me that I never share with anybody. I started at Intel in 2008. I was a double E and I was hired as an electrical engineer. This is me in the sub fab. I really loved this experience because I got to go see all the robots making wafers and things like that. In the sub fab, you only wear half of the equipment, but in the fab fab, you have to put the whole bunny suit on where order matters and if you put your boots on before your hat, you have to redo the whole thing. It’s nuts.

Jenn Luna:  So after four years at Intel, I decided I wanted to jump into software, so that brought me to San Francisco, of course. I found a company called SolarCity. Anybody heard of it? Awesome. They are now Tesla, but for five years, I was at SolarCity working in the solar industry. This is me at Bay To Breakers, just fully embracing the San Francisco culture. I loved it. I ended up moving back to Arizona, but everybody goes here and then goes back somewhere else. Anyways …

Jenn Luna:  So when I started there, it was a super small team. I was in crazy startup mode. This part of my life was so exciting. It was nuts. There were just no requirements. Here’s a picture of one of my first requirements meetings with the CTO and it’s like, here’s what we want, let’s put it on a whiteboard, just spent three hours in a room and if you didn’t take good notes and you don’t build what I want, you’re fired. So this was crazy to me. This was actually my first project. And here’s another one that makes me laugh because what is this? It’s like, squares inside of octagons. I don’t even know how I completed this, but anyways. Yeah, so to make things worse, here’s our software stack. It’s just a giant monolith. We have a database and lots of codes that go to it, but no one can figure out how this thing works, right? So it’s like it was just really intense. Makes you feel like this. Super excited about a moving gif in my presentation.

Jenn Luna:  So every day felt like this, but I truly enjoyed it because I was learning so much. It made the team bond. We sat for late nights together, drinking and trying to figure things out. It was a blast. I learned so much. The point is this was the most valuable five years of my software experience. We had a huge monolith. We built it into microservices. We went from a startup to well-oiled machine. We used to use SourceSafe for source control. Does anyone know what that is? When you check out a file, it’s locked. No one else can check it out. It’s just ridiculous. I don’t know. Maybe I’m the only engineer, is that why I’m the only one that thinks that’s funny? Okay.

Jenn Luna:  So I’m going from this 10-person team on to five years of trying to build this thing out, and then eventually we have 150 people on distributed teams all over the nation. I worked remote from Arizona for three of these years, so I was coming to San Francisco every month. We didn’t have any processes. Like I said, we’d lock ourselves in rooms and then at the end of the day, we had scrum, agile, we were just knocking projects out quick. Requirements were everywhere. There was no way I could forget anything. Everywhere I looked, the requirements were there. So if I messed up, it was on me. Like I said, late-night releases. After that, we had pipelines, you’d push your changes to production, it gets pushed up to actually be released, and your stuff is out there, but not before going through massive amounts of tests. So it was way harder to mess up.

Jenn Luna:  So everything was perfect, right? Sunny days every day, and I get bored. So to quote Miley Cirus, it’s definitely the climb because I really enjoy just trying to get what was messy into something beautiful and it was so much fun and I learned so much. So here’s a quick snapshot of our software stack afterwards. It was just really nice, microservices everywhere.

Jenn Luna:  So since I became bored, I was looking for the next challenge. I had gotten my real estate license with my husband during late night classes. It was just something I was interested in. Him and I bought and sold a few properties together, so I wanted to truly understand this experience and I also didn’t want to pay commission to anybody. So it was nice to just get my license. I don’t know. Why not, you know? One of my things is I just try to do too many things, and it’ll drive me nuts, but at the same time I love it. Uh oh. Oh. Okay.

Jenn Luna:  So I joined HomeLight … back to the monolith, right? It’s seriously not this bad. But it is a monolith. We have a giant code base. Here’s a more realistic representation of our software stack. Our sales app, our HomeLight.com, all the blogs related to that, internal tools, everything’s built on top of the same code base, right? We do most things in Ember, but we’re quickly adopting React. We have Ruby on Rails backend. We use Sidekiq for all of our acing job processing, and then we utilize Redis for things like queuing, and then we use a Postgres HomeLight database.

Jenn Luna:  So HomeLight agent matching. I don’t actually work on the agent matching or the algo, but this, to me, is the core part of our business. I’m more of internal tools, sales app stuff, but because this is the most important part of our business, I wanted to talk about this so I had the engineer that works on agent matching give me the details through a fire hose a few days ago. So it’s definitely more than just swiping left or right. You’ve heard all these ladies talk about matching. We have algorithms, like Silicon Valley. That’s our secret sauce. We have four versions. These versions have over 150 data sources that power them through ETL and we have just under about 50 million transactions that are analyzed for around two million agents. So it’s a lot of data. Also a gif that moves.

Jenn Luna:  So our matching process; I’m just going to quickly talk about it because it’s very involved and very well-thought out and it works extremely well and it’s definitely our pride and joy. We take in raw data, it gets summarized, and very recently we’ve been utilizing elastic search like crazy for scaling abilities. Before this last version, before our scaling was kind of on a vertical level. There was no way we were going to be able to keep searching through all this data as it grows and be productive. It took many seconds, which is bad in software world. So for millions of agents and transactions, we needed some other solution, so that’s what V4 has done for us. Elastic search also has some really great geo-spacial search context and it leverages scoring algorithms and decay functions that basically just helps you search the data better. And then after that, we apply all the basic matching criteria that these ladies talked about, like area, buyer, seller, property details, all the basics. Sorry about that. I don’t know what that was. Yeah.

Jenn Luna:  So then after that, we geo-code the address. What we noticed is that neighborhood knowledge is very effective, so if the agent has had many transactions in an area where this house is being sold or wanting to be bought, we will boost those agents because neighborhood knowledge is just very effective with people. It helps them, they feel more comfortable, and I don’t know, you just kind of know things that maybe you wouldn’t have known if you are just diving into that neighborhood. Then we analyze these agent metrics, we rank the agents, these are based on things like number of transactions, how long it takes them to close a house, just a bunch of stuff like that. And then at the very end, we will apply agent preferences because it’s a two-way street and agents should have preferences. So if they want only sellers with blue hair, hashtag picky agents … so we should also let them choose what they want so that it’s a two-way match. I was going to put a picture of people with a heart but that’s too personal I think.

Jenn Luna:  So lastly, performance is key for our data processing because we have millions of agents in our database, we definitely need to keep scaling correctly. So this Version4 that we have has brought our searching from eight to 15 seconds or so down to under a second, which is pretty incredible. So they can just keep scaling horizontally and it’s going to be totally fine. And elastic search … I think I already said this, I’m going to skip that. And then also, we are constantly refreshing this data every month. We don’t want any stale data. We don’t want agents to be picked up that have retired or don’t have any transactions in the last few months or anything like that. So we make sure that it’s meaningful. Then lastly, we do very slow roll-outs. We will roll something out, see what the results are basically in terms of conversion rate, so if something’s working really well, we’ll keep it but we’ll finally tune these algorithms and then once we have something that we think is working the best for us, we roll it out nationwide.

Jenn Luna:  So that’s it for me. Thank you, guys.

Tina Sellards speaking

Facilities and Administration Manager Tina Sellards gives a talk on “Connecting Data and Technology to the Human Experience” at HomeLight Girl Geek Dinner.

Tina Sellards:  Thanks, Jenn. Hi, guys. We are going to have a dessert bar in the back. You are welcome to grab some now if you like. It’s cookie dough, but definitely something to stick around for.

Tina Sellards:  So a little less on the technical side for me as the Facilities Manager, I’m sure you can imagine. My name is Tina Sellards. I am the Facilities Manager here at HomeLight. That is definitely not all that it encompasses my job. As many people know in a startup, I am an administrative assistant to our CEO, I do a lot of our licensing on our brokerage side and really kind of jumping into our title marketplace side as well, but then also this space that you see here, the food that you’re eating today, all of that stuff is definitely me. Thank you. So the human side is pretty huge to me, as you can imagine.

Tina Sellards:  A little background for you on me. I went straight out of college, graduating from the University of Central Florida, very proud of that, UCF, go Knights. And then went into AmeriCorps right out of that. Really had that if not us, then who, if not now, then when mentality when I came out of school, and was ready to change the world. Learned a lot about government and what kind of bogs down that world as well as I went into that, and decided I wanted to really jump into changing that world as well from the inside and was lucky enough to join the ’08 Obama campaign. Really that network that you build, so huge, your tribe, the people that I met, my AmeriCorps experience helped me bridge that changeover into my work on the Obama campaign in ’08. I ran a region of Florida for them. Everything from getting the volunteers in the door, staffing, getting an office, doing all of those things with no money, really, on that side of things. So super interesting.

Tina Sellards:  But something that was really interesting to me on that ’08 Obama campaign was the data, honestly, and the technology that was being used. This chart right here is from the Pew Research Center. It was really the first campaign that was using Internet as a main source of information for people. As you can see here, from 1996 all the way through 2008, among adult users, Internet usage for your political information went up significantly and they were utilizing a … I don’t know if you all are on it, a service called MyBarackObama.com. MyBarackObama.com actually was a community-based system. They had over 35,000 groups that organized 200,000 events throughout the US to get him elected. In some great foresight, decided to keep that live and use that as a community organization tool throughout his administration and then into the next campaign, which was actually pretty amazing. And then I also noticed something as I was looking at this research, too, which was very interesting to me which was how voters communicated about the campaign. Look at that Twitter down there. In 2008, only one person said they used Twitter to communicate. So just want to let that sit in a little bit as we kind of think about that. On the Twitter side of things, I think we’ve come a long way.

Tina Sellards:  And come into the 2016 campaign. And I make this transition really to talk about Twitter, Facebook, Google, all of those things as we’re using and the data that we’re using, the technology that we’re using, and does it really connect us more. Does it do those things? Do you get the information? Is that the correct information? Are you getting to connect with people in the same group as you, those kinds of things. It was really important to me as I kind of came off of that campaign and started to move into a more kind of people-role in organizations that I was doing, how do we, as a group, as a community, really build that interaction and not silo ourselves into those easy data groups or easy breakup groups that we can kind of put ourselves in. I think one thing that just kind of zoomed in for me was fear. Fear is really kind of a driving factor, right? And why we allow ourselves to be siloed into some of these groups. A fear of maybe that big tech company to breakup your industry, or a fear of the unknown of a different group of people or community than you. Unfortunately, fear really can kind of drive some of these things and I think that’s kind of where we’ve come with some of the data and technology. How do we get away from that is the next question.

Tina Sellards:  I think, and I very much subscribe to Brene Brown. I don’t know if any of you ever listened to Brene Brown or any of that, but vulnerability is how we do that, and leadership with vulnerability is a really key point in the human connection. I think we can really hurt ourselves and break ourselves up by just kind of communicating with the groups that we know and doing the things that we always know. Being vulnerable and letting ourselves be open to that information and being open to other people’s experiences is really how we build these communities and I think something here that I really appreciate about HomeLight and just bringing it together is a core value for us, and it’s not only a core value, it’s something we really live is being a part of our family and really being that open, unique kind of environment. I think it’s super important because I don’t think we’re going to conquer these fears and these issues that we have as a larger society if we don’t start opening up to that and really starting to have those conversations as a group.

Tina Sellards:  So I just wanted to share a little bit about my experience on that and data, and the human connect and hope you all stay vulnerable, open, and communicate as a whole community together, because that’s important in building communities like HomeLight and other … Girl Geek, and things of that nature. Keep those communities open. Be vulnerable.

Vanessa Brockway speaking

Senior Manager of Business Development & Strategy Vanessa Brockway gives a talk on “Data & Emotion in Making Career Decisions” at HomeLight Girl Geek Dinner.

Vanessa Brockway:  Hey, everybody. Vanessa Brockway. I’m on the business development team here at HomeLight, and that means a bunch of different things, but we won’t get into that today. So one thing’s we’re asked about, things we’re passionate about. So in my spare time, love to travel and I love to do interior design. But the thing I’m going to talk about today is using data and emotion and making career decisions. I think that’s probably a common thread among everybody. Often times people are drawn to events like this when they’re thinking about the next move or what they should do next. I’m going to share a bit about my perspective on this and then how that led me to HomeLight.

Vanessa Brockway:  So I think careers are a lot like The Game of Life where it’s not just kind of this up and to the right or corporate ladder, there’s a lot of twists and turns, unexpected events. You kind of sometimes are accelerating, sometimes you’re in cruise control and you can’t always predict everything that’s going to be coming your way. And so when thinking about how to approach your career and how to plan for it or how to decide what the next step is evaluating your life as a whole and the things that kind of get you going and what motivates you. As the qualitative aspects, it’s the emotion to drive the data that will also influence this.

Vanessa Brockway:  So I put some images up here, but do you love to travel? Do you want to be on the go? Is exploring the world something that motivates you? Or is being close to home and being able to have a more flexible location where you are, is that something that’s important in your life that time? How do you define success? Is being on the cover of Forbes or making a 30 under 30 list? Is that what’s going to make you feel valuable and that you’ve done something? Or is it building a passion project or building a company of something that’s really meaningful to you. Is that how you’re going to define success? What is the environment that you want to be at all day? Is it a big company with lots of people, huge market presence? Does that get you going? Or is it smaller office, more intimate relationships with those people that you work with? What is it that you want to be surrounded by everyday? Kind of taking that step back, evaluating what makes you feel motivated as a person, and then turning that into data elements to actually help drive a decision.

Vanessa Brockway: So in thinking about an actual company or industry, and evaluating what is the actual size of a company that’s interesting to me? Where do you want to live? What are the demands of that? And how can you take what you’ve learned about yourself by reflecting and actually put that into specific data? So these are just a couple of examples of … LinkedIn, you can actually see the size and growth trajectory of a company. Where are the locations of their offices? GlassDoor; how do employees feel about that company that you’re looking at? What are the employee sentiment and benefits, and things that people get? Crunchbase; do you want to be potentially at a startup? How do you actually quantify what that looks like in terms of amount of fundraising that’s happened? Who are the investors? And also just looking at articles about that company and being able to gather what is the industry saying about the industry as a whole, but also that company in particular.

Vanessa Brockway:  And then this is another piece. So separate from the company as you’re evaluating company, evaluating the role. So I found this comic online. I thought it was pretty funny. “I’ve always wondered why you decided to be a dog. I was fooled by the job description.” So don’t take a job description at face value. Take a step back and look at okay, what is this job within the company? What does that team look like? Is this going to be a very specific role where you’re going to be a ,subject matter expert, or is this going to be an all around athlete where you’re going to be asked to wear a number of different hats? What does the hiring plan look like? Is this company on a very, super fast growth trajectory and then it’s soon going to change? Or are we kind of more in a steady state? What is the title? Is that something that resonates with me? The comp, the benefits, does this company have cultural values that I identify with? And really looking at the specific role and breaking that down to specific data points that you can then tie back to how you evaluated yourself and looked at what motivated you and what was exciting for you.

Vanessa Brockway:  So my personal story is I started off at a pre-seed stage company, which was Stitch Fix at the time. We were under 10 people, no funding, very different but it’s my first taste of startup life. I absolutely loved it. And then I’ve also spent time at publicly traded large companies like Shutterfly where you’re working towards quarterly earnings, your massive, massive companies. And also Haus, which was a company I worked before here and it was every step of the way, I got a different kind of slice and flavor of tech companies at different growth points in their trajectory. The way I ended up at HomeLight is I realized this was the exact point in time, the type of company that I wanted to be at. I’ll walk through some of those pieces about it of how I made my decision.

Vanessa Brockway:  So for particularly looking at the growth stage of the company, for LinkedIn down in the bottom left. When I joined HomeLight about a year and a half ago, Series B, solid funding, had a runway that was very, very strong, but at the same time, this office sells under 50 people, so you’re able to do a number of different things and step into a bunch of different roles, which is something I really thrive in and really love. In terms of GlassDoor, so people loved working here. That was a huge check mark. Being able to see that the employees that were there go in everyday and working there. The real estate industry, something I’ve always been interested in. As I mentioned, I loved interior design but it’s not something I knew a lot about, so researching, seeing how the industry was talking about HomeLight, how they were talking about prop tech companies. Things like that really just help inform the decision. And on Crunchbase, looking and okay, who are the investors that are actually investing in this company? Actually reaching out and speaking to some of them, like hey, why’d you invest in this company? What do you think about it?

Vanessa Brockway:  All those data points together help to make sure that the position you’re looking at, the company you’re looking at align with what’s important to you, but then also is setting you up for a successful position after you join. Yeah, that’s what I have to say.

Sandy Liao:  By the way, when I saw Vanessa put all the slides, you actually really did a data analysis of HomeLight because she screen-shotted all those images before because nowadays, if you search HomeLight, our ratings, our LinkedIn, everything is different so you’ve actually done all those research prior to you joining and saved it in a document. That’s why you’re able to pull it into your presentation, which I’m like super impressive. It’s also unprompted. It’s impressive to know that someone actually did the work as much as Vanessa did to know, to identify HomeLight as a great place to be before she accepted the offer. So we’re great to have you and thank you for sharing that experience with us.

Sandy Liao:  I’m going to be closing up here before the end of the night and I quickly just want to give everyone here a huge thank you for sticking around again. But nevertheless, I want to give everyone here on the panel a huge round of applause please. While we were going through the preparation for the night, all of us were giving each other ideas, what we’re going to do, what are we going to do. None of us have a full-time job of public speaking and we watch all these tech talk preparation, we’re like, oh my God, we need to find some sort of inspirational speech for all of you guys to take away. But I think that the big piece from all of us speaking here is that our takeaway is we’re all just going through the same thing in different stages and different environments, but hey, we’re all trying to be here to make something work and to see what some of our potential could be. So I appreciate all of you here that’s on the panel tonight to take this time to challenge yourself to make yourself uncomfortably, becoming more and more comfortable sharing your stories and supporting from one another.

Sandy Liao:  So I’m going to quickly here, I’m going to promise to go through this really fast. But I just thought that while we tie in a lot of data and motions and talking about HomeLight, utilizing data to support our consumers, to really find the agent, and going through different marketing channels and career decisions. I think that it’s very important for everyone here who are looking into new career changes to understand what it means internally on a data perspective and what are some of the data metrics that I am looking into and that we are doing here at HomeLight as well.

Sandy Liao:  So anyone here heard the term people analytics? Great. We got a few hands. So this is just like a dictionary definition that I found online. I don’t even know if this accurate one, but it sounds pretty accurate, but people analytics is the use of data and data analysis techniques to understand, improve, and optimize the people side of the business. So analytics is become this huge buzzword, everyone’s talking about it, whatever role you’re in, what is your data, how do you measure your success and all that fun stuff. We also are doing that on the people side. But what’s really important is that we want to start to be able to create data that’s useful and not just creating data for the sake of it, but we want to create something that’s actually meaningful for everybody and for all the business decisions.

Sandy Liao:  So I’m going to share here on the four strategic imperatives for people analytics and especially for a company our stage, right? We can’t compare ourselves to companies like Google, Facebook who has kept millions and millions of data everyday that they can spend time on analyzing it. But what do we do when we’re only about less than 200 employees, we’re in about five different locations, locally, and what do we do with the analytics and the datas that we have?

Sandy Liao:  So first it is essential for us to set alignment. What it means is that alignment, not just between our employees, but making sure that our leadership, our executives are also aligned with all the decisions that we want to make. So from the people side, we want to say, hey, we want to start having educate more, development opportunities, more events like this, but if leadership is not understanding the purpose of it and that we’re not aligned, these things will not happen. So in the very beginning, it’s just essential for finance, for the VP of finance and our CEO and executives to understand what are some of the goals that we’re trying to make on the business side. Example would be what is the revenue we’re trying to achieve for the year? What are some of the headcount goals that we have? Because without knowing the essential of our business goals, as much as I want to say, hey, people first, people first, but we also need to make sure that we’re going to be able to secure ourselves financially well. So setting that alignment from the very beginning is just very crucial for this stage.

Sandy Liao:  And the second piece I want to talk about here is actually developing a data-driven culture. So this is unprompted, we’re not sponsored by this particular company, but at HomeLight, we use an anonymous feedback tool called TinyPolls. What it means is that every week this software will prompt us to ask all of our employees one question. The question could be how are you doing today to a question like this: in your current job, what is the number one thing that inspires you and that makes you happy here and want to work harder?

Sandy Liao:  So this TinyPoll’s feedback was actually created when we started all of our different offices in different countries, right? Have you heard a couple of us spoke, they started in Phoenix, we’re in San Francisco, we now have New York. How do we still gather data from our employee on the regular basis and be able to have that transparent communication between leadership team and everyone individually? I was fortunate enough to come across this platform who serves just that. We just want people to give candid feedback without being feeling like they’re going to be punished or be in trouble if they were to share anything on how they feel. So this feedback tool, we’ve actually implemented for over two years. It’s actually been working very well internally. With this data, we’re able to understand how people are feeling for whatever location you are and also be able to make decisions and programs that’s actually going to surface the direct feedback from everybody internally.

Sandy Liao:  Simultaneously, outside of the anonymous feedback loop, we also want to incorporate our performance data. What it means is that for us as a company, we started doing performance review on an annual basis, and then we also do a year-end check in, but these are not just data that you want to have between you and your manager, but we want to have 360 reviews that we get feedback from all of our peers as well. As much this is an important data between you and your manager, it is also really important for the business because we want to understand, hey, even if it’s not measurable bullet point percentage that we’re looking at, at least on a regular, quarterly basis that you are speaking with your manager to talk about, hey, I want to be able to achieve these five goals for the quarter and are you able to do that. At the end of the quarter, you guys should be sitting down, looking back at all the goals that you have set initially and if you find out that, hey, I’ve able to achieve three out of those five goals, what can the company provide you with, what type of training, or what are some of the resources for you to be able to hit the two bullet points in order for you to fulfill all of the achievement and goals that you had set initially?

Sandy Liao:  So incorporating performance data is just crucial to the business, as well as yourself. So for any of you guys sitting here, if your manager has not spoken with you over the past quarter or past six months about how you’re doing from a performance standpoint, it’s just super, super important to hold that in your hands and make that calendar invite, and make them have that conversation, right? Because especially working in a startup, these things kind of get out of hand when we’re trying to do hundred things at once, but before any of us sitting here analyzing whether or not we’re excited to look for new opportunity or what not, it is just necessary to take that step to have that conversation with people that is mentoring you and that are working with you directly.

Sandy Liao:  And last piece here, I want to incorporate a little fun before we end the night here, but collecting data is actually huge, right? So as I was interpreting how people analytics is becoming this huge thing, we, as a company, can’t share that we have a whole lot of data on the hiring side because so many of our roles are actually brand new to the company. So we have never hire a data scientist before and we are trying to hire that, so we’re trying to get data as we are developing these new roles and so forth. But a really fun data I’m sharing here today. This is actually a real, process, a number that we have from us hiring our most recent female engineer, Raquel, who’s actually here today. She flew in for this wonderful event. And we have actually sourced 448 female engineers around the country to get 26 recruiters screens. Can you imagine us just playing people, I’m sure a lot of you guys gone through this. Throughout those 24 screens, only seven of them made it through the hiring manager call. With the seven hiring manager, only four people actually got to the assessment stage. So out of the seven calls, only four of them were approved by the manger. With the four assessment, we got three on-site and ultimately we found Raquel here today.

Sandy Liao:  So these are the example of data and this is actually one that’s a fairly good example. We have some ridiculous roles that we have opened for a long time and it’s the sourcing number even bigger, but the point is in order for us to make tangible and actionable items based on data, we need to start collecting them regularly, whether it’s phone screens or whatever sourcing number it is, it’s just very crucial to do that.

Sandy Liao:  So the actionable item here, why there’s a dog. This is actually my dog. His name is Cooper. I rescued him about a year and a half ago. This is a significant picture for him because that was the day he got all his shots and he was straight legal to take on the action. He was ready to go. And since then, he’s been a wild, wild dog and I bring him around here once a while and everyone can share that experience with that. But that’s it for me. I hope this was helpful for everybody. We are a little … we ran a little later than expected, but we’re all going to be here hanging out, eating some desserts. We have wine and you guys are all welcome to just hang out and if you have any questions for us, we’re happy to answer them. So thank you so much for coming.


Our mission-aligned Girl Geek X partners are hiring!

Former Salesforce EVP Leyla Seka Speaks Out About Why Women in Tech Need to Ask for More

Leyla Seka, former EVP at Salesforce

The driving force behind Salesforce’s $8.7M commitment to closing the gender wage gap, Leyla Seka built AppExchange from its earliest days, served as General Manager of Desk.com and then Executive Vice President of Mobile and was one of the most senior female leaders in Salesforce company history. In a rare Girl Geek X interview, Leyla recently dished out some much-needed advice for working women: “always ask for more.”

There’s an element of luck to success.

When asked how she got to where she is in her career, Leyla was quick to admit that luck played a role in her success. “Anyone that says they’re successful without acknowledging the luck of being at the right place at the right time, I think is a bit too much of a narcissist.”

Leyla Seka
Former Salesforce EVP and GM of Desk.com, Leyla Seka

“I also worked my butt off and I pushed,” she added. “I just didn’t settle for anything. I just pushed, and pushed and pushed.”

“A lot of it was really, really hard, but it was totally worth it. I don’t sit around and wish or wonder about what if I had asked for this or what if I had asked for that anymore, which is a nice change.”

Despite her success, over time, throughout many companies and throughout her career, she had the sense that the men made more money. It wasn’t something she’d confirmed or had sophisticated research to back up — just a feeling.

Then Salesforce gave her the opportunity to run their Desk.com division. “It was probably the best thing I’ve ever done in my career,” she said, “I had so much fun. I had a team of four people and we grew like crazy.”

The first two years included unbelievable growth. Leyla had a team of four people: two men, two women. When bonus time rolled around, she fought hard to get a lot of money for everyone on her team.

“I really just thought they all deserved an equal amount, so I gave them all the same and I gave them a lot — a lot more than any of them had ever earned before. I worked hard.”

Then she had meetings with the people on her team. Her assistant set up the meetings, and it just happened to be the two women that went first.

In the first meeting, she shared that the woman receiving her bonus was appreciative and overjoyed, thanking her and gushing over the amount.

Then it was the second woman’s turn, and her reaction was similar.

When she told her first male team member the amount of his bonus, she was shocked to hear his response: “I want more.”

Leyla was in disbelief. “I thought in my head, ‘What? What? What?! How could you want more? You’ve never gotten this much!’ But I thought ‘Okay, I’ll ponder that.'”

Then the second man who was akin to a COO and her partner in running the business — her primary partner — was told his bonus amount. He looked right at her, and said, “I want more.”

When Leyla paused to ask him what was going on, he didn’t hesitate. He quipped, “We’ve always been taught to ask for more.”

There was no uncertainty, no doubt or reluctance in his voice. He didn’t shy away from it. He wasn’t scared or embarrassed to ask, and he certainly didn’t fear her response.

Women haven’t been conditioned to ask for anything, let alone MORE.

“It was sort of like someone slapped me across the face because I thought of all the times that I had gotten a bonus or promotion, or a job, or any of these things and I had been like, ‘Thank you,’ because that was the way my mother had raised me.”

Women aren’t accustomed to pushing for more. We’ve been told that it’s “unladylike,” and that NO means NO — in EVERY situation.

We don’t question the seemingly “generous” salaries we’re offered, as long as they’re close to or slightly above what we earned previously. We don’t asked for the bigger bonus when we put in more work, and we often don’t ask for the raise or promotion as our responsibilities increase.

Women need to start asking for more, because men already are — and they aren’t second-guessing themselves about it.

Around the same time Leyla was having this realization, one of her friends was promoted within Salesforce’s HR department. After many discussions on the topic, they scheduled a meeting with their boss, Salesforce CEO Marc Benioff, and joined forced to make a presentation that questioned pay disparities at Salesforce.

In their presentation, they noted that addressing the wage gap could be very expensive for the company, but now that they’d become aware of it, they felt strongly that it needed to be fixed.

Leyla stressed that Marc Benioff is an amazing ally and someone that’s not afraid to do great stuff, so he was like, “Go for it. Do it.”, she said.

They completed the audit, and many amazing things have come from it: it led to Salesforce creating a position for a Chief Equality Officer, the Office of Equality, hosting the first Women’s Summit, and they’ve paid over $8.7 million to reduce the wage gap.

Salesforce has made incredible progress… but it wouldn’t have happened if no one had asked.

In a conversation with former Salesforce SVP of Product Management Jennifer Taylor, who now serves as Head of Products at Cloudflare, both women lamented some of the opportunities they and others missed because they simply hadn’t asked or pushed hard enough.

Jennifer Taylor, Head of Products at Cloudflare and former SVP of Product Management at Salesforce

“I often find when I’m working with people, whether it’s men or women, people sometimes forget that hearing ‘no’ is the beginning of a conversation,” Jennifer shared. “If I had gotten up and walked out of a room every time I heard a no, I think I would have missed a lot of opportunities for growth.”

Leyla in part credits Salesforce’s culture for teaching her the valuable lesson: “Salesforce is a company that definitely teaches us all to push, to keep trying for the next goal. And I do think it’s so funny how many things I didn’t ask for that I would have gotten — and once I did ask, I did get.”

“The dialogue we have inside of our heads often hurts us more than what’s actually going on.”

Jennifer’s closing remarks certainly left an impression on this Girl Geek: “My advice is to ask and put yourself on that journey. Take those risks in asking, because you will learn and grow no matter what the response is.”

Leyla wrapped up the conversation with her own poignant advice for individuals working everywhere, and especially those working in tech leadership roles: “You have a platform, whether you think you do or you don’t. I would actually even challenge you further to ask, how are you using your platform to help people? Are you sponsoring a woman of color, are you trying to mentor a woman of color, are you thinking even beyond just our own fight? Equal pay is super important, but the work I’ve done with BOLDforce [Black Organization for Leadership and Development at Salesforce] in many ways is probably some of the most cutting edge and interesting stuff we’re doing, because we’re really trying to tackle the notion of allyship inside of corporate America.”

“We all can be allies, there’s always someone that can use your help, so it’s important to give that forward. That really helps you find your own path as well.”

The full transcript and video interview these excerpts were taken from is available here.

To hear more from Leyla Seka and other women who are passionate about having a positive impact on the evolution of America’s corporate landscape, check out the dozens of tech talks and interviews shared on the Girl Geek X YouTube channel, and subscribe to the Girl Geek X newsletter.


About the Author

Amy Weicker is the Head of Marketing at Girl Geek X, where she helps companies hand the mic to hundreds of women in tech across 40+ tech talks & dinners in the San Francisco Bay Area each year. She previously ran marketing at SaaStr, where she helped scale the world’s largest community and conference for B2B SaaS Founders, Executives and VCs from $0 to $10M. She also served as Director of Marketing at Sales Hacker, Inc. (acquired by Outreach) which helps connect B2B sales professionals with the tools, technology and education they need to excel in their careers.

“Creating an AI for Social Good Program”: Anna Bethke with Intel (Video + Transcript)

Speakers:
Anna Bethke / Head of AI for Good / Intel
Angie Chang / CEO & Founder / Girl Geek X

Transcript: 

Angie Chang: Hi, Anna.

Anna Bethke: Hello, how are you doing?

Angie Chang: Good. So we are back. I’m going to … there we go. Snazzy background. So we are recording the videos, this is a common question we get, and they will be available later on our website, girlgeek.io. Please tweet, the hashtag is GGXElevate.

Angie Chang: We’ve been sharing selfies of viewing parties, since it’s International Women’s Day, of women gathered, and allies, in rooms and offices, and coworking spaces around the world.

Angie Chang: Next up is Anna. She will be talking about how she developed the AI for Social Good program at Intel.

Anna Bethke: Cool, thank you. I’m super excited to be talking with everyone. I’m assuming my slides are showing, but let me know if that’s not the case. Awesome. I wanted to get into what does AI for Social Good mean, as well as what are some of the projects that we’ve been doing here at Intel, and things that I’ve seen elsewhere in the space, because it’s one that I’m super passionate about, and love.

Anna Bethke: But just first want to talk a little bit about how I got to where I am today. So I am from Colorado, and I also think of things in a geographic sense. Then I studied aerospace engineering at MIT out in Boston, and started on my career path as a geospatial data analyst in sorts, taking imagery from satellites, taking information from that, and then writing some algorithms to find different patterns of life, and anomalies.

Anna Bethke: Did something slightly different, but also geospatially related at Argonne National Labs, and that was in the Midwest, which was lovely, was really close to my husband’s family, but not quite the right fit for us. Moved over, and was doing a data science consulting type of gig at Lab 41, and landed at Intel doing data science work there.

Anna Bethke: So before I took up this role, I was primarily looking at natural language processing, deep learning techniques. How do we make these faster, what are the different things that we can do, what is the state of the art? It was very interesting, but I just have been so inspired by a lot of different projects, and I’ll talk about some of these groups a bit later.

Anna Bethke: Now I’ve been being a volunteer for Delta Analytics, as well as Data and Democracy, two groups that help pair you with some different projects that you can help a non-profit with, or help move the bar on what is important in the world.

Anna Bethke: That’s sort of how I define, and how AI for Social Good is defined, and it’s easiest to talk about specific projects than to say what this is, because AI is super nebulous. What is good is also something that has some subjectivity to it, but basically, the idea is how do we utilize AI hardware, and software, which is a lot of different techniques, and these technologies to really positively impact our world?

Anna Bethke: The thing that I find really promising and fascinating about this is that we can have a very large impact. This is a smorgasbord of some of the projects, and I’ll go into depth for a few of them. But there’s a lot of different verticals. Healthcare is a large one where we can start to be able to take these image segmentation networks, or object detection type of networks, and say, “Okay, well where is a potential tumor?” Or, “Where is a disease?”

Anna Bethke: “Is this something that looks benign or …” what’s the opposite? “Benign or …” sorry, I can’t say that word today. But basically, “Is this cancerous or not?” Taking these types of ideas from our research areas, and putting them into the field.

Anna Bethke: It’s very wide and varied. For earth or our different types of work there, we can do a lot of things, too. So ways to protect our natural resources, ways to protect, also, our man-made resources, so one of the projects was looking at restoring landmarks such as the Great Wall of China, or how do we map our structures and buildings so that we can have a better disaster response?

Anna Bethke: And then ourselves. How do we protect our kids against online threats, or physical threats? This is some work that the National Center for Missing & Exploited Children has been doing for years, and how can we help them with technology so that they can help protect and find potential perpetrators faster?

Anna Bethke: And then how do we help ourselves create these online communities that are better, so like preventing harassing text online, and even mitigating and stopping it.

Anna Bethke: That’s a high level. There’s more on the website, but the interesting thing is really once we get to dive into these projects. One of the ones that I think is really interesting, because it came from one of our software innovators. So these are basically entrepreneurials, individuals that are external to Intel, and they have these ideas of ways that can really help society, or these different projects, and there’s a link at the end of this presentation that you can get more information on this particular project.

Anna Bethke: Basically, this guy Peter Ma, he was looking at the issue that every minute a newborn dies from infection caused by lack of safe water or an unclean environment. This is worldwide, and it’s a very large issue. This was the World Health Organization, but the current systems that we have there are very expensive.

Anna Bethke: They require manual analysis, so you can’t just take your machine, and bring it from one village to another village, and that’s just not possible. But you want to make certain that the water everywhere in the community, and you’re going to need to be measuring this multiple times, because the water quality can change.

Anna Bethke: So what Peter did with some expertise help from Intel, is he built a convolutional neural network, basically. A computer vision model that is able to take a water sample, and using off the shelf products, as well as this Movidius Neural Compute Stick, which you can buy this commercially from a lot of different sites.

Anna Bethke: Don’t know if I have a link on it here. Oh yeah, if you go to the AI for Social Good website, then you can get to see more information on this product, and that has more information on how you can buy these NCS devices. They’re really low weight, you could actually see one in the image, I believe. That USB stick, basically.

Anna Bethke: He was able to build this entire prototype for less than $500, and now it’s even smaller, and less expensive, and it’s more than 95% accurate. So it might not be perfectly accurate, but it tells you a lot of information, and can really start to help communities know where their water is safe to drink, and where is problematic, which can greatly improve people’s lives.

Anna Bethke: Another really interesting project that I love is with a company called Resolve, and we’re building, with them, Trailguard AI, and the idea here is the camera that’s in the picture is this motion capture camera. Motion capture cameras are great. Scientists have been using them for a very long time to be able to monitor the health of animals, and where are animals located.

Anna Bethke: Park rangers are also starting to use this to be able to say, “Okay, when are there poachers in an area,” and how do we help this poaching epidemic, and really turn the tide on it? Because basically right now, National Geographic has identified that an elephant is poached every 15 minutes, or a rate of 3,5000 a year.

Anna Bethke: This presentation that I’m giving is about 15 minutes long, so in all probability, an elephant could’ve been killed during this, which is just really sad to think about. There’s not a lot of park rangers, it’s a massive area that they’re trying to cover, so what can we do more?

Anna Bethke: What we did with Resolve was embedded also the Movidius Vision Processing Unit, so this is the same chip that’s in that USB stick that was in the last project. But basically, we can run an object recognition network on the Edge, here.

Anna Bethke: Everything that is being processed is being done on this VPU, and basically what happens is an image is taken, that goes to the chip, the chip is able to run this CNN for object recognition, and in this case, we’re looking for people in particular, because this is what the park rangers are very interested in.

Anna Bethke: If there’s a person or a vehicle, then it’ll ping the park rangers, and this is really–

Anna Bethke: –Both reduce false alarms, about 75% of the images that the park rangers would’ve originally gotten, wouldn’t have anything in them. So basically, if a tree moves, then this camera goes off, because you want that to be very sensitive.

Anna Bethke: Now, by just sending the 25% that have a person or an animal, so I think people are only in about five or less percentage, depending on the camera, of course. You can greatly reduce the false alarms, as well as extend the battery life.

Anna Bethke: So we’re expecting these to last a year, which really helps, because then it’s harder for the poachers to find. If you’re always blazing a trail between all of these different cameras, then it’s pretty easy to figure out, a poacher can figure out where they are, and avoid them.

Anna Bethke: It’s really interesting. Something that we already have, both of these last two projects, as well as the next one, these technologies that we already have, like object detection is something that is getting more robust, that we’ve been using for a lot of different applications that we’ve been researching for a number of years now.

Anna Bethke: So how do we use it, though, for these really impactful purposes? One of the last projects I wanted to just mention before going into some of the stuff that I’ve learned while building up a program around these types of projects is the Wheelie. This is a project that we worked with a company called HOOBOX, and basically, they are robotics experts.

Anna Bethke: In the last example, Resolve are conservationist experts, so they bring deep knowledge about what the issue is, and we bring the technical expertise. So what HOOBOX saw, was there are a lot of people worldwide that are suffering from spinal cord injuries, and there’s more and more every day.

Anna Bethke: But the offerings for mobility devices can be expensive, complex, difficult to use, and there aren’t as many options as one would like. We developed the Wheelie 7 with them, and it lets users choose the most comfortable facial expressions to command their own wheelchair.

Anna Bethke: You can basically say, if I smile, go forward. If I raise my eyebrows, go backwards. If I open my mouth, go right. If I stick out my tongue, go left; things like that. This is really nice, because every person has different abilities, so by giving choices that extend the range of people that this’ll be really effective for.

Anna Bethke: And again, facial gesture recognition is built on a lot of deep learning applications that we have already looked at in-house, and so how do we apply it to this? What are the different things that we really need to think about in order to make sure that this product works for everybody in a very safe and reliable way, and that people’s privacy is protected, and all of the things that we really need to be considering while looking at this type of project.

Anna Bethke: This all of course is run on the wheelchair too, because you don’t want to be sending this information to the Cloud. That would take a long time, and if you are telling your wheelchair to stop, you want it to stop immediately. So this is run on the Intel NUC, it’s this little miniaturized PC with a customizable board. I think it’s four by four inches, so really small, can fit on the wheelchair, and it also doesn’t pull a lot of electricity, and the facial gestures are captured by this 3D RealSense camera, and that gives more information about the facial gesture than just a normal 2D camera.

Anna Bethke: Again, all these devices are things that we are commercially selling, which is great, because it’s things that are already built, and we can just improve them, make them better by seeing how they work in this new and different environment.

Anna Bethke: This project in particular was supported through a couple different projects. The Software Innovator Project as well, that was that CleanWater example that I gave a couple things ago, and our AI Builders Program. This one’s interesting, it’s sort of tuned for startup companies, and I’ll have a link for that one, too.

Anna Bethke: What does it really mean for me as being the head of this effort? I do a lot of different things. One is research and development, this is a slightly older picture of my cat, is a bit older, but I get to play with code, and do some literature research. That’s a little less now that the program is running, and getting a lot more interest in it, but it’s something that I try to carve out as well.

Anna Bethke: Sometimes I miss doing more of the technical work, but this is something that I am exceedingly passionate about, so I don’t mind not getting to code every day anymore. Another one is Connector. This was a breakfast ideation session last week, or a couple weeks ago, where I talked with a lot of different people like, “What can your companies do? What are the different ways that we can really just raise the bar, even just a little, with our technical expertise, with what our various organizations, or ourselves can offer?”

Anna Bethke: The last is Advocate. Talking at conferences, speaking to all y’all. One of the things I really hope to communicate is that this type of work is really important, and also really interesting and fun, and has a lot of very good business use cases that might not be as prevalent either.

Anna Bethke: I think a lot of us really want to do this type of work, because it’s the right thing to do, but there’s a lot of benefits too. It’s great for marketing, of course, as you could likely imagine, but it’s also really great for hiring and retention.

Anna Bethke: A lot of people want to be doing these types of projects, so the more that we offer them, the more that our companies can hire in this type of talent, and keep us all happy. Then the third for us specifically, being a hardware company, I think I eluded to it a bit, is that we really see a larger number of use cases of how can we apply technology, and then that helps us make certain that we are designing our technology in such a way that it is robust, and that there is a larger user base, basically.

Anna Bethke: So I’ve been learning all these different things along the way, but it’s interesting. There’s some other things too, so I wanted to share a few lessons. Asking for work that inspires you. The role that I’m in now didn’t exist, and I am so grateful for my manager, as well as the leadership here, that they’ve been really supportive in me taking on this position.

Anna Bethke: Helping me get the resources that I need, as well as helping to find what are the things that we can do now, in a couple months, where are the places that we really should be looking?

Anna Bethke: The second is that there are a lot of people who want to help, whether it’s helping plan meetings, whether it is doing the engineering work, being a contact coordinator. A lot of the projects that we have been developing are ones that one of my colleagues, or a colleague’s colleague has a friend who is doing this thing, and they are having this issue. Can we help out with that?

Anna Bethke: Those connections are wonderful. Low hanging fruit are wonderful as well. I say wonderful a lot. It’s very true. A lot of times we really try to go for, I think they’re called moon shots, but what is the coolest and the best thing that we could possibly do?

Anna Bethke: And while those are important too, there are things that we can do today, or in the next month, that are potentially quite easy for us to move the bar a little bit, but can have a really large impact in somebody’s life, or some animal’s life, the environment’s state.

Anna Bethke: Those are important to continue to look at, and to consider. But it’s also okay to say no, and this is one of the hardest things, and it’s really actually necessary to say no sometimes. I have been having to come to terms with the fact that I’m not able to do everything that I want to do, and we’re not able as an organization, or a company, to do everything, to help everybody, and so it’s sort of making sure along the way, that I’m preserving my own health and wellbeing, and sanity, and spending time with my cats, and my husband, and all of that at the same time, too.

Anna Bethke: And then who can help? Redirecting it to other resources, or saying, “I’m sorry. I can’t help at this moment, but maybe this person can.” Doing that redirect. But yeah, it’s hard. Then finally, you’re here for a reason. Whatever position you’re in, it’s awesome.

Anna Bethke: Imposter syndrome is one of my good friends now. I have definitely doubted myself along this way, when I was a data scientist, now as a head of a program, it crops up all the time. This is something that I remind myself of, and I think that we all should elevate each other.

Anna Bethke: I love communities like this, because I really feel like we do that. And then actually last, is that if you’re helping to debut hardware, there’s a high likelihood that they will take a picture of you holding the hardware.

Anna Bethke: This picture, I had no clue was going to be taken. This was right before the holidays, and I had just repainted my nails, and I had never painted them this bright and shiny, but I’ve come to terms with this too, and loving it.

Anna Bethke: I think it’s funny, so I have to laugh at myself a little bit, but I actually really like the super pink sparkly nails. How do you get started in this? Hopefully I have shared enough inspiration, and project examples. There’s more at AI for Social Good, at intel.ai, and this really follows a similar process as most other projects, so getting your ideas, finding the partners.

Anna Bethke: So the partners are someone that has the ability to really implement this into action, and that really varies, and then getting your research together, the data, the compute. There’s some things that could help, like at this AI Developer Program, and that actually gives you links to both the Software Innovator Program, as well as our AI Academy, if you’re a student, but there’s some Dev Compute there, which could be helpful.

Anna Bethke: And then your algorithm development, so how am I actually going to analyze all this data, and make sense of the world? And then testing and deployment. This is really important, of course, to make sure that the system is working before it goes out into the wild. Aibuilders.com is the startup company connector, if that’s something that you’re in, and then one of the really important things as we’re doing this, is to talk about, and think about how do we do project management, project deployment in a responsible way, so there’s a bunch of different resources that are out there. There’s a lot of toolkits, so this goes everywhere from checklists like Deon from DrivenData, which just [inaudible 00:23:00]. These other things you should be considering as you go through, to more algorithmically based mechanisms like the IBMs 360 Fairness Toolkit, or the What If tool from Google.

Anna Bethke: Take a look if you are doing a project. Either for a socially impactful, or anything else, and there’s a large discussion around this right now, which I love. I’ll leave you all this sample of various social good volunteer organizations.

Anna Bethke: This is definitely a growing area of interest, so something in the Bay that I’ve been involved in is Delta Analytics. This is mostly San Francisco, but the other ones are either completely based in the United States everywhere, or also global, so DataKind, Data for Democracy, Code for America, Visualization for Good is really cool if you’re more on the visualization side, and then there’s a lot of different hackathons and challenges that you can join, too.

Anna Bethke: So yeah, that’s it.

Angie Chang: Thank you, Anna. This has been a great, very informative, resourceful talk. There’s been a lot of chatter and questions. I don’t know if we have time for … I’ll send you the questions, and you can maybe answer them on Twitter. I think you’re pretty active on Twitter, and we can get all the questions answered, with helpful links.

Angie Chang: Our next session will be starting soon, so thank you so much for joining us.

Anna Bethke: Thank you, yeah, I’ll definitely answer them there.

Episode 11: Introversion, Shyness and Being You

Transcript:

Angie Chang: Welcome to Girl Geek X podcast, connecting you with insights from women in tech. This is Angie.

Sukrutha Bhadouria: This is Sukrutha.

Gretchen DeKnikker: I’m Gretchen.

Rachel Jones: I’m Rachel, the producer of this podcast, and we’re the team behind Girl Geek X. This podcast brings you the best of Girl Geek X events, dinners, and conferences, where we’ve been elevating women in tech, for over 10 years.

Angie Chang: Today we’re talking about personality, and how it shows up at work.

Rachel Jones: Is this something that’s come up at Girl Geek dinners? In questions in some way, or?

Angie Chang: I don’t think it really comes up in questions, but you can definitely tell that when a Girl Geek is speaking on stage, and, or off that they have their performance self, and then they have their authentic self.

Angie Chang: So, as an introvert, I’ve always enjoyed discovering who is also an introvert, and how they’ve managed to either shine on stage, shine in meetings, how they manage to speak up.

Sukrutha Bhadouria: I don’t think it’s been a discussion point, but we’ve definitely seen it.

Gretchen DeKnikker: I think as we go through this today, also thinking about that, there’s a difference between being an introvert, and being shy, or being uncomfortable, or sort of lacking confidence in one area. Right? Those are all not the same thing but, they sort of all get wrapped into one.

Gretchen DeKnikker: There are lots of introverts that aren’t shy, and there are lots of extroverted people who freak out at having to get up and speak in front of people, too. I always get annoyed, as an introvert also, that there’s this thing of, you’re shy, and so you’re not talking to me, or you lack confidence, and therefore, you’re being shy, and it’s like no, I’m just introvert. I find the discussions I’m having with myself in my head, way more fascinating than the discussion we’re about to have, and it doesn’t really have anything to do with being shy or anything else.

Rachel Jones: That’s something that I think is really interesting in this conversation. I think so often, personality isn’t even who you are. It’s who everyone else, decides that you are-

Gretchen DeKnikker: Mm-hmm (affirmative).

Rachel Jones: And it really affects the way that people treat you. So, as soon as they decide that you’re an introvert, then that totally affects the way that they’re interacting with you, and kind of affecting how they perceive your ability to change, or what you’re capable of doing.

Gretchen DeKnikker: Mm-hmm (affirmative).

Rachel Jones: Like, oh, this person is this type of way, so they won’t be able to do this, or they won’t be interested in that, or this is the reason why they’re acting this way. I think the labels that people put on each other around personality in the workplace can definitely have damaging effects.

Gretchen DeKnikker: Or that being an introvert is somehow some sort of disability. Like, oh, well you’re just gonna have to learn how to be an extrovert. It helps if you can play an extrovert on TV but, honestly introverts bring so much to the table, and should do so unapologetically.

Sukrutha Bhadouria: I think even the evolution when, Angie, you first started doing the intros, before a Girl Geek dinner, the talks would start, then when I started to join you, we were like so raw, and we were so awkward, and uncomfortable, and now we’re more confident.

Sukrutha Bhadouria: While I do think in a lot of ways I’m an extrovert, I’m very uncomfortable to the point when first started working, and you know you’re introduced to like Agile and Scrum, and you’re to give your update, and say what you had worked on the day before, and what you plan to do that day. I was so intimidated when I would have to give my update. Before, I would look down, and talk, and pretend I was looking at my notebook because I was so awkward.

Sukrutha Bhadouria: You know, I just decided there are things I need to do, and I forced myself. I need to know people at work. It’s really important to have those water cooler conversations, to get to know more people, and because I don’t feel comfortable talking right away, I ask a question, and they’re talking. So, that’s how I kick off a conversation. So, even though I’m shy when I first meet people, I found a work around, I guess.

Gretchen DeKnikker: I’m the opposite because I’m an introvert but, I spent my whole life doing dance, and singing, and acting, and so, I’m way more comfortable presenting in front of 100 people, than I am with three people I’ve never met, that I’m forced to have a conversation with. That’s terrifying to me, so I think it’s just sort of your comfort zone, right? What you’ve done previously that has helped you gain confidence to do it, right?

Gretchen DeKnikker: I think those smaller conversations are still, like, but now I think I can just say this is super uncomfortable, right? And bear with me, which I would never have done before. Angie, I feel like you have a lot to add on this because I feel like you are both shy, and introverted, and yet you’ve chosen this career path where you’re this role model, and sort of creating this whole environment for women, and you’re doing it all while you’re struggling with being shy.

Angie Chang: I like to think that the job found me [crosstalk 00:05:28]. I think most of what I actually do is, I’m more articulate writing. So I’ve done a lot of writing, and then when we do all the introductions at Girl Geek dinners, and having to give talks on diversity, and women in tech over the years. It’s really helped sharpen, and it’s definitely taken about 10 years to get to a place where I can actually just give talks now, rather easily but, it did take that decade of really grating through some pretty embarrassing experiences, to get to a place where I can now give a talk, and not think too much about it.

Gretchen DeKnikker: Is that your advice? That you just have to keep doing it until–

Angie Chang: You’ve just got to keep doing it for a long time, and I think I’ve definitely seen other people do this as well. Sometimes we see someone be super confident, and then we don’t really think about how long they’ve been doing this. If not in this particular job or industry, maybe they had 10 years of improv experience under their belt, and you just didn’t know it, and that’s why they’re so great with their gesticulations, or something.

Sukrutha Bhadouria: You know just the other day someone who’s really shy at work, told me that they wanted advice. They were talking about how they admire how I could just talk to anyone, and I’m like, “Listen, listen, this wasn’t always how it was.” I feel like I need to know everybody, like I said earlier, and I just force myself to talk to people.

Sukrutha Bhadouria: Now about what you just said, Angie, being able to give talks, something I struggle with myself. I was saying just giving an intro at a Girl Geek dinner, why I started to force us to do it, is because we needed to get better at it, and we did. I do think that obviously you got better at it faster than I did, but I had to, and everybody takes varying amount of time, but I had to really practice. I would initially write out what I planned to say, and researched the speakers, just in case I needed to say something about them, and I would come prepared. Same thing now, it’s so much easier for me to mentally prepare to give a talk, and I always say yes when I’m asked to speak. I just never say no.

Angie Chang: I like that, over preparing although you don’t have to, and that gives you an extra source of great things to say about people, and be great at giving talks.

Gretchen DeKnikker: [inaudible 00:07:57] quote from Disha Gosalia, who gave a talk at our Quantcast Girl Geek dinner, on how she navigated her career, as a shy engineer. It’s great. You guys should check it out. Now she was recently promoted to the VP of Service Operations and Support.

Disha Gosalia: Growing up in India, when you’re somebody who’s a straight A student, or academically focused, you’re kind of placed on a pedestal, and you always make your parents proud, and so it doesn’t matter if you’re a loner out there. I never realized that I was a shy, loner kid.

Disha Gosalia: So imagine my surprise when after I completed my software engineering degree in Computer Science, and went for my first job as a software engineer. In my first half yearly performance review, when your boss goes through all the great 10 things you did, but that one area of development that you always think about.

Disha Gosalia: He actually asked me, “So are you a little introverted? I never see you walking around the desks of your colleagues, or chatting up with them, and you actually don’t even talk much in team meetings.” And I’m like, “Am I supposed to talk much in team meetings? Well, I’m new. Should I not be listening more?” But, that was honestly the first time I realized that my personality did have a part to play in my career.

Disha Gosalia: Fast forward several years, now as I parent to fairly sensitive kids who are often called shy and quiet. I grapple with this thought on a daily basis. How do I raise confident young adults, who can accept themselves as what they are, but at the same time also has this growth mindset?

Sukrutha Bhadouria: I totally get that. I’ve struggled speaking up in meetings, even though I had a lot to say. I just realized that there were ways in which I could prepare myself to be able to speak. One is, of course, I would practice. You always know the content, or at least the goal of a meeting, before you go in. At least most of the times you know, so you can go in prepared with the points that you want to bring up. If someone else brings it up, that’s fine. You can add to it.

Sukrutha Bhadouria: So that sort of thing is really important for people to at least get a sense of what your opinion is. You can’t grow in any role without people really knowing what your opinion is, and whether or not you agree or disagree. You can’t always be silent. I don’t know if I would have given the feedback to her, the way her manager did, but I can totally see this feedback being necessary, and valid.

Angie Chang: When she was talking at the Girl Geek dinner, I was like, that is so me. I was called out, at least a few times, for being quiet in meetings. Then I’d actually actively have to work on being able to jump in and give an opinion or talk, give an update, make myself known in the room. So that’s definitely something to recognize, and overcome but it is definitely overcome-able.

Gretchen DeKnikker: I mean I think I come at it from, maybe the opposite angle, right? I have lots of opinions, as anyone who knows me will tell you. I think where I really struggled was, I’m a person who, I can live in this deeply analytical world, and I can live in this very, very creative world. That sometimes that allows me to see solutions, and patterns, and things that are really weird, and non-obvious. It took me probably until I was in my early 40s, to stop hiding that, or apologizing for it, or saying things like, “This might be really weird, but.”

Gretchen DeKnikker: Or I would spend days testing something out, to see if it would work before I would suggest it to anyone, and to really be like, “You know what? This is actually what makes me unique. This is the unique thing that I’m bringing to the table.” And to not be afraid to speak up in that way. To be like I know that my brain doesn’t work like, I think there’s some test out there. It’s like 90% of people don’t think that way, right?

Gretchen DeKnikker: So I was like, “Well, I’m awesome. You guys are so lucky to have me here, even if you think my ideas are crazy.” Right? And being okay with, it’s wrong. It’s okay if it doesn’t work out. It’s better to speak up. I don’t think you’ve told us, if you’re an introvert, or not.

Rachel Jones: True. I was thinking about that, when you were talking about what your introversion looks like, and how you are more comfortable presenting than talking with a room of three people. I was thinking, yeah, I’m more comfortable presenting to a room of 100 people than just mingling in a room of 100 people.

Gretchen DeKnikker: Right?

Rachel Jones: That’s the scariest thing-

Gretchen DeKnikker: Yeah.

Rachel Jones: To me but, I think one thing that was really interesting that stood out to me from Disha’s quote is how she said she didn’t realize that she was a shy, loner kid-

Gretchen DeKnikker: Yeah.

Rachel Jones: She didn’t realize until it was told to her, and when that feedback was given to her, it was really just behaviors that her boss was seeing, and not really who she was as a person. So I think thinking of personality as a fixed thing, versus things that you can learn, or things that you can challenge yourself to do. Yeah, I think that kind of complicates this question of personality, and how much it limits a person, or defines a person.

Sukrutha Bhadouria: I think as a manager, you need to be equipped to understand various personality types. You know the contradictions, where it feels like maybe not everybody knows it. So they don’t give the best feedback, and they don’t call attention to an area, along with what the benefits would be if they were to, sort of, acquire a different way of expressing themselves. If you don’t do that, it gets very confusing, and it feels very strange that you’re being told that you’re quiet, and that’s why that’s going to hinder your growth. That’s something that we need to be a little bit more sensitive about.

Sukrutha Bhadouria: I do want to say, though, there are certain tracks and career–study tracks–that you will find more introverts in because you know, especially when you’re a software engineer, a lot of what you do has to be a little bit more solo. You’re not working in a group. You’re not presenting as often as one would if you were in business school.

Sukrutha Bhadouria: There’s this quote I read somewhere that said, “If you’re looking for an introvert in business school, you’re not going to find one because even they know how to navigate the system, and come across like an extrovert.” So it’s not surprising to me that it took her that long to be told that she was shy because that’s just–the fact that she came from Computer Science, it’s not entirely surprising that it wasn’t called out earlier, or she didn’t draw attention to it earlier.

Angie Chang: I completely identify with being a shy, loner kid. I was that person in middle school, but I think that you have to know how you operate best, and even though I’m a shy introvert, I find one on one with people is really not intimidating. So it actually doesn’t bother me to go into a room and be like okay, I just need to talk to one person, and find that one friendly looking person to walk over to, and put my food down next to, and ask if that seat’s taken, or whatever, and just be like, “What are you doing here?” Making it more minimal to you, and not thinking about it as a room of 100 people. That’s also a handy tip from Girl Geek dinners, and having gone to many.

Sukrutha Bhadouria: Yeah, I watched you do that. Man, I would always be very awkward when people were in their own groups already, at a networking event, or even at a Girl Geek dinner. I remember you told me, like, walk over to who you want to talk to, and just look at them, and then they’ll automatically turn to you. Even if they’re talking amongst each other, then you can ask a question, or sneak in a word, or a sentence, and join the conversation. Even if after that you’re just listening, you’re joining the conversation.

Angie Chang: Yeah, I think we’re always feeling like we’re the most awkward person in a room but, I think that everyone feels like they’re awkward person in the room. So they’re usually happy, when you talk to them. They’re like, “Oh yes, I have someone to talk to now. They broke the ice.”

Sukrutha Bhadouria: Yeah, I think it’s been good, for me, to identify when I get uncomfortable, and when I do feel comfortable, and so in situations when I know I’m going to be uncomfortable, I bring whatever’s going to make me comfortable, so I’m my best self.

Gretchen DeKnikker: Jumping back what you guys were saying a little bit earlier, I’m curious–being told to speak up…. What advice would you give to the manager because that isn’t super helpful, right? But if you, like Sukrutha, you must have people on your team now, that are kind of shy. How do you manage that, and what advice would you give to managers?

Sukrutha Bhadouria: So first, I don’t think it’s been very helpful to tell someone, “Hey, you need to speak up more.” I think I need to be very, very aware of who has opinions, and who’s not speaking up because they’re just uncomfortable. I need to feel responsible, that I should draw them into the conversation. Once they started talking, they’re usually fine, and then slowly as time goes by, I need to be mindful to also draw attention to the fact that their thoughts were really, really helpful, and useful in the conversation, and then automatically they’re more comfortable bringing it up. I don’t know if pointing it out, and telling someone hey, you’re too quiet, just speak up, has ever been helpful, honestly.

Angie Chang: Right. That was great advice from the manager in the room [crosstalk 00:18:26].

Gretchen DeKnikker: I feel like if you’re giving that advice as a manager, like you need to speak up, then people are going to feel like, they need to add something just to say something, and then-

Sukrutha Bhadouria: Right.

Gretchen DeKnikker: How awkward, and detrimental is that, right? To just be saying anything to have your voice be heard in the room. That’s worse right?

Sukrutha Bhadouria: That is the worst because I do see a lot of women think that they just have to do that, and you hear them saying the most random things-

Gretchen DeKnikker: Right.

Sukrutha Bhadouria: But, you’re like you could not have thought that’s going to be useful in the conversation.

Gretchen DeKnikker: Yeah, but it’s the advice, that they’re, yeah-

Sukrutha Bhadouria: Yeah.

Gretchen DeKnikker: I think it’s more of a function of the advice, than anything.

Sukrutha Bhadouria: Totally, and that’s why it became more noticeable to me. There was mostly women doing that, and someone has told them, “Hey, you know you’re going to get overlooked, if you don’t say what you’re thinking.” You can’t always be thinking something useful, so you just say anything.

Sukrutha Bhadouria: So all of what we’ve been talking about, relates a lot to our quote from Product Design Lead, Eunice Noh. She shares a story of learning to speak up confidently, during our dinner with Blend.

Eunice Noh: Speaking up in meetings was really hard for me. There is an analogy I like to use. I’m not sure if everyone feels this way but, when I’m on a plane, in like the window seat. I think the entire time on the plane, wondering when am I going to use the bathroom. I’m like, “Okay I’m going to go now. Should I wake up my neighbor? Am I going to get stuck in front of the beverage cart?” By the time I convince myself that I’m ready to go to the bathroom, they’re like, “Seat belts, please.” And you gotta hold it for another 30 minutes, and it’s not a good feeling. It’s really not.

Eunice Noh: I think that’s similar in meetings, you know? I work myself in my head, and I’m trying to figure out exactly what I’m going to say, and making sure that what I say is not stupid. Should I talk now? Should I wait until the person’s finishes talking, and a million other questions in my head, and by the time I get the confidence to actually say something, the meeting has ended, and everyone’s already moving out of the room.

Eunice Noh: That’s, it’s very vulnerable. I was very, very, very vulnerable, and luckily, different from many of the past companies I’ve worked for. I had a great support system through my manager, and I had an outlet to talk to a lot of women during our Blend ladies’ night out.

Eunice Noh: So having people to help build that confidence, and give you those light nudges, and support from people around me. As a manager today, I strive to be like my manager, to give support, and help build confidence for every single person on my team. I’ve realized that building confidence, differs per person, that being thrown into the deep end, might not always be for everybody.

Rachel Jones: So it sounds like Eunice actually gives a lot of advice that was related to what Sukrutha said about how to tell someone they should speak up. How it’s more about building confidence, and knowing that not everyone will thrive if they’re just thrown straight in with knowing kind of, what approaches work for different people.

Angie Chang: What are your thoughts on this?

Sukrutha Bhadouria: I thought it was interesting that she called out that, at least at Blend, she was able to get a really good support system, through their Blend ladies’ night out, but maybe it can be a little bit more deliberate. Where you are trying to build your own support system, without waiting for your manager to set it up, or without your company’s ladies’ night.

Sukrutha Bhadouria: That will help build the confidence for you to then speak up at these meetings. Especially when you have something to say, that’s when you want to speak up. I like that she called that out, but you won’t always have those opportunities, where a manager’s helping us, or that we have a ladies’ night.

Gretchen DeKnikker: I think another thing that has happened to me in meetings, which is different from having a hard time speaking up, is having a hard time being heard. I’ve even tested this, because men will keep talking. If you go to say something at the same time, and they’ll keep talking, and so I’ve literally tested, if I keep talking, and they keep talking, will eventually they stop, and I’ve never talked long enough, that they’ll stop. That’s super frustrating.

Gretchen DeKnikker: This episode made me think about, there was this time, I’d been traveling a lot, we’d gone to the East Coast for a week, and then we’d literally flown on Friday night, and got there on Saturday, and then in Israel, the work week starts on Sunday. You’re on a weird time zone, or whatever. So, I was already extra cranky. Then we went in for a meeting, and this was my domain, and the two men in the room just kept talking, and I kept trying to add something. This was not their domain.

Gretchen DeKnikker: So, at some point, and this is not my finest moment, was I just got up, and I was like, “Well, it seems like you guys all have it figured out, I’m going to the hotel.” Up until very recently, I felt really bad about that, and I was forced to apologize to one of the people in the room, and I just thought about it recently. I was like, wait, hold on, and I felt embarrassed about that, and probably wouldn’t have shared it in this forum. Then I’m like wait a minute, no. That was a perfectly acceptable, I probably was not a nice as I could have been about it because I was tired, and I was so annoyed, but, at the same time, if you’re not going to let me speak, why am I in this room because I’d much rather be sleeping.

Angie Chang: I think that was radical candour, and that was a good job.

Gretchen DeKnikker: I want to go back, and be like, dude, I’m actually not–because I also felt like forcing someone to apologize to you, who obviously doesn’t want to apologize–

Rachel Jones: Like, what [crosstalk 00:24:17]-

Gretchen DeKnikker: Is also some ego driven nonsense, anyway.

Sukrutha Bhadouria: The danger, for me, when I was learning to speak up, is I started to talk over other people, though.

Gretchen DeKnikker: Mm-hmm (affirmative).

Sukrutha Bhadouria: That was something, and I still struggle with that, when I really rack my brain, that I want to say something, and I’m going to be heard, I tend to talk over people. I don’t want to be part of the problem.

Gretchen DeKnikker: When I feel myself very anxious, to speak, I think, “Why are you so anxious about this, why can’t you let that thought finish,” right? Most of the time it’s just like because I know what they’re saying is wrong, and I just want to be more efficient, or at least I think. Right [crosstalk 00:24:59]? Also, just let them finish their thought, but I think it’s different when you’re in this, when you’re having a conversation, and you’re interrupting people versus, it’s a room full of people, and everyone’s talking over everyone, and literally, you can’t be heard. Right?

Gretchen DeKnikker: I’ve raised my hand before, of just like, when you guys are done, and you want to stop talking all over each other, I will wait my turn, but if this is the only way that I’m ever going to be able to say anything, is if I just raise my hand. Which makes people laugh, and they think I’m a little bit weird, but I also get to speak at some point, without having to yell over the top of everybody.

Rachel Jones: I relate to that a lot, and I definitely relate to Eunice’s quote also because I’m definitely the person–My statement has to be completely perfect, before it can leave my mouth, and enter a room. So, I will spend so much time, just analyzing, reanalyzing the thing that I’m about to say, and then yeah, usually by the time I’m ready to say it, then we’re off that topic, or the conversation’s over, or the meeting is over. Yeah.

Rachel Jones: I’ve just really envied people who feel free to just say whatever, whether it’s valuable, or not.

Gretchen DeKnikker: The people who are willing to say their thought process out loud. The speak, then think versus the think then speak.

Rachel Jones: And then ponder over it, for the night, and then come up with the perfect thing to say, and be able to say it to no one.

Angie Chang: Bring it up in the hallway the next day, in a smart email.

Sukrutha Bhadouria: So, someone told me this, which I’ve never been able to do it, but they said, in a meeting, if someone, or whatever, in a casual conversation, too. Someone is saying something that you don’t like, and you already know that, they’re saying something that’s not right, you have to say, in order for it to not turn into a back and forth. You have to say something like, tell me more, and that’ll give you more of a chance to compose yourself and find the right words to put it back on them, and find the right ways to disagree, in a very civil way. I’ve never been able to do it, though. Once I do try it, I’ll let you know if it works.

Gretchen DeKnikker: In my better moments I do, but it’s mostly like, okay. It literally, tell me more about this ridiculous thing [crosstalk 00:27:19], that seems to have come out of your mouth, but that’s also me, stopping myself from saying something ridiculous that’s not helpful.

Sukrutha Bhadouria: Right. Right, I think that’s what I want to do because I sometimes just react. Not sometimes, a lot of times.

Rachel Jones: Is it the same for you, at work, and outside of work?

Sukrutha Bhadouria: As I work with the same set of people a long time, yeah, then this–the layers shed, and I end up reacting the same way, I suppose. Some of my peers I’m working with right now, I’ve worked with for three years. So, I’m starting to be less polite when someone says something awful, or does something that’s not cool. When I see in our peer group, a guy cutting someone else off, then I’m the first one to react, unfortunately. Whether it’s in my head, or verbally, I’m probably the first one to react.

Angie Chang: That’s not unfortunate.

Gretchen DeKnikker: Yeah.

Sukrutha Bhadouria: There are various ways to shut down conversations, and there are more constructive ways to make the person who is being foolish, very obviously foolish to everyone else but, if I join in that, and I play in the mud with them, then-

Gretchen DeKnikker: Right. [crosstalk 00:28:48]

Sukrutha Bhadouria: I look as foolish-

Gretchen DeKnikker: Right.

Sukrutha Bhadouria: That’s what I don’t want.

Gretchen DeKnikker: I think as I get older, it’s more of the same person. Right? I think, I didn’t bring a lot of my personality, my fun side, to work, for a really long time. Now I think, I spent so many years, at very early stage startups that now I think, me at work is more intense, and I have to pare that back because I’m so used to working on these crazy deadlines, and whatever, that I get overly aggressive about things that aren’t that big of a deal, but I just have to uncondition myself because I’m not that way with my friends. It’s like it’s gone in this further direction, the opposite way, as I’ve gotten older.

Angie Chang: I find work personalities, and every day personalities, that’s something that I’m kind of working on. Sometimes it’s intimidating to go to an office and hear everyone chatting around the water cooler, talking about things, often times pop culture, which I’m really bad at. I’m trying to find something to say, but I’ve nothing to say, nothing to say about sports, and I know if I want to talk about something that excites me, it’s the death app, of five inspirational quotes of the week, and I’m like great, I’m going to talk about death now. So, I think it’s–

Gretchen DeKnikker: And if you’re not understanding her, yes she said death app, like dying people app, yes–

Angie Chang: Yes, like a goth. So, I think it’s definitely some part of me is like how do I be one of the cool kids, and have this great conversation [inaudible 00:30:28] off the bat, about the very topical things. I think it’s just a matter of really making the points when they’re needed, and being memorable with them, instead of just the everyday chatter. The [inaudible 00:30:40] everyday chatter is helpful in greasing the wheels, to get what you want at work.

Gretchen DeKnikker: I think it’s just finding, points of commonality, and as you get older, and you get more comfortable with who you are, and you bring more and more of that to work, then I think you sort of find those different connections with different people, and I’m sure that not everyone wants to talk about the Kardashians all the time, Angie–

Angie Chang: Thank the Lord.

Gretchen DeKnikker: Just Sukrutha and I.

Sukrutha Bhadouria: Well, there’s so much to talk about. I’m kidding.

Gretchen DeKnikker: So, youngster.

Rachel Jones: Yeah.

Gretchen DeKnikker: What do you bring to work?

Rachel Jones: Well, thinking about this, I think a lot about just different work contexts, only giving you so much space to be your authentic self.

Gretchen DeKnikker: Mm-hmm (affirmative).

Rachel Jones: So, I was working with high schoolers for a few years, and the version of myself that I had to be, in order to do that job, was so different from my natural self. There were so many parts of me that I couldn’t reveal, so many things that I had to force myself into, like being more authoritative, and speaking more strongly.

Rachel Jones: It’s definitely nice to be able to feel like you can be fully yourself at work, but I think a lot of spaces are not designed for that, and I think even, just as a woman, and as a woman of color, thinking about how office culture in general, just puts a lot of limits on those kinds of expression.

Rachel Jones: I think the question of bringing your full self to work isn’t always just about your own comfort, or the extent to which you know yourself. It is a lot about the environments that you’re in, and you can definitely work to challenge that, or work to really just show up, regardless of the space that you’re existing in, but it definitely has varying levels of success with people in different situations.

Gretchen DeKnikker: Yeah. That makes me think of 22 years ago, when I started in tech, and how you’re just trying to be in the room, and not remind them that you’re a woman, and how much anonymity I started over the years, and now it’s not just, oh I can let them know I’m a girl sometimes. It’s like yeah, I’m a girl, and what? That’s my superpower, what do you have? Right. I think that the industry has evolved, right? That’s nothing that I changed personally. That’s just sort of, how things have gone.

Angie Chang: I want to see it on our shirt. Girl Geek X, that’s my superpower.

Angie Chang: This ties in well with our quote from Sandra Lopez. She brought up a great point about personality while sharing some advice for her younger self during our Elevate Conference this year. Sandra Lopez is a VP at Intel Sports.

Sandra Lopez: I joined Intel 2005. 2006 I had a meeting with an individual by the name of Early Felix, and Early Felix was pulling together executive leaders that happen to be Latinos, and he asked me a question. He asked me this, what does it feel like to be a corporate Latina? A Latina working in corporate America. I was just like, what are you talking about? I was asking what are you talking about because I never made my ethnicity, or my gender, an issue. Yet it was bothering me because I couldn’t answer the question. So each day would go by, I would take showers, I would think about it.

Sandra Lopez: Several months later I was in the shower, and I realized something. I realized that I was just never myself, and so, in the spirit, I wanted to discover who I was. I began to shed the skin that society influenced me to wear, such as the pant suit, and I began to be more familiar with who I was. Who Sandra Lopez was, in her own skin. Five feet, two inches tall. I was destined to wear feminine clothes. I wanted to wear those red suede pump shoes that you see on the PowerPoint, with three inch stilettos. I wanted to wear dresses that would accentuate my Latina curves, because that would be my ability to embrace my unapologetic self.

Sandra Lopez: If I were to advise my younger self, and do it all over again, is to be your unapologetic you, and I say that because in the process of understanding who you are, and what makes you special, you’ll discover your own depth, and what you’re capable of. You get confidence, you’ll know your place in society, in this world, and because I discovered who I was, over 10 years ago, arguably my career started to succeed.

Gretchen DeKnikker: Can I first just say, I love her so much. It’s just, we just need so many women like her to stand up and just say, these are all of the things that make me, me, and I’m going to bring them in with me, and I think it’s different for everybody, right? Figuring out who that is, and who you are, and then not feeling bad about it, anymore.

Angie Chang: I guess, after listening to Sandra, I feel inspired to think about what I do want to talk about at the water cooler, and bring myself to work in a way that works, and also less, I think Sukrutha said, to not force myself to open my mouth in a meeting, when there’s nothing particular to say.

Sukrutha Bhadouria: I think it was when Microsoft sponsored a Girl Geek dinner, one of the speakers said that, instead of working on your weaknesses, you should work on your strengths. So, when I listen to Sandra, I thought about that because she decided to just embrace who she was, and continue on, and that’s when she ended up seeing her career really start to succeed. I find that for myself too.

Sukrutha Bhadouria: Luckily we work in an environment now where people just dress any way they want, and when they’re in tech, but I’m sure at that time, for her, it must have been really, really hard, to be herself. So, yeah, more power to you if you want to wear something different, or whatever makes you comfortable.

Angie Chang: When my sister got tattoos, mom was like, “You’ll never work in an industry”, and we’re like wait a second. Now, I think tattoos are everywhere. Having hair that’s a different color than the one you were born with, is completely natural, all these things–

Gretchen DeKnikker: I hope so, my hair grows purple.

Angie Chang: Right, and all these things, all these ways we differentiate ourselves, and we celebrate our differences, has made us a more interesting workplace, and industry.

Sukrutha Bhadouria: Yeah, you need all sorts of perspectives, make a good product.

Angie Chang: All different personalities. Yeah.

Gretchen DeKnikker: Yeah, I think that’s what attracted me to tech, right? Because I am very rough around the edges, and I could figure out how to get along, and be more polished, and whatever, but instead, I found a place that would take me as is. I don’t go into big corporate environments where I would be just, pretending to be somebody else, in my ways, I stick to smaller, scrappier environments, where a personality like mine works, and so, it’s sort of, rather than trying to change me, I just kept trying out different environments, until I found one where I could just be myself.

Rachel Jones: I think that’s an interesting way to think about it. I really like that Sandra uses the word unapologetic.

Gretchen DeKnikker: Mm-hmm (affirmative).

Rachel Jones: Because to me, that really sounds like I’m going to show up, regardless of [crosstalk 00:38:33] how well my personality fits into this space, or how anyone feels, or how much space they’re leaving for me, to do this. So, it’s interesting just thinking about that perspective, versus I’m going to find a place that actually supports my personality.

Gretchen DeKnikker: Mm-hmm (affirmative).

Rachel Jones: I think, yeah, they’re both definitely valuable, and I think, yeah, it would work differently for different people, but it’s really interesting hearing both perspectives.

Gretchen DeKnikker: The thing that is sort of counterintuitive about it, is that you think if you are bringing more of yourself, that you’re being more vulnerable, and people are going to judge, and reject that. Actually what I found, and what I think Sandra found, is that people really embrace that because there’s something very genuine about it, and they know what they’re getting with you, and love me, or hate me. There’s not much mystery to what a daily interaction with me is going to be like, right. I think that people appreciate that, they just appreciate transparency.

Rachel Jones: I like thinking about this, discovering your authentic self, and bringing your authentic self being kind of a key to unlocking success in your career.

Gretchen DeKnikker: And understanding that it’s a work in progress. Right. You’re not one day just going to wake up, and be like, “Oh, I know exactly who I am now”, right? I feel like I’ve made this analogy that you’re like a jigsaw puzzle, and so I thought, around the time I turn 40, which is also around the time I became a venture-backed founder, that not all the pieces fit together perfectly, but that I had a pretty good picture of who I was.

Gretchen DeKnikker: Then as a founder, I was just put in these situations, and this intensity, that I’d never experienced, and I felt like I was suddenly handed new puzzle pieces, and that it wasn’t just, oh, I have to figure out where this goes. It’s like I almost have to reshape this whole thing, to figure out, because there’s essentially extra pieces, that’s all. You get more confident, and more comfortable, but you’re always going.

Angie Chang: When we think about people, and their authentic selves, I am always grateful for those people who came to their authentic self, and do things like quit working in finance, and now open a bakery. That’s super popular in San Francisco.

Angie Chang: Recently I’ve been constantly inspired by Brit Marling’s work. She turned down a job at Goldman Sachs to make films, and pursue her love for writing and directing and acting. So, I look at, not just people being an authentic self to be an executive, but also just be good at a craft, or good at their technical self. Obviously I’m a huge fan of The OA right now.

Angie Chang: Thanks for listening to this episode of the Girl Geek X podcast. We’ll be back soon with more advice from women in tech.

Rachel Jones: This podcast is produced by me, Rachel Jones, with recording help from Eric Brown. To learn more about Girl Geek X, or buy tickets to our next dinner, visit GirlGeek.io. You can also find videos and transcripts from our events. If you’re interested in hosting a Girl Geek dinner, email Sponsors@GirlGeek.io.

Angie Chang: This podcast was sponsored by Quantcast, a global leader in artificial intelligence technology. Quantcast is using machine learning to drive human learning, to help brands grow in the AI era.

Angie Chang: This podcast is sponsored by Blend, a Silicon Valley technology company, propelling the $40 trillion consumer lending industry into the digital age, through partnerships with banks, lenders, and other technology providers.

“The Gendered Project”: Omayeli Arenyeka with LinkedIn (Video + Transcript)

Speakers:
Omayeli Arenyeka / Software Engineer / LinkedIn
Gretchen DeKnikker / COO / Girl Geek X

Transcript:

Gretchen DeKnikker: Hey, everybody. Welcome back. Our next session here is with Omayeli Arenyeka. Arenyeka, tell me I’m saying it right.

Omayeli Arenyeka: Arenyeka.

Gretchen DeKnikker: All right.

Gretchen DeKnikker: This is important to get people’s names right. So she is a software engineer at Linkedin. She is also an artist and a poet from Nigeria. She submitted her talk to us through our speaker submissions on how she built a gendered dictionary and we thought it was so interesting that we invited her to come here and share it with you guys today, so …

Gretchen DeKnikker: Also, the videos will be available later. Don’t forget to tweet with hashtag, #GGXElevate. We’ve got the Q&A going in the bottom. And just after this session we will give away some more socks, so stay tuned.

Omayeli Arenyeka: That’s good?

Gretchen DeKnikker: Yep. I see you.

Omayeli Arenyeka: Okay.

Omayeli Arenyeka: I’m Yeli. Thank you so much for having me. My talk is about building a gender dictionary. But before we get into all the technical stuff I wanted to play a little word game. And it’s simple, you don’t have to do anything but think really hard.

Omayeli Arenyeka: So the game is, I say a word and you think of an image that’s associated with it.

Omayeli Arenyeka: Okay, here we go. Superhero. Ninja. Hacker. Rockstar.

Omayeli Arenyeka: And then now I want you to consider the images that came up, if they were of humans, whether those images were of a man or a woman or someone who doesn’t exist in those binaries. And this isn’t to shame anybody, it’s just an opportunity to reflect on biases because those biases we have they make their way into things that are supposed to be objective. So when given the option, translating from English to French, machine assisted language translation systems, like Google Translate, code the word nurse as feminine.

Omayeli Arenyeka: So in the Turkish language they use a gender-neutral pronoun that covers he, she, it. So when Google Translate goes from Turkish to English it has to decide whether the gender-neutral pronoun means he or she or it.

Omayeli Arenyeka: So this poem is written by Google Translate on the topic of gender, and is a result of translating Turkish sentences that use that gender neutral pronoun into English, so some of the lines are, he’s a teacher. He’s a soldier. She’s a teacher. He’s a doctor. She’s a nurse. She’s a nanny. He’s a painter. He’s an engineer. He’s a president. He’s an artist. He’s a lawyer.

Omayeli Arenyeka: And so, the algorithm in basing its translations on a huge corporate set of human language, so it’s reflecting the bias, a gender bias that already exists in the English language.

Omayeli Arenyeka: Another example of the effect of gendered language was highlighted be the augmented writing platform, Textio. They found that the gendered language in your job posting can predict a higher … can predict the gender of the person that you hire.

Omayeli Arenyeka: So thinking about this and other ways that our everyday gendered language communicates ideas we might not mean to, I decided I wanted to create something to allow [inaudible 00:03:45] for gender language, and that’s what this talk is about, Building a Gendered Dictionary.

Omayeli Arenyeka: So specifically, I wanted to make an API and a tool where you could find all the gendered words, you could find the equivalent of a gendered word. To be clear, what a gendered word is, they’re words that apply to a certain gender. So, lady/gentlemen, prince/princess, so some of them like lady and gentlemen, prince and princess, they have equivalents and some of them don’t. Some of them, like actor, are not gendered in definition but might be gendered in practice.

Omayeli Arenyeka: So the first question that I had to ask was, where and how do I get this data? So there are some existing data sets of gendered words. One of those examples is from a team of Boston … a team of researchers from Boston University and Microsoft Research. They created a data set that’s part of their work into removing the sexist biases that exist in corpus in data sets that train algorithms, like Google Translate. So they were trying to remove the bias from platforms like Google Translate.

Omayeli Arenyeka: But unfortunately, all the data sets I found, including that one, were not substantial enough. At most they had 1,000 words, and a lot of the words were false positives, so they weren’t actually gendered words. So I decided I would use these methods, API, static data, and web scraping to get the data.

Omayeli Arenyeka: So to start with, I had to determine what a gendered word was, so what I would I tell the computer that a gendered word was? So to start, it was all the words in the dictionary that have at least one of these terms in it, so woman, female, girl, lady, man, male, boy. For example, businessman has the word man in its definition and archeress has the word female, so both of them would count as gendered words.

Omayeli Arenyeka: Then I started looking for some APIs. So I found one of the largest … the biggest online English dictionary by number of words, Wordnik has an online API and it has a free … it has a reverse dictionary feature, which means find all the words that have one of those terms in their definition. So you can see on this screenshot, the reserve dictionary of woman is all the words in the dictionary that have the word woman in their definition. So airwoman would have the word woman in its definition, so it would count as a reverse dictionary term.

Omayeli Arenyeka: So Wordnik has a client for interacting with the APIs, so I just used that to make a call to their reverse dictionary. You can see that happening in line seven. I have all the terms and then I make the call to the Wordnik API in line 10.

Omayeli Arenyeka: So I got about 400 words back, which was kind of confusing because the API said that there were over 3,000 words that were … that had the word woman in their definition. So I had to find another data set, so I stored the 400 words I got from the Wordnik reverse dictionary API and then moved on to the second way of getting data, static data sets.

Omayeli Arenyeka: So I looked on GitHub and I found a dictionary in JSON format and I read that in using Python. So Python has a JSON module that you can just import. So I loaded that in for filtering and I got all the definitions of the word, as you can see on line six.

Omayeli Arenyeka: So, like I said, if a word has one of these terms in its definition, then it’s a gendered word. So how do we check that? With Python you can say, “If string in definition.” So if woman in definition, or female in definition, or lady in the definition, but then you have this long list of conditions. So instead of doing that we can use RegEx. So, for example, my name is Omayeli, but a lot of people often misspell it, so I could use RegEx to create one pattern that matches my name and all the misspelling of my name.

Omayeli Arenyeka: So I created a RegEx pattern for all of these terms, so I could search them in definitions and see if the word was a gendered word. So in RegEx the pipe symbol represents or, so this is saying match woman or female or girl. And then, if you find any of these strings … if you find any of these words in the string, you can see patterned at search definition, it’s searching the definition for one of those patterns. And if you find it in the definition, then we do something. But the issue with that is that it wasn’t looking for whole words, so sub-strings also count. You can see on the right the words that are matched, human, manhole, so these are not gendered words. They have the word man in them but it’s just a part of the word and not the full word.

Omayeli Arenyeka: So I had to use word boundaries, so word boundary allows you to perform a “whole words only” search. So now it’s looking for whole words and not just part of a word. So you can see on the right, it no longer matches manhole and manatee. It only matches man and boy at the bottom.

Omayeli Arenyeka: But then I also want words like grandfather, so what do I do? So word characters, which matches a word character. So anything from A to Z, zero to nine. So now it finds father and all the words that are combinations of father and another word in front.

Omayeli Arenyeka: So going step-by-step through the patterns, these parentheses are for grouping a pattern together as one. This character set says, “Match anything in this set.” So you don’t have to match all of them, but you just have to match one thing in that set. This is, like I said, matching a word character. This is matching a dash. And then this is saying it’s optional, so there can be something before the word but there doesn’t have to be.

Omayeli Arenyeka: And these are the final RegEx patterns. They’re pretty long. So after I finalized the pattern I went through the dictionary and for each entry in the dictionary, if the definition contained one of those terms then I added it to the list of gendered words. So in line eight it’s checking if any of those terms are in the definition, then we add that to our list of gendered words.

Omayeli Arenyeka: So when I add that together, the words from Wordnik and Webster and some other files, it came to about 8,000, which is great. Much more the 400 that I started with. But then when I went through the list there is words that did not belong there, words like lioness. So for my definition of what I wanted this gendered dictionary to be, it was a collection of gendered words for human beings, so not animals. So this was not a word that I wanted in my word set.

Omayeli Arenyeka: So instead … So I decided I would start to look for patterns in the incorrect words, so find … what were the common things in the definitions of the words that were not supposed to be in the set? So one of the patterns of incorrect words that I found was that in some of the incorrect words the definition included the gender term being used as the object of a preposition. So, for example, in the definition of waterfall it says, “An arrangement of a woman.” In the definition of Peter is says, “A common baptismal name for a man.” So it’s not a name that is describing a man. It’s a name for a man, so Peter shouldn’t be a gendered word. And you can see in the other definitions, “Short cape worn by woman,” or, “The position of a man.”

Omayeli Arenyeka: So how would I remove words that fit this category, and the category being the gendered word is being used as the object of a preposition? So first, I had to isolate the part of the string that I wanted to look at, and that was everything before the gendered word. So you can see, the highlighted portion is everything before the word … before and including the word man. So we can use … in Python, we can use RE module, which is for handling RegEx expressions. So the RE search method in line four will search through the text for any of those terms, any of our gender terms in line two. So in this case we have the string definition in line eight, so it’s looking for the word man and it will find the word man.

Omayeli Arenyeka: And then we get … When we get the location of where the word is, you get the end index. So in line nine you can see the search method in RegEx returns the index of where the word was found. So, from there we can get the end index and then we can use that to trim the string. So in line 11 you can see that the word, it’s now a common baptismal name for a man and it doesn’t include everything after.

Omayeli Arenyeka: So after we trim it, remove any punctuation, we use the string class in Python. The string class has a list of punctuations, so we use that to filter in line five and then we return the string without any of the punctuations. So if there was a punctuation it would remove it. So if there was a string in line seven, the return string would be line nine, so, which no punctuations.

Omayeli Arenyeka: So now that we have this trimmed definition we can use NLTK to find where the preposition in is the string. NLTK stands for Natural Language Toolkit. It’s used for processing the English language. So the first thing we do is we tokenize it, so tokenization is the process of chopping up a string into different pieces that are called tokens, and then throwing away certain characters like punctuation. So you can see on the right we pass in … This is an online version of a tokenizer. We pass in a common baptismal name for a man, and then it breaks it up into different tokens.

Omayeli Arenyeka: After we tokenize we can use something called a part of speech tagger, so I load in the part of speech tagger in line … in two, it’s part of NLTK. In line five I tokenize the string. So you can see, in line six, it has … the definition is chopped up into different pieces. And then in line eight we use the tokenizer from NLTK, which gives every token a part of speech. So you can see, common is an adjective, baptismal is an adjective, man is a noun. And I know those because I looked up what the tags were in NLTK. So you can see, NN represents a noun, so man is a noun. JJ represents adjectives, so baptismal is an adjective. And then after that, first of all, we remove the a’s and the and’s and the the’s because we don’t really care about them, so we remove them in line four and then we get … In line nine we get the word before the gendered word. So we know that the gendered word is the last word in the sentence. It’s the last word in the string, so we get the word before that, and then we check to see if that’s a preposition.

Omayeli Arenyeka: So in that case–this case, a baptismal name for a man, the word before man is for, which is a preposition so it returns false and says, “This is not a gendered word.”

Omayeli Arenyeka: Another pattern was that there were a lot of clothing items. So you can see skivvies, pajama, loose-fitting trousers, all of these are not gendered words. So I found a list of clothing items so I can remove any words that has one of these clothing items in the definition. Unfortunately, the website where I found the list, I had to apply for an API key. I did apply like six months ago and they didn’t get back to me, so I decided I would scrape their website. So web scraping is a tool for extracting information from websites that involves grabbing the html that makes up the website. And for doing this in Python there are two libraries I usually use, urllib.request and Beautiful Soup.

Omayeli Arenyeka: So the first thing you have to do is figure out how the data you want is structured in the dom, which you can do using the inspector tab of your browser. So we see that the data I want is in a link. That’s the child of a span element with a class TD. So I open the URL of the page in line two. In line three I add it to Beautiful Soup. In line four, and then in line five I look for the specific elements. So I’m trying to find links that are the children of spans with class TD. And then I get all the texts for them and I have the list of clothing items. So I use that to filter the dataset to remove any clothing items that are disguising themselves as gendered words.

Omayeli Arenyeka: And it came down to about 4,000 words. The last thing I wanted to do was find gender opposites, so I wanted to match words with their opposites, king/queen, father/mother. I can use that … I can do that using something called Word2Vec. So Word2Vec is an algorithm that transforms words into vectors. So back in 2013, a handful of researchers at Google set loose a neural net on a large corpus of about three million words taking from Google News texts. So the goal was to look for patterns in the way words appear next to each other. So you can see in the graph, microwave is close to refrigerator and it’s far from the word grass. Grass is close to garden, is close to hose and sprinkler. So the Google team discovered that it could represent these patterns between words using vectors and vector space. So words with similar meanings would occupy similar parts of the vector space and the relationships between words could be captured by simple vector algebra.

Omayeli Arenyeka: So these relationships are known as word embeddings and the dataset is called Word2Vec. It’s based on the idea that a word is characterized by the company it keeps, so a word is close to another word in space if they appear in the same context. For example, if we give the algorithm this text, since salt and seasoning appear within the same context, the model it creates will indicate that salt is conceptually closer to seasoning than, say, chair. And with that model, and with those word vectors we can do stuff like getting the similarity of words. You can see woman and rectangle are not very similar. Their similarity value is less than 0.1, whereas the similarity value for woman and wife is 0.8.

Omayeli Arenyeka: And word analogies, so you can do it for woman. You can do woman is to queen as man is to king.

Omayeli Arenyeka: In Python, if you wanna use models and the Word2Vec algorithm you can use a library called Gensim. So I mentioned earlier that they used … they set loose a neural net on a model of like three million words, so you can load that model of three million words into Python using Gensim. So it’s called Google News Vectors, so you can see in line two we have a model, Google News Vectors, and we load that in and then we have … we can call the models most similar method in order to get the equivalent of a word. And this isn’t perfect, but I think it works for most of my cases.

Omayeli Arenyeka: So in line nine we pass in woman, wife. So positive, woman is to wife, and then we pass in man, and then the results we get if the score is greater than 0.6 then we say it’s an equivalent. And so, we have …

Omayeli Arenyeka: And that was it. I got an initial word set using APIs and finding static dataset, and then I cleaned and filtered the dataset using RegEx, web scraping and NLTK, and then I used Word2Vec to find the equivalent for words that have them. And then I created a website and an API to house the data. So you can see, woman is to wife as man is to husband. So when navigating the site users can learn what words are specific to a gender, what words have gender equivalents, what words don’t, and which ones significant imbalances exist. And you can see what words have undergone semantic derogation, which is a process where [inaudible 00:20:25] take on more negative connotations. For example, the word mistress was more … was once the equivalent of the word master but over time it’s taking on a new meaning.

Omayeli Arenyeka: So in summary, the words that we don’t and do have matter. They reflect our biases and the ideas that we value [inaudible 00:20:42] risk reinforcing perpetrating those biases if we don’t [inaudible 00:20:42] the words we use and why.

Omayeli Arenyeka: Thank you.

Gretchen DeKnikker: Thank you so much, Omayeli. This was great. I wish you … Go back in and read the comments, because everyone was so excited about how you broke down this search methodology and the test you did at the beginning. Everyone was like, “Oh, I thought of a man, too.” So everyone really, really enjoyed it. Unfortunately, we don’t have time for Q&A and I know we’re missing those a little bit today, so don’t worry everybody. We have a list of the questions and we can go back and do more in-depth interviews with all of the speakers later. So your questions will get answered at some point. Thank you again.

Girl Geek X Xilinx Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Jayashree Rangarajan,  Lori Pouquette, Jennifer Wong, Ambs Kesavan

Xilinx girl geeks: Jayashree Rangarajan, Lori Pouquette, Jennifer Wong and Ambs Kesavan discuss the latest trends in accelerating computation for real-time machine learning at Xilinx Girl Geek Dinner in San Jose, California.

Speakers:
Eva Condron-Wells / Senior Manager of Talent Development / Xilinx
Niyati Shah / Senior Software Engineer / Xilinx
Changyi Su / Staff Design Engineer / Xilinx
Uma Madhugiri Dayananda / Senior Software Engineer / Xilinx
Tom Wurtz / Senior Director, Documentation & Program Management / Xilinx
Ambs Kesavan / Senior Director, Software Infrastructure Engineering & DevOps / Xilinx
Lori Pouquette / VP, Global Customer Operations / Xilinx
Jayashree Rangarajan / Senior Director, Software Development / Xilinx
Jennifer Wong / VP, FPGA Product Development / Xilinx
Angie Chang / CEO & Founder / Girl Geek X

Transcript of Xilinx Girl Geek Dinner – Lightning Talks & Panel:

Angie Chang: Okay. Here I go. Hi, thanks to you all for coming to Xilinx. My name is Angie Chang and I’m the founder of Girl Geek X. How many of you here, it’s your first Girl Geek Dinner? Okay. A good amount. All right, so Girl Geeks Dinners have been happening up and down the San Francisco Bay Area for the last 10 years. About every week, we’re in a different company. And I’m really excited to be here tonight at Xilinx.

Angie Chang: Personally, I was really excited when I looked up the website and I was like “I can’t wait to come here, check out the technology”. And the demos have been really awesome, and I encourage you to hangout and look at them. And walking down that hall of patents is very inspiring. And of course the food was amazing.

Angie Chang: So, thank you to everyone for coming. One thing I want to encourage everyone who comes to Girl Geek Dinner is to network. And I know that’s a kind of scary term, we’re always told to network. But one thing that I always try to do, even as an introvert, is to meet people. At least one or two people over a dinner, when you guys are chewing your food, just talk to people, ask them what they’re doing, if there’s anything you could do for them, and if there’s anything they could do for you. Because people are often looking for new opportunities, they’re looking to learn about the technology and also just to make friends.

Angie Chang: ‘Cause when you have friends in your workplace, or in your industry, that’s how we can stay in tech. We all know that women are dropping out of the workforce over time, and that’s really why we continue to do these dinners, is to encourage woman to continue coming out, meeting each other, encourage each other, and helping each other stay leaning in or whatever you like to call it, to your career.

Angie Chang: So please enjoy yourselves, and thank you again, Xilinx, for hosting.

Eva Condron-Wells speaking

Senior Manager of Talent Development Eva Condron-Wells welcomes the crowd to Xilinx Girl Geek Dinner in San Jose, California.

Eva Condron-Wells: Wonderful, thank you so much, Angie.

Eva Condron-Wells: So welcome to Xilinx. My name is Eva Condron-Wells and am a senior manager in human resources. I have the pleasure of helping employees at Xilinx listen, give, and feel as though they belong.

Eva Condron-Wells: I’m a social scientist by background, and I grew up in this very specific niche of the semiconductor industry, at another well known company. Most of my career has been in this particular slice of the semiconductor industry. And I think, in this moment, I just want to reflect on the fact that we have many things to be grateful for. And I want to start this evening with thanking a number of people.

Eva Condron-Wells: So first and foremost, thank you to the Girl Geek team for creating this type of forum for us to connect. So thank you so much, Girl Geek. Thank you to our Xilinx greeters, gurus, and guides. I hope that they’re making your visit meaningful. If you haven’t already seen other aspects of our campus, we welcome you to join us in the demo room later on. And, of course our guests, right? So you took time out of your day to come meet us, we know that you work hard and we’re all different types of Girl Geeks, but regardless of that difference, we’re here together to celebrate that brilliance of who you are and share who we are. So thank you for taking time out of your busy days to join us.

Eva Condron-Wells: And speaking of Xilinx, how many of you knew what Xilinx is or does before you saw the Girl Geek invitation? Raise your hands.

Eva Condron-Wells: Wow, that’s pretty impressive. You must be in fairly technical households. Because Xilinx, sometimes called X-links, but we help people understand how to pronounce it, isn’t necessarily a household name. That said, we affect the lives of people everyday. As we’re nested in technology that’s used broadly around the world, affecting everyone’s lives.

Eva Condron-Wells: Xilinx is a 35 year old company. Founded, what I like to say, on friendship. It was founded on friendship and a very incredible idea that is continuing to create many innovations, now and in the future. Our mission is to build the adaptable intelligent world. We have 4,000 employees plus, worldwide, and 4,000 plus patents. Giving us a one-to-one ratio of people to patents.

Eva Condron-Wells: When I articulate the pride I have in where I work, and who I work with, I genuinely mean we have some of the most brilliant people in the world. And you will get to hear from some of them tonight.

Eva Condron-Wells: So we invented the field programmable gate array. Not something everyone hears everyday, but this is a device that makes the life of an engineer easier. It speeds up their ability to perform their responsibilities and have chips work in various ways, right, thousands of applications can be used. But these engineers are empowered to reprogram chips in the field in hours instead of waiting weeks to get a completely new product. So this is the power of what we were founded on, the concept of enablement, empowerment, and acceleration.

Eva Condron-Wells: We have over 60 industry firsts, and our latest product, Alvio… Which is absolutely gorgeous, and is in our demo room, you can see the board… Is plugged into data centers, giving a 90x acceleration over a CPU. This nested technology is inside of thousands of products, changing the way we live, love, work, and play.

Eva Condron-Wells: We are not only diverse in our thought, that creates this technology, we are seekers of innovation, and we welcome brilliant minds who want to play in this space as well.

Eva Condron-Wells: Tonight our focus is on acceleration. You will gain insights into acceleration by the people who enable it. We will have three lightning talks, a panel, and Q&A where we look forward to hearing your questions to dive a little bit deeper into this topic. After 8 pm, we’ll transition to desert for those of you who haven’t already taken advantage of that. Some more demos, just around the corner, and some more networking.

Eva Condron-Wells: This is a very special evening for us, and we’re thrilled to have you. Now onto acceleration. Let’s let the lightning talks being. May I have our first presenter? So please welcome one of our first of three presenters, Niyati Shah. Thank you so much, Niyati.

Niyati Shah speaking

Senior Software Engineer Niyati Shah gives a talk on compilers for adaptable compute acceleration at Xilinx Girl Geek Dinner.

Niyati Shah: Good evening, everyone. So let me begin my introducing myself. My name is Niyati and I work in the logic optimization group. I primarily focus on software architecture and [inaudible] design. Outside of work, I enjoy weightlifting, and traveling, and doing what I love, both at work and outside is what makes me a girl geek.

Niyati Shah: So let me start by asking you all a question. How many of you here are hardware engineers? Brilliant. And how many are software engineers? Perfect.

Niyati Shah: So, as we can see right in this room, we have a range of distribution of engineers from hardware background and software background. And that, I believe, presents the direction in which Xilinx has been headed. Traditionally Xilinx is a semiconductor and FPG chip company, and most of our customers used to be hardware developers. But as we are moved into the data center and acceleration markets, more of our customers are coming from software backgrounds.

Niyati Shah: And, what I’d like to do today, is give you an overview of the tools that we have that help our developers, regardless of their background, use our FPS to run their designs.

Niyati Shah: So we have the Hardware Developers who code using RTL, Verilog, VHDL. And for them, we have our signature product, which is the Vivado Design Suite. Next we have our Hardware Aware System Software Developers, who use C/C++ and SystemC. And for them, we provide them the Vivado HLS Compiler. We also have our Software Application Developers, who tend to use FPS mostly for accelerating their products or their designs. And for them, we have the SDAccel environment. And finally, for our Data Scientists who use frameworks such Caffe and TensorFlow, we have the AI Compilers or Edge Compilers.

Niyati Shah: In the next few slides, I’m going to go into a little bit more detail onto each of these different tools, so that you can get an idea of how they meet the needs of the targeted developers.

Niyati Shah: So as I mentioned, most of our hardware engineers work with RTL, Verilog, VHDL. But their figure is a piece of hardware, and it’s only going to understand binary numbers or bitstream. And so we have to take the RTL through a process to generate that bitstream. And the best analogy that I could give is kind of a translator. So, if you and your neighbor were to speak in, say, different languages, and the translator will go from your language and convert it to a language that your neighbor can understand. And so we take the RTL through a similar process. We start with synthesis. But [inaudible] the RTL, we apply synthesis. And the job of synthesis is to create a logical netlist , which is technology mapped to the targeted FPGA.

Niyati Shah: Once that is complete, then we need to optimize the netlist. And that’s where I come in. My team and I work on optimizing the logical netlist. We’ve optimized the design for power, area, timing, depending on the developer need. And finally create an optimized netlist, which will better use the resources that are on the FPGA.

Niyati Shah: Once that part is complete, we run placement. And the placement essentially takes these logical blocks here, and puts them on physical locations on the FPGA. Finally, we have the router which connects those physical blocks together. And after placement and routing are complete, we have a bitstream that we load on the FPGA to run the design.

Niyati Shah: Now, instead of starting an RTL, if our developers were to start with say, C/C++ or SystemC, then an additional step gets added because we have to first convert those languages to RTL before they can be fed into the backend tools.

Niyati Shah: And that’s where our Vivado HLS Compiler steps in. The HLS Compiler provides an eclipse ID, and allows our customers to design and develop using our product… The Vivado HLS Compiler and be part of this setting on top of the Vivado Design Suite. So, once the C/C++ is converted into RTL then we feed it into the Vivado Design Suite and generate a bitstream. And that allows us to provide a comprehensive solution for people starting with C/C++ or SystemC.

Niyati Shah: Now so far the tools I have talked about address the needs of our hardware developers. But for our software application developers, we also have a tool, which is our SDAccel Environment. The SDAccel Environment and Compiler sits on top of the existing Vivado HLS Compiler and the Vivado Design Suite, and allow users with no FPGA background, no hardware expertise, to take their designs and run them on our FPGAs. They will also allow us to support heterogeneous applications. So most of our software application developers are trying to accelerate their designs using the FPGAs. And so the designs will have a software component and they will have a hardware component. And the best example I can give you, is that of computer vision.

Niyati Shah: So I’m sure some of you have security cameras at home. And, in that case, you know that there are multiple parts to that. There is recording live video, but there is also object detection, where it will tell you there’s your dog is running around outside, or say there’s a robber in your house. And in that case, the object detection part is what gets accelerated using hardware.

Niyati Shah: The SDAccel Environment provides multiple different tools to our customers. We have a debugger, profiler, libraries ranging all the the way from low level mat libraries to high performing DSP libraries, but the star of the show is the compiler. The compiler helps us provide a comprehensive solution for both the software parts and for the hardware parts. The compiler compiles the software part so that it can be ran on the x86 machine. And it compiles all the individual hardware components, the kernels, so that they can go onto the FPGA. The kernels are compiled using the xocc compiler, which internally leverages the Vivado HLS Compiler and the Vivado Design Suite to generate bitstream for the parts that need to be accelerated.

Niyati Shah: So once the compilation is complete, then the software part will run on the [inaudible] machine, whereas the hardware part, which are the kernels, will run on the FPGA and they will communicate with used each of their using Xilinx runtime tools.

Niyati Shah: Now this SDAccel Environment and compilers are very crucial and critical building blocks in our next set of compilers, which are aimed towards our data scientists, who provide us models and frameworks such as Caffe and TensorFlow.

Niyati Shah: Our AI Compilers… The object here is to take the models that the customers have provided and optimize them with Deep Neural Network pruning and quantization, so that we can get optimized models. And these optimized models are then targeted using our SDAccel Compiler, so that they can go into the FPGA and generate a bitstream. Finally, we have our SDI Compilers, which provides bitten interfaces. And so we can integrate very easily with the Caffe and TensorFlow families.

Niyati Shah: What I would like to leave you with is that regardless of your background, whether it be software, hardware, or anything between, we at Xilinx have a tool that will allow you to take your design and run it on our FPGAs easily and efficiently. Thank you.

Eva Condron-Wells: Thank you. Our next presenter is Changyi Su.

Changyi Su speaking

Staff Design Engineer Changyi Su gives a talk on machine learning platforms at Xiliinx Girl Geek Dinner.

Changyi Su: Thank you for your introduction. My name is Changyi Su. I’m a design engineer from the device power and signaling technology team. My job is focusing on memory interface, timing analysis, and where a [inaudible].

Changyi Su: So today, I will go over one aspect of machine learning, the memory. So I will explain why memory is one of the enabler for machine learning and how Xilinx can be part of the solutions.

Changyi Su: We know that deep learning is one of the many approaches of machine learning, which try to simulate the human brain working with neurons. The neural network has a layered structure, each of the layers, the dot, simulate the neuron. The line between the nodes is a wait. So all the neuron network does is a computation. To compute the output of each of the node by multiplying the wait and the each of the input nodes, and then plug into the activation function. So as shown in this figure, compared to other machine learning algorithms, deep learning algorithms scale up much better with small data. Therefore, the performance of deep learning algorithm is limited by the need for a better hardware acceleration for scaling up data size and algorithm size.

Changyi Su: Recently, FPGA become a very strong competitor to GPUs, to serve as well based accelerator for machine learning. So with a programmable, flexible, how well configuration FPGA often provide better performance per watt to GPUs. Xilinx FPGAs support many types of memory technologies, either internal or external to the device. Compared to the off-chip memory, the on-chip memory has lower latency, lower power consumption, but a higher data bandwidth. FPGA devices offer the industrial leading 500 megabit on-chip memory storage space. So this allow the users to create on-chip memory of real size to suit their applications and also eliminate some of the external components. However, the on-chip memory is very expensive, and hard to expand the capacity. Therefore, the hardware accelerators still have to depend the external memory to meet the storage requirement for machine learning. And also the bandwidth of the off-chip memory is a bottleneck.

Changyi Su: So, with Xilinx’s devices, engineers are able to optimize the memory solution for different applications. For example, for machine learning, the intermediate data… Activation data is usually stored in on-chip memory, to reduce the data movement between the processor and the off-chip memory. The HBM and DDR can be used to store the input data in a right [inaudible], the way to write a parameters.

Changyi Su: So, when we’re working on the memory solution, one big challenge is the trade off between the memory and the computer resources, to achieve the best performance with the lowest latency and lowest power consumption. So, this is a most amazing part of my job in Xilinx. And this makes me a Girl Geek.

Changyi Su: As I mentioned in the previous slide, due to the capacity limitation on-chip memory, how the accelerator still have to rely on external memory to provide the massive storage for machine learning. So over the years, the DRAM Chip density is scaling up. Therefore, the DDR memory capacity upgrade is one of the easiest way to immediately improve the system performance. So we can increase the memory density by using high density DRAM chips, or multiple die package of DRAM chips. Also, the dual in-line memory model–DIMM–is very effective to increase memory capacity with minimum PCP space. So also, DIMM is a model which can turn several DRAM chips on one side or both side of a small circuit board. DIMM can be also config to a multiple RAM configuration to further increase memory capacity. However, with multiple loading, the signal integrity of the memory channel is severely degraded. So the entire system may not operate reliably at higher data rates. Fortunately, most of this can be solved by optimizing the channel configuration with the efforts and expertise from memory design system engineers.

Changyi Su: Over the past 10 years, the DDR memory data bandwidth capability did not evolve quickly enough to keep pace with the bandwidth demanding from applications such as machine learning, video transcoding. So to bridge the bandwidth gap, Xilinx introduce high bandwidth memory, HBM. HBM take advantage of circuit stacking technology that puts FPGA, DRAM, side by side in the same package. So, the whole package DRAM structure together with one thousand data bandwidth, HBM not only can provide extra more storage space, but also enable terabyte per second data bandwidth. So with HBM enabled FPGA devices, fewer DDR components are needed. For some extreme case, like this example, without any external memory components, Xilinx’s HBM solution can provide the same capacity, but much higher data bandwidth, and the better power efficiency.

Changyi Su: So the takeaway of my presentation today is evolving machine learning workloads demand varying bandwidth requirements. Xilinx’s diverse memory technologies enable it. Thank you.

Eva Condron-Wells: Thank you Changyi. And our next speaker and final lightening talk presenter is Uma Madhugiri Dayananda.

Uma Madhugiri Dayananda speaking

Senior Software Engineer Uma Madhugiri Dayananda gives a talk on real time video transcoding at Xilinx Girl Geek Dinner.

Uma Madhugiri Dayananda: Hello, everyone. Today we’re going to explore how real time transcoding can be accelerated on FPGA in the context of data center. Can you guys take a guess on what kinds of media applications these are?

Uma Madhugiri Dayananda: Sorry?

Audience Member: Facebook.

Uma Madhugiri Dayananda: Yeah. Any others?

Uma Madhugiri Dayananda: So, that’s Facebook Live and Twitch Live Streaming, that’s used for gaming.

Uma Madhugiri Dayananda: And, it’s just not these two applications, you also have YouTube Live, and LinkedIn just announced their live streaming application this couple of weeks ago. And it’s not just limited to these applications, live video is just everywhere and it’s growing rapidly.

Uma Madhugiri Dayananda: What’s happening behind these live videos? We have a huge distribution of clients here for example, our cellphones, the tablets, the PC, the TVs, are connected all via wireless networks and each of them have their own resolution and network characteristics. And we want to download the live video to each of these clients, so… To download the live video to each of these clients, the live video input is pre-encoded with the HEVC Encoder, at a different resolution and a bitrate of… And using video transcoding. So, essentially video transcoding is the conversion of one video encoding format into another.

Uma Madhugiri Dayananda: What is the advantage of video transcoding? It provides savings in terms of bandwidth and storage [inaudible]. So how many of you have seen live videos then? You experienced video stars by watching live video? Video consumes a lot of processing resources and power, and for live video applications, latencies involves milliseconds, so…

Uma Madhugiri Dayananda: Here’s a plot that shows the encoder quality preset versus the performance. I’m taking a specific example of x265 preset slow, with the quality preset on it’s supposed to be very good, like when you look at it visually. So, comparing that… If it’s encoded with CPU, you get 10 frames per second, but if the same application is being run on FPGA it’s like 120 frames per second. And for data centered context, it’s not just the quality and performance, you also have to consider the power as well. So considering all the three in the equation, you can see that it is like 72x acceleration.

Uma Madhugiri Dayananda: How does the FPGA solution compare to GPU solution? So here is an example of FPGA based HEVC Encoder compared against NVIDIA GPU. HEVC Encoder and see at the same quality level there’s 35% bitrate savings. Which translates into your bandwidth and streaming cost reduction.

Uma Madhugiri Dayananda: Another reason for using FPGA is probably your transcoding. So the video codec world is changing with the introduction of new codecs every few years. If you the timeline, from 2010 to 2020, there’s four new codecs. We already have HEVC and VP9, and AV1 was just standardized last year, and VVC is going to be standardized next year, so… You have a hardware or a custom chip for each of these codec, if you have that then it’s going to be a lengthy design process and also you can’t use the hardware, so… If you use FPGA, then the applications can be adapted… On the same FPGA you have HEVC application be running and also the VP9 application, so FPGAs are adaptable and reusable.

Uma Madhugiri Dayananda: What is happening behind the scenes? How is the video transcoding happening on the FPGA? We have the live video coming in from the host’s CPU and that’s being decoded on the FPGA using H.264 Decoder, and scaled into multiple resolution using adaptive [inaudible] scaler. And these scaled resolution videos are again to be encoded with the better quality encoder, and sent out back to the host.

Uma Madhugiri Dayananda: Here is this video transcoding stack that Xilinx offers. I work end to end on this pipeline. Building FFmpeg applications, XMA plug-ins, testing these applications on different SDAccel boards, targeting different devices. And not just these, I also work on video algorithms, including the quality, for improving the quality, and benchmarking encoders from partners according to customer requirements, so… I guess that makes me a Girl Geek. I work on things that I’m passionate about, video compression technologies.

Uma Madhugiri Dayananda: What I would like you to take away from this talk is FPGAs give you better performance, better adaptable and reusable, and you don’t necessarily have to be a hardware engineer to use FPGA, you can be a software engineer and still use FPGA to isolate your application. Thank you.

Eva Condron-Wells: Thank you so much, Uma, and to all of our lightening talk presenters. Let’s shift gears to our panel of senior Xilinx leaders. Here to lead our panel discussion is Tom Wurtz, Senior Director of Documentation and Program Management. And our distinguished panel.

Tom Wurtz: So tonight we’re going to explore the topic of acceleration. We’ve got a great panel of senior Xilinx leaders here to enjoy the [inaudible] discussion. So we’re going to start with Jayashree Rangarajan. She’s a Senior Director of Software Development. And her Girl Geek power is simplifying solutions to complex engineering ideas. We’ve also got Lori Pouqeutte, who’s our Vice President of Global Customer Operations, and her Girl Geek power is knowing what the customer wants and needs before they know what they want and need. All right. Next up is Jennifer Wong, and she is a Vice President of FPGA Product Development, and her Girl Geek power is optimizing results for both engineering and management. And finally, we have Ambs Kesavan, who’s our Senior Director of Software Infrastructure, Engineering & DevOps, and her Girl Geek power is improving the development efficiency of using tools both in the cloud and on something.

Tom Wurtz: All right, so let’s talk about this acceleration, that’s a pretty wide topic. We’re going to take it in a couple of different directions. First, computational acceleration. And then we’re going to make it a little bit more personal and talk about careers, as well as teamwork. So, we’re going to start with the hard stuff. So we’re going to geek out a little bit on computational acceleration. You heard Changyi talk earlier about machine learning. So this is things like image classification, motion detect, and speech recognition. So Ambs, I’m going to have you go first, and I’m going to have you talk us through some of the bottlenecks and challenges in this.

Ambs Kesavan

Senior Director of Software Infrastructure Engineering and DevOps speaking at Xilinx Girl Geek Dinner.

Ambs Kesavan: Thanks, Tom. I actually view this as an opportunity rather than a challenge. So, I’ll talk about the opportunities here and I’ll explain why I view that way. So we are in an era of big data, and there is a lot of statistics about big data. And I was looking at a recent article that said every single second, we generate about terabytes of data from connected devices and sensors around us, and 70% of this data is video. And that amounts to about 800 million hours every single day or something like that, and businesses are trying to take advantage of this data. They want to mine the data, to be able to look through things. One is for better customer service and also in [inaudible].

Ambs Kesavan: So machine learning applications are getting innovated at that rapid pace, in every single industrial segment, whether it is retail, or finance, or healthcare, Uma talked about video transcoding, speech recognition, name it. Every single industry is going through innovation. And these machine learning applications, they actually have the most algorithms for actually doing the machine learning. And these algorithms, if you run on CPU, that is no longer sufficient. It is not a scalable given the massive volume of data that we are looking at.

Ambs Kesavan: So acceleration is the way to go. And innovation needs to happen both in hardware, also in software, in order to accelerate this machine learning applications and algorithms. And that’s the opportunity, Tom. And Xilinx is well positioned Tom, [inaudible].

Tom Wurtz: Thanks. Lori, maybe you can join in with a few more thoughts.

Lori Pouquette speaking

VP of Global Customer Operations Lori Pouquette talks about opportunities and challenges in supply chain for applying machine learning to the business at Xilinx Girl Geek Dinner.

Lori Pouquette: Sure, and I’ll take this more from the practical application in business. So we’re building a lot of capabilities to accelerate machine learning, but we actually have to apply that too in our business. And in supply chain… There’s quite a bit of opportunity in supply chain for applying machine learning to the business. And it provides good ROI to the business. A couple of the areas that we’re looking at are, what I call predictability or or predictable analytics. And one of the applications would be being able to understand the profile of the die that is coming out of the fab very early on, so you can actually match that to the demand much earlier in the work stream and optimize the use of your materials. Another predictable application is actually in the back end, when you talk about equipment. One of the things that shuts us down is unplanned machine downtime. So if we can anticipate and understand the profiles of the machinery so that we prevent it from going down in the first place, it’ll definitely propel the business. The challenges with all of this, though, are the massive amount of data that you have to gather to really train your models and make sure you have the right algorithms, so you get the ROI out of it.

Tom Wurtz: So there’s clearly a lot of opportunity is this space, so… You see companies like Google and Intel, they’re doing these dedicated AI chips, there’s dozens of startups that are actually going down this path as well. And maybe Jayashree, you can walk us through kind of what you see in terms of where is the market going from here.

Jayashree Ranga: You all heard Ambs talk about the terabytes of data getting generated. And there’s also… She was mentioning about algorithms that need to be developed… And specifically, if you look at the computing today, CPUs generally are meant for general purpose computing. When you talk of AI, and machine learning in particular, you have two types of learning. There is training, and then there is inference, right. And then training, you use all of this data that you have accumulated and we are training certain models to do a particular task, [inaudible] domain, or [inaudible] exploration, or whatever. But then, when you have to deploy that model, actually for doing the inferencing, you want this to be done in such as way that the computation happens a lot faster and the CPUs are not scaling… You probably have heard at many conferences that [inaudible] are not scaling anymore.

Jayashree Ranga: So there is a need for us to be looking at how can we accelerate the solutions that are targeted for these specific applications. So, now you’re seeing companies looking for like… Especially the examples that, Tom you gave about the GPUs and the… That’s primarily because they saw a need for how do I accelerate this? I can’t do it just with software, I need to be building custom hardware. And, it doesn’t just stop the building that accelerator, but you’ll need to have some surrounding support in the system because data that needs to come into that accelerator, and how’s the communication going to happen? So there is a need for these specialized architectures to be built and that’s why are seeing a lot of these startups getting funded to find that next big accelerator architecture that we can build.

Jayashree Ranga: Second thing with machine learning also is you probably are hearing new networks getting created everyday, right. Which means you want an architecture that you don’t build for one network, but two years later you’re not able to use it. So, you want to a hardware architecture that is scalable with the needs. And that’s an area where Xilinx’s provided solution, which is adaptable and reusable, you saw it in the presentations earlier, it provides solutions that people can use for building networks that gets scaled with new models that come in. Especially with the startups that you’re talking about, I’m waiting to see as well, which ones succeed, which ones get taken over.

Tom Wurtz: Yeah. So adaptability, programmability, is part of the Xilinx DNA. So, when we think about programmability, there’s the idea of buying a chip off the shelf and programming it to do what you want. But there’s also the notion of, once that chip is actually in operation, being able to turn off part of it, and reprogram just that part to take on a different workload. And it’s part of the general philosophy of Xilinx, and as we look forward to the next generation of chips, it becomes even more impressive what we’re planning to do, so… Jennifer, maybe you can walk us through what Versal looks like.

Jennifer Wong: Earlier I see a lot of hands go up when we ask “Who knows about Xilinx?” So I saw a lot of hands go up. Then I heard, “How many people are hardware engineers here?” So I do see quite a few hands go up. So I wouldn’t be surprised that some of you here are some of our customers, or you have used our products, it depends… When you were in school… So you must be very familiar with our earlier product lines like UltraScale or UltraScale Plus.

Jennifer Wong: So Versal, it’s a significant step up of UltraScale, UltraScale Plus in terms of performance. So what is Versal? Versal is our first 7 nanometer product, TSMC’s latest process node, and is the industry’s first ACAP. ACAP stands for adaptive compute acceleration platform. But this is truly a platform device, not your old FPGAs. Though I should say this is the really revolutionary architecture. So this revolutionary architecture combines a scalar engine, an adaptive engine, an intelligent engine, to give us this significant performance improvement. The performance improvement can go up to about 20x of today’s GPU, or 100x up to today’s CPUs.

Jennifer Wong: So, now , how do we achieve this kind of a performance improvement. It’s pretty impressive. So, given today’s focus topic is machine learning and compute acceleration, I’m going to talk a little bit about the intelligent engine. Intelligent engine is also known as AI engine. Internally, we have a name called AI engine. So this is specially designed for compute intensive applications, like machine learning and wireless operations. Now go to the next level, what is AI engine made up of? It is really a wide array of integrated DSP engines, which are capable of [inaudible] and complex MAC operations. Now we have all these very very powerful engines. We need to think about how to connect them together in order to take advantage of them in terms of hardware acceleration.

Jennifer Wong: So you have used our products before, you must know MPSoCs. So, in MPSoCs we have a processor subsystem sitting alongside with the FPGA fabric. And these two entities are sitting side by side with some interface in between them, but the bandwidth is relatively limited. So, when they operate, they are operating fairly independently. So the difference between the older generation products and Versal is we added a very powerful NoC engine in between all these powerful engines. NoC stands for network on chip. This is not new in the industry, so NoC is very standard in ASIC. But what we are doing here is applying it to our architecture in order for us to leverage their compute acceleration.

Jennifer Wong: So what is compute acceleration, or hardware acceleration? It is… What it is a design when you can partition the area that are very very performance critical. So you partition it out, and put it into these powerful engines, and use compute acceleration to make the performance improve. And then, after the partition, you can have the slower function continue to run on the processor’s subsystem. And that’s how you achieve the big performance gain. And I’m going to stop right here, there are other innovations in Versal architecture. I’ll be happy to talk to you, if you’re interested, later this evening.

Tom Wurtz: Thanks, Jennifer. Versal is definitely an example of technology escalating at an incredible pace. So I’m going to ask each of you to kind of give some thoughts on where you think the things are going to go in the next three to five years in the acceleration space.

Jennifer Wong: Maybe I’ll just follow up on what I just said. So today’s product, what we have today is pretty big. If you look at our die size, it’s huge, and power is pretty high. So, they’re going to data center, into the cloud. And I see this intensive computation not stopping, because this is in the very early stage and I see that continue to go. But in three or five years, as our process becomes more mature, our technology become more advanced, I see more and more functions going to the edge devices, like mobile phones in your hands today. And I think in future, that’s where it will go.

Ambs Kesavan: So, Jennifer let me add a slightly different viewpoint. We talk quite a bit about computer acceleration. Even tn the presentations we heard about computer acceleration, and here what happens is you’re transferring massive amounts of data from storage to compute, doing that acceleration. And then transferring the results back to storage, and there is lot of data movement happening back and forth. And that’s not necessarily very efficient, you’ll run into [inaudible] storage, and networking. And every single data center, whether it is cloud data center or on ground data center, you’re going to have compute, storage, and networking. So, what if you do the computation closer to storage? So you’re actually doing the acceleration closer to storage instead of doing the data transfer back and forth. And that’s the area Xilinx is innovating and that’s… One example that I can give is the smart SSD announcement, that happened couple of months ago, when Samsung had it’s tech day. And that precisely is doing acceleration at the storage itself. And there is also similar innovation happening on the networking with SmartLynq. So it essentially, it’s converge solution with computer acceleration, storage acceleration, and networking acceleration, and that’s where the industry, I think, will benefit a lot.

Tom Wurtz: Maybe you could talk about the software a little bit, Jayashree.

Jayashree Rangarajan speaking

Senior Director of Software Development Jayashree Rangarajan speaks on a panel discussion at Xilinx Girl Geek Dinner.

Jayashree Ranga: So, if you look at where a lot of this machine learning development is happening, it’s happening on the cloud. And they are the software developers. Throughout [inaudible] Niyati’s presentation, data scientists are looking at these massive amounts of data and looking at how am I going to write the right algorithm to solve this problem, right, and they are operating at higher levels. So what I see happening with the software is many layers of libraries being built, where these are libraries probably optimized to work for the hardware architecture that we’re targeting. Not necessarily done by the data scientists, but it’s provided by the company that is also delivering the hardware. Plus, there’s probably going to be AI specific library. For instance, we talked about video transcoding. So if you have open CV libraries that need to be provided for you. If your are software operator, you will understand this, right. Because we don’t… Anymore write string compares and stuff. You use [inaudible] libraries or SDL. So you are going to see stacks of libraries built, which are the highest level the application developer… Whether they are in a Caffe framework or a FFmpeg framework if they’re dong video transcoding. They’re going to leverage these libraries.

Jayashree Ranga: So I see a lot of innovations happening in that realm. And companies will be providing their own libraries. I see open source development or certain APIs that can be leveraged by people who are trying to address a lot of machine learning problems and various [inaudible].

Lori Pouquette: And then, if you take that from the engineering world to “Where is this all going to be used?” Xilinx has long been serving multiple end market segments, from automotive, to communications, tested measurement, medical, but our new focus now is on the data center area. The proliferation of the data, the video, it’s all going to be needing to be stored up in the cloud or on the premises. So we’ve really now got this strong focus on data center. And as part of that, in addition to selling our semiconductor devices, we’ve also started to sell boards. So we have Alveo product, which you heard Eva talk about earlier, you can see it in the demo room. So this is now making the acceleration capability, from just the semiconductor to enabling the customer with a board. So they can use the board and then go on to their design work for their entire solution much faster. They don’t even have to just start with the FPGA.

Lori Pouquette: Now, as we go out into time, what’s going to be really good for everybody is, we’re still serving all these other in markets, and as in markets like automotive really get into advanced autonomous drive or advanced applications, they’re all going to need to store that information somewhere. And they’re all going to need to process that information, and they may need to do that at the edge very rapidly, or they may be able to do that in a central place. But, basically what we’re doing here with acceleration isn’t just going to serve the base acceleration market, but all of our markets.

Tom Wurtz: Well we got pretty hard there to the extent that we pretty much used up most of our time. But I do want to take one moment to ask each of your, if you were to put into one single word, what does it take to actually accelerate a team, what would that be? And let’s start with you, Ambs.

Ambs Kesavan: Communication.

Jennifer Wong: Teamwork for me.

Lori Pouquette: Focus.

Jayashree Ranga: I’m going to go with two words, I would say it’s the winning attitude.

Tom Wurtz: All right. Thank you very much for all of our panelists.

Eva Condron-Wells: Thank you so much to our panelists and to our lightening talk presenters. At this time we’d like to open it up to the floor. And I’ll need to steal a microphone from one of our panelists, so we’ll share. Pardon me. Thank you.

Eva Condron-Wells: So, you’ve heard a lot of different insights, different perspectives. We’d like to hear a few questions. We probably have time for… We’ll take three. So… And we’ll continue the conversation afterwards. So if you’re still burning… You have a burning question in your mind, please know that we are planning to be here until 9:00, and we have plenty of demos to share too. So… And don’t forget to take your gift on your way out, I want to say that before you mentally leave… While I have your attention, that we have a gift for all of you, that we would like you to take on your way out, in the lobby of when you came in. So, please be sure to take that with you.

Eva Condron-Wells: So, that said, I’ve given you a little bit of time to think about your question, and I have a microphone right here. Come on up, let’s have… If you don’t mind saying your name and your question to our panel.

Sara Biyabani: Sara Biyabani, GridComm CTO. So the question I have, for engineering… We talked about acceleration, so I want to start with FPGA is a [inaudible] processors, you know, they’re part of the screen, right? They’re not the… Well, I mean they could be the star, they could be the diva. So then there’s the processor, right, and you’re not going in the space, competing with x86 or R. So what does that architecture look like? What’s your ideal architecture?

Jennifer Wong: I think we have a pretty good… We do have a processor. So we have a… So for the Versal we have the A72 cores, we a have a dual A72 core in the processor subsystem, in the scalar engine, what we call the scalar engine these two processors, subsystem, you a two A72 cores and then two R5 cores in there. So from a processing standpoint, we do have capability of doing that. And now we added a lot of these architecture that we can accelerate functions. So that is what we think were our niches. So we can partition it where very critical functions, we can put it into the more expensive side. We use a lot of silicon area for acceleration hardware. So we would smartly do that, and say, okay, “What is the important one that we can partition out, put it into hardware acceleration, and leave the processor still running?” So that is what we think where the niche. Everything is on one piece of silicon. So think about it, if you’re doing it outside, you have processors and other components. So, the interface takes a very long time. The key is integration for us. Everywhere, the smaller the footprint, the more integration you do, the more performance you… Both performance and power. So, whenever you got our chip, power is a big deal. So, performance and power both are advantage to integration. And that’s where we think we going in future.

Eva Condron-Wells: Thank you, Jennifer. We have actually three questions just popped up, right all in the same area. So we’re going with that energy, and please, your name and your question.

Sylvita: Hi, I’m Sylvita. First of all, thank you to Xilinx for hosting this event. Given all the other Girl Geek Dinners I’ve been, it’s nice to see somebody on the hardware chip design side kind of take a lead here. So my question is related to that. Because most of the discussion in the big data, enterprise SaaS, or AI has been around the software and the algorithms. The first time I saw a focus on hardware was actually a Startup Grind where one of the presenters talked about , don’t write off the chip guys here, because there’s a next revolution coming where we’re going to see a lot of custom built chips for AI applications. And given that we’re talking about… First of all, this is a Girl Geek Dinner, and given that we’re talking about the fact that we need women to be on an equal footing in the way that the AI is going to evolve, what are some of the ideas you might have for some of the younger folks to continue in this space?

Jayashree Ranga: Mic check? Can you hear me okay?

Jayashree Ranga: I can think of a couple of things, right. Computer architecture, ’cause as I was talking earlier about, hey there’s many architectures that need to… They’re probably waiting to be innovated, right, because we are talking of machine learning in many many domains and stuff. So I do think, as young women who are looking to hey, what do I want to do if I want to enter into hardware, I think having a good grounding in computer architecture principles is going to go a long way. And just learning the hardware aspect, alone, is not going to be sufficient. You also need to understand which domain you’re targeting. So you need to know the end customer that you’re going to be influencing. So you need to learn about the software component, also. So I think, if you want to excel in both sides, as you are getting your early education, having good grounding in both hardware principles as well as software programming principles help you better understand the needs on both sides. But as you go further, and you are looking to specialize, then at that point, it’s an individual style choice as to whether hardware attracts you more or software attracts you more. But I think it’s good to keep your options open.

Eva Condron-Wells: Great. Thank you so much. We’ll take two more questions, and then continue with our networking.

Mung: Okay, and my name is Mung. I’m a software engineer working hardware design company. You already addressed a little bit about that, and I just wonder if you can address a little bit more regarding into FPGA application in machine learning and hardware acceleration to win over ASIC in that field.

Jennifer Wong speaking

VP of FPGA Product Development Jennifer Wong speaking at Xilinx Girl Geek Dinner.

Jennifer Wong: I think the jury is still out, but I think everybody is working very hard. This is a very hard space. And I think everybody is trying a very different route. And Xilinx has Xilinx’s niche. And I think what we give here… What our specialty is, is we are reconfigurable. Aside from being able to partition… That’s definitely a big deal, we can allow software to run on software, hardware fabric to run on past hardware… The bigger part, I think, is the reconfiguring part, which I didn’t talk about earlier. So we talked about many many different workloads today. During the day, the data center can be running one kind of work load, in the evening it’s a different workload. So, what we excel, here, is we allow reconfiguration. So even you can… Within the ACAP, you can run different workloads at different times, with the exact same piece of hardware. So we believe that is a very big niche that we can further leverage in this particular space.

Eva Condron-Wells: Thank you.

Lori Pouquette: I just want to add that, beside the reconfigurability, there’s the costs. So the costs of doing ASIC is becoming prohibitive for many applications. So on higher volume applications it’s still an option. But many of our customers who might’ve classically done ASICS, are moving away from it because it’s just not making financial sense. So there’s that piece of it too.

Eva Condron-Wells: Thank you, Lori. And our final question.

Wolfgang: Hello, my name is Wolfgang. I’m a hardware engineer. And I want to thank you that I have the chance to ask a question even though I am not a Girl Geek. So, my question is about reliability. When we have Ultra high speed computations in applications such as control of industrial processes or autonomous driving, it is not only imperative that those computations are very fast, but the results need to be very reliable. And we don’t always have the option to just [inaudible] processing memory because we have a sensor somewhere, a camera, it sends data to some processor, and that processor has to make a decision and send it to some machine that needs to act and do the right thing. So what are your thoughts about reliability, or maybe a hint, it has to do with something with high speed interfaces and a robustness against big errors, but there are also many other aspects to that.

Eva Condron-Wells: Thank you.

Jennifer Wong: Okay, so reliability has always been a big issue for us. In the past, what we do is we allow [inaudible] reliability and we do things very carefully with our foundry. Because foundry give us some specs that we have to follow through. TSMC is pretty well known in terms of it being rather conservative. So, they give us specs and we obviously negotiate with them. There is a little but of push you can do, because everybody is competing for performance and power. The more you can push the foundry, the more you can gain the advantage. So, this is maybe against what you are asking, but what I’m trying to say here is we do a very [inaudible] balancing act in terms of balance versus reliability. We don’t completely ignore reliability. We go for performance, but we always make sure we can meet our reliability. And we do a very very thorough quality QA in the end. And, also, we have qualification… Pretty substantial qualification… Different time, maybe you can add to it.

Lori Pouquette: Yes. So on the quality side, we have many in markets we serve. So we have commercial grade, we have automotive grade, and we even have an aerospace… So all of those have different levels of quality and reliability qualifications that we go through. So we definitely are very attuned to the upcoming challenges of the technology to be able to respond quickly in those types of situations you’re describing. But we definitely offer a variety, and do quite a bit of different qualifications depending on the markets that we’re serving.

Eva Condron-Wells: All right. Thank you all so much for your very thoughtful questions. And thank you all, our guests, our speakers, panelists, for your insights. We genuinely appreciate you taking the time to share your insights with this team and group. So, that said, we are officially closing our technical talk. But we are not done with our evening yet, and we are happy to stick around and answer more questions. Some of our Xilinx employees are wearing “ask me about” stickers, you’re welcome to engage with them. I will be wearing one too, so find me later. And of course, we have our dessert, and networking, and demos. Thank you all so much. This concludes the technical talk, thank you.

applause

Applause from the girl geeks at Xilinx Girl Geek Dinner after the panel discussion. Thanks for joining us in San Jose!


Our mission-aligned Girl Geek X partners are hiring!