Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at Palo Alto Networks and check out open jobs at our trusted partner companies.
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Girl Geek X team: Gretchen DeKnikker and Angie Chang share their excitement for Palo Alto Networks Girl Geek Dinner in Santa Clara, California. There was a photo booth for making girl geek flipbooks!

Speakers:
Varun Badhwar / SVP of Products and Engineering / Palo Alto Networks
Liane Hornsey / Chief People Officer / Palo Alto Networks
Nir Zuk / Founder & CTO / Palo Alto Networks
Citlalli Solano / Director, Engineering / Palo Alto Networks
Meghana Dwarakanath / Manager, SQA Engineering / Palo Alto Networks
Archana Muralidharan / Principal Risk Analyst / Palo Alto Networks
Paddy Narasimha Murthy / Senior Product Manager / Palo Alto Networks
Angie Chang / CEO & Founder / Girl Geek X
Gretchen DeKnikker / COO / Girl Geek X

Transcript of Palo Alto Networks Girl Geek Dinner – Lightning Talks:

Angie Chang: My name is Angie Chang, founder of Girl Geek X. I want to thank you all so much for coming out tonight to this beautiful… Thank you so much to the Palo Alto Networks for sponsoring. I’ve never been here and the campus is amazing. This space is beautiful. We’ve had so much fun meeting people here, checking out the demos, eating delicious food and drink. Now we’re really excited tonight to meet some of the people who work here, talking about their expertise.

Gretchen DeKnikker: Angie was going to tell you this part but I’m going to tell you. Okay, did you guys see all the cool stuff? There’s a photo booth here, over there and you can do that. Then you can make a little flip book where you move and then they’ll make you a little flip book on the spot.

Gretchen DeKnikker: There’s demos back there, which seem to be fabulously popular. Got the little cards. There are recruiters all over so if it seems awesome here, and it seems pretty awesome, the food’s awesome, everything’s awesome so far, right? Yes?

Audience: Yes.

Gretchen DeKnikker: Yes. Okay, so if you want to work here, they’re also hiring. Then you can work with some of these amazing women that you see here tonight. Girl Geek, we do these every week. Who’s their first time here?

Gretchen DeKnikker: Okay, cool. We’ve been doing these for about 10 years. We’ve done over 200 of them. We do them every week now up and down the Peninsula, in San Francisco. If this is fun, make sure you’re on the mailing list and come to the next ones.

Gretchen DeKnikker: We also got a podcast, we just got going. I think we’re on episode eight. What was it about? Tech stayers. Yes. Don’t leave. Tech stayers. But there’s ones on impostor syndrome and mentorship and career transitions and learning styles and just like, so check it out, it’s on all of your usual podcasting places, and then let us know if you like it or what we could do to improve it because we’re not podcasters, we have no idea what we’re doing. We’re just talking, like I am right now.

Gretchen DeKnikker: Without further ado, thank you guys so much for coming tonight and I don’t know who I’m turning this over to.

Varun Badhwar: I’ll take it.

Gretchen DeKnikker: Okay. Let’s welcome the man to the stage.

SVP of Products and Engineering Varun Badhwar warms up the crowd at Palo Alto Networks Girl Geek Dinner for talks on secure development, secure design, how security is just such a core part of development life cycles.

Varun Badhwar: Thank you. Privilege to be here. Good evening, everyone. Really an honor to have all of you here. My name is Varun Badhwar. Figured I’d just spend a few minutes sharing a little bit of my story. I’m six months or so new into Palo Alto Networks. Came in through an acquisition, actually. We recently, Palo Alto Networks acquired a company called RedLock. I was a founder CEO of that company. As a startup, one of the biggest things that makes you successful, brings the teams together is really the culture, right?

Varun Badhwar: A lot of people, there’s no product, there’s very little salary we can pay people, the office isn’t as nice as this, but ultimately we join companies for the people who are going to work with and really a grand vision of a problem we’re going to solve. That problem for us was securing the cloud. As you’re doing that and as you’re building… For those of you who are not familiar with cybersecurity, there’s normally in this industry, if you’re successful, you likely end up getting acquired by just potentially six or seven companies that can do that.

Varun Badhwar: For me, a lot of people have said, we were only three years into building RedLock, sort of how we ended up here? Why did we make that decision? Ultimately, Palo Alto Networks, for those of you not very familiar with the company, has always been at the very top of that list for us for a couple of reasons.

Varun Badhwar: One, is you’re going to hear from our founder, Nir and the team. Just incredible pace at which Palo Alto Networks has disrupted the market, has taken a leadership position, now is the largest pure play cybersecurity company in the planet.

Varun Badhwar: More importantly, all of that has been done with only 6,000 people in the company, right? Larger companies in security have 80, 100,000 people. For us, it’s been fantastic. You come in, you get the best of feeling like a startup operating really rapidly yet having a culture, and having values that are very startup like. Everything from empowerment for the teams, empowerment down to individuals working in this company to–

Varun Badhwar: I’ve just been fascinated with how important diversity has been to this company. Obviously this is one small commitment towards that. But as I can come in here and I’m asked to go work towards our annual conference, which is Ignite that’s happening next month. From the number of attendees, our customers that are coming and tracking diversity statistics there to how many speakers we’re bringing to the table, have these advanced diversity there, diversity in hiring. and diversity obviously is is about professional backgrounds–

Varun Badhwar: Maybe, heck, if you look at our CEO, he never worked in cybersecurity before this, he came from Google, right? He’s first timer in cybersecurity, so for those of you say, “This feels like too geeky of a space.” Not really. I think we really appreciate diversity. Whether you’re coming from a consumer background, enterprise background, you get race, ethnicities, values as well.

Varun Badhwar: I don’t want to take up too much time here. Just articulating a couple of things. One, phenomenal company. We’ve loved the last six months. My teams tell me we are working harder than we did at a startup and having a lot more fun. The fact that the values are so aligned to–what I think a lot of us love probably the companies we’ve worked for. The intersection of just cybersecurity, and specifically for my part, cloud is just so fascinating. It’s a cat and mouse game, security. You’re never done building products that work well. You’re always against forces that are from–Maybe Nir will touch on, it’s a good topic for him.

Varun Badhwar: There’s people that are putting more and more emphasis. Attackers are trying every which way to get into people’s environments. They just need to be right once, we need to be right every time with the products that we build, right? Really an amazing career opportunity. Again, want to thank you all for coming here. Hopefully, you’re going to learn a lot about secure development, secure design, how security is just such a core part of development life cycles. I will pass that over to Liane and Nir.

CPO Liane Hornsey talks about the the inclusion agenda at Palo Alto Networks Girl Geek Dinner.

Liane Hornsey: Hello, everybody. I am the Chief People Officer of Palo Alto Networks and honestly, I love moments like this, and I really love moments like this because I’m in the company of a lot of technical women. I always feel, oh gosh, they’re so much better than me. They’ve got all these skills I haven’t got. They know how to code, they know how to build product, they know how to make things happen, and I work in HR. But I also think and feel very, very humble when I’m with technical women, that it is also harder for women in technical jobs to come to work each and every day. It’s harder for them than it is for me.

Liane Hornsey: When I walk through this door, I walk into a function and I’m surrounded by other women. I am not surrounded just by people who are different to me. So I don’t carry that burden of being different as I walk through the door, in the same way as many of you do. Because you work as a minority in many of your teams. I’m doubly in awe, because you can do all these things and you have all these skills that I don’t have, and you have that additional, not burden, but that additional concern of being different.

Liane Hornsey: Now, I have only been at this company for seven months, a little like Varun, about the same time. I joined this company, honestly, not really knowing much about this company. But truly, I have come to love this company with all my heart. I really love this company, and I want to tell you why. Partly it’s this, I truly believe you have to work for a company that is doing good stuff.

Liane Hornsey: Every night I drive home and I turn on my radio and I feel just that little bit less safe. Every day I think about my children online, who are a bit bigger than your children. But I think of them online, and I think about their safety. Every day, particularly as a European, I think about the importance of cybersecurity. I know I am working for the good guys and I’ve got to tell you, that feels really very, very nice, but it’s not just about what we do. What I really love about this company is how we do it. When I first came here I met my team and the first thing they said to me is, “We don’t have D&I here. We don’t have diversity and inclusion.” I’m like, “Whoa, bit weird,” and they said, “No, we have inclusion and diversity.” Then I thought, yeah, yeah, that’s a bit of a fad. You’re just changing the words around.

Liane Hornsey: But I do want to impress something upon each and every one of you. Diversity and inclusion does not work. There is no point putting in more underrepresented minorities into companies that can’t change and make them feel welcome. There is no point pulling more women into technology, if you can’t make them feel like they can bring their whole selves to work each and every day.

Liane Hornsey: For me, it’s not the diversity agenda. It’s about the inclusion agenda, and that is what Palo Alto Networks is going to be known for. That is what we are going to do that is different and unique. I don’t believe there are companies in the valley that have solved the diversity and inclusion issue. We all know we’re spending a ton of money each and every year trying to encourage minorities, trying to encourage more women, trying to encourage difference and we’re failing. And we’re failing, I realize now, because we are doing it wrong.

Liane Hornsey: It’s about time we understand, each and every person in this room, even if we’re united largely in agenda, we are not the same. I am not the same as every other woman in this room. I am an individual. I am unique, and I am special as are each and every one of you, and that is what Palo Alto Networks is going to be about. It’s going to be about over the next couple of years, making sure that each and every individual that joins this company feels special, feels that they’re doing amazing blooming work, work that will change this world and work that will make everybody safe, and that we can all be whoever we want to be at work. I think if we crack that, we’ve done something pretty darn good.

Liane Hornsey: I’m not going to say much more to you. I am so glad that you’re here. I am so glad you can see everything, that’s wonderful. But I’m most glad for you that you can hear from our founder. In my career I have worked with a number of founders, and I’ve got to tell you, it is not always a joy. They can be a little unusual. This time it’s an absolute, an amazing joy, and I’d like to introduce Nir.

Founder and CTO Nir Zuk talks about the past, present and future of cybersecurity at Palo Alto Networks Girl Geek Dinner.

Nir Zuk: Thank you, Liane. Thank you all for being here. I’m Nir, I started this mess about 14 years ago. We got funded about 13 and a half years ago. We’ve been selling products for about 12 years. Like Varun said, we are the largest cybersecurity vendor in the world today, we’re also the largest cybersecurity business in the world. Even businesses inside other large companies are smaller than us and these businesses have been around for 25, 30 years, sometimes even more than that.

Nir Zuk: How does a company that’s only been selling products for 13 years, becomes larger than companies like Cisco and Juniper and Semantic and other large cybersecurity vendors? Of course, it would through disruption, right? To disrupt the market, you completely change the market, and maybe I’ll say few words about disruption.

Nir Zuk: The first thing about disruption is that it’s a weird thing. It’s not like it’s… The way you disrupt the market is not by building a product and starting to sell it and then figuring out, wow, I disrupted the market. It’s actually the other way around. You find the market that’s ready for disruption. You find the reason why it’s ready for disruption and you address that, right?

Nir Zuk: If you think about it, some of the companies that you work with and a lot of the companies that have changed things like the taxi industry, and the hospitality industry, on the consumer side, and then companies that have changed the way we do HR and the way we do salesforce management and CRM and the way we do IT operations and so on, they were all going into markets that have been doing the same thing again and again and again and again for many, many years and found the reason to disrupt those markets, disrupted the markets, and have been successful at that. We’ve done the same thing.

Nir Zuk: The next question that I always get asked is, how do you make sure that nobody comes behind and disrupts you? It’s not easy. The thing about disruption is that when you face disruption as a large company, it’s very, very difficult to deal with that. It’s very difficult to deal with disruption because you have two pretty much bad options. The first option you have is to embrace the disruption, meaning to say, wow, this is very disruptive. Everything I’ve done so far is irrelevant. Let’s embrace the disruption. The challenge, especially as a large company, as a publicly traded company and so on, is that that really kills your business, and you have to start again. It’s not that you start from scratch, but it’s enough that your revenues go down 2, 3% and you’re done. Right?

Nir Zuk: Embracing disruption is hard because you have to start convincing the markets that you are disruptive and then you have to buy and sell them something new while they don’t buy your old thing. Then you can fight the disruption, but if the disruption is real and true, then you’re going to eventually end up staying behind, which is really what happens to our competitors when we started disrupting the market. They all fought the disruption, they all went through the five stages or first denial, right? Nobody needs it. Then we do it too, and then eventually it’s, okay, let’s go and find something else to do.

Nir Zuk: To make sure that we don’t get disrupted ourselves, the only logical way to do it is to disrupt ourselves. Keep looking why the market is ready for disruption and going and disrupting it at the risk of hurting your existing business, which we do. We keep doing that all the time and we don’t have time to talk about it right now and today, but we keep disrupting the market, we keep changing the market and changing the way the cybersecurity market works. I think that that’s the first thing that we’ve done.

Nir Zuk: The second thing that’s interesting about the cybersecurity market is that when we started, it was made of two types of companies. It was made of very large vendors. Again, I mentioned some names, Cisco and Symantec and there’s another company that they used to work for in the past called Checkpoint out of Israel, which is also a very large vendor in the industry. There’s McAfee and Juniper used to be a large vendor in the industry and there are a few others, and they all sell products that are very, very successful, but really aren’t doing anything to secure their customers. In fact, they all sell products that we call firewalls, you’ve probably heard about firewalls and everybody knows that you need the firewall, it’s just firewall is not going to make anyone secure.

Nir Zuk: Firewalls are not a security product, they are a hygiene product. Saying that the firewall is security products is like saying that soap is going to make you healthy. It’s a hygiene products, it’s not going to make you sick, but it’s not going to make you healthy. You’re not going to prevent some or the most important diseases. Right? That’s one set of companies.

Nir Zuk: The other set of companies that we saw when we started the company 14 years ago was the innovative companies, the companies that actually do something for their customers to stop the bad guys and to make the world safer, but those companies just never took off. There was this disconnect between the two when… and part of it is because it’s very hard for customers, especially very hard for organizations to tell what’s working and not working, what’s not working. Like how are you going to evaluate a cybersecurity product? They’re going to hire a bunch of hackers and pay them a lot of money and go create an attack against yourself and then see if the product… Nobody does that, right? Usually you get a script from the vendor and you followed the script, then guess what? It works, right?

Nir Zuk: When we started the company, we decided to be different. We decided that we’re going to build the product that is both going to be big, and is going to actually do something for our customers, and that’s part of our culture. There are other things in our culture that I think are very important. But I think… and I’ll talk about him in a second, but I think the most important thing in our culture, or about culture is that we strongly believe is that cultures create companies and not vice versa. Meaning it’s the culture that you have when you start a company, and if you work really hard and make sure it doesn’t change much, it’s the culture that you have over the years that’s great in your company versus your company creating a culture. Okay? If your culture is to be disruptive, then you’re going to be disruptive. If your culture is going to be, you’re going to invest in sales and marketing to convince the world that the products that you build that aren’t doing anything, actually do something, then that’s going to be your culture.

Nir Zuk: There are a few very important things that we created in the culture of the company that I think have brought us to where we are, and the largest vendor in the cybersecurity industry, and we’re also growing much faster than everyone else. There are areas where we’re by far the largest vendor, well, there are areas where we’re bigger than everybody else combined. Doing really well, and again, it’s our culture and it’s things like being disruptive. It’s things like we always… we don’t solve simple problems. Meaning, if there was something in cybersecurity that we think someone else is already doing well or we don’t know how to do better than them then we’re not going to do it.

Nir Zuk: Customers keep asking me, “Why aren’t you doing the… Distributed Denial of Service protection?” Whatever that means, right? Because I just don’t know how to do it better than others that are doing it today. They ask me, “Why aren’t you doing web application firewall?” I just don’t know how to do it better than others, so why would I do that? Okay? The things that we do here are things that we know how to do better, or we think at least we know how to do better than everyone else. That’s in our culture. Like when we make a decision whether to do something or not, that’s a very important criteria. Criteria, right? There are other important criteria. That’s one part of our culture.

Nir Zuk: The other part of our culture is to always do the right thing for the customer. Now, of course, every company that you work for will say that they are doing the right thing for the customer, but as an example that I used just a moment ago, if you invest in sales and marketing to convince your customers that the stuff that doesn’t work, doesn’t do much for your customers actually does, then that’s not your culture. Your culture is not to do the right thing for the customer and… For us to do the right thing for the customer, I think the way we think about it, the way we’re presenting this, we always do… we only do things that we can be proud of. Okay? I cannot be proud of selling a customer a product that doesn’t do what the customer thinks that the product does. So, we’re just not going to do that. We’re going to do the right things so that we are proud of what we do so that customers eventually will get the benefit of the products, and that’s very important for us.

Nir Zuk: Another area that’s very important for us is self-awareness, okay? Many companies just aren’t self-aware when it comes to the issues that they have. Whether in their structure or in their products or in whatever it is. We are very very self aware. I mean I’m not going to, of course, wash the dirty laundry here, but in meetings we always talk about the issues that we have. We always talk about competitive issues that we have, we always talk about organizational issues, we always talk about the different things that are going to make us not successful, or are making us not successful in some areas, and we’re very self aware of that and we fix it. We certainly don’t kill the messenger up, we promote the messenger here, and take care of that. Those are important things that just don’t exist in many companies. When you look back 14 years ago and we look at the set of companies that we compete against today, they just have a very, very different culture than we have today.

Nir Zuk: The last thing, which you’ve already heard, that’s important for us in the culture is diversity. Diversity is not just gender diversity, which is very important. I think among the first 25 employees of the company, about a quarter, 25th and a quarter were women. But it’s not just gender, it’s also underrepresented minorities. It’s also diversity as to where people come from in terms of the companies that they come from. We don’t just hire from two companies, we hire from as many different companies as possible, so we get as many different opinions as we can. When we think about diversity, we think about diversity across everything. It’s really an important part of our culture. Like if you walk around and you see the list of things that are in our culture, which are posted in various areas of our buildings, that’s one of them. Being diverse is very, very important for us. We just think that it makes us better and it makes us build better product for our customers. Okay?

Nir Zuk: Maybe the last thing I want to talk about is would… like Liane said, not too many people know about Palo Alto Networks, especially if you’re not in the enterprise space and not in cybersecurity space. You don’t know much about Palo Alto Networks other than maybe you every now and then you’ll hear about our financials or things like that. But if I look at the things that we’re proud of and the things that are somewhat unique to us, we are one of the large… we have one of the largest infrastructures in the world, or certainly building one of the largest infrastructures in the world. Cybersecurity is becoming more and more, and that’s something we’re driving, but it’s becoming more and more a data problem. The amount of data that you need to deal with in order to find the bad guys and stop them, it’s just unbelievably huge. We’re talking… I mean, if we today had to collect or are able to collect all the data that we need from our customers, we’re talking about several billions of events per second. Okay? This is the kind of infrastructure that we need to build. We’re talking about many, many, many, many exabytes of data in order to make our customers secure. I mean, I’m not saying we’re there yet, but that’s something that we need to build, and over the next few years we need to build.

Nir Zuk: Cybersecurity is becoming a data problem and we’re leading that. We’re very large infrastructure company.

Nir Zuk: One of the things that we’ve done, and one of the disruptions that we brought to the market is we have transformed the cybersecurity market from a market where you buy a lot of products. A typical organization and typical enterprise will have dozens and sometimes more than 100 different cybersecurity products that they deploy in their infrastructure. We transform that into a market that’s delivered via SaaS. Okay? That’s another thing that’s important about Palo Alto Networks, and yeah, there’s also the cybersecurity aspects. You’re going to be a cybersecurity expert to work at Palo Alto Networks. The number of people here that actually know cybersecurity, probably 200 or 300 of our employees, actually are cybersecurity experts. All the rest are data experts, and service delivery experts, and operations experts, and of course, that’s in the engineering department and we have many other organizations within the company. Okay? That’s what I had to say. I’ll stick around if you have some questions. We don’t have time for questions right now, and I guess next one is Citlalli. Thank you.

Director of Engineering Citlalli Solano talks about loving where she works because she identifies with Palo Alto Networks’ mission of securing “our digital way of life.” She is a proponent of the security-first mindset.

Citlalli Solano Leonce: Hello, everybody. My name is Citlalli Solano Leonce. I am a director of engineering here at Palo Alto Networks. I’m a software development and I really couldn’t be more proud of having you guys here tonight. A little bit about myself, so let me share a little bit of my story.

Citlalli Solano Leonce: I grew up in Mexico City, back in the day there were no cell phones, no tablets, no flat TV screens, no nothing, right? No, internet even, and I remember vividly how my mom would take me with her to the bank, right? The old big computers with black screens and green letters and characters going around. I always wonder what is happening behind? How could that person type something, and then some magic happens? Right? Fast forward a little bit. I got… that’s what I… got me to study computer science.

Citlalli Solano Leonce: Finally right out of college, my first job was at the central bank in Mexico. I finally, my dream come true. I was able to understand what was happening behind the scenes, but there was a slight difference back then, and is that I was not only understanding what was happening, but I was in the driver’s seat. Here I am, 21, 23 year old, building systems for my country.

Citlalli Solano Leonce: I was developing in C++ and my modules eventually ended up in the payment systems. Now people are able to transfer money from one bank to another immediately. I was paving the way for the digital transformation of my own country. That was… at that time probably, I didn’t realize that impact, but looking back it’s like, really I was a key player there.

Citlalli Solano Leonce: Fast forward a little more. Here I am standing in front of all of you in the middle of the Silicon Valley. A day in the life, you wake up, your phone plays a nice tune for you. With the internet of things, you can have your coffee machine make coffee for you, and then you wake up to the very nice smell of coffee beans. You can say, “Alexa play, what’s the weather today? What’s my stock options? You take your car, the car drives you wherever you want, right? Like that, so it’s amazing, right? All these transformation, I can’t believe I have been fortunate in life to live this revolution, right? But there’s another side to that. What is happening with all of that? Now, we all have our lives in the digital world. Raise up hands, how many of you do your banking online? Probably everybody, right? How many of you do video gaming or your kids do video gaming? Right? Now, even that is online.

Citlalli Solano Leonce: Some of us are doing… those DNA test, 23andMe, okay? Then we can share and, or maybe we’re cousins, we’re third cousins or whatever. Right? That’s amazing. But, where do you think all these data is going? Everything is hosted in the cloud, right? We are leaving our digital fingerprint over there, and it’s not only the data, these services themselves are deployed in the cloud. They’re either running in AWS, Azure, GCP. Who knows? right?

Citlalli Solano Leonce: Amazing. But we also have a big responsibility. Everything is interconnected. How do we prevent the bad guys from getting that? It’s not only just your little blog post, it’s now your financial information, it’s now your DNA information, right? Who knows what’s going to happen in a few years.

Citlalli Solano Leonce: Let’s look at how we are developing those various systems, and something that Nir was referring to, it’s not only about cybersecurity and cybersecurity professionals, right? We at Palo Alto Networks happen to make software that secures the enterprise. But security is responsibility of everybody. Who is building that 23andMe mobile app? Probably one of us. Right? Who is building those banking applications? One of us, right? What are we doing to prevent that from being vulnerable? It shouldn’t be an afterthought and in and out of the job of the InfoSec guys. Archana, here, who specializes in InfoSec, can tell you a lot more about the security practices, but that should start before.

Citlalli Solano Leonce: Looking at this SDLC, it’s something that’s probably very, very familiar to many of you. What do you think here is missing? Any ideas? We have the the planning, we have architecture and design, implementation, testing, deployment, maintaining, anything that is missing here?

Audience Member: Security.

Citlalli Solano Leonce: Security. Where do you think security should go? What circle are we missing? Where do you think that goes?

Audience Member: Everywhere.

Citlalli Solano Leonce: Everywhere? Yeah. Oh, you guys are too good for me. Yeah, spoiler alert. Yes, security is everywhere. It’s not, oh, QA should test for security, and Meghana can tell you a lot more about all our QA security practices. But this goes before, even as we are designing, Paddy here also will talk to you about product management, but it’s everybody’s responsibility.

Citlalli Solano Leonce: Circling back, we are living in this amazing world. We have all these services at our fingertips, right? Everybody’s now, now our kids, everybody is. But also we have a big responsibility. I personally love working here because I really identify with our mission of securing our digital way of life. I truly believe that. As the previous presenters were saying, it’s truly our responsibility and we are hoping for a better world one day after another. I’m hoping that tomorrow is going to be a little safer than today, so that the world that I leave to my kids and my legacy is much better than what I’m living right now. I invite you all to adopt security as your own, and let’s build that secure world together. Thank you very much.

SQA Engineering Manager Meghana Dwarakanath says, “We have to continuously rethink our role and what we need to do in our roles to be successful. This mindset is not only encouraged here at Palo Alto Networks, it is expected, and that is what I love the most about working here” at Palo Alto Networks Girl Geek Dinner.

Meghana Dwarakanath: Hello, everybody. My name is Meghana Dwarakanath. I’m the Software Quality Assurance Manager for public cloud security here at Palo Alto Networks. Now, I have been able to contribute across three different products here at Palo Alto Networks: WildFire, which is our malware protection as a service, Aperture, which is our data loss prevention as a service for SaaS applications, and now with public cloud security product RedLock. I’m sure you all know already all about it, with all the demos you’ve attended.

Meghana Dwarakanath: How did I get started? I like to tell people that I’ve worked my way up the networking stack. I started off on CDMA. Then IP, TCP, SUDP, finally landed in the cloud, and out of pure curiosity took a right turn into security. This is a story you’ll hear a lot more from people who are working here. Because, initially when you think security, or at least when I heard about security, the first thing that comes to your mind is some Mission Impossible scene. There’s lot of screens, hackers. But then working here I came to know it takes a lot of people from a lot of different backgrounds and expertise to come together and make a good security product. Now, if you take my team, for example, I have people from DevOps background. I have people from Dev background, of course QA background, security company experiences, non-security company experiences, and with all those different perspectives, we are able to build a much more secure and successful QA process, which is what I’m going to talk about today.

Meghana Dwarakanath: One of the axioms in security is, you’re only as strong as your weakest link. Now let me ask all of you something. What do you think is the weakest link in your companies? Maybe you don’t want to say it out loud. But the answer should be, nothing. We are all strong, we’re all doing good, and we agreed, right? Now, of course, when it comes to our production environments, we are very thoughtful about protecting them, and we should be. Because it has our customer data, it has our reputations, and it needs the protection. By the time we come to our QA environment, it kind of tapers a bit, right? Why? Because you’re thinking it’s QA.

Meghana Dwarakanath: We don’t have customer data in there, hopefully. It’s an afterthought, we really don’t think about it. But if you really think about the challenges we have and the kind of products we are testing today, we need to think about why we need to secure QA environments. Because when somebody gets to your QA environment, there are a lot more things they can get out of it, apart from customer data. For example, they can get an insight into your system internals. They can figure out how your systems and services are talking to each other and you’re literally helping them make a blueprint to attack your production environment. You have proprietary code, of course, that is running in your QA and staging environments, and so there’s a potential loss of intellectual property there.

Meghana Dwarakanath: Again, hopefully you don’t have customer data in your QA environments. I really hope you don’t, because here at Palo Alto Networks, the InfoSec team, Archana will tell you more, they’ll find out, come hunt you down, and take that data. Then, of course, if you’re the unfortunate victim of something like a bitcoin mining and that, you get a very massive bill at the end of the month, a very, very unpleasant surprise, right?

Meghana Dwarakanath: This is just your test environment. What is the other aspect of testing? Test automation, right? Anybody who is testing the SaaS service will tell you they test against production. Every time you release, you want to make sure that your production is doing okay. All the features are doing okay. So what do you do? You run your test automation against production, which means your test automation now has credentials that can access your production environment. You probably have privileged access because you want to see better what you’re testing, and now you’re co-located next to customer data, which is a very–potentially–a very unsafe mix.

Meghana Dwarakanath: How do you do the security? One of the ways we have been able to do this successfully here, is to consider test as yet another microservice that is running in your production. All those production microservices that you deploy, test is just another one of them. How do your microservices store credentials? That is exactly how you test automation will store credentials, the same SDLC process that Citlalli talked about, where security is not an afterthought. The same thing applies to your test automation code as well. You deploy monitoring for your test automation services just like you would do for your production services, and then whatever deployment automation you have, your IS automation code you have, you first test deployment into the same very architecture, and now you have all the added protections that your production microservices are getting.

Meghana Dwarakanath: There are a lot of fun new QA testing concepts, right? AB testing, blue-green testing. How do you test this global? With this, we are able to be in every single stack we deploy, test continuously, and get continuous feedback about our test and production environments.

Meghana Dwarakanath: Now we have the right people, we have the right mindset, we have the right intentions. We just want to ensure that our intentions have the right impact. What do we do? We just happen to have a set of world-class microservices at our disposal, so we don’t put our own environments, which means for example, all my test environment are monitored by RedLock, to see if we have any security vulnerabilities there so that I can immediately know about them and then I can make sure they’re secured, right? This is a win, win situation, of course, because we are in the same cycle of continuous feedback. We tell how the product is doing, the product is securing us.

Meghana Dwarakanath: Now, from this talk, I really want all of you to have two takeaways from this. One, of course, to really go and think about how your QA practices are, are they secure? And what needs to be done to make them secure. The second thing is to realize that we are in an ever changing landscape and there are different and new challenges, right? We have to continuously rethink our role and what we need to do in our roles to be successful. This mindset is not only encouraged here at Palo Alto Networks, it is expected, and that is what I love the most about working here. Thank you.

Principal Technical Risk Analyst Archana Muralidharan reiterates that “security is not a one-time concept. It’s a continuous process” at Palo Alto Networks Girl Geek Dinner.

Archana Muralidharan: Good evening, everyone. I am Archana Muralidharan, I work as Principal, Technical Risk Management, here at Palo Alto Networks, InfoSec department, and the same function of a lot of people refer to now. I feel more responsible now to deliver what exactly we do put into product security here.

Archana Muralidharan: Before we get into the specifics of how we do stuff at Palo Alto Networks, let me share with you some fun facts about me. I was born and raised in Chennai, a city in southern part of India. Where the weather is really hot all through the year, 365 days a year. There are only three seasons, according to us, hot, hotter and hottest.

Archana Muralidharan: There is there are no cold seasons that are known to us. If you ever see me wearing a jacket when it’s 70 degrees outside, you know why it is. My childhood dream was to actually become, any guesses? Was a Bollywood singer. Honestly, I still learn Indian classical music just for the fact that I couldn’t become one. But destiny was something else, I completed my engineering and I ended up becoming a software engineer.

Archana Muralidharan: It was by accident, I would say that I got into information security, because I didn’t even know what it was like 12, 13 years ago when I started my career in InfoSec. After having been in InfoSec for so long, I really, really love the domain. It is so interesting because it throws a unique set of challenges and problems for us to solve. Now that we heard a lot of our leaders, Citlalli, Meghana, touching upon how important is security to be incorporated as part of SDLC, let me dive deep into that.

Archana Muralidharan: There are some debates here and there in terms of the actual estimates, but all research does confirm the cost and time involved to remediate the vulnerability, grows exponentially over the different phases of SDLC. That’s why it’s really important for us to start thinking about security, during the initial phases.

Archana Muralidharan: Especially when you’re delivering the cybersecurity product, we want to be doubly sure, triply sure, over cautious sometimes to ensure the way we develop [inaudible] makes us really secure, because we have commitment to protect the digital way of life.

Archana Muralidharan: Our approach here at Palo Alto Networks is to embed security as part of every phase of [inaudible] like how you just heard from a lot of the speakers who spoke previous to me. As part of requirements, we make it a point that we collect security requirements as an NFR, meaning non-functional requirements, in addition to the normal performance ahead of requirements. We ensure that they are understood, well documented so that we could potentially prevent a lot of vulnerabilities creeping down street.

Archana Muralidharan: As part of Design phase, being from InfoSec, worked very closely with the product architects to understand the architecture and review it from a security perspective to ensure… to look for all possible attacks, incorporate possible mitigations well in time to prevent design flaws that would otherwise result in vulnerabilities in the product.

Archana Muralidharan: As part of Build phase, we primarily do two activities. The first one being static code analysis where we look for vulnerabilities and remediate in the custom code what we developed, as part of product development. The second piece being, using this open source vulnerability assessment tools to figure out the vulnerabilities in the open source libraries and frameworks worth the use in our product. But it’s really important that we understand what we sign up for.

Archana Muralidharan: During Testing phase, we do something called as application integration testing to find vulnerability that we had missed as part of Build phase. For instance, when we do static code analysis during Build phase, the code doesn’t run, [inaudible] it is static. When more components are integrated, come together, there could be a possibility of more vulnerabilities, which we typically find, specifically targeting areas, some of the stack like versus logic errors, privilege escalation, which no static analysis tool can find as of today.

Archana Muralidharan: As part of Deployment we perform deployment architecture review. This is very similar to what we do as part of design phase. The only reason is because we are in the [inaudible]. We follow [inaudible] frameworks, we build stock so fast. There’s always a chance that they may miss the actual design, what was approved versus the actual design one gets to plan, finally may differ and you want to be really sure that the final architecture, what gets deployed is indeed what was approved.

Archana Muralidharan: Finally, I’m very sure all of us would be aware and agree that security is not a one-time concept. It’s a continuous process. We monitor… We scan our product environments for vulnerabilities in infrastructure, web application, API, SQA configuration, so forth and so on, and remediate those vulnerabilities well in time.

Archana Muralidharan: Aside from that, there could be a situation where a vulnerability is out in the market, but it’s not part of your scan cycle, so we don’t want such vulnerabilities to be executed and end up being in a breach situation. We use runtime application, self protection to detect those vulnerabilities and lock it from getting executed during run time. These are all the activities that precisely we do as part of software development life cycle. We take security really very serious.

Archana Muralidharan: Before [inaudible], I want to share, why do I love working for Palo Alto Networks? Trying to give a basic… I would like to share my personal story when I interviewed with Palo Alto Networks. Having been in consulting for 10 plus years, the very fact, the very idea that I’m going to work for a cybersecurity product company, really thrilled me, really excited me.

Archana Muralidharan: I applied for a job, I went for an interview, everything went well, and always we have this feeling that we could have done probably a little better. Any of you think like that? After any interview? I felt the same. But all well, I get a call from the hiring manager, did a very appreciative of all the great qualifications, what I have and I was super excited. I thought I was [inaudible] the job, but then there’s a slight twist to it. I did not get the job. Difference, I was not a right fit for that job. Instead, they offered me a totally different job, which in their opinion, they believe that’ll be a better fit for me. As you all are confused now, I was completely lost and confused, because never in my career of 15 plus years, there was ever a situation where something like this happen. It is always either a yes, or a no.

Archana Muralidharan: With a lot of confusion, I agreed. I accepted the offer. Glad that I made the decision, no regrets whatsoever after that. It has been a great learning experience here. The reason why I’m sharing this with you today is to reemphasize that teams here at Palo Alto Networks think very differently to solve the problem statement. That’s what makes this place unique and a great place to work. Thank you so much for your time.

Senior Product Manager Paddy Narasimha Murthy talks about PMs and Security PMs, and how PMs work at Palo Alto Networks Girl Geek Dinner.

Paddy Narasimha Murthy: Thank you, Archana. Hi, everyone. Glad to be here with all of you. I’m Paddy Narasimha Murthy. I’m a product manager on the Cortex team, an engineer turned product manager. That’s a very brief introduction about me and I’m here to talk about the perspective of PM-ing at Palo Alto Networks. What does that mean? You heard from a development manager’s perspective, QA, and InfoSec perspective, and now this is the PM perspective. But before I go into the details, I want to go over… what does a PM do, and then what does a Security PM do, and finally, why is it fun working here at Palo Alto Networks?

Paddy Narasimha Murthy: Many of you might’ve seen this image. This is a classic image where you see different perspectives for the exact same element, and here the element is elephant. What a PM does is basically intuit what a customer wants. Let me go over that with an example. Let’s say Customer A comes to you and they say, “I have certain data set and I want only the senior management to actually have access to that data set.” Okay, great. As a PM, you make an order for it and you say, “Okay, this is probably how I’m going to go build that feature.” But in the meanwhile, Customer B comes to you and says, “I have a data set, but I only want my support engineers to access that data set,” and you go, “Okay, that’s also great and I can build a feature for that.”

Paddy Narasimha Murthy: Imagine if you are a PM and you were to build a feature that satisfies that Customer A, and another feature that actually satisfies Customer B. Do you think that’s going to be sustainable? Because it’s very soon you are going to run into a situation where there’s going to be dozens of customers and probably even hundreds of customers asking for something very similar saying, different people need to have access to it. That is where you as a PM come into a picture and where you actually help draw this elephant. In this particular case, you could solve this problem by building something called a role-based access control, for example, where you can actually have a common solution that would satisfy with Customer A, Customer B, as well as thousands of customers who could have the same need in the future.

Paddy Narasimha Murthy: Role-based access control system, just a brief introduction is basically setting up rules, which are privileges, and those privileges tell you what users have access to and what they don’t have access to, and you can assign these privileges to users. That is one way of solving this problem. This is what a PM [inaudible] does.

Paddy Narasimha Murthy: The other important aspect is that PM also helps understand teams the big picture. For example, different teams when they are building different features, what they see is just a tiny part of it and probably an isolated view of that feature because they might only be integrating with one more team or in some cases just couple of more teams. They’re a very isolated view of the work. So a PM’s job is to step in and actually help draw the elephant where you tell teams that, “Hey, this is what we’re building and here is how your piece is going to fit in.” That’s the job of a PM.

Paddy Narasimha Murthy: That said, what does a Security PM do? A Security PM does all of what I mentioned, and a little more. The first factor is security actually takes time. It’s not a one or zero or, okay, let’s do this feature or let’s not do this feature. There’s a cost to it. As a PM, what you do is typically you try to figure out how to build a secure product. If building a secure product, let’s say, you can only… ship five features in a product instead of 10 features, then so be it, because that would actually make your product a lot more secure because you’re able to spend more time on security related features.

Paddy Narasimha Murthy: Next is it’s actually an investment. By that, what I mean is it pays in the future. Let me explain that with an example. Many of you might’ve heard two factor authentication. It is basically you put in your password as well as another form… another factor for authenticating yourself. We all know 2FA is important and many online services and companies and so many others online accounts offered 2FA. But how many of you go turn it on, or how many companies even go turn it on. Even though this is a new feature and explain it to the customers as to why this is important, it actually go ahead and build this feature and explain it to the customer as to why this is important, because this is going to prevent them from the ever evolving threat landscape, and pushing this feature out in the next 10 years or so is not really going to benefit our customers. That’s what you may have trade off.

Paddy Narasimha Murthy: Next is, it is ongoing. Let’s say as a PM you decide that your product is reasonably secure, so you put the secure stamp on it, you ship it out into the world, and that’s it, your job is done. No, it’s not. Because security is constantly ongoing and you have to evaluate, is your product continuing to secure the customers? Is there more to it? If I were to extend the 2FA example, what I would be doing for the ongoing aspect of it is to now figure out, maybe I should be offering multi-factor authentication and not just two-factor authentication, because that is how I am going to protect my customers from the ever evolving threat.

Paddy Narasimha Murthy: Moving onto why PM in Palo Alto Networks. Security up and down the stack. Because Palo Alto Networks, we have a wide suite of products that our different speakers alluded to earlier. We have firewalls, so ranging from the physical devices up to the cloud, we have a whole suite of security products. If you were to join as a PM or in any role, you would actually get to work across the stack of products.

Paddy Narasimha Murthy: Product-oriented engineering is another factor. Where we don’t just stack up products because some customer came and told us, “Hey this feature is cool and I would like to see this feature in my product.” That’s not how we go about it. Everything that we build here at Palo Alto Networks starts with a problem statement. PM sits down and write a very cohesive problem statement. We start our process on there, and with that problem statement would be good, we actually sit together with the PM teams and we go over that problem statement and we convince ourselves, is this the right thing to do? Is this the right problem that we would need to solve? Is this the right thing for our customers. We attack the problem from different perspectives to make sure that we’re actually going after a problem that really needs to be solved. That is another factor that I really like.

Paddy Narasimha Murthy: The last one here, everyone here wants to do the right thing. Palo Alto Networks is a pretty large organization. We have several different teams. If you think about it, different teams have their own ways of doing things. We have different priorities, and they have different incentives too. But in a lot of cases when we have to work together, there is going to be conflict. But it’s really easy to work together because everyone in the room really wants to do the right thing. That is the biggest reason why I really enjoy working here. With that, I’m done, so are all the speakers. Really want to thank every one of you for coming here and spending your evening with us. Thank you all, we’ll be hanging out here, so happy to answer any questions you have. Thank you once again.

Pictured: Citlalli Solano (Director of Engineering) at Palo Alto Networks Girl Geek Dinner 2019.

---

Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at Palo Alto Networks and check out open jobs at our trusted partner companies.
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at SurveyMonkey and check out open jobs at our trusted partner companies.
• Read about the inspiring event on the SurveyMonkey Design blog!
• Find more SurveyMonkey Girl Geek Dinner photos – please tag yourselves!
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

COO Gretchen DeKnikker of Girl Geek X introduces CTO Robin Ducot at SurveyMonkey Girl Geek Dinner in San Mateo, California.

Speakers:
Robin Ducot / CTO / SurveyMonkey
Sarah Cho / Director of Research / SurveyMonkey
Sarah Goldschmidt / Product Design Manager / SurveyMonkey
Mala Neti / Software Engineer / SurveyMonkey
Shilpa Apte / Engineering Manager / SurveyMonkey
Jing Huang / Director of Engineering, Machine Learning / SurveyMonkey
Erica Weiss Tjader / VP, Product Design / SurveyMonkey
Gretchen DeKnikker / COO / Girl Geek X

Transcript of SurveyMonkey Girl Geek Dinner – Lightning Talks & Panel:

Gretchen DeKnikker: Hi everyone. I’m Gretchen. I’m with Girl Geek X. Thank you so much for coming tonight. Welcome to SurveyMonkey. How awesome is this space?

Gretchen DeKnikker: It’s so good. I love it. I love it. Also, I’m going to introduce a unicorn in one second after I do my little spiel.

Robin Ducot: Spiel?

Gretchen DeKnikker: Spiel.

Robin Ducot: Spiel.

Gretchen DeKnikker: How many first Girl Geek dinner?

Gretchen DeKnikker: A lot. Welcome. We do these every week. We’ve been doing them for ten years. We’ve done like 250 of them so far. We also record them. They’re on YouTube. We do a podcast. It’s in the podcast places, so definitely check those out. Something we were talking about a little earlier, I was thinking the reason that we do these is to give women the opportunity to get up on stage and talk about being awesome rather than talking about being women. One thing I’ll challenge you guys with is find yourself a woke male and bring him next time, because there’s nothing gendered about this content, and building ally-ship is a really good thing.

Gretchen DeKnikker: Without further ado, I know I’m going to get comments on this, but I can’t wait to hear what the feedback is. Send it my way. I’m ready.

Gretchen DeKnikker: Now–

Robin Ducot: Hello.

Gretchen DeKnikker: Hi. I don’t know if you’re ever met a female CTO live and in person, but you’re about to. She’s amazing. In the like three minutes, we’ve just decided we were separated at birth and are going to be best friends now. Right?

Robin Ducot: We’re going to get caught saying something inappropriate at least once. We know this.

Gretchen DeKnikker: Yeah. We were really happy our mics were muted is basically what happened. Without further ado, please welcome the CTO of SurveyMonkey, Robin.

Robin Ducot: Hi. Nice to see everybody here. I’m so glad y’all could come. I’m Robin Ducot. I’m the CTO of SurveyMonkey, and we’ve got a great set of talks tonight. I guess we’re just, without further ado, we’re going to get started. Please, at the end we’re going to be taking questions right after the panel. We hope that you guys will have some interesting questions for us because we are excited to talk to you then and afterwards.

Robin Ducot: Why don’t we get started? Yeah? You ready? Why don’t I bring on stage Sarah. She is from our survey research team, and she is going to talk about surveys.

Director of Research Sarah Cho gives a talk on “Surveys: Why Do They Matter?” at SurveyMonkey Girl Geek Dinner.

Sarah Cho: Hello. Thanks. Hi. I’m Sarah Cho. I’m Director of Research here at SurveyMonkey. You’re probably wondering what a director of research at SurveyMonkey actually does, but what we do is we consult with all the different areas of the business from product to engineering to marketing to sales to give them advice on what is actually good survey best practices. You can think of our team as basically being the biggest survey nerds, or I guess for this crowd I should call them survey geeks, at this company. Of course, we can’t invite you to SurveyMonkey and not talk about surveys, so that’s what I’m going to be talking to you about today is first, let’s level set a little bit about what I mean about at survey. There’s a lot of different things that the word survey can mean. While this photo looks amazing and awesome, like I bet a lot of us would like to be up high on this mountain top, and we’re not talking about land surveying here. We’re really talking about survey in questionnaire form.

Sarah Cho: What I like to kind of describe surveys as is you’re having a conversation with a very specific purpose at massive scale. First, I’m going to walk through a couple of different ways that you can do surveys, talk about the different types of surveys that you can conduct, and then hopefully you’ve been inspired that you can go home and do some surveys on your own. I’m going to give you a couple of tips on how to make sure that your surveys are going to yield the highest quality data because it’s a little bit of art and it’s a little bit of science.

Sarah Cho: Surveys. They come in many, many different shapes and sizes. Has everyone taken a survey before? Yes. It could be on the phone. These are like the surveys that call you when you’re sitting down at dinner and you’re right about to take your first bite of food and they’re like, “Hey, will you take a survey?” It could come in person. Maybe it’s someone who’s knocking on your door or maybe you’re at the doctor’s office and they’re asking you a little bit about your medical history or increasingly, obviously what we deal with is surveys that are online. The great thing about online surveys is that they can be provided in a variety of different avenues. One of the more interesting things is surveying kind of where people are working, so utilizing Slack if you guys utilize Slack, and surveying people in Slack or maybe surveying people in Facebook Messenger. Really going to where the people are rather than trying to bother them in the middle of dinner or maybe in their face knocking on their door.

Sarah Cho: The beauty of surveys is that they can really be on any kind of topic. The most famous survey is probably the US Census, or the decennial census. Actually next year in 2020 is going to be the next census, so for everyone regardless of whether you are a citizen or not, that is kind of one of those myths they tell you about the census. Everyone needs to respond. If you don’t respond, technically you could go to jail, although I wouldn’t worry too much about it because they actually haven’t prosecuted anyone since 1970. It’s very highly unlikely, but the census is a really important survey. Things like congressional seats, which eventually turn into who is going to be in the White House, which we’ll leave politics out of it, but anyways, obviously very important decisions are made from the census.

Sarah Cho: The census is a really big government–Surveys aren’t necessarily meant for just only for big government or big business. You really can use them in many, many different contexts. If you know me in my social life, I tend to spam you with surveys a lot, all for various things like organizing camping trips. For this screenshot, which is a little bit blurry, I’m trying to organize friend’s birthday dinner at Beretta in San Francisco if any of you guys have been and just trying to figure out what to offer on the prix fixe menu. Very practical. Very helpful.

Sarah Cho: Another good way that people have utilized surveys in sort of a personal context that I haven’t really shown you here in this deck, but one of my co-workers have actually utilized surveys to ask people about tell me three words about myself on what you think about me. There was a lot of words that came up, things like thoughtful, really considerate, but what was actually more revealing in terms of thinking about her own professional development and growth is the words that weren’t represented. You can also utilize surveys for your own professional growth and development.

Sarah Cho: I talked a little bit about the government context, but another interesting government context is actually how you can utilize surveys to form opinions, sorry, to gauge opinions and make changes, whether it be in the government context with this where they were asking people who were serving in the military about what they thought about serving with a gay or lesbian service member. What they overwhelmingly said was that actually we think that regardless of someone’s sexual orientation, they can effectively do their job. That eventually was evidence that congress used to repeal Don’t Ask Don’t Tell. You see this in the workplace a lot. A lot of organizations are asking employees about what they feel about their workplace. Employers are actually taking that information and making meaningful changes.

Sarah Cho: We do that, for example, at SurveyMonkey. Obviously we like to eat our own dog food or drink our own champagne, whatever the better phrase is for you guys. For example, we did a survey about do people feel like they are included within our SurveyMonkey community? Do they feel like they belong? Actually, because of that, we found that a lot of people didn’t feel like there was a good path to growth within the company, so we made a lot of different changes like having a career ladder, changing from a yearly review cycle to a quarterly what we call growth impact and goals cycle. We’re able to make a lot of these changes and a lot of organizations are doing similar things.

Sarah Cho: Then finally, a lot of people use them for market research. I like to use this example because it’s a silly example. Maybe they should have used some market research here. This is a product called a UroClub. In case you’re ever on the golf course–I don’t golf myself so maybe it’s a bad example, but in case you’re ever on the golf course and you need to go to a restroom and there is none available, you can utilize this golf club to relieve yourself. Clearly they didn’t use any market research for that, so obviously it’s good to use surveys to make sure that you are doing well in that area.

Sarah Cho: Now going onto a few tips about how to create better surveys for you. The first thing is questionnaire design. You can see here, if I were to read this entire question, that would take up the rest of my 10 minute lightning talk, so I’m not going to, but you can see already visually without even reading any of the words, it’s a bad question. There’s a lot of words, TLDR. Remember when you’re writing survey questions, surveys are super visual. If you have a huge block of text, whether it be in the questions or a lot of answer options, people don’t like to read through that, not because they’re being lazy, but you really should be designing with your respondents in mind.

Sarah Cho: The next thing that I like to point out is jargon or industry terms. A lot of times we are so specialized in our industry, so in the survey industry there is this thing called acquiescence bias. Has anyone heard of it? No.

Sarah Cho: If I asked you guys about acquiescence bias, one person maybe, yeah. Most people would be like, huh, what. That’s the whole point. You shouldn’t use terms that only people in your industry are aware of, or the worst is acronyms. Here it’s how would you rate our POS system? Has anyone worked in retail or restaurants? You probably know what a POS system stands for. It stands for point of sale. If you don’t work in restaurant or retail, POS might mean something very different, like are they asking me about the toilet? Instead, you should be very clear. You really want to know how was the checkout process? Instead of using any acronyms or jargon, really stick to something that’s really clear that can be understood by everyone. A good rule of thumb is things are at an eighth grade reading level. That’s generally a good rule of thumb for surveys.

Sarah Cho: You can spend a lot of time on your questions and a lot of time looking at your answers, but then, sorry, you already got the joke. Sometimes you also need to pay attention to your response options, not just the questions themselves. This is a good example. It’s probably someone who maybe was asking about ethnicity and accidentally included Chinese in sexual orientation, but clearly that’s not right. This is also a key for you guys just to remember if you are doing surveys, always have someone else preview your survey for you because it’s like writing a term paper when you’re in college. You don’t realize there’s a typo in the first sentence of your paper that’s like 20 pages long and you’ve read it 15 times. A lot of the times, if someone else takes a look at your survey, they’ll be able to point out things like this that you may have missed.

Sarah Cho: You could write the perfect survey with the perfect response options and then kind of bungle the data analysis. In this particular example, does anyone know what might be wrong here?

Sarah Cho: Yes. It adds up to more than 100. It adds up to 120 and the last time I checked, a closed ended question like this typically adds up to 100. Just make sure that you’re double checking what we like to call number checking or fact checking. That is a crucial step. Another, again, I’m sorry I’m picking on Fox News, but their graphics department is pretty horrible. You can see here that they’re showing a difference between, it looks like a really big difference, right, like between now and January 2013. It looks really big, right, but actually it’s only 4.6 percentage points. It’s 35 to 39.6. That is actually where someone is playing with the axis and making a difference much more magnified than it actually is. This is a good example of not only things you should avoid, but other things you can kind of see when you’re interpreting other people’s data to see if they’ve kind of messed around with the interpretation of it.

Sarah Cho: Just to wrap up because I only have around a minute left, or actually I’m over a minute, sorry. I don’t know how to use this timer. Here are a couple use cases that you can think about. We’ve talked a little bit about them, but remember you can always survey your customers. Even if you don’t have customers, say you’re a teacher, you have students. Survey your students. You can always gain product feedback. Say you are thinking of starting your own company, but you’re probably going to get very biased opinions if you ask your friends and family member. They’re either going to say that’s a really awesome idea, because they don’t want to offend you, or they’re going to say maybe you should stay away from that because they think you shouldn’t take that risk. Always ask for feedback from people you don’t know.

Sarah Cho: We talked a little bit about employee engagement, but there’s a lot of different ways to make sure your team is happy. We talked about inclusion, but there’s always from the minute they step in your door as a candidate to the minute they leave as someone who is no longer an employee of your campus. You can always, we like to say we want to empower the curious, so satisfy curiosities whether it’s thinking about what markets you can potentially expand to or maybe if you want to go back into academia or just put your academic hat on and just think a little esoterically about one specific problem, you can always think about what is the survey component to that.

Sarah Cho: Finally, I talked a little bit about the example about utilizing it for personal and professional growth, but just to wrap up, this is just saying that even if you don’t want to create your own survey, you can still help us survey nerds out there by being a respondent. Make sure if someone sends you a survey request, just take a few minutes out of your time, respond to that survey because that’s really going to help the people out who are sending it out. I’ll be here around in the networking hour and for Q&A. Thanks for listening.

Sarah Cho: I’m also going to now turn it over to Sarah on our product design team. We’ll keep it easy with the same names.

Sarah Goldschmidt: We good? Hello? Hi everyone. Good evening. Thanks for coming. I have a slide, I promise.

Sarah Goldschmidt: It’s not? I’m holding the clicker. Duh. Anyway, user experience. Thanks all.

Product Design Manager Sarah Goldschmidt gives a talk on “Human Readable: Designing Data for Carbon-Based Lifeforms” at SurveyMonkey Girl Geek Dinner.

Sarah Goldschmidt: I’m a product design manager here at SurveyMonkey, and I’m here today to talk about designing data for carbon-based lifeforms. What does that mean? I know, it’s funny. You’re laughing. It’s designed to laugh. That’s great. We’re going to talk about designing for carbon-based lifeforms, which is us, people.

Sarah Goldschmidt: I want you to take a moment and just think about in 2019 the way we think about data has really evolved from the output of a product to kind of the product itself. I want you to hold that in your mind for the next ten minutes because it’s really the crux of the conversation that we’re going to have today. We as makers of products, whether we’re designers or engineers or product managers or marketers, all play a role in defining the relationship between the product that we make and the data that it collects, creates and disseminates into the world. What I’d like you to walk away with is kind of a call to be more curious and more creative and more expansive in your thinking about how we define this relationship.

Sarah Goldschmidt: This is a chart. Everybody has probably seen a chart like this at some time in their life. You’ve probably made one. You maybe made one in elementary school. Charts and data visualizations are a powerful way to make data, particularly numeric data, visible to the human eye. They can also be a crutch. We can over index on them. I say this as a designer who has probably been guilty of this. We receive data that we’re told is important for a product. Users must use this somehow. We spend a lot of time designing the container for that data. We visualize it. We make really beautiful lines and then we kind of leave it there. We don’t ask more questions. We don’t curate the palate of data that we might be working with.

Sarah Goldschmidt: Charts are great for making data visible, but not human readable. That requires meaning. What’s this idea of human readability? I will confess, I made it up. I will define it for you. Data becomes human readable when through the power of design we wrap that data in meaning. Meaning is how we get from 1, 2, 3, to aha. That aha being how people understand how to feel about the data, how they react to the data, and how they can act with it.

Sarah Goldschmidt: I want to talk a little bit about two products here that are really wonderful examples of thinking more expansively about data as a product and about making that human readable. One of my favorite examples is Spotify. Spotify is not typically what you think of when you think of a data company, but Spotify is sitting on mountains of data, millions of users and all the data associated with how they listen to music. They don’t surface this in the app. You probably don’t want to know how many times you’ve replayed the Justin Timberlake song over and over and over. It probably wouldn’t be good for their business model, but somebody at Spotify decided to sift through that data and thought maybe it could become a product itself. Thus, the Wrapped report was born.

Sarah Goldschmidt: I don’t know how many people here get a Wrapped report. I get a Wrapped report. You get a Wrapped report? Awesome. For those of you who haven’t gotten one or don’t know about it, what happens is at the end of the year, in this case, 2018, Spotify is going to send you out this beautifully designed, curated mini-site that essentially tells you the story of your year through the way you interacted with music. There’s really lovely insights. Sometimes it’s funny. Sometimes it’s cringe worthy, but it’s really designed to share. Of course, this exploded into a phenomenon. We’re posting it all over buildings. We’re talking about it on the news, and of course, we’re sharing it with our friends. One of the remarkable things about this is that it wasn’t successful because the data was solving a really huge problem or being used in some kind of scientific theory. It was making people laugh. It was making people reflect, and it was helping them connect with each other. That’s human readable data turned into a product.

Sarah Goldschmidt: This is one of my favorite examples. They went further, right? They were farther and somebody sifted through all that data that Spotify has to come up with these advertisements that went a step further and told the story of the cultural year of 2018. Some of you might have had the great pleasure of seeing shark do-do-do-do-do, a viral kids video that’s actually quite annoying. This is just one example of this advertising campaign that they’ve rolled out. I encourage you to look it up. They are really funny, but again, taking what is actually just a pile of data somewhere in a database and telling a story with it, making it meaningful and having it touch people in various ways. Making them laugh.

Sarah Goldschmidt: Another example that might be more relevant to us in the room, those current product managers, designers, and engineers or aspiring ones, this is SurveyMonkey Engage. Engage is a stand alone product that we make here at SurveyMonkey. It’s designed to help employers connect with their employees to better understand what’s going on and improve their workplace culture. The data that Engage collects and disseminates is almost completely numerical, so we’re stuck in that position of saying we’re going to have a lot of charts because we have to display that data. We have to display the relationships between that data, allow people to filter it, et cetera, et cetera. What you also see in this interface are a lot of words. That was the secret to making this data more human readable.

Sarah Goldschmidt: Where we got to this is we decided people are changing human culture. What’s important to them? It’s probably not sitting around and looking at a bunch of numbers. Our users want to spend more time connecting with their employees and creating change, and numbers don’t create culture change. People do. Conversations do. Relationships do. We thought how do we apply that to numeric data? And the secret was in what we call the core factors of engagement. We did a bunch of research, looked at all the data points we were collecting, and we saw that they grouped into conceptual chunks. What that allowed us to do is instead of giving a bunch of data points related to how people interact in your workplace assigned to a number of questions in a table, we wrapped it in words. That’s team dynamics. Then we named a bunch of areas of engagement, purpose alignment, visible future. We created language. The focus for our users, instead of nitpicking at the data itself, which is available when you want it, you have shared language with your employees to be able to talk about what’s going on. The data, the numeric data displays that kind of supporting role to be able to talk about direction at a later date.

Sarah Goldschmidt: Something that these two really have in common, you’re probably hearing me say the word story. It’s not just meaning that we need to make data human readable, it’s story. When you string meaning together, you really get a story. That’s what a story is. Stories have been used for thousands of years for humans to be able to understand their world, their place in it, and how they can act. It provides that extra oomph. When design transforms data into story, that’s when the magic happens. The magic is that connection and action.

Sarah Goldschmidt: What I think is really important for the take away today is that anybody can make the magic happen. It’s all about starting a conversation and providing perspective. I’m going to give you four steps that anybody in this room can take to start the conversation in your organization or your school or your product company to get data from being kind of just a set of numbers in a table, maybe, to human readable data.

Sarah Goldschmidt: Step one, find what’s important to your people. I use the word people very specifically here, not users. Users are users when they happen to be using your product, but they’re always people. Ask yourself what’s important to them when they’re using the product. Sure, it might be in a business context, but what’s important to them as a human being? Maybe they’re particularly concerned about what their boss thinks of them. That’s nerves. Maybe they’re stressed out. Maybe they want to buy a really cool shirt and your data is going to help them do that. I don’t know, but that’s the questions. When you find out what’s important to your people, it tells you what kind of meaning you need to wrap your data in.

Sarah Goldschmidt: Step two, prepare your palette. Your data palette. As a painter, I want to be able to paint with as many colors as I possibly can to open up more and more types of stories that I can tell. Don’t cut yourself off early by assuming you know what kind of data you need to display and what you can do with it. Take it all, but remember to do it ethically and responsibly.

Sarah Goldschmidt: Step three. Design a story, not a chart. Those charts are going to come in and really help you tell a story visually, but think about the story first. What kind of meaning do you want to string together? What do you want somebody to be able to do, and then build your charts after that to help support the story.

Sarah Goldschmidt: Step three. Validate the aha. All I know for designing with data for many years is that every time I think I’ve really got it down and I’ve got a universal story that everyone is going to understand, it’s absolutely not true. Users find out a new thing to do with it. I get to learn from that, so always validate your aha and watch your story evolve. It’s a participatory sport.

Sarah Goldschmidt: With those four steps, you’ve really got what it takes to start working with human readable data like a champ. I’m going to encourage you today to be curious, have fun, and make something meaningful. Thanks so much for having me today and for listening. I will be around for questions if you want to talk about product design or data or this weird concept of human readable data. Thank you so much.

Sarah Goldschmidt: I’d like to call my colleague Mala up here to talk about engineering.

Software Engineer Mala Neti gives a talk on “Transforming SurveyMonkey’s Front-end Platform with GraphQL” at SurveyMonkey Girl Geek Dinner.

Mala Neti: Hi everyone. I am Mala. I’m a software engineer here at SurveyMonkey. Today I’m very excited to share with all of you our journey of transforming SurveyMonkey’s front end with GraphQL. Let’s start by talking about 2019. 2019 was a pretty fun year for us here at SurveyMonkey in part because we were able to completely reimagine our front end architecture. What did this mean for us?

Mala Neti: To start, we were able to work towards consolidating our numerous individually deployable and siloed micro webs into just a handful of apps built on React and Node.js. We were able to introduce a very slim aggregation layer with GraphQL that sits between our front end and our back end collection of rest APIs where all of our business logic and application logic are hosted.

Mala Neti: Today I wanted to focus on the aggregation layer portion using GraphQL, but before I do that, I would love to get a gauge from the audience. How many of you have heard of GraphQL?

Mala Neti: Nice. How many of you have worked with GraphQL in any capacity? Not bad. More than I was expecting.

Mala Neti: For those of you who haven’t, GraphQL is simply a query language for APIs. You can think of it as making it easier to build out your front end by providing a declarative way of fetching data without having to have knowledge of the entire system. In that way it kind of separates the back end–innovation of the back end, from the front end and kind of makes them completely independent. In addition, graph APIs are organized in terms of types and fields rather than endpoints, so the client can get as many resources as possible in one single request to one single endpoint or as I like to call it one endpoint to rule them all. I did not come up with this even though I think it’s pretty genius.

Mala Neti: I think it’s easy to see why GraphQL or the performance benefits that probably come from using GraphQL with this declarative and efficient type of data fetching. Spoiler alert, it worked. For the pages that we have today powered with GraphQL, we saw a huge reduction of payload and a huge reduction of data sent to the client, and a huge improvement in our time to first interaction. This is a really big win for us and for all of our users of SurveyMonkey product.

Mala Neti: Now that we know that it works, let’s talk a little bit about how it works. We have our GraphQL client and our GraphQL server. At the center of our GraphQL server is our schema. Our schema is basically just the contract between our client and our server telling the client how it can access the data. It does this via API operations and data types. The API operations can be read and write operations like queries and mutations, which are basically sort of top level entry points into the graph, and data types like [inaudible], scalers, objects, and the list goes on.

Mala Neti: Facades are then R functions, sorry, resolvers are then R functions that map our API operations to our back end services code. Once we have our schema and our GraphQL server sort of built out, we can easily then build out our GraphQL client knowing what data is available to us and how it’s available to us. As we talk about building out our GraphQL client, it’s also very important to talk about the amazing tooling ecosystem that GraphQL provides us. At SurveyMonkey, we use a GraphQL playground which is basically an in browser IDE and it allows me to hit my GraphQL server and build my queries directly in here in this tool and see my responses. If I didn’t have any sort of involvement in actually designing my schema, I have all of my schema details documented right here. I have my queries and mutations that are available to me, my type details, what fields are nullable and non-nullable and so much more.

Mala Neti: Not only do we have an amazing performance benefit that we talked about earlier from GraphQL, but we also have an increased developer discoverability and productivity, especially from a front end perspective, which is pretty exciting. Now that we have a little bit of context as to what the GraphQL server entails and some of the tools that we can use to build on top of it, let’s dig a little bit deeper into building out our GraphQL client.

Mala Neti: Say I was building out this My Surveys page and I wanted to fetch data that told me what surveys to render. Looking at my tooling and my schema code, I know sort of that I have some options and what’s available to me. I can start by building out my query. This is just a simple query. I have named my query for the purposes of debugging and discoverability. As you can see, I pass in an argument which is language ID, which we have set to be a non-nullable integer, and I pass that in as variables as well as in pagination properties to my survey categories nested field. Under the hood, this basically dictates how my data is going to be resolved and what data I get back.

Mala Neti: Then, as you can see, all I asked for, the only subfield I asked for on items is ID and name. Nothing more and nothing less. I’m truly just asking for all I need. As you can see from my response, the structure of this response mimics the structure of my query. This is really great because I ask for something and I get predictable results back. One of the things to keep in mind as you’re building out your GraphQL client is that at the end of the day, GraphQL can be driven by the data requirements of your products. The people who are building, or the developers who are building your UI can have a little bit of that responsibility as to how they get the data that is building their UI.

Mala Neti: Now that I have my query, how do I actually, or I’ve built out my query, how do I actually incorporate it into my front end? How do I incorporate it into my React application? At SurveyMonkey we use the Apollo client platform, which is basically just a GraphQL implementation. It exposes me to a lot of cool things, one of which is my query component. I can basically pass in the query invariables that I wrote and built earlier as props to my query. I can also pass in this render prop function, which as you can see, exposes me to my various different query states, whether that’s loading, error, and so on. Based on my query states, I then can intelligently render my UI, which is pretty awesome.

Mala Neti: One sort of best practice that we like to follow at SurveyMonkey when we’re building our components and their respective queries is modularity, the idea of modularity and reusability. GraphQL works very well with that because it allows me to co-locate my data with my components. The data requirements of my components can live right beside it. That’s important because if I needed to add fields, delete fields, delete a component, really make any sort of change, GraphQL, doing it this way allows me to do that. I don’t have to worry about going up the component tree and having a query that basically powers multiple different components. I can just deal with the component at hand and the data that belongs with it.

Mala Neti: Another sort of application of this concept is component boundaries. If I have a network error or error fetching my GraphQL data, rather than having my entire page fail or multiple components fail that are dependent on a particular query, all I have to worry about is gracefully degrading my components that are dependent on that query, which is great because this is a much better user experience.

Mala Neti: All of this is awesome, but this means that by default every component will have to make their own network request, and we did just talk about how one of the selling points of GraphQL is minimal round trips. Because of this, GraphQL has actually come up with various ways to deal with this problem. What we are currently using today is query batching. That essentially allows all of my queries to be combined into one request. This has advantages and disadvantages. There’s alternatives to this, but we found as of now this works for us because it allows us to think about modularity while still reducing the number of requests and kind of gaining that performance benefits from GraphQL.

Mala Neti: From this example, I hope that it’s easy to see why we at SurveyMonkey love GraphQL. I think, A, our developers are happy because we have exposure to an amazing tooling ecosystem. We have a predictable developer experience. Our back end engineers are also happy because we have, they can sort of iterate on their code independently of sort of the new requirements that are coming in on the front end. My customers are happy because I have a performant robust product. If my developers are happy and my customers are happy, we have a really happy SurveyMonkey.

Mala Neti: With that, I wanted to say thank you so much. We’re really excited about what we’re working on here at SurveyMonkey. If any of this sounds exciting to you, please come hang out with me after and I’d love to chat. Thank you.

Robin Ducot: Now we’re going to have a little chat with some of our leaders here at SurveyMonkey about career topics. I’d like to welcome to the stage Mala. Not Mala. Mala just left. Shilpa, Jing, and Erica. Thanks, guys.

SurveyMonkey girl geeks: Robin Ducot, Shipla Apte, Jing Huang and Erica Weiss Tjader share advice and stories at SurveyMonkey Girl Geek Dinner.

Robin Ducot: I have a few questions, but let’s get started with introductions. I’m the CTO here at SurveyMonkey. I have been, let’s see, 30 years doing technology and about 15 of those years, last 15 years as an executive. I’ve managed everything from product, program, engineering, QA, ops, I don’t know, security, IT, all different types of areas from companies as big as Adobe to companies that you’ve never heard of that are gone now. I love engineering, and I’ve been here at SurveyMonkey 18 months. It’s just been a really exciting journey. Fun fact. Fun fact, I used to play in punk rock bands. People will tell you this and laugh. Also, 25. This number is the metric I use to measure the number of engineering leaders I’ve created in Silicon Valley. I love coaching and mentoring leadership skills for technologists and helping people grow into leaders of technologists is one of my passions, so 25 in Silicon Valley.

Robin Ducot: With that, why don’t we get started with some introductions of the other folks. Shilpa?

Shilpa Apte: Sure. Hello? Can you hear me?

Robin Ducot: Yeah.

Shilpa Apte: Sorry. Hi. I’m Shilpa. I’m an engineering manager here at SurveyMonkey for the respondent experience team. We’re the team that manages and innovates on the survey taking page. It’s the page you see when you take a survey. I’ve been at SurveyMonkey since I was an intern in 2012, so just about seven years. Let’s see. Highlight of my career has been getting to move to Dublin for two years to set up the engineering team there. That’s Dublin, Ireland, not Dublin, East Bay. My fun fact is that while I lived abroad, I traveled to–I did 17 weekend trips to countries I’d never been to before.

Jing Huang: Hi everyone. My name is Jing Huang. I’m a director of engineering focused on [machine learning here at SurveyMonkey. I joined SurveyMonkey for three years. Fun fact, I did a hiking trip to Everest base camp with a 14 day hike. If you’re ever hiking in a high altitude you’ll know like if you get a cold, it’s devastating. I got a cold in day five, but luckily I’m here, so I survived. I was able to complete the hike. The highest point I was able to reach was more than 18,000 feet, which is something I get to blab a lot. Some highlights of my career, I studied robotics and artificial intelligence. After graduation, I didn’t find a job in ML that was like more than a decade ago within a prime time for ML, but I was able to work on different applications from network security appliances to cloud management to big data. Finally I got an opportunity to work on ML here in SurveyMonkey, so that’s where I am.

Erica Weiss Tjader: Hello. Good evening. I’m Erica Tjader and I’m the VP of Product Design.

Erica Weiss Tjader: We had a joke in rehearsal that I wasn’t even going to need a mic I’m so loud, and they turn mine off. That’s a funny joke.

Erica Weiss Tjader: I’m Erica Tjader. I’m the VP of Product Design here at SurveyMonkey. I have been here two years. The baby is due August 1st. I thought I’d just get that out of the way so we can be done with the awkward like is she, is she not. Anyway, highlight of my career was landing this job at SurveyMonkey. I know that sounds really cliché. I promise they’re not paying me to say that. Tom, my boss, isn’t here, but I think the reason why it stands out as such a highlight to me is not just because of the great role that it is, but actually what it represents in terms of me doing some things that were really out of character for my personality. One was risk taking. When I first kind of, well I don’t know if I applied for the role, but when I first pursued the role, it was a really big step up for me in my career. I definitely did not meet all of the requirements in the job description, so it was a big risk to kind of throw my hat in for it. I think that was one big piece.

Erica Weiss Tjader: The second was intentionality. When I was starting my job search, I made a list of two, just two, criteria that I absolutely must have in my next role. I was really specific about looking for that and not settling for a role that had less. Then I think the third is patience, not my strong suit. Over the course of a couple of years, the first time I passed on SurveyMonkey, the second time they passed on me, the third time was a charm. Just the highlight of my career just I think because of how I’ve reflected on what it says about me.

Robin Ducot: Thanks, Erica. All right. Let’s start with some questions. How about we talk about your current work life. One of the things I think is really, really important is that for you to be successful, if you don’t, aren’t really engaged in what you’re doing, if you don’t really like it, it’s really, really hard to be successful because you’re just not going to be excited about putting the effort in that it really takes to make a success of yourself. What I’m curious about is what is one thing you do in your job function that gets you really, really excited to come to work? Why don’t we start with you, Shilpa.

Shilpa Apte: I love coaching and mentoring people. I love getting to know people and figuring out what their aspirations are, what skills they want to learn, where they want to take their careers and kind of seeing that progress being made over time. I see every single day as a new opportunity to learn something new about someone on my team mainly because people are complex and they change. I just think it’s an interesting problem.

Robin Ducot: Jing, what about you?

Jing Huang: I mentioned I studied robotics and I was a SciFi fan ever since I was little. I was always curious about how AI will shape our future. Getting to work on machine learning here is just dream come true. We have more than 40 billion people [inaudible] data collector here at SurveyMonkey, so this is really a dream job.

Robin Ducot: Erica?

Erica Weiss Tjader: For me, I think one of the things that I really enjoy about this role is the strength it plays to. Anybody familiar with the StandOut strengths assessment? It’s a survey template you can use in our library, but anyway, they also didn’t pay me to say that. Imagine that. I’m a connector. That’s one of the strengths that always stands out for me. I think it’s something that I get to do a lot of in my role. Whether it’s the day-to-day of connecting designers on my team with new challenges or mentors or growth opportunities, whether it’s working with teams to find ways to connect teams with other teams that have similar products or project challenges, and of course even in the nature of our product itself which is about connecting our customers with their customers for feedback. I think it’s just a role that I really get to leverage the strength a lot and that’s really energizing for me.

Robin Ducot: Thanks, Erica. Why don’t we talk a little bit about your career journey. One of the things that’s really, really important is being able to advocate for yourself. How have you guys advocated for yourselves throughout your career and what tactics have really been helpful in making you be heard? We’ll start with you, Jing.

Jing Huang: Sure. To be honest, I’m an introvert, so self advocating wasn’t natural to me. My pivotal point came when I realized self advocating is not only about self, it’s actually about advocating for your team. It’s also mutually beneficial for you and for your manager. There are two tactics I want to share today that will be helpful. I think it was helpful. Number one is to do regular updates of your work progress. Doesn’t matter if your manager asks or not, but do it. You can do it through email. You can do it through your one-on-ones. Do keep those cadence. That’s just going to be mutually beneficial because your manager wanting to know your work. That’s the number one thing.

Jing Huang: Second thing is share your knowledge with your peer, with your cross functional partners. Do tech talks. Do lunch and learn. When you go [inaudible] try to talk in conferences. This is also a mutual beneficial thing. It’s self advocating in one way, but it’s also sharing your knowledge. Everyone else is going to learn from you. Those are two things I wanted to share.

Robin Ducot: What about you, Shilpa?

Shilpa Apte: Yeah, for me self advocacy, for me it was a pivot when I realized that if you do it a little bit over time, it’s much easier to have important conversations when you really need to rather than trying to do a big one all at once. The way in which I do that is, one, to just believe in myself. I think if you believe in yourself, build confidence in your abilities, it trickles into everything you do. Two, understanding what your manager expects of you and meeting those expectations, it’s just easy points. That’s not to say that you should feel restricted by those expectations, but I think that’s kind of a baseline. Beyond that, you should kind of like, what Jing said, be giving regular updates about what you’re working on. The third is really being able to articulate what value your work is creating for the business and for your team and making sure that your manager understands that as well.

Robin Ducot: Those are great suggestions. It’s funny, for me, one of the things I’ve noticed is that developing a very strong sense of entitlement, which sounds negative, but I swear it’s positive, developing a sense of entitlement that you are allowed to be in the room, especially for female technologists I’ve noticed this is an issue. I was fortunate in that I’m third generation technology leader in my family, so I didn’t realize women weren’t supposed to have a voice around technology. I’ve noticed that it’s really, really important that you feel entitled to speak up. Your voice is just as important as anybody else’s. If somebody interrupts you, interrupt them back. You know?

Erica Weiss Tjader: Well said. Well said.

Robin Ducot: All right. Moving on. Mentorship. People talk a lot about mentorship, but what’s interesting is some of the most influential relationships that you end up having because they’re more common are the allies that you develop in the workplace. I’m a little bit curious about, for you, Erica, tell me a little bit about some of the experiences you’ve had with allies.

VP of Product Design Erica Weiss Tjader shares tips for making allies in the workplace, for your professional success, at SurveyMonkey Girl Geek Dinner.

Erica Weiss Tjader: It’s an honor to have Robin Ducot ask me this question because she is one of my greatest allies here at SurveyMonkey, but that’s not what I’m going to talk about. I was actually trying to think of a really good example, and I found the list was so long I couldn’t even remember the last names of half of the people that were coming to mind, which I think is either indicative of my memory or actually the nature of allies. Unlike mentors, allies are not big investments in relationships over time. They are episodic. They are based on a specific purpose at a specific place and time. As a result, they can have a really much bigger impact on something you’re trying to achieve at the time.

Erica Weiss Tjader: A good example that I thought of is in a previous role I was a design leader of a smaller team. One of my biggest challenges that I was facing was making inroads with our engineering leadership around the notion of, the importance of front end development, design systems, some of the topics that design and leaders and engineering leaders often talk about. I was having a hard time getting traction. It was one of those tough problems because it was probably the most important thing to my team and yet the thing I had the least direct control over. This was an example. I have to influence because I don’t own the answer to the problem. This particular ally was a new engineering manager that joined the organization. In my initial just meet and greet with him, I learned that he had some expertise around developing front end teams and design systems and sort of an interest. Perhaps most importantly, I learned that he had a personal relationship with our CTO who was the person I was having the hardest time making inroads with.

Robin Ducot: Wasn’t me.

Erica Weiss Tjader: They were personal friends. No, it was a different story. I’ve got a lot of stories.

Erica Weiss Tjader: What I did is I really just started out by befriending this guy. I’m like I’m going to make your transition into this company really easy. I’m going to introduce you to people. I’m going to tell you all the secrets. We had lunches. We had coffees. We started to build a relationship, and in a very short period of time we were able to transition that relationship into finding a mutually beneficial place where he was able to leverage his expertise and his influence in the engineering organization to start a front end team and I was able to give him disproportionately more resources and support from the design team to really improve the value and success of that.

Erica Weiss Tjader: I think it’s just a great example of an alliance that was very intentional, but looked very different than a mentorship relationship because it was really about a place and a time and a need and a relationship right in that moment.

Robin Ducot: That’s a wonderful story. I mean I think that’s something that’s really, really important. I know as an engineering leader, one of the allies that I always develop when I first join a company is the person who runs the customer support team, making sure that they understand that I am there to help them when we screw up because obviously the site does have issues on occasion. Making sure that you have that relationship so that they support you when you inevitably screw up and then you can support them. I’ve noticed over the years that this is a pattern that really, really works.

Robin Ducot: I do have a question. We are in sort of the center of the feedback economy, SurveyMonkey is. I feel like feedback is–one of the reasons I love working here is because I love data, I love feedback, I love learning new things and driving insights from data. One of the things, though, that I’m curious about is feedback is so important in developing our career. What is some feedback that you’ve gotten? We’ll start with you, Shilpa, that you’ve gotten that really made a material impact on you?

Shilpa Apte: It really can be summed up in two words. Be reliable. That’s not to say I was unreliable, but you commit to something and it starts slipping and you don’t tell the person that you told you would do it in time that it’s slipping. I don’t know. These little habits that build over time I think I kind of realized that you can be the smartest person and the most talented person in the room, but if people can’t rely on you long term, they’re not going to want you to be on their team. I think once I realized that, that kind of just staying on top of things and making sure, even if I’m not going to complete something that I said I was going to, just communicating that out made my relationships and trust between colleagues a lot stronger. That was kind of a career shift for me.

Robin Ducot: That’s great. For me probably, if I think of some of the best feedback I’ve ever gotten, I have a lot of imagination and I like organization structures and redefining things. As a leader, I got the feedback from an executive coach some time ago that, you’ve got to stop reorging every week. You’ve got to stop all that shenanigans with the reorgs because people, you may be able to surf over the top of all this change and this chaos and you like change, but people, a lot of people like stability. Stop. It was really, really helpful to me because it helped me see the world slightly differently than the way I’d been seeing it. I was like but it’s so exciting all this change. They were like actually you’re killing people. I think that getting feedback, soliciting feedback and listening to it is a really, really important part of developing your career.

Robin Ducot: Advice for others. What skills and experience have been the most valuable to you in developing your career growth? What advice would you give to somebody if they wanted to turn to a career in tech? Jing, we’ll ask you this question.

Jing Huang: Sure. When I think of this question, I actually think of it from a different angle because tech, or the industry we’re in, is fast growing, ever changing. It is not any particular skill or experience that will determine our successful in this industry. It is what I believe is the growth mindset that we all should have and the willingness to learn new things and the courage to actually take on new challenge. I think if you have that, you will be able to learn. Just be sure that you’re actually enjoying doing tech in the first place, right? If you’re sure with that and learn, you will be successful. It’s just time.

Jing Huang: One thing I liked about what Steve Jobs said, long pause, this is an old sentence, but I think it always inspired me is really you cannot look on how dots connected when you look forward. You only see the connections backwards. When you see a new challenge, when you see a new opportunity, take it. Do it. You will find how that’s connected after some years. Robin would know better.

Robin Ducot: Because I’m old, yeah. Lots of connections.

Robin Ducot: I think one thing that’s really important, if you’re thinking about a job in tech is that you have to like solving problems. As an engineer, that is your job. Your job is to come up with interesting and creative ways to solve problems. If you don’t like solving problems, if you think problems are a pain, then you’re not going to really enjoy being an engineer. To Jing’s point about start first to see whether basically this looks like an interesting career to you because if you don’t enjoy it, if you don’t enjoy solving problems, you’re not going to enjoy being an engineer.

Robin Ducot: With that, our final question we’ll take to Erica about finding the right company, the right place to work. What advice would you give the women in the audience who are looking to change companies?

Erica Weiss Tjader: First, I’d say the recruiting booth is over there. No, in all seriousness, I think the first question I’d ask is why are you looking to change companies? Are you running away from something or to something? I think it’s really important to be true to ourselves and be really specific with ourselves when we start to think about making an important change like that and start with the foundational assumption that there’s no perfect job, there’s no perfect company, there’s only the right job and the right company for you right now. I think how do you get to that assessment? I always encourage people to start with a list of what are the top three most important criteria for your next role. You only get three. There’s somebody in the audience I know I’ve interviewed recently, so she’s familiar with the question. Don’t worry. She’s joining.

Erica Weiss Tjader: Anyway, I digress. Start with your three criteria and then be really intentional about how you go out and look for a role that meets that criteria. As women in technology in the Bay Area, we all have the luxury of recruiters hitting us up all the time, of going to events like this where hiring is a big focus. Talk to those people who are seeking you out, but also seek out companies and people who you think are interesting or who you think might meet your criteria well. Have coffees. Have lunches and casual conversations. Find that role that really speaks to what you’re looking for. I often say that the best reason to look for a new job, not the only reason, but probably the best reason is because you’re looking for a very specific type of growth or opportunity that your current company can’t give you at this time. Usually it’s no fault of the company. It has to do with the size or the stage or the priority.

Erica Weiss Tjader: Have those specific criteria, be deliberate, and then go out and look for a job and do not accept a job that meets any less than two of your three criteria. I think just be really true to yourself there. I think the last thing I would say in terms of advice here is to really be open minded and flexible because after you’ve gone through that process of thinking through your criteria, of talking to people, you actually might find that you look at your current company through a different lens or see opportunities there in a different way. It might be that right now that company is the place that meets your criteria the best. Really just be open minded in that search and clear about what you’re looking for and what you’re running away from or running towards.

Robin Ducot: That’s really, really great advice. Another thing I would also think about is that you are joining a group of people and if you don’t like them or if you don’t like the person you’re going to work for, doesn’t matter how great the company is, your job is going to suck. They really are not going to be good advocate for you. If you don’t connect with them, if you don’t feel like they are somebody who is going to really understand your deal and what’s good and strong about you in particular, they don’t appreciate it, then find somebody else to work for. That’s my small piece of advice on finding companies to work for.

Robin Ducot: That is our last formal question. What we’d love to do is get questions from the audience. We are, I’m just wondering, do we have a mic we can hand around? All right.

Erica Weiss Tjader: Bernice is over there with a mic.

Robin Ducot: We have a question here.

Natalia: Hi. I’m Natalia.

Robin Ducot: Hi, Natalia.

Natalia: Should I stand up so everybody can see? I’m really curious about how you approach difficulty to make people feel in the survey. We’ve already heard about best practices, how to run great survey, but how to encourage people to look into it? Maybe let me also explain the question where it’s coming from. Five years ago whenever I received a survey, I just filled it in because it was new and it was so cool. Now I receive surveys like in my career and then private ones and then some place for my husband and so on and so forth. There’s a lot of them. How do you make people start the survey?

Robin Ducot: I think I’m going to have Sarah, if she’s okay with that, come up here and maybe answer that question just as our survey research expert.

Erica Weiss Tjader: Sarah, I’ve got a mic for you. You thought you were getting away.

Sarah Cho: It’s a little hard to hear over there. Do you mind repeating what you had to say?

Natalia: Can you hear me?

Sarah Cho: Oh, how to increase response rates. Was that the question?

Natalia: Yeah, so how do you make people actually start the survey because [inaudible] so many of them from various sources.

Sarah Cho: That’s a really good question. A lot of people worry about response rates, so there’s a couple of things that you can think about. Number one, how are you inviting them? How many of you guys, every workplace has this, has this type of person. We utilize Slack here, so I’m sure you guys might utilize this. I hate it when someone Slacks me and says, “Hey.”

Sarah Cho: Then you’re forced to say, “Hey, how can I help you?” I find that, that’s like one of my pet peeves. It’s the same thing with survey invitations. If you just say, “Hey, take my survey” what’s the incentive for anyone to actually take your survey? That’s not engaging. That’s not personalized. That’s nothing, right? One of the things is to really think clearly about your invitation. It’s best if you can customize it. If you know the person’s name, utilize the person’s name. If you can disclose what the survey is about, give a quick one sentence, not too long again, description of what the survey is about. The other thing to remember is the first impression is the biggest impression that you can make. Again, thinking about the invitation, you don’t want to use too much text because people are going to get overwhelmed and not read it. Same thing goes for the first question in your survey.

Sarah Cho: If you start with a really hard question, we actually see this because we collect so many responses. We can actually go through our database and see what decreases response rates and what increases it. The first question is actually one of the most crucial. If you start with an open-ended question, which is really hard for people because you have to type out your answer, your completion rate automatically drops on average around 15 percentage points. That’s a lot. If you just start with an easy question, even if it’s like a soft opener that you kind of need to throw away, that’s always best. Start with a multiple choice easy question.

Sarah Cho: Those are two small things that you can do. Number one, personalize how you’re inviting people in, and number two, make your first question easy. There’s a lot of other things that you can consider, like incentives, but that’s a whole ‘nother talk. I’ll save that for later.

Shilpa Apte: I would also just add on that within the product itself when you’re creating a survey, there’s a tool called Survey Genius that actually tells you, you’re welcome, that tells you, kind of scores your survey and gives you feedback on how you can improve response rates based on question ordering and length and all that.

Robin Ducot: Who’s got the mic? Bernice.

Audience Member: Can you hear me? My question is if we’re not necessarily sure our three, oh sorry. I won’t yell. All right. If we’re not necessarily sure what our three criteria are, what we want to do, what we want our next step to be, how can we go about finding that next step and just kind of a general what do you want to be when you grow up more?

Erica Weiss Tjader: A couple of us might have thoughts here. My first thought is don’t try to make one of your criteria like this is what I want to be 20 years from now when I grow up. That’s like this well thought through career that probably won’t even exist 20 years from now, right? I think when I talk about criteria, I’m actually talking about really specific attributes of your role. For example, one of my criteria when I was joining SurveyMonkey was I had a 10 month old at home. I knew that eventually I was going to want to have another child. I knew that I was going to work a lot of hours no matter where I went, so I knew short commute was super important to me. One of my two was short commute.

Erica Weiss Tjader: I think it’s about what is important to you in your life right now. If you’re earlier in your career, you might say what I really like is I’m a really outgoing person and I really get a lot of energy from talking to people. If I don’t know what function I want to be in, I know that it’s super important to me to have a role where I interact with people for more than half of my day. Then you can start having the conversations and you’ll find out which roles actually fit that criteria and which don’t, but don’t start out with like I must be in sales or I must be a front end engineer, like trying to get too specific about the career path. I think that will emerge for you as you think about what kind of work or environment gives you energy and you’re passionate about.

Jing Huang: One thing I think I can add on is when you really don’t know what you really like, I think when I was in school this was a question. Look at what other opportunities are out there. Try to do them. You can always think about you like or not, but that’s just thinking. You have to do it and actually test if you actually like it or not. Don’t pass an opportunity thinking you may not like it. If there’s an opportunity, actually catch it and try to do something about it.

Robin Ducot: Experiment.

Audience Member My question is how do you solicit constructive feedback? I find, personally I find asking directly is not the most effective way. Actually, I’ve been thinking about sending out survey. I’m wondering if you have any insights.

Robin Ducot: We’re running a feedback survey right now in engineering. Anonymous is good. If you really want to find out, for an organization or for something that you’re doing, anonymous is good. If it’s personal, if you’re trying to get feedback from people personally, you just have to show that–it takes time to get people to trust you, that you won’t freak out. Ask for feedback and it’s still too sugar coated, poke a little. Have it be a little less sugar coated. Poke a little, less sugar coated. Not reacting. Just keep asking questions. Then people will start being willing to give you the kind of feedback that you really need because you need people to believe that you can handle it, right? That’s why people sugar coat things because they don’t want to deal with people’s emotions when they upset them.

Robin Ducot: Present yourself as somebody who is just interested in the facts and it really, really helps.

Erica Weiss Tjader: I think another trick I’ve called on for some of the folks on my team, I’ve had people reporting to me who are the type of people who always get positive feedback. Every time we send the anonymous peer feedback 360 survey, everybody says they love working with me. By the way, that’s true. That means that you’re great and you should feel really good about that, but obviously it’s not as constructive. I think there’s a couple tricks. One is ask a different level of person for feedback. Perhaps you’re asking people who are your current peers, not the people who you would like to be your peers in your next role, right, in terms of thinking about asking the next level.

Erica Weiss Tjader: The other thing is asking the question in a different way. Rather than saying, let’s just say your ambition is to be a more influential leader. Rather than asking somebody what room for improvement do I have or some sort of generic question, ask I’m really focused on trying to have more influence as a leader. What are some areas where you think I could be more influential? What could I do differently to have more influence? Really ask about the thing that you have identified as the thing you want feedback on specifically as opposed to just general feedback questions.

Robin Ducot: Yeah. I’ve noticed also real time feedback. You’re in a meeting and the meeting’s over. Asking the person do you think that thing I said in the meeting was okay, where you’re right there in the moment and they might be more inclined to give you feedback because you just had this experience. It’s hard also for people to remember if you’re asking things that are too general. It might be another thing that could help you.

Robin Ducot: We’ve got a question back there in the orange.

Audience Member Can you hear me?

Robin Ducot: Yes.

Audience Member Thank you again for organizing this. I have used SurveyMonkey for many years for my own events. I used to be in marketing, so thanks for this opportunity to see you in person. I used to be an engineer who moved into marketing, product management, sales ops, kind of seen all the 360 of the business. One of the comments Robin made around problem solving, that one aspect that I learned in engineering has stayed with me throughout my career, so I just wanted to say that to the audience and also to anybody who is in tech, if you don’t like problem solving, then your career, you basically are lost.

Audience Member: The one question I have is in engineering especially, given that I’ve done everything, I’m realizing now that if I wanted to come back, what sort of strategic leadership roles could one aspire to? Is that even possible?

Robin Ducot: You mean you want to move laterally from where you are, the level that you’re at now, back into engineering?

Audience Member: Yes.

Robin Ducot: Probably you’d have to go join engineering at the level you left, not the level that you are, just because engineering leadership, usually you need to have the respect of the engineers that are working for you, and that means having been sort of through the ranks of leadership and engineering. At least that’s been my observation. If you’re willing to go back and if you left as an engineering manager, joining as an engineering manager again and working your way up, at least that’s what I would guess unless you go to a start up. A start up, everything’s possible. You can be a CTO at a start up with an accounting degree. It’s totally fine. Just write some code.

Robin Ducot: Sometimes I’ll tell people that. You get somebody who is like I want to be a CTO now. I’m like go join a start up. You can do that. Knock yourself out. Anyway, I digress. I think if you really want the leadership role early, start ups are great for that.

Audience Member: Hi. I have a question. What if you like what you’re doing, but you just don’t like the people you’re working with?

Robin Ducot: Get a new job. Get a new job.

Robin Ducot: We got recruiting over there.

Robin Ducot: Life is too short. If you hate the people you work with, get a new job. You’re not going to be able to fix them. I’m just saying. I don’t know if somebody has a more sophisticated answer than that.

Audience Member: Hi there. My name’s Vidia. It’s fantastic to be here and see all these women at the panel here. It’s been great. Robin, my question is for you. It’s great to see a female CTO. My world is full of the other gender, so it’s very nice seeing this. Can you talk about your journey? You said you’ve been in tech for 30 years? What were your pivotal moments when you look back now and said okay this was a game changer? I’m sure you have a few.

Robin Ducot: Oh boy. I was really lucky. Like I said, my mother ran a huge research team at MIT, so I didn’t know. I didn’t know. I didn’t know women weren’t supposed to be engineers. Nobody told me that. I also liked being an outsider. I was a punk rocker. I didn’t mind being the only female engineer in the room, and again didn’t realize I wasn’t supposed to be there because nobody told me and I didn’t mind being an outsider. I was really, really lucky. I think that along the way things that I had to learn that were hard to me, when I realized that after getting into fights with more than one boss over multiple companies realizing that it was me, that it wasn’t them, that it was me, and that I had to work for people that I liked and who understood my special snowflake self.

Robin Ducot: I think that everybody’s got a certain set of strengths and weaknesses. Over the years, developing a career around my strengths and then hiring from my weakness. It’s not one specific story. It’s sort of a constellation of experiences that have led me to believe that if you focus on your strengths and hire for your weaknesses, you could actually be really, really successful. I don’t know. It’s kind of a long, windy answer to that because I don’t really have specific things that come to mind as oh my God that time when. There have been moments in time of pieces of advice I’ve gotten. Because I will have a street fight about things, I got the advice once of, Robin, sometimes you just got to keep your head down. Keep your head down and just keep moving. All those people that you want to kill, they’ll be gone and you’ll still be there. Then you get to run the show.

Robin Ducot: The guy who gave me that advice is my boss here. I worked with him 20 years, I mean we’ve been working together on and off 20 years. That was a long time ago. It was great advice. Anyway, there’s been lots and lots of little experiences that have brought me where I am, but I do think focusing on your strengths and being resilient. You’re going to get your ass kicked every day. Just keep showing up. That’s been my experience. Again, like I said, I liked being an outsider so I didn’t really mind. I always thought this is my fight to have, but what I’m realizing and what I’ve come to realize as trying to help other women move through, not everybody likes being an outsider. Not everybody wants to have that fight and they shouldn’t have to. You shouldn’t have to have that fight. It shouldn’t be part of the deal. You should just be able to be a woman in tech and not have it be a big deal.

Robin Ducot: It kind of annoys me actually sometimes. I feel like I’m a prized poodle that’s being trotted out. Female technology leader. I want that to be the norm. I want that to be everyone can be a CTO and be female. It’s like that’s not a thing, but we’re not there yet. I recognize that. That’s just my long winded answer to your question.

Audience Member: Hi. My question is SurveyMonkey consider yourself like Uber in survey industry? If not or yes, what is your major competition?

Shilpa Apte: Do we consider ourselves to be Uber in surveys and who is our major competition?

Robin Ducot: We are the Uber of surveys. Oh yes. Oh yes.

Robin Ducot: It depends on what market, actually who our biggest competitors are. It depends on whether it’s the consumer market, sort of low end surveys, basic surveys, and then you have Google forms and things like, I don’t know, Type Form. That’s who I was thinking of. Type Form. See how much I worry about them? Then at the high end in market research you have Qualtrics. One of the things I love about SurveyMonkey is that our product works for all of these markets, for the big and the small, and that’s actually what makes us unusual. Type Form doesn’t work for market research and Qualtrics doesn’t work for the basic user.

Robin Ducot: Yes, we are amazing. I totally think that.

Angelica: I had a question. I actually had the reverse question of someone here who asked a question. My name is Angelica and I’ve been in the tech industry for about ten years now. I’ve been fortunate to have a very supportive boss and team. My situation is I don’t really like what I’m doing anymore. It’s a short glass ceiling where I’m at and there’s not really a lot of room for growth or advancement. Just asking for anyone’s opinions about being in that kind of situation, having a supportive team and a supportive boss, but not really being happy with what you’re doing.

Erica Weiss Tjader: I think we call that the curse of being too comfortable or something like that. Again, I started with the there is no perfect job, there is no perfect company. I think it depends where you are in your life. I think you might be at a point in your life where you’re like these are the years where I want to work hard and I want to charge ahead and I’m willing to pay the price to do that. Therefore, staying in a role where you know that there’s no growth for you is probably not a good idea. You may be at a stage in your life where you’re like you know what I need? I need to feel supported. I need to feel happy going to work every day because the people I’m working with. At this stage right now maybe actually climbing that ladder is not my top priority.

Erica Weiss Tjader: I think it really depends where you are, but I think if you are at a place where growth is important in terms of your–personally and you recognize that is not what you have at this company, I guess the first question is are there other roles within the company where you could find growth and other avenues to explore there? If not, that’s probably the right time to start looking at what else is out there.

Robin Ducot: Absolutely.

Audience Member: Hi. Thank you very much for sharing your advice today. I came from a finance marketing background and being in the Bay area, obviously tech is where it is. Want to know what’s your perspective on boot campers as well as if you have any advice for reframing other industry experience for transitioning to tech. Thank you.

Robin Ducot: I have some thoughts about this, but I don’t know if you guys want to share. I think that boot camps are typically really, really great feeders for small startups and tend to be less effective for getting jobs at larger companies who have big internship programs, but startups usually don’t. They are very excited about taking boot camp folks. I’ve worked at all different size companies and that’s been the pattern is that typically small companies eventually grow out of the boot camp feeder just because there is a lot more risk. As companies get bigger, they’ll take the CS degrees from their intern programs as the sort of feeder into the low level engineers.

Robin Ducot: I love startups. I think the experience you get at startups is amazing. I think that going through a boot camp and ending up at a startup is an amazing way to start your technology journey.

Jing Huang: One thing I think of, like your background in finance, right? Cross functional. If you think about tech today, having a knowledge of a different industry is actually very valuable. Taking tech applied for finance. Think about, for example, data science. There’s a big application on data science for finance. Those are areas where you actually have a unique strength to segue into tech if that’s what you want to do.

Audience Member: Everybody heard about the bro culture here in Silicon Valley. Is there a thing in SurveyMonkey or any other job you had before and how did you deal with it? How did you handle situations like mansplaining or competing for role, climbing the ladder?

Robin Ducot: Mansplaining? Sarcasm. Really? Really?

Audience Member: What about the career ladder?

Robin Ducot: For career ladder, I mean SurveyMonkey does not have a bro culture. It’s actually one of the wonderful things about SurveyMonkey. Do you guys agree with that? Yeah. I think that’s one of the things that actually was a little bit interesting of a transition for me when I joined was that I didn’t have to have that kind of, didn’t have to whip out the sarcasm quite as frequently. I think from the career ladder, I think it’s really just important to advocate for yourself. Men ask. I always asked. Always. Every single promotion I’ve ever gotten, I asked for. Every single one. It’s not because they didn’t think I was good. It’s just that people pay attention to people who are asking. That would be the thing that I would say is that the bro culture thing, I don’t know what to do about that.

Robin Ducot: I just tell them to knock it off and move on. I don’t know if you guys–

Shilpa Apte: Getting more women into tech.

Robin Ducot: I think that’s actually the best way to solve for it is if you’re a leader is to bring more women in. Then it gets diluted and, I don’t know, slows down. As an individual that doesn’t have the power to change it, if the company is really obnoxious and doesn’t fit your cultural values, then maybe look for some other place. Also, just tell people to knock it off. That’s my favorite thing to do. Whether you feel comfortable doing that is really sort of your thing. As for career growth, you really have to just keep, I mean one of the things that Jing mentioned, which is so important, is making people aware of what you’re doing. Just making sure that your competency is visible and whether it’s tech talks, speaking, things like that so that people become aware of your value to the organization.

Robin Ducot: I don’t know if that’s helpful. I don’t know what to say about the actual bro culture, I mean to fix it. It’s mostly just bringing more women in and not tolerating it when people are obnoxious.

Olga: Hello everyone. My name’s Olga. My question is about technical excellence. Whenever you come as a technical lead or an architect to the company, to the new company, you always have to prove yourself. Unfortunately, I am doing my engineering for 15 years. I built system that always run whenever I leave the companies, but still, like five years ago when I joined Salesforce, I had to prove myself by designing this complex system and building it myself. Then people realized what I really worth and was very surprised that I did this. Unfortunately, I do the same at Google right now. I still have to prove myself and building this complex system and designing this myself because I find that people, maybe they trust me, but they don’t trust as much as I would like. I see they trust some other people. They’re just not the same kind of expertise and the same bar that they expect from the technical lead.

Olga: I was wondering, is it going to happen at all the companies, just like when I join any new this is what happens? You try to stay at the same company for longer or there’s some other skills you can do so you can prove without building this complex system, like writing the code yourself?

Robin Ducot: I think storytelling is a gift that it takes, it’s useful to develop. I’m not sure if you guys agree with this, but I think that one of the challenges, and I don’t think this is a male/female thing actually. I think this is maybe an introvert/extrovert thing. You’ll get people who are just really good at telling their story and selling themselves. These are skills you can develop. Storytelling, selling skills, being able to tell the story of the things you’ve done in the past and being confident about it, feeling a sense of being entitled to be in the room and telling your story is really, really an important skill that will shortcut some of that. I think in engineering, though, engineering tends to be a show me kind of culture. It’s this intersection of being able to talk about technology and learning how to tell the story of all the things you’ve done in the past so that people will listen and telling it in a way that they will listen, not the way that you want to tell the story, but the way they will hear it.

Robin Ducot: You can definitely get coaching in that specific area if it’s something that you’re finding frustrating. I mean some of it is just when you start a new company you do have to tend to prove yourself a little bit. That’s just sort of part of it. I don’t know if you guys have some thoughts about that.

Jing Huang: Totally agree. I was an engineer developer myself. Like I said, self advocating was not natural. Same thing for you. It just feels like we need to do the work to prove ourselves, but in a lot of cases, we have peers or male peers that came to them more naturally where they’d just be able to tell the story about the work they have done instead of having to build something again and again doing the same job to prove what we already been able to do. Learn the skill to really be able to sell your story, self advocating. If that doesn’t come naturally, like Robin mentioned, get some coaching. Really just improve that set. Be confident.

Robin Ducot: Strategically placed solving other people’s problems also becomes a story that people will tell about you. One of the ways when I was still writing a lot of code, I would come in and help somebody solve a problem that they were having. You do that enough and people will trust you without you having to be designing something from scratch and building a whole up system, but being able to troubleshoot their systems because if you’re good at writing a system, if you can really build such a system from scratch, then you might be able to help somebody else fix theirs. Helping people fix their problems makes you very, very popular. Very popular.

Robin Ducot: What’s going on back there?

Audience Member: First of all, thank you for sharing your experiences as inspiring technology leaders. Looking back at your career journey, what was the best decision you made and the worst decision you made and why?

Jing Huang: It’s such a hard question.

Robin Ducot: Don’t try to get your boss fired. Don’t try to get your boss fired. It does not work. Even if they suck and they deserve to be fired, don’t try to get them fired because…. Complex, yeah. How about that?

Robin Ducot: Jing, you got something more?

Jing Huang: I couldn’t think of any like really worse decision you could make. I think every decision, there is different perspective. There is different outcome, but there is always a way out. I think that’s an opportunity. Nothing really bad could happen. Either you stay with a job or you leave for another company. It’s a choice. It’s an opportunity that you will make because of your decision. There’s nothing really bad that’s going to happen.

Robin Ducot: Yeah. It’s actually interesting. The most important thing is how you get back up if you, so take risks. You’re going to fail. It’s okay. You learn. I mean in my experience you learn from failure, not from success. Take everything as a learning experience and get back up and do it again. I don’t know. I’ve had so many learning experiences that have helped me learn.

Robin Ducot: Okay, we are at 8:30. Holy mackerel.

Robin Ducot: One more question right here I guess, and then we’ll-

Audience Member: I’m actually looking for advice. Imagine a manager kind of experienced outsider looking for a new position. This manager, let’s say me, I find a position in a company that I really admire. I look through the qualifications and I look through this job responsibilities and I know that I’m going to excel at this job, but all of this preferred qualification may not match what you actually have on resume. How do you grab this attention? How do you break through? How do you be noticed by either a recruiter or a hiring manager when you apply to this job?

Robin Ducot: I mean the question is, so you have the qualifications, but your resume doesn’t illustrate that you have the qualifications?

Audience Member: [inaudible] some specifics. For example–

Robin Ducot: Yeah, what’s an example of a specific? A specific programming language or something?

Audience Member: Yeah. It’s like a particular experience or, let’s say, experience in network or experience of working with healthcare for example.

Robin Ducot: You know, it’s funny, men will just apply. You just apply. This is one of the things we talk about. Actually when we create job descriptions, we’re really careful to make them so that they’re inclusive, that women are not going to automatically exclude themselves because there’s too much specificity in them. If you reduce the amount of specificity, then you’re going to get a wider range of people because the reality is that you don’t really know what you want exactly. You write a job description and then people show up and you’re like, hey, I kind of like you. You really don’t match the job description exactly, but you have something that’s special that will really resonate with the team. Apply anyway.

Audience Member: [inaudible].

Robin Ducot: Sorry, I can’t hear you.

Erica Weiss Tjader: I think, how to stand out. One thing, I’ll take on that, the reality of, depending where you’re applying, most companies the resumes that are coming through the application system are being reviewed by the recruiters and the recruiters, while great partners to the hiring managers, they know what they’re supposed to look for based on the resume. If it doesn’t match, you’re most likely to get passed on. Doesn’t mean you shouldn’t apply and you shouldn’t go through the recruiter. I think it’s a yes and figure out how to get in to a company, how to talk to real people, how to network. Again, I think it’s a lot harder. Let’s say you were interested in a design manager role on my team, which hopefully you’re not because then this will become awkward, but let’s just say you reach out to me and you’re like I really want to be considered for the role. Can we talk?

Erica Weiss Tjader: I might look at your resume and go I’m not sure, but if you reach out to me and say your experience looks really interesting, I’d love to pick your brain. Can I buy you a coffee? I might be like, oh yeah, okay, maybe, yeah. Then while we’re having coffee, you might somehow slip in the fact that actually I’m looking for a job. Would you ever consider somebody like me? Again, hopefully I didn’t just play out some future scenario that we’re going to have because that would be really weird, but I think it’s like if you come at it from I’m a job seeker, I’m a job seeker, I’m a job seeker, you might find that’s not the best way to build a relationship. Talking to people in the organization and finding your way through is how you’re going to stand out either because your resume looks exactly the same as everybody else’s or not quite the same as everybody else’s.

Robin Ducot: See if you can find out who the hiring manager is and then use LinkedIn to see if you’re connected to them. I mean that’s probably the most straightforward way to sort of do an around the process. Yeah. LinkedIn is your friend.

Robin Ducot: We are being kicked off the stage. Thank you so much and please enjoy the rest of the evening.

Thanks to all the SurveyMonkey girl geeks who helped make the SurveyMonkey Girl Geek Dinner possible! We had a fantastic time.

---

Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at SurveyMonkey and check out open jobs at our trusted partner companies.
• Read about the inspiring event on the SurveyMonkey Design blog!
• Find more SurveyMonkey Girl Geek Dinner photos – please tag yourselves!
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at HomeLight and check out open jobs at our trusted partner companies
• Read about the empowering event on the HomeLight blog!
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Head of Talent, Culture & People Operations Sandy Liao kicks off a HomeLight Girl Geek Dinner in San Francisco, California. HomeLight girl geeks share how they use data and human emotion to empower decision making.

Speakers:
Sandy Liao / Head of Talent, Culture & People Operations / HomeLight
Mary Remillard / Talent Operations & Culture Specialist / HomeLight
Molly Laufer / Director of Offline Marketing / HomeLight
Sam Ryan / Product Manager / HomeLight
Ames Monko / Product Designer / HomeLight
Jenn Luna / Senior Software Engineer / HomeLight
Tina Sellards / Facilities & Administration Manager / HomeLight
Vanessa Brockway / Senior Manager of Business Development & Strategy / HomeLight
Gretchen DeKnikker / COO / Girl Geek X
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X

Transcript of HomeLight Girl Geek Dinner – Lightning Talks:

Gretchen DeKnikker:  From Girl Geek, thank you guys so much for coming tonight. If you guys want to come in and have a seat, we’re going to get started. So we’ve been doing these for about 10 years and this is over 200 that we’ve done so far, so we do them every week. How many people is this their first one? Cool. So stay on the mailing list. Come, you can do these every week up and down the peninsula into the South Bay. And this is Sukrutha.

Girl Geek X CTO Sukrutha Bhadouria encourages people to sponsor and rally their teams to plan a Girl Geek Dinner to increase your visibility in the organization and boost your career.

Sukrutha Bhadouria: Hi. I’m glad that there’s no more feedback. It was driving me crazy. Welcome, everyone. Like Gretchen said, this has been going on for 10 years. We branched into virtual conferences and podcasts, so you can listen to our voices some more if you go to whatever your favorite podcasting service is. A little history; we started off in 2008 and it was just a way to get women from various companies together. You got a sneak peak into the company and what they’re working on and the companies had access to these amazing women and these amazing women had access to each other. You don’t often get to hear about these wonderful accomplished women like those who are sitting behind me as easily if you don’t attend an event like ours.

Sukrutha Bhadouria:  So we’re hoping that you’re going to be building your network tonight and making connections. The other thing is that we’re always looking for ideas and content that you might want, so do send us your recommendations and your requests, but please do get your companies to sponsor because that just increases your visibility within the company. I can tell you I got a lot of visibility and access to my CTO in my company when I got Salesforce to sponsor. It was great. You don’t get to get access to them as easily. As it is out of it, it just made my career trajectory improve drastically. So you want to do that.

Sukrutha Bhadouria:  More than that, you want to, like I said, make connections tonight and please, there’s a lot of people who wanted to be here and couldn’t make it because we filled up and also because it’s a weeknight, so please tweet and share on social media. Our hashtag is GirlGeekXHomeLight, and I hope to see you at more events. We have one every week, like we probably already said. But we’re filling up really quickly this year, so hope to see you soon. All right, thank you. Sandy.

Sandy Liao:  Hi, everyone. We’re going to … it’s always funny at these events where there’s no one ever sits in the first row or the first seat next to the speaker. But make yourself comfortable. Move up if you have to for anyone else in the back. So quickly, I just want to introduce myself. My name is Sandy. I’m actually the head of Talent and People Operations here at HomeLight. This is actually super exciting personally. I’ve actually been a fan of Girl Geek for almost two years now. Thanks, Jenn here, who introduced me to the group. I’ve had the opportunity of attending multiple events. These things are really … people ask me, is this for recruiting, is this for anything? To me, it really is just for yourself. It’s all of you guys taking the extra time and after a long day of work, after a long week, you guys are taking the extra time to come out, to network with other women in different positions also working in the technology world. I just want to thank you guys all for coming. This is our very first women’s sponsored event at HomeLight. I’ve envisioned this for a long time and I couldn’t have expected a better outcome. So thank you all for coming and welcome to our office here.

Sandy Liao:  Great. So I want to kind of start off … we actually have a really awesome panel tonight. The way how we structure our panel is that we want everyone here to get a different flavor of the people that we have and all of our team, so from facilities to operations to marketing to product and engineering, we want everyone here to be able to understand a little bit about everyone’s role here at HomeLight and how we’ve all kind of came through all of our stories to become in the role that we are today. So we have a ton of buzz words in this valley talking about the word “diversity.” To me, I truly believe outside of diversity, it really means more than that. It means a balanced culture, it means a culture where you’re learning a little bit about everyone’s experience and background. It shouldn’t just be about your ethnicity or your race or anything else. It really should be about how everyone came about to become the person that they are today, and that’s how I define diversity and what it means to have a balanced culture. Very proud; I’ve been here for about three years. I’m very excited, I’m proud to have the team that we have today and I hope all of you guys are going to have a chance to meet some of our team member, and have a good time. So, thank you.

Sandy Liao:  A little story before we begin to pass it on to the panel, I kind of want to quickly introduce myself as well as tell you guys a little bit about HomeLight. So who here has actually heard of HomeLight before the event today? Okay, nobody. No worries. That exactly was my response three and a half years ago. Who here has actually gone through a home-buying or selling experience in the past? Awesome. There’s a few of you.

Sandy Liao:  So, the story of how I came about HomeLight was that three and a half years ago, I was actually in the process of buying my first home in San Francisco. I grew up in the city and I thought that I knew a lot about San Francisco, I know what neighborhood’s great, and I also consider myself fairly tech savvy. You know, I know how to use the Internet, I can go on and search for a realtor and do all that fun things. My mom was actually telling me, oh, use my friend. She’s great. She’s been a real estate agent for 25 years, but she’s never sold a home in San Francisco. And that’s the interesting part about this real estate world is that all of us really envision an idea of what the process would look like to one of the most important buying decisions or selling decision your life, but the truth is is a very complicated and emotional process.

Sandy Liao: So while I was going through it myself, I went online and I searched a couple of different real estate company online, and to be tech savvy that I am, I put in my contact information online and I was like, all right, I’m going to get some help here. And I did. I got 27 missed calls from random numbers within the same day. No idea who these people were. I got a whole lot of voicemails. Everyone’s introducing themselves, they know my name, they were telling me how wonderful they can be to help me, but the truth is I can’t … I don’t know who to call back. I can’t decide who’s actually going to be great. So I didn’t end up responding to anybody.

Sandy Liao:  So that was my story initially while I was going through the process. Through a very similar networking event like this, I actually met a woman who was actually in a viewing at HomeLight and I was telling her that I’m looking to buy a house and I’m in the process of looking, she end up referring me to HomeLight.com, and she said, “Hey, maybe you should check out this company. I just interviewed with them. I don’t know what I’m going to …” She didn’t end up working here, but she was the reason that I’m here today.

Sandy Liao:  HomeLight’s mission is that we empower people to make smarter decisions during one of life’s most important moments: buying and selling their home. Our goal is we analyze millions of home selling datas to be able to find you the best performing real estate agent in your area, so that you can make a great decision when you’re looking to buy or sell. Our CEO, Drew, who founded the company eight years ago was going through a very similar process with his wife, also looking to buy a house in San Francisco, and just find out how challenging and difficult it is to go through that process. With everyone here, we all believe there is a better way of doing this, and buying and selling a home should be a very, very exciting process in everyone’s lives. So we’re here to make that better.

Sandy Liao:  Before I pass it down, today our topic here is to talk about how we utilize our data as well as our human emotions to empower all of our decision making. I’m going to hand it off here to Mary on the talent team who’s going to share with you guys a recent experience of hers utilizing HomeLight, our platform. So, thank you.

Talent Operations & Culture Specialist Mary Remillard gives a talk on “How the Marriage of Data & Human Connection Resulted in my Successful Home Purchase” at HomeLight Girl Geek Dinner.

Mary Remillard:  Thanks, Sandy. Hi, everyone. Thanks for being here today. I’m actually on Sandy’s team. I’m the Talent Operations and Culture Specialist, which essentially means that I spend most of my days focused on recruiting on our higher needs across the board. I also do have a hand in our HR and administrative tasks, as well as our cultural and employee engagement initiatives, so a lot going on there. But I truly have been passionate about HomeLight since Sandy reached out to me, two plus years ago on LinkedIn and introduced me to the company and the people and the culture and our mission, which is to empower folks to make a smart decision as they’re going through one of life’s most important moments, which is buying or selling their home.

Mary Remillard:  I have, in my two plus years, been pitching HomeLight as a recruiter literally thousands of times. I did the math, which is crazy. It’s been thousands of times. I have always felt like I genuinely appreciated the service that we offer, and I understood why we have this purpose of giving this service to folks. However, it was only until recently when I went through the buying process myself with my husband that I was able to fully comprehend the emotions and the weight of such a big moment in your life of buying your very first home.

Mary Remillard:  So my husband and I, we’re super frustrated. We have always rented and we were probably touring our maybe 12th or 13th complex, and we were just never able to find that perfect situation for us. It was either too far from work, too expensive, not clean, the folks at the front desk were too grumpy for us. Whatever it was, we just figured out, you know what, maybe it’s time we stop buying … or renting, rather. So we’re very proud upstate New Yorkers, never thought we would leave home, but we found ourselves in Scottsdale, Arizona, and decided that, you know what, we are not getting any sort of return on investment as we’re continuing to rent. Why not put down some roots in the desert here and embark on this buying process.

Mary Remillard:  So luckily, having been an employee for two plus years, I knew exactly where to turn. So within an hour of my husband, Will, and I making that decision, we were on our couch and we were engaging with Kimmy, who’s one of our home consultants in the Arizona office. She just asked us some quick questions around the home that we envision ourselves in. She asked us price point, location, timeline we were working with, those things of that nature. And basically she’s gathering that information to then plug into our own algorithm that analyzes this really robust database that we’ve been building since 2012. That’s how we’re able to determine who are going to be these best agents for Will and me as we go through this crazy task of buying our first home. But by the end of the day, we had engaged with two phenomenal agents and then we had this whole other issue where we had to decide between two great people. That’s a great problem to have. Unfortunately, a lot of folks don’t get to have the experience of that being a problem.

Mary Remillard:  So ultimately, Will and I went with a gentleman by the name of Chris Benson, who’s an agent in Arizona. He’s lived there for many, many years, knows the area like the back of his hand, has closed 425 transactions, has nearly two decades of experience and he specializes in single family homes. So Will and I, we went with Chris because it was clear to us that he was going to be someone who would hold our hand and answer our seemingly endless list of stupid questions and just not make us feel bad about it, and make himself available to us.

Mary Remillard:  So within 30 days, and mind you, Will and I decided pretty quickly that we’re sick of touring, we don’t want to rent, let’s buy a home, but our lease was coming up in less than a month. So we did not want to have to figure out a situation between different homes. We wanted to be able to move in immediately. So I don’t know if many of you can relate to that, but when I told people I wanted to buy a home and move in within a month, I usually got a pitying laughter, like good luck, lady. But we were able to do it, and the thing is why we were able to accomplish that is because HomeLight’s algorithm did its job.

Mary Remillard:  Chris Benson was able to help Will and I get into this beautiful townhome that we’re so excited about because he’s closed homes with our needs, we wanted to buy a home around the 200,000 mark, which believe it or not in Arizona that’s doable. And he’s closed the majority of his homes averaging at 214,000. So the reason why he was so good for us is because he’s done this 425 times. He could do it with his eyes closed. So not only are we having a great experience, but so is Chris working with HomeLight because we’re basically teeing him up to work with people whose areas of need are right in his areas of expertise.

Mary Remillard:  So we obviously had a really great time, but what also made this so wonderful of an experience is because Chris is a human. Chris has raised two daughters who he absolutely glows when he gets to talk about them, he knew the highways that I just wouldn’t tolerate for traffic and a commute, I live a mile away from work now. He speeds on the 101, he’s proudly proclaimed that, and he’s a Cubs fan, so him and my husband were able to go back and forth while I was just like, okay, baseball, woohoo.

Mary Remillard:  But long story short, Will and I now are so excited about our future in Arizona because HomeLight did its job, its algorithm worked, it matched us with Chris, and because Chris is a phenomenal agent who’s experienced, and again, he’s done this 425 times. So that’s really the HomeLight difference right there, and I’m so proud to work for a company that now I know firsthand is truly making a huge difference.

Director of Offline Marketing Molly Laufer gives a talk on “Offline Performance Marketing: Using Art and Science to Drive Response and Revenue” at HomeLight Girl Geek Dinner.

Molly Laufer:  Apparently I need a new headshot. That photo was like, eight years old, one toddler and many gray hairs later. I’ll work on that. So hi everyone, my name is Molly Laufer. I’m the Director of Offline Marketing here at HomeLight. I’ve been here for about seven and a half months, so I think on the panel I’m probably relatively newer to the team from everyone else. Here at HomeLight, I’m responsible for channels like TV, radio, podcast advertising, out of home, direct mail, and some of our large scale brand sponsorships. Now here at HomeLight, we utilize these channels not just to drive top funnel awareness and sort of general brand awareness and market share, but also to actually drive immediate performance. We utilize these channels as performance channels to drive leads and revenue, and I’ll get into that in a little bit more detail in a minute.

Molly Laufer:  All right, so we were prompted here to talk about some of our passions and our hobbies that we have as well outside of work. When I thought about it and realized that I have a gregarious 17 month old, I realized that most of my current interests outside of work really are just around keeping my toddler alive and trying to main some semblance of balance, which to be honest, I don’t know how that works. So don’t ask me for any advice on that. But professionally outside of offline marketing and customer acquisition, the things that are really important to me specifically are around supporting veterans and their transition to the tech world from the military through networking and storytelling, as well as finding community and support for myself and other working-out-of-the-home moms, especially when they’re making their transition back into the tech world.

Molly Laufer:  I started my professional career in 2007 as a surface warfare officer in the US Navy. I spent four years deployed as part of Operation Enduring Freedom, Operation Iraqi Freedom, as well as a handful of counter-narco terrorism missions as well. I was the ordinance officer and force protection officer onboard the USS Samuel B. Roberts where I was one of three women on a ship of about 280 men. So being in this room with this many women in the inverse is freaking awesome. This is a really cool balance. And then I was also a training and readiness officer onboard the USS Nimitz, which is the photo here. This was probably back in 2010.

Molly Laufer:  So anyways, that’s how I started my career. In 2011, I made what I think, looking back, was a very clumsy transition to the civilian world in technology, specifically in Silicon Valley. I joined a pre-revenue, pre-funding, pre-product, pre everything startup. I was the first employee at the direct to consumer e-commerce net company called NatureBox where I worked on community, I did a lot of our social media, our initial paid social media marketing, as well as influencer marketing, which then, for me, really pivoted into focusing on offline, like podcast and TV and radio. I was at Nature Box for about four years, and then I made a kind of non-traditional transition to the agency world. I worked at an offline performance marketing agency for a handful of years, and it was a great experience. I got to work with a lot of really interesting businesses and a lot of really interesting business models, but ultimately I was really eager to get back to be really hands on internally. I found HomeLight about seven months ago, and I couldn’t be any happier.

Molly Laufer:  So I like to make this joke in some of my interviews which was in the Navy, the closest point of approach is when two ships are passing in the night and you want that CPA to be as high as possible, otherwise you have collisions at sea. When I came to the tech world and realized that CPA actually meant Cost Per Acquisition, and if you were doing your job right, you wanted it to be lower, that was a huge surprise for me, but that’s a whole ‘nother TED talk for another day.

Molly Laufer:  So I’m going to talk a little bit specifically about how we approach, or how at least I approach, offline media planning and it’s really interesting that our topic today is around sort of using data and emotion to make decisions because I’ve always said offline marketing is this real mix of art and science. There’s a lot of data, there’s a lot of those really concrete, quantitative information that can go into making a media plan. But at the end of the day, like every decision that we make, whether it’s buying a home or even just looking at your grocery budget, you have to make some sacrifices, you have to make some decisions in your media plan because most of us work in areas of business where you can’t … money doesn’t grow on trees, you can’t afford to do anything.

Molly Laufer:  So, you know, at HomeLight, I’m going to speak to a couple of these things. This is certainly not exhaustive. I listed a couple of different attributes that go into building an offline media plan. But some of the things that we do here at HomeLight that I think are really unique, we’ve been in business for about seven years, so we have really robust consumer data, and we can take that lead data and we can input that into traditional media planning tools like Nielsen and like MRI, and we can actually get really great personas about who our customers are, how old they are, generally where they live, what type of media habits they consume, are they more likely to watch TV on Roku or an AppleTV or Hulu versus a traditional linear buy, what types of stores do they shop at, and these are all the different types of inputs that we can start to input into a media plan.

Molly Laufer:  Because when you think about buying cable TV, there’s hundreds of channels, there’s many different approaches. You have to start to narrow down the way that you think about what are going to be the right buys to attract the customer that’s right for HomeLight. Things like seasonality is also really important when it comes to doing an offline media buy. You know, there’s … every business has its own unique seasonality that they tend to see better efficiency for their business, but the thing that’s really challenging is that the media landscape also has its own unique seasonality. So for example, things like political campaigns, Black Friday, the end of quarter, and all of the local markets, you’ve got all the guys that are selling mattresses and trucks and they’ve got to get them off the lot at the end of the month. All of these things where you think yeah, I’m really jazzed up because I’m going run this amazing campaign at the last week of May, well guess what? Every single car dealership in America is trying to sell cars on Memorial Day weekend, and so you might end up being kind of SOL if you’re really banking on certain weekends like that.

Molly Laufer:  So there’s all these factors that are kind of outside of your control that you need to have a really good grasp on before it comes to planning a media campaign. The other one that I’m going to touch on here before I move on is specifically around the competitive landscape, and I’ll talk a little bit more about this later on, but what’s really interesting is that in certain offline marketing channels, the competitive landscape either can work to your advantage so you can see where your competitor’s advertising, and you can take the move to maybe follow them. In absence of data where you haven’t advertised before, you could look at competitors or like-minded companies, see where they’re advertising, and choose to do the same thing. However, this approach really doesn’t work in other media channels. For example, podcast advertising or radio endorsements where you have an actual human, a person who’s standing up and saying all right, now onto a word from our sponsors. Those types of ad placements, they can really only have room for one type of product at a time. It would be very, what’s the word I’m looking for? I don’t know. It would be very inauthentic if a person were to endorse, say, one mattress company and then the next week turn around and advertise for another mattress in a box company. So things like competitive landscape and this sort of winner-take-all in the space can be really important.

Molly Laufer:  These are just a couple of the facets that go into planning an offline campaign. The output that you see here, which I realized just looks like a bunch of dots and bar charts, because everyone’s impressed by dots and bar charts. No, but in all seriousness, what this tells us is this gives us an output of who our customer is, what types of media are they watching, and where are we going to be more likely to not only reach a higher percentage of our audience, but as you can imagine those are the placements that tend to be really expensive. It’s no surprise that most of our customers and probably all of yours are watching ABC and NBC and CNBC because guess what? That’s what all of America is watching. And so you get a lot of really interesting data down here on the other end when you look at well, what are some of the smaller networks that the audience is also watching? Can I add frequency and can I add additional touch points for our brand using lower reach, but very low-cost and high efficiency media.

Molly Laufer:  So again, those are some of the factors that go into when you’re actually looking at a media plan. When you are using these tools and you get an output, at the end of the day, you can’t buy everything on a media buyer. You have to use some sort of prioritization and rankers. It’s different for every business. It’s different if you’re a national company versus a geo-based company. But those are some of the factors that I use, at least, here at HomeLight.

Molly Laufer:  Another chart with lots of dots and bars. But this is really interesting. So I thought a lot and hard about how can I talk about offline media measurement. I could take an hour, and I think I have seven and a half minutes and I’ve probably already burned through five of them right now telling you about crazy stuff I did before I joined HomeLight. So I wanted to use this specific example because I like to be specific when possible. So the team knows this. I was in the Navy, so I use really nerdy, nautical analogies that no one really understands. But what’s really interesting is when you’re in the Navy, there’s actually two places that you can drive a ship from. The first is you can be in the bridge, right up there with a little wheel. It’s not really big like you see on the Titanic. It’s actually a little wheel that’s this big. It’s super anti-climatic.

Molly Laufer:  So you can either be up there on the bridge looking out, seeing, hey, I see a ship over here off the port side, hey, I see a ship over here off the starboard side, and you can use your eyes and drive the ship, right? You can also, this is crazy, you could not have anyone on the bridge of the ship. You could all be farther down in the ship in the combat information center and using your radars to drive a ship as well.

Molly Laufer:  Now I wouldn’t necessarily recommend that because you lose that eye contact to actually see what’s out there, but the analogy that I always like to make in offline marketing, and I promise there’s a good analogy here, is that when it comes to offline marketing, we have a really tangible way to get sort of directional signal-based indication of what type of media is working better than others. I would really equate that to sort of being downstairs in the combat information center, being able to just look at what the radar is telling me and using that to make navigation decisions. You’re certainly not going to get the full picture, and there’s no substitute for actually going above deck and putting your eyes out and saying, does that ship actually look like it’s pointing in the direction that the radar says it is?

Molly Laufer:  But for us, specifically on … I use this example for TV because I think it’s really visual, but what we do here is, and this is … I wouldn’t say this is necessarily unique to HomeLight, I think this is pretty common in offline marketing, but a lot of people don’t know. We’re all sitting at home, we’re all watching TV. If you’re like me, you’ve probably seen a million e-commerce, direct to consumer companies pop up on TV over the last couple of years. And what’s really cool, and I had a stock image of it but it was kind of cheesy but I took it off, it was basically a couple sitting on the couch watching TV while also scrolling on their iPhones. Because let’s be honest, who does that with their spouse or their friend every night?

Molly Laufer:  Yes. I love that. That is an offline marketer’s dream, right? Because when they’re watching a commercial and they see something really interesting, they just start Googling it. So this is the type of signal that we get when our TV commercials air. We can start to see directionally, well what type of response do we see when we air a spot on CNN at noon? How does that compare to a spot that we air on HGTV Property Brothers at 8 p.m.?

Molly Laufer:  That’s certainly not going to give you the full picture of the impact of your media buy and I would probably need another two hours to go into that, but this is giving you some really good signal-based direction that we can use to make media optimizations. If you have a background in digital marketing, I would say this is the equivalent … this is about the closest thing that you would get to a direct click. In digital media as well as sort of some older types of advertising where you’ll see phone numbers on TV. You still see that today if you’re looking at a lot of lawyers and there’s a lot of local businesses that will really utilize phone numbers, and that’s what we use here at HomeLight.

Molly Laufer:  All right. So I’m going to pivot and just sort of close with that’s all great, but if you don’t do offline marketing, how is this actually going to be interesting for you? So I kind of took a step back and thought, all right, what do I do when I have a decision to make? I like to use all of the data. We all do. But guess what? As we’ve talked about here and as you’re going to continue to hear, the data only goes so far when it comes to making a decision. So I thought I’d kind of leave with four pieces of advice that I try to follow myself when I have a decision to make, and I don’t necessarily have all of the information that I need.

Molly Laufer:  So the first … maybe I should have put this last, but this is my favorite is building a professional or a personal board of advisors in your general role or industry that you can turn to to help when you’re facing a tough decision and you just need a little bit of outside perspective. So you certainly wouldn’t want to go to a competitor, you certainly wouldn’t want to ask a agency who’s working on a competitive product. You’re not going to maybe give them all the answers, but this has been really helpful for me and it’s events like Girl Geek, it’s events like even just talking to some of the partners that you work with. For example, at HomeLight, we do a couple of key large national sponsorships.

Molly Laufer:  And so even just reaching out to those folks and saying hey, I noticed that you also have Wayfair sponsoring. Can I talk to the person who runs offline marketing at Wayfair? Hey, I noticed that Visa’s a sponsor, too. Would it be possible for you to be put me in touch with the person who has my role at that company? More often than not, people tend to want to be helpful and give advice in areas that they have experience in, especially if it’s not competitive. So trying to build up that board of advisors wherever you go in your career, it’s always been really helpful for me to get that outside perspective from someone outside of HomeLight.

Molly Laufer:  This is interesting. So evaluating the risks and having worst case scenario planning. I’m a very positive person, but when it comes to making a decision, my mind first thing goes to what if this is the wrong decision and it completely fails? Not everyone’s like that and if you’re not, teach me your ways. But if you are and you tend to go to the worst case scenario, I like to think well, could I handle that? What would be the worst case? What would be the worst case scenario? And then what would I do about it? So when I made the decision to leave a very fast growing startup to go to an ad agency, I was really, really worried because I thought, God, this could be a career killer for me. Everyone says don’t go to agency side, you don’t get the hands-on experience, you’re going to be working crazy hours, it’s going to be crazy. They were right. They weren’t lying.

Molly Laufer:  But I said, okay, what if they were right and I’m absolutely miserable in this role? It was just the worst decision I’ve ever made. I said well, I would leave and I would find another job. I said, huh, you probably can’t do that every career move, right? If you start to do that over and over again, you just become a career hopper or a serial hopper, but I thought if that’s the worst case scenario, I could handle that. Now what I didn’t do is evaluate what would be the best case scenario and the best case scenario, to be honest, was I think what ended up happening which was I got great experience, I touched different business models, I touched different products, I got my hands on media channels that I would have never otherwise had the opportunity to work on. And now, to be honest, that’s a strategy that I use in general, personal and professional.

Molly Laufer:  This one I think is really interesting. I’m going to use an example from HomeLight which is when in doubt, let your values, whether it’s a personal decision or your company’s decision, guide what you do. If you’re ever at a turning point and it’s yes or no, you say what do my values tell me? And the example that I would use for this most recently was we recently announced a sponsorship as a title sponsor of the US ski and snowboard team, which we’re super excited about as you’ve seen probably from our conference rooms, all of our conference rooms are named for different ski resorts because one of our values here at HomeLight is work hard, ski hard. In my case, it’s work hard, mom hard. I don’t do a lot of skiing right now. The point is when we evaluated this proposal from the US ski team, we used a lot of data, we looked at what were overall CPMs, what type of response rates do we think we could get from the live and broadcast opportunities. But there was all this unknown that we weren’t quite sure about how we were going to measure or was it going to work.

Molly Laufer:  So after evaluating the worst case scenario planning and saying if we were wrong, how will this impact our bottom line, we said, let’s let our values really guide us. One of our mottos here at HomeLight is work hard, ski hard. Let’s do this. And so that has been an area where I think whether it’s in your personal life or in your career specifically, taking a minute to think about the company values, which we all have on our walls and we talk about in all hands, but when you ever need to make a decision, let the values guide you because that’s technically what values should be for is for guiding decision making, not just for putting on a wall and using it in recruitment.

Molly Laufer:  And the last thing I’ll say is trust your gut. I know I’m running over so I don’t think I’m going to give a specific example of this. Only just to say that personally, since becoming a mother in the last year and a half, I’ve realized that out of all the … you can go as evidence-based as you want on everything, but at the end of the day, there’s no one right way to be a parent, just like there’s no one right way to do your job. At the end of the day, trust your gut because it’s probably a lot better than you think it is, and have confidence in what your gut tells you. So, thanks for letting me chat.

Sandy Liao:  If anyone wants a refill on wine, feel free to do so. I’m going to do that myself, so help yourself.

Product Manager Sam Ryan talks about her career journey and product management at HomeLight Girl Geek Dinner.

Sam Ryan:  Hey guys. Sorry. Apologies in advance. I’m suffering from a little bit of a cold right now. But, Molly, thank you for that inspiring talk. Hard to follow up on that. But hi, I’m Sam Ryan. I’m a Product Manager at HomeLight. Thank you all for coming here. It’s actually quite unbelievable that we’re actually hosting this event in this office. So today I’m really excited to talk to you about a little bit of my journey at HomeLight into product, and a little bit about what products and engineering looks like at HomeLight.

Sam Ryan:  So I was hired at HomeLight in 2016 by Sandy and I think I was employee number 38 at HomeLight generally and employee number four or five, I believe, in our Phoenix office. So not only was I hired at HomeLight, but this cemented my move from New York City to Scottsdale, Arizona, which I never expected. I had a very memorable first day in which I was tasked with building my desk, my IKEA desk, and hooking up my computer. But you know what? It really inspired, I think, my journey up until today.

Sam Ryan:  So I was actually hired as what they called I think at the time, Sandy, you can correct me, but I think an experimental account executive/sales person. But really my job was I talk to agents for 40 plus hours a week because those are our users, and talk to them about what they like, what they dislike, what type of problems are they facing not only within the HomeLight platform, but generally, and what we could do better to support them. I was trying to solve a problem. We would introduce highly motivated buyers and sellers to top performing real estate agents across the country, but agents did not really like the HomeLight platform, and therefore we were having a lot of trouble getting updates from them in terms of the progress that they were making with the clients that we were introducing to them. So this caused inefficiencies not only within the sales organization, but obviously company wide.

Sam Ryan: So after a few months of this, I was obviously overflowing with feedback. I would turn to all of my teammates any time we would have someone from our San Francisco office at the time visit and I would just be like, guys, we have a problem and we need to improve the product or we can’t solve the problem at hand. So I think it was a few months into my career at HomeLight as an unofficial product manager, because we didn’t have a product team at the time, I had my first release. So I worked with one of our awesome UX designers, Wally, and one of our killer engineers, Charlie, who both are still on my team today almost three years later, which is quite amazing, in redesigning the referral manager or the CRM type product that our agents use to update us on the current status and progression of the clients that we introduce to them at HomeLight. And the users loved it, which was pretty crazy.

Sam Ryan:  So I think it was really maybe a week after my year anniversary at HomeLight where I was officially moved to the product team, and I think I was employee number two. And the one thing I carried with me, or team member number two, the product team, and I think the one thing that I carried with me from working this experimental role where I talked to a lot of agents to being actual product manager was there is nothing more important to being close to your customer. However, HomeLight moves quickly. We release, and I think rapid cycles is putting it lightly, we release at a pace that’s unreal sometimes. Our team is just here for it and work so hard to do it. But, it’s really hard to use the traditional surveys and interviews, though we still do, to get rapid customer feedback.

Sam Ryan:  So I think it’s so important. I spend many hours on the Internet scouring relevant news articles, forums, threads, reading the comments on these news articles, digging into Reddit. I have spent way too many hours on the real estate subReddit, just understanding, querying every type of HomeLight query that is possibly out there and just trying to dig in to what people are talking about, not only for HomeLight but the industry in general. Trying to embody my user but I don’t have time to be a real estate agent, though I worked in the industry before in New York City. To try to understand and this is an industry where we’re constantly innovating and the tech industry’s constantly innovating and real estate agents have a thought of fear and a thought of let’s embrace this, and what are we doing. HomeLight’s here to empower them and that’s super exciting. So staying close to the user.

Sam Ryan:  Things that we do at HomeLight to kind of embrace the user and embrace their feedback and experience, but also release rapidly. I kind of gave a brief strategy and I kind of compare those and it’s funny that Molly talked about working on a ship and kind of some things there because I compared it to a rocket ignition system, which I had to caption because I kind of just Google image searched a launch button and this came up and I’m like, oh, there’s a lot of switches here and that makes sense. Because what we try to do because you have to release rapidly and we’re trying to gather all this feedback and all of these plans and just go, we have to have a lot of checks and balances in place to ensure that we’re not going to burn down the house, and we haven’t yet, which is super exciting. It’s been two years, so maybe I should knock on some wood somewhere.

Sam Ryan:  But I wanted to give you guys, I don’t know, some of the strategy that my team uses to just ensure that we can release quickly, efficiently, but also keep the user at top of mind and ensure the success of the user and ensure the success of our product. So some of the things that we’ve done; we find users who love our product generally, and users that are … this might not be the best advice, but users who might be a little bit more tech savvy than the average user, and are willing to use things until they break and are willing to struggle a little bit to provide us feedback. We have small focus beta groups that we can roll out to. We really utilize blind, human testers.

Sam Ryan: Actually, before every single deploy at HomeLight, we employ our global app testers, GAT. It’s a great product and they employ actual human testers all across the country that will go through and follow step by step directions and use your product and provide you very, very granular feedback and the test usually fail because of silly things like copy or you accidentally said next and it says, “submit.” But it does point out, I don’t know, very interesting insights about the product and things that you can improve upon, fix, or fix your instructions.

Sam Ryan:  Also, our support and sales team, so for every major release, I really encourage, generally, not only in support and sales, actually company wide, who wants to be involved in this testing process? I got this spreadsheet going, let’s go bug bash. So I really try to widely encourage company-wide involvement in those types of things, not only in forums that people who are talking about these things, but it helps me be informed about my product.

Sam Ryan:  We utilize roll outs often. Any type of risky, major feature, we really like to utilize roll-out flags so that we have that after-launch protection. So worst case scenario, again, hasn’t happened yet, but we can roll back.

Sam Ryan:  I also … In the planning stage, we implement tracking so that there are those flags that pop any time that something could go wonky even a little bit. So I really like utilizing user events, maybe a little bit too much, but I always have a dashboard the day before release that’s ready to go the second that we release and can trigger anything that could go awry at that time. Sometimes it’s triggering nothing. That’s best case scenario.

Sam Ryan:  Also, we use Sentry for error tracking, which we use for all of our staging environments as well as production and consistently monitoring that around these major releases as well. Again, around release time, I stay very close to our sales and support team. They’re generally headquartered in our Arizona office, so we just made a major app release last week, so I was there in the Arizona office literally sitting at the desk next to them. It’s like, okay guys, let’s go. What’s happening? You guys are talking on the phone. What’d they say? Have they used it? Have they used it? They’re like, we’re making them download it now. I’m like, good. Let’s call them back tomorrow. But I think that’s some of the most valuable feedback that I get, of course.

Sam Ryan:  Really encouraging direct channels of feedback, and making it not weird. You’re not bothering me. If I don’t answer you, it’s not you, it’s me. I’ll answer you eventually or maybe I won’t, but again, I really value all of it. I personally like utilizing Slack for these things, but of course not everyone loves Slack, so email, whatever. Just get in touch with me and get in touch with me two or three times if you need to. I don’t know, just encouraging that direct feedback loop. One thing my team embraces and the ping pong emoji is something that we use back and forth on Slack often. The ball is in your court. HomeLight generally really encourages ownership, not only over the products that we manage, but over the release and the success of those products. It’s not only me personally, it’s the team. So I currently oversee what we call the pro’s team, and that’s the professional experience at HomeLight that not only spans real estate agents, but other real estate professionals generally within the industry.

Sam Ryan:  So we kind of pass this emoji back and forth because if the product fails, it’s not only I failed or you failed or he failed, it’s all of us. So we really like saying the ball is in your court. Like hey, I wrote this back, but … this name tag keeps falling off. But hey, you perform again, Sam, we’re going to test together and if I miss something, it’s all of our fault. But I don’t know, we like passing the ping pong emoji back and forth. But anyway, it’s been great having you guys in our office and it’s amazing that HomeLight has grown to the size that we are at today to be able to host this type of event. And Sandy, thank you again for introducing me to the HomeLight family. It’s been a great almost three years. Thanks.

Product Designer Ames Monko gives a talk on “Using Design and Empathy to Create Joyful Product Experiences” at HomeLight Girl Geek Dinner.

Ames Monko:  Hello. My name is Ames and I’m a product designer. Tonight I’m here to talk to you a little bit about how I use design and empathy to create joyful product experiences. So I was going to ask this question, but Sandy stole my thunder, about if any of you have had gone through the process of buying a home, or know a friend who has gone through it. It sucks. It’s like the worst thing. So … you can go to the next slide. And I also need to go to the next slide. Manually. It’s mine. Ugh, this is the worst. Okay.

Ames Monko:  So we all know, or if you don’t know, mortgages are a time consuming, confusing, and overall stupid daunting, bureaucratic process. Fun fact; traditional lenders don’t actually care about you or the experience that you have. Mortgages are technically, I would say, a very small percentage of their actual revenue making, they make money in tons of different ways. So they just kind of choose not to fix the process and then in turn, will just make you go through their very inundated, crappy process. Essentially they attempt to try to innovate, but by innovating, they kind of put a shiny UI on the top of the funnel, like put in your name in this and it’s so sleek and it’s modern because it’s the Internet. And then once they get you, you’re kind of just thrown back into their very clunky, not … just very cold process.

Ames Monko:  Since the subprime mortgage crisis, home buyers know they deserve a higher quality of experience and making one of the biggest financial decisions of their lives.

Ames Monko:  Oh yeah, I should put that there. So the already anxiety-provoking experience of borrowing money is currently made worse by multi-step process riddled with mortgage jargon, AKA anything you’ve ever seen from Rocket Mortgage, it’s not a rocket. It’s not. Sorry if anybody … you work at Quicken. HomeLight’s hiring. Just saying. My approach is like, what if in addition to streamlining the process … next slide. We could approach our design from a place of compassion and empathy. Throughout the past six years, I’ve worked in the mortgage tech industry with the goal of demystifying the process. I spent four years prior to being at HomeLight at Better Mortgage. I was one of the first initial employees of that whole project. It was myself, one front-end engineer, one quote unquote mortgage professional, he was just sent to go learn about mortgages, and one back-end engineer. We sat in a very small conference room in New York. In three months, we basically built what is the backbone of Better Mortgage.

Ames Monko:  So that’s when I started … my Aries brain was like, oh, this is a pretty tough problem to solve and it’s kind of holding my attention and I would never say that in a million years, like I’m really passionate about mortgages. Because talk to me eight years ago, I’ll be like, what? I don’t care about mortgages. It’s not a thing. But as an empathetic person myself, having seen people, friends of mine, family members, go through this process, I’m like, oh, maybe I can use my expertise in design and also my empathy as a human being to try to start fixing this process. Next slide.

Ames Monko: From a design perspective, to remedy any potential pitfalls and offer support when needed. My approach is you want hands on? You can have hands on. You’re tech savvy like Sandy? You don’t even have to talk to anybody. If you can figure it out? Cool. You can do it all by yourself.

Ames Monko:  These are millennials. Apparently I’m considered … I was born in 1980, which I’m apparently a millennial, but I didn’t really get the Internet until, I don’t know, I was graduating from high school, which is in 1999. I didn’t get my first cell phone until 2004, which was a cool flip phone. So the fact that … I was like, I need to find a picture of millennials and I just put these together. They look like millennials, I think. But more importantly, more than any other group, they are relying on financing for their home purchases. Many have already been confused and sort of let down by the student loan process, but nonetheless are still willing to borrow. But they expect the former archaic, home financing process to be simplified, transparent, and pleasurable? Which I don’t think we’re there yet. Next slide.

Ames Monko:  So there’s this really great article. I put a link in here and I was like, oh wait, but these people are probably not going to get the link. But Adam Grant and Erin Henkel wrote a piece for the Harvard Business Review. In it they say that the first step in empathizing with your customer is to gather insights and ask what is broken, frustrating, surprising, or uncomfortable for your customer. The second that you can train employees to put a customer first, it will dictate how you build and design a product. From a design’s perspective, fixing these problems in a visual way that makes people laugh, feel reassured, or feel like their needs are being met or anticipated is a solution that builds trust in my work, or our work.

Ames Monko:  When we take something tedious and scary and turn it into a pleasurable experience, we make the applicant feel valued. If you can give a customer the tools, they feel empowered by that. This is their most important decision and if you can help them get there by just simply adding a more confetti button, do it.

Ames Monko:  My goal has always been rooted in keeping joy at every step. People should feel excited about buying a home, not dreading it. I want them to look back and think about how great it was to buy a home, not the horror of the experience. It will definitely make meeting up with your friends less interesting because they don’t have anything to complain about and talk like, ugh, that was just like the worst thing in the whole world. So that will go away. Unfortunately, you’ll have to talk about more positive things. Next slide.

Ames Monko:  The future direction of HomeLight at the helm is one where curiosity about customers’ experience gives us a unique perspective to stay connected to them. And with all the technical difficulties, I am now done.

Sandy Liao:  Thanks, Ames. They flew all the way here from New York just to join us for the evening, so thank you so much for being here.

Senior Software Engineer Jenn Luna gives a talk on “Engineering’s Software Stack and How We Power our Matching Algorithm” at HomeLight Girl Geek Dinner.

Jenn Luna: Hey, everybody. Software Engineer, introvert, so I’m going to do my best. I was really nervous to go after Ames because her slides were so beautiful and mine basically look like a 10-year-old’s book report, and not a gifted 10 year old, just a regular 10 year old. So bear with me. Okay, so these are the things that make me who I am. Software Engineer for, I think, almost 10 years now, which is absolutely crazy. I am also a real estate agent on the side. I only do it for friends and family because I don’t have time. I’m a new-ish mom. New-ish because she’s almost nine months old and time flies. I’m all these things; I’m a teammate, an employee. In my free time I like to snowboard. I have dance lessons on Wednesdays and travel is pretty much the most important thing to me sometimes, besides my daughter, of course.

Jenn Luna:  So here’s the most embarrassing picture of me that I never share with anybody. I started at Intel in 2008. I was a double E and I was hired as an electrical engineer. This is me in the sub fab. I really loved this experience because I got to go see all the robots making wafers and things like that. In the sub fab, you only wear half of the equipment, but in the fab fab, you have to put the whole bunny suit on where order matters and if you put your boots on before your hat, you have to redo the whole thing. It’s nuts.

Jenn Luna:  So after four years at Intel, I decided I wanted to jump into software, so that brought me to San Francisco, of course. I found a company called SolarCity. Anybody heard of it? Awesome. They are now Tesla, but for five years, I was at SolarCity working in the solar industry. This is me at Bay To Breakers, just fully embracing the San Francisco culture. I loved it. I ended up moving back to Arizona, but everybody goes here and then goes back somewhere else. Anyways …

Jenn Luna:  So when I started there, it was a super small team. I was in crazy startup mode. This part of my life was so exciting. It was nuts. There were just no requirements. Here’s a picture of one of my first requirements meetings with the CTO and it’s like, here’s what we want, let’s put it on a whiteboard, just spent three hours in a room and if you didn’t take good notes and you don’t build what I want, you’re fired. So this was crazy to me. This was actually my first project. And here’s another one that makes me laugh because what is this? It’s like, squares inside of octagons. I don’t even know how I completed this, but anyways. Yeah, so to make things worse, here’s our software stack. It’s just a giant monolith. We have a database and lots of codes that go to it, but no one can figure out how this thing works, right? So it’s like it was just really intense. Makes you feel like this. Super excited about a moving gif in my presentation.

Jenn Luna:  So every day felt like this, but I truly enjoyed it because I was learning so much. It made the team bond. We sat for late nights together, drinking and trying to figure things out. It was a blast. I learned so much. The point is this was the most valuable five years of my software experience. We had a huge monolith. We built it into microservices. We went from a startup to well-oiled machine. We used to use SourceSafe for source control. Does anyone know what that is? When you check out a file, it’s locked. No one else can check it out. It’s just ridiculous. I don’t know. Maybe I’m the only engineer, is that why I’m the only one that thinks that’s funny? Okay.

Jenn Luna:  So I’m going from this 10-person team on to five years of trying to build this thing out, and then eventually we have 150 people on distributed teams all over the nation. I worked remote from Arizona for three of these years, so I was coming to San Francisco every month. We didn’t have any processes. Like I said, we’d lock ourselves in rooms and then at the end of the day, we had scrum, agile, we were just knocking projects out quick. Requirements were everywhere. There was no way I could forget anything. Everywhere I looked, the requirements were there. So if I messed up, it was on me. Like I said, late-night releases. After that, we had pipelines, you’d push your changes to production, it gets pushed up to actually be released, and your stuff is out there, but not before going through massive amounts of tests. So it was way harder to mess up.

Jenn Luna:  So everything was perfect, right? Sunny days every day, and I get bored. So to quote Miley Cirus, it’s definitely the climb because I really enjoy just trying to get what was messy into something beautiful and it was so much fun and I learned so much. So here’s a quick snapshot of our software stack afterwards. It was just really nice, microservices everywhere.

Jenn Luna:  So since I became bored, I was looking for the next challenge. I had gotten my real estate license with my husband during late night classes. It was just something I was interested in. Him and I bought and sold a few properties together, so I wanted to truly understand this experience and I also didn’t want to pay commission to anybody. So it was nice to just get my license. I don’t know. Why not, you know? One of my things is I just try to do too many things, and it’ll drive me nuts, but at the same time I love it. Uh oh. Oh. Okay.

Jenn Luna:  So I joined HomeLight … back to the monolith, right? It’s seriously not this bad. But it is a monolith. We have a giant code base. Here’s a more realistic representation of our software stack. Our sales app, our HomeLight.com, all the blogs related to that, internal tools, everything’s built on top of the same code base, right? We do most things in Ember, but we’re quickly adopting React. We have Ruby on Rails backend. We use Sidekiq for all of our acing job processing, and then we utilize Redis for things like queuing, and then we use a Postgres HomeLight database.

Jenn Luna:  So HomeLight agent matching. I don’t actually work on the agent matching or the algo, but this, to me, is the core part of our business. I’m more of internal tools, sales app stuff, but because this is the most important part of our business, I wanted to talk about this so I had the engineer that works on agent matching give me the details through a fire hose a few days ago. So it’s definitely more than just swiping left or right. You’ve heard all these ladies talk about matching. We have algorithms, like Silicon Valley. That’s our secret sauce. We have four versions. These versions have over 150 data sources that power them through ETL and we have just under about 50 million transactions that are analyzed for around two million agents. So it’s a lot of data. Also a gif that moves.

Jenn Luna:  So our matching process; I’m just going to quickly talk about it because it’s very involved and very well-thought out and it works extremely well and it’s definitely our pride and joy. We take in raw data, it gets summarized, and very recently we’ve been utilizing elastic search like crazy for scaling abilities. Before this last version, before our scaling was kind of on a vertical level. There was no way we were going to be able to keep searching through all this data as it grows and be productive. It took many seconds, which is bad in software world. So for millions of agents and transactions, we needed some other solution, so that’s what V4 has done for us. Elastic search also has some really great geo-spacial search context and it leverages scoring algorithms and decay functions that basically just helps you search the data better. And then after that, we apply all the basic matching criteria that these ladies talked about, like area, buyer, seller, property details, all the basics. Sorry about that. I don’t know what that was. Yeah.

Jenn Luna:  So then after that, we geo-code the address. What we noticed is that neighborhood knowledge is very effective, so if the agent has had many transactions in an area where this house is being sold or wanting to be bought, we will boost those agents because neighborhood knowledge is just very effective with people. It helps them, they feel more comfortable, and I don’t know, you just kind of know things that maybe you wouldn’t have known if you are just diving into that neighborhood. Then we analyze these agent metrics, we rank the agents, these are based on things like number of transactions, how long it takes them to close a house, just a bunch of stuff like that. And then at the very end, we will apply agent preferences because it’s a two-way street and agents should have preferences. So if they want only sellers with blue hair, hashtag picky agents … so we should also let them choose what they want so that it’s a two-way match. I was going to put a picture of people with a heart but that’s too personal I think.

Jenn Luna:  So lastly, performance is key for our data processing because we have millions of agents in our database, we definitely need to keep scaling correctly. So this Version4 that we have has brought our searching from eight to 15 seconds or so down to under a second, which is pretty incredible. So they can just keep scaling horizontally and it’s going to be totally fine. And elastic search … I think I already said this, I’m going to skip that. And then also, we are constantly refreshing this data every month. We don’t want any stale data. We don’t want agents to be picked up that have retired or don’t have any transactions in the last few months or anything like that. So we make sure that it’s meaningful. Then lastly, we do very slow roll-outs. We will roll something out, see what the results are basically in terms of conversion rate, so if something’s working really well, we’ll keep it but we’ll finally tune these algorithms and then once we have something that we think is working the best for us, we roll it out nationwide.

Jenn Luna:  So that’s it for me. Thank you, guys.

Facilities and Administration Manager Tina Sellards gives a talk on “Connecting Data and Technology to the Human Experience” at HomeLight Girl Geek Dinner.

Tina Sellards:  Thanks, Jenn. Hi, guys. We are going to have a dessert bar in the back. You are welcome to grab some now if you like. It’s cookie dough, but definitely something to stick around for.

Tina Sellards:  So a little less on the technical side for me as the Facilities Manager, I’m sure you can imagine. My name is Tina Sellards. I am the Facilities Manager here at HomeLight. That is definitely not all that it encompasses my job. As many people know in a startup, I am an administrative assistant to our CEO, I do a lot of our licensing on our brokerage side and really kind of jumping into our title marketplace side as well, but then also this space that you see here, the food that you’re eating today, all of that stuff is definitely me. Thank you. So the human side is pretty huge to me, as you can imagine.

Tina Sellards:  A little background for you on me. I went straight out of college, graduating from the University of Central Florida, very proud of that, UCF, go Knights. And then went into AmeriCorps right out of that. Really had that if not us, then who, if not now, then when mentality when I came out of school, and was ready to change the world. Learned a lot about government and what kind of bogs down that world as well as I went into that, and decided I wanted to really jump into changing that world as well from the inside and was lucky enough to join the ’08 Obama campaign. Really that network that you build, so huge, your tribe, the people that I met, my AmeriCorps experience helped me bridge that changeover into my work on the Obama campaign in ’08. I ran a region of Florida for them. Everything from getting the volunteers in the door, staffing, getting an office, doing all of those things with no money, really, on that side of things. So super interesting.

Tina Sellards:  But something that was really interesting to me on that ’08 Obama campaign was the data, honestly, and the technology that was being used. This chart right here is from the Pew Research Center. It was really the first campaign that was using Internet as a main source of information for people. As you can see here, from 1996 all the way through 2008, among adult users, Internet usage for your political information went up significantly and they were utilizing a … I don’t know if you all are on it, a service called MyBarackObama.com. MyBarackObama.com actually was a community-based system. They had over 35,000 groups that organized 200,000 events throughout the US to get him elected. In some great foresight, decided to keep that live and use that as a community organization tool throughout his administration and then into the next campaign, which was actually pretty amazing. And then I also noticed something as I was looking at this research, too, which was very interesting to me which was how voters communicated about the campaign. Look at that Twitter down there. In 2008, only one person said they used Twitter to communicate. So just want to let that sit in a little bit as we kind of think about that. On the Twitter side of things, I think we’ve come a long way.

Tina Sellards:  And come into the 2016 campaign. And I make this transition really to talk about Twitter, Facebook, Google, all of those things as we’re using and the data that we’re using, the technology that we’re using, and does it really connect us more. Does it do those things? Do you get the information? Is that the correct information? Are you getting to connect with people in the same group as you, those kinds of things. It was really important to me as I kind of came off of that campaign and started to move into a more kind of people-role in organizations that I was doing, how do we, as a group, as a community, really build that interaction and not silo ourselves into those easy data groups or easy breakup groups that we can kind of put ourselves in. I think one thing that just kind of zoomed in for me was fear. Fear is really kind of a driving factor, right? And why we allow ourselves to be siloed into some of these groups. A fear of maybe that big tech company to breakup your industry, or a fear of the unknown of a different group of people or community than you. Unfortunately, fear really can kind of drive some of these things and I think that’s kind of where we’ve come with some of the data and technology. How do we get away from that is the next question.

Tina Sellards:  I think, and I very much subscribe to Brene Brown. I don’t know if any of you ever listened to Brene Brown or any of that, but vulnerability is how we do that, and leadership with vulnerability is a really key point in the human connection. I think we can really hurt ourselves and break ourselves up by just kind of communicating with the groups that we know and doing the things that we always know. Being vulnerable and letting ourselves be open to that information and being open to other people’s experiences is really how we build these communities and I think something here that I really appreciate about HomeLight and just bringing it together is a core value for us, and it’s not only a core value, it’s something we really live is being a part of our family and really being that open, unique kind of environment. I think it’s super important because I don’t think we’re going to conquer these fears and these issues that we have as a larger society if we don’t start opening up to that and really starting to have those conversations as a group.

Tina Sellards:  So I just wanted to share a little bit about my experience on that and data, and the human connect and hope you all stay vulnerable, open, and communicate as a whole community together, because that’s important in building communities like HomeLight and other … Girl Geek, and things of that nature. Keep those communities open. Be vulnerable.

Senior Manager of Business Development & Strategy Vanessa Brockway gives a talk on “Data & Emotion in Making Career Decisions” at HomeLight Girl Geek Dinner.

Vanessa Brockway:  Hey, everybody. Vanessa Brockway. I’m on the business development team here at HomeLight, and that means a bunch of different things, but we won’t get into that today. So one thing’s we’re asked about, things we’re passionate about. So in my spare time, love to travel and I love to do interior design. But the thing I’m going to talk about today is using data and emotion and making career decisions. I think that’s probably a common thread among everybody. Often times people are drawn to events like this when they’re thinking about the next move or what they should do next. I’m going to share a bit about my perspective on this and then how that led me to HomeLight.

Vanessa Brockway:  So I think careers are a lot like The Game of Life where it’s not just kind of this up and to the right or corporate ladder, there’s a lot of twists and turns, unexpected events. You kind of sometimes are accelerating, sometimes you’re in cruise control and you can’t always predict everything that’s going to be coming your way. And so when thinking about how to approach your career and how to plan for it or how to decide what the next step is evaluating your life as a whole and the things that kind of get you going and what motivates you. As the qualitative aspects, it’s the emotion to drive the data that will also influence this.

Vanessa Brockway:  So I put some images up here, but do you love to travel? Do you want to be on the go? Is exploring the world something that motivates you? Or is being close to home and being able to have a more flexible location where you are, is that something that’s important in your life that time? How do you define success? Is being on the cover of Forbes or making a 30 under 30 list? Is that what’s going to make you feel valuable and that you’ve done something? Or is it building a passion project or building a company of something that’s really meaningful to you. Is that how you’re going to define success? What is the environment that you want to be at all day? Is it a big company with lots of people, huge market presence? Does that get you going? Or is it smaller office, more intimate relationships with those people that you work with? What is it that you want to be surrounded by everyday? Kind of taking that step back, evaluating what makes you feel motivated as a person, and then turning that into data elements to actually help drive a decision.

Vanessa Brockway: So in thinking about an actual company or industry, and evaluating what is the actual size of a company that’s interesting to me? Where do you want to live? What are the demands of that? And how can you take what you’ve learned about yourself by reflecting and actually put that into specific data? So these are just a couple of examples of … LinkedIn, you can actually see the size and growth trajectory of a company. Where are the locations of their offices? GlassDoor; how do employees feel about that company that you’re looking at? What are the employee sentiment and benefits, and things that people get? Crunchbase; do you want to be potentially at a startup? How do you actually quantify what that looks like in terms of amount of fundraising that’s happened? Who are the investors? And also just looking at articles about that company and being able to gather what is the industry saying about the industry as a whole, but also that company in particular.

Vanessa Brockway:  And then this is another piece. So separate from the company as you’re evaluating company, evaluating the role. So I found this comic online. I thought it was pretty funny. “I’ve always wondered why you decided to be a dog. I was fooled by the job description.” So don’t take a job description at face value. Take a step back and look at okay, what is this job within the company? What does that team look like? Is this going to be a very specific role where you’re going to be a ,subject matter expert, or is this going to be an all around athlete where you’re going to be asked to wear a number of different hats? What does the hiring plan look like? Is this company on a very, super fast growth trajectory and then it’s soon going to change? Or are we kind of more in a steady state? What is the title? Is that something that resonates with me? The comp, the benefits, does this company have cultural values that I identify with? And really looking at the specific role and breaking that down to specific data points that you can then tie back to how you evaluated yourself and looked at what motivated you and what was exciting for you.

Vanessa Brockway:  So my personal story is I started off at a pre-seed stage company, which was Stitch Fix at the time. We were under 10 people, no funding, very different but it’s my first taste of startup life. I absolutely loved it. And then I’ve also spent time at publicly traded large companies like Shutterfly where you’re working towards quarterly earnings, your massive, massive companies. And also Haus, which was a company I worked before here and it was every step of the way, I got a different kind of slice and flavor of tech companies at different growth points in their trajectory. The way I ended up at HomeLight is I realized this was the exact point in time, the type of company that I wanted to be at. I’ll walk through some of those pieces about it of how I made my decision.

Vanessa Brockway:  So for particularly looking at the growth stage of the company, for LinkedIn down in the bottom left. When I joined HomeLight about a year and a half ago, Series B, solid funding, had a runway that was very, very strong, but at the same time, this office sells under 50 people, so you’re able to do a number of different things and step into a bunch of different roles, which is something I really thrive in and really love. In terms of GlassDoor, so people loved working here. That was a huge check mark. Being able to see that the employees that were there go in everyday and working there. The real estate industry, something I’ve always been interested in. As I mentioned, I loved interior design but it’s not something I knew a lot about, so researching, seeing how the industry was talking about HomeLight, how they were talking about prop tech companies. Things like that really just help inform the decision. And on Crunchbase, looking and okay, who are the investors that are actually investing in this company? Actually reaching out and speaking to some of them, like hey, why’d you invest in this company? What do you think about it?

Vanessa Brockway:  All those data points together help to make sure that the position you’re looking at, the company you’re looking at align with what’s important to you, but then also is setting you up for a successful position after you join. Yeah, that’s what I have to say.

Sandy Liao:  By the way, when I saw Vanessa put all the slides, you actually really did a data analysis of HomeLight because she screen-shotted all those images before because nowadays, if you search HomeLight, our ratings, our LinkedIn, everything is different so you’ve actually done all those research prior to you joining and saved it in a document. That’s why you’re able to pull it into your presentation, which I’m like super impressive. It’s also unprompted. It’s impressive to know that someone actually did the work as much as Vanessa did to know, to identify HomeLight as a great place to be before she accepted the offer. So we’re great to have you and thank you for sharing that experience with us.

Sandy Liao:  I’m going to be closing up here before the end of the night and I quickly just want to give everyone here a huge thank you for sticking around again. But nevertheless, I want to give everyone here on the panel a huge round of applause please. While we were going through the preparation for the night, all of us were giving each other ideas, what we’re going to do, what are we going to do. None of us have a full-time job of public speaking and we watch all these tech talk preparation, we’re like, oh my God, we need to find some sort of inspirational speech for all of you guys to take away. But I think that the big piece from all of us speaking here is that our takeaway is we’re all just going through the same thing in different stages and different environments, but hey, we’re all trying to be here to make something work and to see what some of our potential could be. So I appreciate all of you here that’s on the panel tonight to take this time to challenge yourself to make yourself uncomfortably, becoming more and more comfortable sharing your stories and supporting from one another.

Sandy Liao:  So I’m going to quickly here, I’m going to promise to go through this really fast. But I just thought that while we tie in a lot of data and motions and talking about HomeLight, utilizing data to support our consumers, to really find the agent, and going through different marketing channels and career decisions. I think that it’s very important for everyone here who are looking into new career changes to understand what it means internally on a data perspective and what are some of the data metrics that I am looking into and that we are doing here at HomeLight as well.

Sandy Liao:  So anyone here heard the term people analytics? Great. We got a few hands. So this is just like a dictionary definition that I found online. I don’t even know if this accurate one, but it sounds pretty accurate, but people analytics is the use of data and data analysis techniques to understand, improve, and optimize the people side of the business. So analytics is become this huge buzzword, everyone’s talking about it, whatever role you’re in, what is your data, how do you measure your success and all that fun stuff. We also are doing that on the people side. But what’s really important is that we want to start to be able to create data that’s useful and not just creating data for the sake of it, but we want to create something that’s actually meaningful for everybody and for all the business decisions.

Sandy Liao:  So I’m going to share here on the four strategic imperatives for people analytics and especially for a company our stage, right? We can’t compare ourselves to companies like Google, Facebook who has kept millions and millions of data everyday that they can spend time on analyzing it. But what do we do when we’re only about less than 200 employees, we’re in about five different locations, locally, and what do we do with the analytics and the datas that we have?

Sandy Liao:  So first it is essential for us to set alignment. What it means is that alignment, not just between our employees, but making sure that our leadership, our executives are also aligned with all the decisions that we want to make. So from the people side, we want to say, hey, we want to start having educate more, development opportunities, more events like this, but if leadership is not understanding the purpose of it and that we’re not aligned, these things will not happen. So in the very beginning, it’s just essential for finance, for the VP of finance and our CEO and executives to understand what are some of the goals that we’re trying to make on the business side. Example would be what is the revenue we’re trying to achieve for the year? What are some of the headcount goals that we have? Because without knowing the essential of our business goals, as much as I want to say, hey, people first, people first, but we also need to make sure that we’re going to be able to secure ourselves financially well. So setting that alignment from the very beginning is just very crucial for this stage.

Sandy Liao:  And the second piece I want to talk about here is actually developing a data-driven culture. So this is unprompted, we’re not sponsored by this particular company, but at HomeLight, we use an anonymous feedback tool called TinyPolls. What it means is that every week this software will prompt us to ask all of our employees one question. The question could be how are you doing today to a question like this: in your current job, what is the number one thing that inspires you and that makes you happy here and want to work harder?

Sandy Liao:  So this TinyPoll’s feedback was actually created when we started all of our different offices in different countries, right? Have you heard a couple of us spoke, they started in Phoenix, we’re in San Francisco, we now have New York. How do we still gather data from our employee on the regular basis and be able to have that transparent communication between leadership team and everyone individually? I was fortunate enough to come across this platform who serves just that. We just want people to give candid feedback without being feeling like they’re going to be punished or be in trouble if they were to share anything on how they feel. So this feedback tool, we’ve actually implemented for over two years. It’s actually been working very well internally. With this data, we’re able to understand how people are feeling for whatever location you are and also be able to make decisions and programs that’s actually going to surface the direct feedback from everybody internally.

Sandy Liao:  Simultaneously, outside of the anonymous feedback loop, we also want to incorporate our performance data. What it means is that for us as a company, we started doing performance review on an annual basis, and then we also do a year-end check in, but these are not just data that you want to have between you and your manager, but we want to have 360 reviews that we get feedback from all of our peers as well. As much this is an important data between you and your manager, it is also really important for the business because we want to understand, hey, even if it’s not measurable bullet point percentage that we’re looking at, at least on a regular, quarterly basis that you are speaking with your manager to talk about, hey, I want to be able to achieve these five goals for the quarter and are you able to do that. At the end of the quarter, you guys should be sitting down, looking back at all the goals that you have set initially and if you find out that, hey, I’ve able to achieve three out of those five goals, what can the company provide you with, what type of training, or what are some of the resources for you to be able to hit the two bullet points in order for you to fulfill all of the achievement and goals that you had set initially?

Sandy Liao:  So incorporating performance data is just crucial to the business, as well as yourself. So for any of you guys sitting here, if your manager has not spoken with you over the past quarter or past six months about how you’re doing from a performance standpoint, it’s just super, super important to hold that in your hands and make that calendar invite, and make them have that conversation, right? Because especially working in a startup, these things kind of get out of hand when we’re trying to do hundred things at once, but before any of us sitting here analyzing whether or not we’re excited to look for new opportunity or what not, it is just necessary to take that step to have that conversation with people that is mentoring you and that are working with you directly.

Sandy Liao:  And last piece here, I want to incorporate a little fun before we end the night here, but collecting data is actually huge, right? So as I was interpreting how people analytics is becoming this huge thing, we, as a company, can’t share that we have a whole lot of data on the hiring side because so many of our roles are actually brand new to the company. So we have never hire a data scientist before and we are trying to hire that, so we’re trying to get data as we are developing these new roles and so forth. But a really fun data I’m sharing here today. This is actually a real, process, a number that we have from us hiring our most recent female engineer, Raquel, who’s actually here today. She flew in for this wonderful event. And we have actually sourced 448 female engineers around the country to get 26 recruiters screens. Can you imagine us just playing people, I’m sure a lot of you guys gone through this. Throughout those 24 screens, only seven of them made it through the hiring manager call. With the seven hiring manager, only four people actually got to the assessment stage. So out of the seven calls, only four of them were approved by the manger. With the four assessment, we got three on-site and ultimately we found Raquel here today.

Sandy Liao:  So these are the example of data and this is actually one that’s a fairly good example. We have some ridiculous roles that we have opened for a long time and it’s the sourcing number even bigger, but the point is in order for us to make tangible and actionable items based on data, we need to start collecting them regularly, whether it’s phone screens or whatever sourcing number it is, it’s just very crucial to do that.

Sandy Liao:  So the actionable item here, why there’s a dog. This is actually my dog. His name is Cooper. I rescued him about a year and a half ago. This is a significant picture for him because that was the day he got all his shots and he was straight legal to take on the action. He was ready to go. And since then, he’s been a wild, wild dog and I bring him around here once a while and everyone can share that experience with that. But that’s it for me. I hope this was helpful for everybody. We are a little … we ran a little later than expected, but we’re all going to be here hanging out, eating some desserts. We have wine and you guys are all welcome to just hang out and if you have any questions for us, we’re happy to answer them. So thank you so much for coming.

---

Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at HomeLight and check out open jobs at our trusted partner companies
• Read about the empowering event on the HomeLight blog!
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at Xilinx and check out open jobs at our trusted partner companies.
• Find more Xilinx Girl Geek Dinner photos from the event – please tag yourselves!
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Xilinx girl geeks: Jayashree Rangarajan, Lori Pouquette, Jennifer Wong and Ambs Kesavan discuss the latest trends in accelerating computation for real-time machine learning at Xilinx Girl Geek Dinner in San Jose, California.

Speakers:
Eva Condron-Wells / Senior Manager of Talent Development / Xilinx
Niyati Shah / Senior Software Engineer / Xilinx
Changyi Su / Staff Design Engineer / Xilinx
Uma Madhugiri Dayananda / Senior Software Engineer / Xilinx
Tom Wurtz / Senior Director, Documentation & Program Management / Xilinx
Ambs Kesavan / Senior Director, Software Infrastructure Engineering & DevOps / Xilinx
Lori Pouquette / VP, Global Customer Operations / Xilinx
Jayashree Rangarajan / Senior Director, Software Development / Xilinx
Jennifer Wong / VP, FPGA Product Development / Xilinx
Angie Chang / CEO & Founder / Girl Geek X

Transcript of Xilinx Girl Geek Dinner – Lightning Talks & Panel:

Angie Chang: Okay. Here I go. Hi, thanks to you all for coming to Xilinx. My name is Angie Chang and I’m the founder of Girl Geek X. How many of you here, it’s your first Girl Geek Dinner? Okay. A good amount. All right, so Girl Geeks Dinners have been happening up and down the San Francisco Bay Area for the last 10 years. About every week, we’re in a different company. And I’m really excited to be here tonight at Xilinx.

Angie Chang: Personally, I was really excited when I looked up the website and I was like “I can’t wait to come here, check out the technology”. And the demos have been really awesome, and I encourage you to hangout and look at them. And walking down that hall of patents is very inspiring. And of course the food was amazing.

Angie Chang: So, thank you to everyone for coming. One thing I want to encourage everyone who comes to Girl Geek Dinner is to network. And I know that’s a kind of scary term, we’re always told to network. But one thing that I always try to do, even as an introvert, is to meet people. At least one or two people over a dinner, when you guys are chewing your food, just talk to people, ask them what they’re doing, if there’s anything you could do for them, and if there’s anything they could do for you. Because people are often looking for new opportunities, they’re looking to learn about the technology and also just to make friends.

Angie Chang: ‘Cause when you have friends in your workplace, or in your industry, that’s how we can stay in tech. We all know that women are dropping out of the workforce over time, and that’s really why we continue to do these dinners, is to encourage woman to continue coming out, meeting each other, encourage each other, and helping each other stay leaning in or whatever you like to call it, to your career.

Angie Chang: So please enjoy yourselves, and thank you again, Xilinx, for hosting.

Senior Manager of Talent Development Eva Condron-Wells welcomes the crowd to Xilinx Girl Geek Dinner in San Jose, California.

Eva Condron-Wells: Wonderful, thank you so much, Angie.

Eva Condron-Wells: So welcome to Xilinx. My name is Eva Condron-Wells and am a senior manager in human resources. I have the pleasure of helping employees at Xilinx listen, give, and feel as though they belong.

Eva Condron-Wells: I’m a social scientist by background, and I grew up in this very specific niche of the semiconductor industry, at another well known company. Most of my career has been in this particular slice of the semiconductor industry. And I think, in this moment, I just want to reflect on the fact that we have many things to be grateful for. And I want to start this evening with thanking a number of people.

Eva Condron-Wells: So first and foremost, thank you to the Girl Geek team for creating this type of forum for us to connect. So thank you so much, Girl Geek. Thank you to our Xilinx greeters, gurus, and guides. I hope that they’re making your visit meaningful. If you haven’t already seen other aspects of our campus, we welcome you to join us in the demo room later on. And, of course our guests, right? So you took time out of your day to come meet us, we know that you work hard and we’re all different types of Girl Geeks, but regardless of that difference, we’re here together to celebrate that brilliance of who you are and share who we are. So thank you for taking time out of your busy days to join us.

Eva Condron-Wells: And speaking of Xilinx, how many of you knew what Xilinx is or does before you saw the Girl Geek invitation? Raise your hands.

Eva Condron-Wells: Wow, that’s pretty impressive. You must be in fairly technical households. Because Xilinx, sometimes called X-links, but we help people understand how to pronounce it, isn’t necessarily a household name. That said, we affect the lives of people everyday. As we’re nested in technology that’s used broadly around the world, affecting everyone’s lives.

Eva Condron-Wells: Xilinx is a 35 year old company. Founded, what I like to say, on friendship. It was founded on friendship and a very incredible idea that is continuing to create many innovations, now and in the future. Our mission is to build the adaptable intelligent world. We have 4,000 employees plus, worldwide, and 4,000 plus patents. Giving us a one-to-one ratio of people to patents.

Eva Condron-Wells: When I articulate the pride I have in where I work, and who I work with, I genuinely mean we have some of the most brilliant people in the world. And you will get to hear from some of them tonight.

Eva Condron-Wells: So we invented the field programmable gate array. Not something everyone hears everyday, but this is a device that makes the life of an engineer easier. It speeds up their ability to perform their responsibilities and have chips work in various ways, right, thousands of applications can be used. But these engineers are empowered to reprogram chips in the field in hours instead of waiting weeks to get a completely new product. So this is the power of what we were founded on, the concept of enablement, empowerment, and acceleration.

Eva Condron-Wells: We have over 60 industry firsts, and our latest product, Alvio… Which is absolutely gorgeous, and is in our demo room, you can see the board… Is plugged into data centers, giving a 90x acceleration over a CPU. This nested technology is inside of thousands of products, changing the way we live, love, work, and play.

Eva Condron-Wells: We are not only diverse in our thought, that creates this technology, we are seekers of innovation, and we welcome brilliant minds who want to play in this space as well.

Eva Condron-Wells: Tonight our focus is on acceleration. You will gain insights into acceleration by the people who enable it. We will have three lightning talks, a panel, and Q&A where we look forward to hearing your questions to dive a little bit deeper into this topic. After 8 pm, we’ll transition to desert for those of you who haven’t already taken advantage of that. Some more demos, just around the corner, and some more networking.

Eva Condron-Wells: This is a very special evening for us, and we’re thrilled to have you. Now onto acceleration. Let’s let the lightning talks being. May I have our first presenter? So please welcome one of our first of three presenters, Niyati Shah. Thank you so much, Niyati.

Senior Software Engineer Niyati Shah gives a talk on compilers for adaptable compute acceleration at Xilinx Girl Geek Dinner.

Niyati Shah: Good evening, everyone. So let me begin my introducing myself. My name is Niyati and I work in the logic optimization group. I primarily focus on software architecture and [inaudible] design. Outside of work, I enjoy weightlifting, and traveling, and doing what I love, both at work and outside is what makes me a girl geek.

Niyati Shah: So let me start by asking you all a question. How many of you here are hardware engineers? Brilliant. And how many are software engineers? Perfect.

Niyati Shah: So, as we can see right in this room, we have a range of distribution of engineers from hardware background and software background. And that, I believe, presents the direction in which Xilinx has been headed. Traditionally Xilinx is a semiconductor and FPG chip company, and most of our customers used to be hardware developers. But as we are moved into the data center and acceleration markets, more of our customers are coming from software backgrounds.

Niyati Shah: And, what I’d like to do today, is give you an overview of the tools that we have that help our developers, regardless of their background, use our FPS to run their designs.

Niyati Shah: So we have the Hardware Developers who code using RTL, Verilog, VHDL. And for them, we have our signature product, which is the Vivado Design Suite. Next we have our Hardware Aware System Software Developers, who use C/C++ and SystemC. And for them, we provide them the Vivado HLS Compiler. We also have our Software Application Developers, who tend to use FPS mostly for accelerating their products or their designs. And for them, we have the SDAccel environment. And finally, for our Data Scientists who use frameworks such Caffe and TensorFlow, we have the AI Compilers or Edge Compilers.

Niyati Shah: In the next few slides, I’m going to go into a little bit more detail onto each of these different tools, so that you can get an idea of how they meet the needs of the targeted developers.

Niyati Shah: So as I mentioned, most of our hardware engineers work with RTL, Verilog, VHDL. But their figure is a piece of hardware, and it’s only going to understand binary numbers or bitstream. And so we have to take the RTL through a process to generate that bitstream. And the best analogy that I could give is kind of a translator. So, if you and your neighbor were to speak in, say, different languages, and the translator will go from your language and convert it to a language that your neighbor can understand. And so we take the RTL through a similar process. We start with synthesis. But [inaudible] the RTL, we apply synthesis. And the job of synthesis is to create a logical netlist , which is technology mapped to the targeted FPGA.

Niyati Shah: Once that is complete, then we need to optimize the netlist. And that’s where I come in. My team and I work on optimizing the logical netlist. We’ve optimized the design for power, area, timing, depending on the developer need. And finally create an optimized netlist, which will better use the resources that are on the FPGA.

Niyati Shah: Once that part is complete, we run placement. And the placement essentially takes these logical blocks here, and puts them on physical locations on the FPGA. Finally, we have the router which connects those physical blocks together. And after placement and routing are complete, we have a bitstream that we load on the FPGA to run the design.

Niyati Shah: Now, instead of starting an RTL, if our developers were to start with say, C/C++ or SystemC, then an additional step gets added because we have to first convert those languages to RTL before they can be fed into the backend tools.

Niyati Shah: And that’s where our Vivado HLS Compiler steps in. The HLS Compiler provides an eclipse ID, and allows our customers to design and develop using our product… The Vivado HLS Compiler and be part of this setting on top of the Vivado Design Suite. So, once the C/C++ is converted into RTL then we feed it into the Vivado Design Suite and generate a bitstream. And that allows us to provide a comprehensive solution for people starting with C/C++ or SystemC.

Niyati Shah: Now so far the tools I have talked about address the needs of our hardware developers. But for our software application developers, we also have a tool, which is our SDAccel Environment. The SDAccel Environment and Compiler sits on top of the existing Vivado HLS Compiler and the Vivado Design Suite, and allow users with no FPGA background, no hardware expertise, to take their designs and run them on our FPGAs. They will also allow us to support heterogeneous applications. So most of our software application developers are trying to accelerate their designs using the FPGAs. And so the designs will have a software component and they will have a hardware component. And the best example I can give you, is that of computer vision.

Niyati Shah: So I’m sure some of you have security cameras at home. And, in that case, you know that there are multiple parts to that. There is recording live video, but there is also object detection, where it will tell you there’s your dog is running around outside, or say there’s a robber in your house. And in that case, the object detection part is what gets accelerated using hardware.

Niyati Shah: The SDAccel Environment provides multiple different tools to our customers. We have a debugger, profiler, libraries ranging all the the way from low level mat libraries to high performing DSP libraries, but the star of the show is the compiler. The compiler helps us provide a comprehensive solution for both the software parts and for the hardware parts. The compiler compiles the software part so that it can be ran on the x86 machine. And it compiles all the individual hardware components, the kernels, so that they can go onto the FPGA. The kernels are compiled using the xocc compiler, which internally leverages the Vivado HLS Compiler and the Vivado Design Suite to generate bitstream for the parts that need to be accelerated.

Niyati Shah: So once the compilation is complete, then the software part will run on the [inaudible] machine, whereas the hardware part, which are the kernels, will run on the FPGA and they will communicate with used each of their using Xilinx runtime tools.

Niyati Shah: Now this SDAccel Environment and compilers are very crucial and critical building blocks in our next set of compilers, which are aimed towards our data scientists, who provide us models and frameworks such as Caffe and TensorFlow.

Niyati Shah: Our AI Compilers… The object here is to take the models that the customers have provided and optimize them with Deep Neural Network pruning and quantization, so that we can get optimized models. And these optimized models are then targeted using our SDAccel Compiler, so that they can go into the FPGA and generate a bitstream. Finally, we have our SDI Compilers, which provides bitten interfaces. And so we can integrate very easily with the Caffe and TensorFlow families.

Niyati Shah: What I would like to leave you with is that regardless of your background, whether it be software, hardware, or anything between, we at Xilinx have a tool that will allow you to take your design and run it on our FPGAs easily and efficiently. Thank you.

Eva Condron-Wells: Thank you. Our next presenter is Changyi Su.

Staff Design Engineer Changyi Su gives a talk on machine learning platforms at Xiliinx Girl Geek Dinner.

Changyi Su: Thank you for your introduction. My name is Changyi Su. I’m a design engineer from the device power and signaling technology team. My job is focusing on memory interface, timing analysis, and where a [inaudible].

Changyi Su: So today, I will go over one aspect of machine learning, the memory. So I will explain why memory is one of the enabler for machine learning and how Xilinx can be part of the solutions.

Changyi Su: We know that deep learning is one of the many approaches of machine learning, which try to simulate the human brain working with neurons. The neural network has a layered structure, each of the layers, the dot, simulate the neuron. The line between the nodes is a wait. So all the neuron network does is a computation. To compute the output of each of the node by multiplying the wait and the each of the input nodes, and then plug into the activation function. So as shown in this figure, compared to other machine learning algorithms, deep learning algorithms scale up much better with small data. Therefore, the performance of deep learning algorithm is limited by the need for a better hardware acceleration for scaling up data size and algorithm size.

Changyi Su: Recently, FPGA become a very strong competitor to GPUs, to serve as well based accelerator for machine learning. So with a programmable, flexible, how well configuration FPGA often provide better performance per watt to GPUs. Xilinx FPGAs support many types of memory technologies, either internal or external to the device. Compared to the off-chip memory, the on-chip memory has lower latency, lower power consumption, but a higher data bandwidth. FPGA devices offer the industrial leading 500 megabit on-chip memory storage space. So this allow the users to create on-chip memory of real size to suit their applications and also eliminate some of the external components. However, the on-chip memory is very expensive, and hard to expand the capacity. Therefore, the hardware accelerators still have to depend the external memory to meet the storage requirement for machine learning. And also the bandwidth of the off-chip memory is a bottleneck.

Changyi Su: So, with Xilinx’s devices, engineers are able to optimize the memory solution for different applications. For example, for machine learning, the intermediate data… Activation data is usually stored in on-chip memory, to reduce the data movement between the processor and the off-chip memory. The HBM and DDR can be used to store the input data in a right [inaudible], the way to write a parameters.

Changyi Su: So, when we’re working on the memory solution, one big challenge is the trade off between the memory and the computer resources, to achieve the best performance with the lowest latency and lowest power consumption. So, this is a most amazing part of my job in Xilinx. And this makes me a Girl Geek.

Changyi Su: As I mentioned in the previous slide, due to the capacity limitation on-chip memory, how the accelerator still have to rely on external memory to provide the massive storage for machine learning. So over the years, the DRAM Chip density is scaling up. Therefore, the DDR memory capacity upgrade is one of the easiest way to immediately improve the system performance. So we can increase the memory density by using high density DRAM chips, or multiple die package of DRAM chips. Also, the dual in-line memory model–DIMM–is very effective to increase memory capacity with minimum PCP space. So also, DIMM is a model which can turn several DRAM chips on one side or both side of a small circuit board. DIMM can be also config to a multiple RAM configuration to further increase memory capacity. However, with multiple loading, the signal integrity of the memory channel is severely degraded. So the entire system may not operate reliably at higher data rates. Fortunately, most of this can be solved by optimizing the channel configuration with the efforts and expertise from memory design system engineers.

Changyi Su: Over the past 10 years, the DDR memory data bandwidth capability did not evolve quickly enough to keep pace with the bandwidth demanding from applications such as machine learning, video transcoding. So to bridge the bandwidth gap, Xilinx introduce high bandwidth memory, HBM. HBM take advantage of circuit stacking technology that puts FPGA, DRAM, side by side in the same package. So, the whole package DRAM structure together with one thousand data bandwidth, HBM not only can provide extra more storage space, but also enable terabyte per second data bandwidth. So with HBM enabled FPGA devices, fewer DDR components are needed. For some extreme case, like this example, without any external memory components, Xilinx’s HBM solution can provide the same capacity, but much higher data bandwidth, and the better power efficiency.

Changyi Su: So the takeaway of my presentation today is evolving machine learning workloads demand varying bandwidth requirements. Xilinx’s diverse memory technologies enable it. Thank you.

Eva Condron-Wells: Thank you Changyi. And our next speaker and final lightening talk presenter is Uma Madhugiri Dayananda.

Senior Software Engineer Uma Madhugiri Dayananda gives a talk on real time video transcoding at Xilinx Girl Geek Dinner.

Uma Madhugiri Dayananda: Hello, everyone. Today we’re going to explore how real time transcoding can be accelerated on FPGA in the context of data center. Can you guys take a guess on what kinds of media applications these are?

Uma Madhugiri Dayananda: Sorry?

Audience Member: Facebook.

Uma Madhugiri Dayananda: Yeah. Any others?

Uma Madhugiri Dayananda: So, that’s Facebook Live and Twitch Live Streaming, that’s used for gaming.

Uma Madhugiri Dayananda: And, it’s just not these two applications, you also have YouTube Live, and LinkedIn just announced their live streaming application this couple of weeks ago. And it’s not just limited to these applications, live video is just everywhere and it’s growing rapidly.

Uma Madhugiri Dayananda: What’s happening behind these live videos? We have a huge distribution of clients here for example, our cellphones, the tablets, the PC, the TVs, are connected all via wireless networks and each of them have their own resolution and network characteristics. And we want to download the live video to each of these clients, so… To download the live video to each of these clients, the live video input is pre-encoded with the HEVC Encoder, at a different resolution and a bitrate of… And using video transcoding. So, essentially video transcoding is the conversion of one video encoding format into another.

Uma Madhugiri Dayananda: What is the advantage of video transcoding? It provides savings in terms of bandwidth and storage [inaudible]. So how many of you have seen live videos then? You experienced video stars by watching live video? Video consumes a lot of processing resources and power, and for live video applications, latencies involves milliseconds, so…

Uma Madhugiri Dayananda: Here’s a plot that shows the encoder quality preset versus the performance. I’m taking a specific example of x265 preset slow, with the quality preset on it’s supposed to be very good, like when you look at it visually. So, comparing that… If it’s encoded with CPU, you get 10 frames per second, but if the same application is being run on FPGA it’s like 120 frames per second. And for data centered context, it’s not just the quality and performance, you also have to consider the power as well. So considering all the three in the equation, you can see that it is like 72x acceleration.

Uma Madhugiri Dayananda: How does the FPGA solution compare to GPU solution? So here is an example of FPGA based HEVC Encoder compared against NVIDIA GPU. HEVC Encoder and see at the same quality level there’s 35% bitrate savings. Which translates into your bandwidth and streaming cost reduction.

Uma Madhugiri Dayananda: Another reason for using FPGA is probably your transcoding. So the video codec world is changing with the introduction of new codecs every few years. If you the timeline, from 2010 to 2020, there’s four new codecs. We already have HEVC and VP9, and AV1 was just standardized last year, and VVC is going to be standardized next year, so… You have a hardware or a custom chip for each of these codec, if you have that then it’s going to be a lengthy design process and also you can’t use the hardware, so… If you use FPGA, then the applications can be adapted… On the same FPGA you have HEVC application be running and also the VP9 application, so FPGAs are adaptable and reusable.

Uma Madhugiri Dayananda: What is happening behind the scenes? How is the video transcoding happening on the FPGA? We have the live video coming in from the host’s CPU and that’s being decoded on the FPGA using H.264 Decoder, and scaled into multiple resolution using adaptive [inaudible] scaler. And these scaled resolution videos are again to be encoded with the better quality encoder, and sent out back to the host.

Uma Madhugiri Dayananda: Here is this video transcoding stack that Xilinx offers. I work end to end on this pipeline. Building FFmpeg applications, XMA plug-ins, testing these applications on different SDAccel boards, targeting different devices. And not just these, I also work on video algorithms, including the quality, for improving the quality, and benchmarking encoders from partners according to customer requirements, so… I guess that makes me a Girl Geek. I work on things that I’m passionate about, video compression technologies.

Uma Madhugiri Dayananda: What I would like you to take away from this talk is FPGAs give you better performance, better adaptable and reusable, and you don’t necessarily have to be a hardware engineer to use FPGA, you can be a software engineer and still use FPGA to isolate your application. Thank you.

Eva Condron-Wells: Thank you so much, Uma, and to all of our lightening talk presenters. Let’s shift gears to our panel of senior Xilinx leaders. Here to lead our panel discussion is Tom Wurtz, Senior Director of Documentation and Program Management. And our distinguished panel.

Tom Wurtz: So tonight we’re going to explore the topic of acceleration. We’ve got a great panel of senior Xilinx leaders here to enjoy the [inaudible] discussion. So we’re going to start with Jayashree Rangarajan. She’s a Senior Director of Software Development. And her Girl Geek power is simplifying solutions to complex engineering ideas. We’ve also got Lori Pouqeutte, who’s our Vice President of Global Customer Operations, and her Girl Geek power is knowing what the customer wants and needs before they know what they want and need. All right. Next up is Jennifer Wong, and she is a Vice President of FPGA Product Development, and her Girl Geek power is optimizing results for both engineering and management. And finally, we have Ambs Kesavan, who’s our Senior Director of Software Infrastructure, Engineering & DevOps, and her Girl Geek power is improving the development efficiency of using tools both in the cloud and on something.

Tom Wurtz: All right, so let’s talk about this acceleration, that’s a pretty wide topic. We’re going to take it in a couple of different directions. First, computational acceleration. And then we’re going to make it a little bit more personal and talk about careers, as well as teamwork. So, we’re going to start with the hard stuff. So we’re going to geek out a little bit on computational acceleration. You heard Changyi talk earlier about machine learning. So this is things like image classification, motion detect, and speech recognition. So Ambs, I’m going to have you go first, and I’m going to have you talk us through some of the bottlenecks and challenges in this.

Senior Director of Software Infrastructure Engineering and DevOps speaking at Xilinx Girl Geek Dinner.

Ambs Kesavan: Thanks, Tom. I actually view this as an opportunity rather than a challenge. So, I’ll talk about the opportunities here and I’ll explain why I view that way. So we are in an era of big data, and there is a lot of statistics about big data. And I was looking at a recent article that said every single second, we generate about terabytes of data from connected devices and sensors around us, and 70% of this data is video. And that amounts to about 800 million hours every single day or something like that, and businesses are trying to take advantage of this data. They want to mine the data, to be able to look through things. One is for better customer service and also in [inaudible].

Ambs Kesavan: So machine learning applications are getting innovated at that rapid pace, in every single industrial segment, whether it is retail, or finance, or healthcare, Uma talked about video transcoding, speech recognition, name it. Every single industry is going through innovation. And these machine learning applications, they actually have the most algorithms for actually doing the machine learning. And these algorithms, if you run on CPU, that is no longer sufficient. It is not a scalable given the massive volume of data that we are looking at.

Ambs Kesavan: So acceleration is the way to go. And innovation needs to happen both in hardware, also in software, in order to accelerate this machine learning applications and algorithms. And that’s the opportunity, Tom. And Xilinx is well positioned Tom, [inaudible].

Tom Wurtz: Thanks. Lori, maybe you can join in with a few more thoughts.

VP of Global Customer Operations Lori Pouquette talks about opportunities and challenges in supply chain for applying machine learning to the business at Xilinx Girl Geek Dinner.

Lori Pouquette: Sure, and I’ll take this more from the practical application in business. So we’re building a lot of capabilities to accelerate machine learning, but we actually have to apply that too in our business. And in supply chain… There’s quite a bit of opportunity in supply chain for applying machine learning to the business. And it provides good ROI to the business. A couple of the areas that we’re looking at are, what I call predictability or or predictable analytics. And one of the applications would be being able to understand the profile of the die that is coming out of the fab very early on, so you can actually match that to the demand much earlier in the work stream and optimize the use of your materials. Another predictable application is actually in the back end, when you talk about equipment. One of the things that shuts us down is unplanned machine downtime. So if we can anticipate and understand the profiles of the machinery so that we prevent it from going down in the first place, it’ll definitely propel the business. The challenges with all of this, though, are the massive amount of data that you have to gather to really train your models and make sure you have the right algorithms, so you get the ROI out of it.

Tom Wurtz: So there’s clearly a lot of opportunity is this space, so… You see companies like Google and Intel, they’re doing these dedicated AI chips, there’s dozens of startups that are actually going down this path as well. And maybe Jayashree, you can walk us through kind of what you see in terms of where is the market going from here.

Jayashree Ranga: You all heard Ambs talk about the terabytes of data getting generated. And there’s also… She was mentioning about algorithms that need to be developed… And specifically, if you look at the computing today, CPUs generally are meant for general purpose computing. When you talk of AI, and machine learning in particular, you have two types of learning. There is training, and then there is inference, right. And then training, you use all of this data that you have accumulated and we are training certain models to do a particular task, [inaudible] domain, or [inaudible] exploration, or whatever. But then, when you have to deploy that model, actually for doing the inferencing, you want this to be done in such as way that the computation happens a lot faster and the CPUs are not scaling… You probably have heard at many conferences that [inaudible] are not scaling anymore.

Jayashree Ranga: So there is a need for us to be looking at how can we accelerate the solutions that are targeted for these specific applications. So, now you’re seeing companies looking for like… Especially the examples that, Tom you gave about the GPUs and the… That’s primarily because they saw a need for how do I accelerate this? I can’t do it just with software, I need to be building custom hardware. And, it doesn’t just stop the building that accelerator, but you’ll need to have some surrounding support in the system because data that needs to come into that accelerator, and how’s the communication going to happen? So there is a need for these specialized architectures to be built and that’s why are seeing a lot of these startups getting funded to find that next big accelerator architecture that we can build.

Jayashree Ranga: Second thing with machine learning also is you probably are hearing new networks getting created everyday, right. Which means you want an architecture that you don’t build for one network, but two years later you’re not able to use it. So, you want to a hardware architecture that is scalable with the needs. And that’s an area where Xilinx’s provided solution, which is adaptable and reusable, you saw it in the presentations earlier, it provides solutions that people can use for building networks that gets scaled with new models that come in. Especially with the startups that you’re talking about, I’m waiting to see as well, which ones succeed, which ones get taken over.

Tom Wurtz: Yeah. So adaptability, programmability, is part of the Xilinx DNA. So, when we think about programmability, there’s the idea of buying a chip off the shelf and programming it to do what you want. But there’s also the notion of, once that chip is actually in operation, being able to turn off part of it, and reprogram just that part to take on a different workload. And it’s part of the general philosophy of Xilinx, and as we look forward to the next generation of chips, it becomes even more impressive what we’re planning to do, so… Jennifer, maybe you can walk us through what Versal looks like.

Jennifer Wong: Earlier I see a lot of hands go up when we ask “Who knows about Xilinx?” So I saw a lot of hands go up. Then I heard, “How many people are hardware engineers here?” So I do see quite a few hands go up. So I wouldn’t be surprised that some of you here are some of our customers, or you have used our products, it depends… When you were in school… So you must be very familiar with our earlier product lines like UltraScale or UltraScale Plus.

Jennifer Wong: So Versal, it’s a significant step up of UltraScale, UltraScale Plus in terms of performance. So what is Versal? Versal is our first 7 nanometer product, TSMC’s latest process node, and is the industry’s first ACAP. ACAP stands for adaptive compute acceleration platform. But this is truly a platform device, not your old FPGAs. Though I should say this is the really revolutionary architecture. So this revolutionary architecture combines a scalar engine, an adaptive engine, an intelligent engine, to give us this significant performance improvement. The performance improvement can go up to about 20x of today’s GPU, or 100x up to today’s CPUs.

Jennifer Wong: So, now , how do we achieve this kind of a performance improvement. It’s pretty impressive. So, given today’s focus topic is machine learning and compute acceleration, I’m going to talk a little bit about the intelligent engine. Intelligent engine is also known as AI engine. Internally, we have a name called AI engine. So this is specially designed for compute intensive applications, like machine learning and wireless operations. Now go to the next level, what is AI engine made up of? It is really a wide array of integrated DSP engines, which are capable of [inaudible] and complex MAC operations. Now we have all these very very powerful engines. We need to think about how to connect them together in order to take advantage of them in terms of hardware acceleration.

Jennifer Wong: So you have used our products before, you must know MPSoCs. So, in MPSoCs we have a processor subsystem sitting alongside with the FPGA fabric. And these two entities are sitting side by side with some interface in between them, but the bandwidth is relatively limited. So, when they operate, they are operating fairly independently. So the difference between the older generation products and Versal is we added a very powerful NoC engine in between all these powerful engines. NoC stands for network on chip. This is not new in the industry, so NoC is very standard in ASIC. But what we are doing here is applying it to our architecture in order for us to leverage their compute acceleration.

Jennifer Wong: So what is compute acceleration, or hardware acceleration? It is… What it is a design when you can partition the area that are very very performance critical. So you partition it out, and put it into these powerful engines, and use compute acceleration to make the performance improve. And then, after the partition, you can have the slower function continue to run on the processor’s subsystem. And that’s how you achieve the big performance gain. And I’m going to stop right here, there are other innovations in Versal architecture. I’ll be happy to talk to you, if you’re interested, later this evening.

Tom Wurtz: Thanks, Jennifer. Versal is definitely an example of technology escalating at an incredible pace. So I’m going to ask each of you to kind of give some thoughts on where you think the things are going to go in the next three to five years in the acceleration space.

Jennifer Wong: Maybe I’ll just follow up on what I just said. So today’s product, what we have today is pretty big. If you look at our die size, it’s huge, and power is pretty high. So, they’re going to data center, into the cloud. And I see this intensive computation not stopping, because this is in the very early stage and I see that continue to go. But in three or five years, as our process becomes more mature, our technology become more advanced, I see more and more functions going to the edge devices, like mobile phones in your hands today. And I think in future, that’s where it will go.

Ambs Kesavan: So, Jennifer let me add a slightly different viewpoint. We talk quite a bit about computer acceleration. Even tn the presentations we heard about computer acceleration, and here what happens is you’re transferring massive amounts of data from storage to compute, doing that acceleration. And then transferring the results back to storage, and there is lot of data movement happening back and forth. And that’s not necessarily very efficient, you’ll run into [inaudible] storage, and networking. And every single data center, whether it is cloud data center or on ground data center, you’re going to have compute, storage, and networking. So, what if you do the computation closer to storage? So you’re actually doing the acceleration closer to storage instead of doing the data transfer back and forth. And that’s the area Xilinx is innovating and that’s… One example that I can give is the smart SSD announcement, that happened couple of months ago, when Samsung had it’s tech day. And that precisely is doing acceleration at the storage itself. And there is also similar innovation happening on the networking with SmartLynq. So it essentially, it’s converge solution with computer acceleration, storage acceleration, and networking acceleration, and that’s where the industry, I think, will benefit a lot.

Tom Wurtz: Maybe you could talk about the software a little bit, Jayashree.

Senior Director of Software Development Jayashree Rangarajan speaks on a panel discussion at Xilinx Girl Geek Dinner.

Jayashree Ranga: So, if you look at where a lot of this machine learning development is happening, it’s happening on the cloud. And they are the software developers. Throughout [inaudible] Niyati’s presentation, data scientists are looking at these massive amounts of data and looking at how am I going to write the right algorithm to solve this problem, right, and they are operating at higher levels. So what I see happening with the software is many layers of libraries being built, where these are libraries probably optimized to work for the hardware architecture that we’re targeting. Not necessarily done by the data scientists, but it’s provided by the company that is also delivering the hardware. Plus, there’s probably going to be AI specific library. For instance, we talked about video transcoding. So if you have open CV libraries that need to be provided for you. If your are software operator, you will understand this, right. Because we don’t… Anymore write string compares and stuff. You use [inaudible] libraries or SDL. So you are going to see stacks of libraries built, which are the highest level the application developer… Whether they are in a Caffe framework or a FFmpeg framework if they’re dong video transcoding. They’re going to leverage these libraries.

Jayashree Ranga: So I see a lot of innovations happening in that realm. And companies will be providing their own libraries. I see open source development or certain APIs that can be leveraged by people who are trying to address a lot of machine learning problems and various [inaudible].

Lori Pouquette: And then, if you take that from the engineering world to “Where is this all going to be used?” Xilinx has long been serving multiple end market segments, from automotive, to communications, tested measurement, medical, but our new focus now is on the data center area. The proliferation of the data, the video, it’s all going to be needing to be stored up in the cloud or on the premises. So we’ve really now got this strong focus on data center. And as part of that, in addition to selling our semiconductor devices, we’ve also started to sell boards. So we have Alveo product, which you heard Eva talk about earlier, you can see it in the demo room. So this is now making the acceleration capability, from just the semiconductor to enabling the customer with a board. So they can use the board and then go on to their design work for their entire solution much faster. They don’t even have to just start with the FPGA.

Lori Pouquette: Now, as we go out into time, what’s going to be really good for everybody is, we’re still serving all these other in markets, and as in markets like automotive really get into advanced autonomous drive or advanced applications, they’re all going to need to store that information somewhere. And they’re all going to need to process that information, and they may need to do that at the edge very rapidly, or they may be able to do that in a central place. But, basically what we’re doing here with acceleration isn’t just going to serve the base acceleration market, but all of our markets.

Tom Wurtz: Well we got pretty hard there to the extent that we pretty much used up most of our time. But I do want to take one moment to ask each of your, if you were to put into one single word, what does it take to actually accelerate a team, what would that be? And let’s start with you, Ambs.

Ambs Kesavan: Communication.

Jennifer Wong: Teamwork for me.

Lori Pouquette: Focus.

Jayashree Ranga: I’m going to go with two words, I would say it’s the winning attitude.

Tom Wurtz: All right. Thank you very much for all of our panelists.

Eva Condron-Wells: Thank you so much to our panelists and to our lightening talk presenters. At this time we’d like to open it up to the floor. And I’ll need to steal a microphone from one of our panelists, so we’ll share. Pardon me. Thank you.

Eva Condron-Wells: So, you’ve heard a lot of different insights, different perspectives. We’d like to hear a few questions. We probably have time for… We’ll take three. So… And we’ll continue the conversation afterwards. So if you’re still burning… You have a burning question in your mind, please know that we are planning to be here until 9:00, and we have plenty of demos to share too. So… And don’t forget to take your gift on your way out, I want to say that before you mentally leave… While I have your attention, that we have a gift for all of you, that we would like you to take on your way out, in the lobby of when you came in. So, please be sure to take that with you.

Eva Condron-Wells: So, that said, I’ve given you a little bit of time to think about your question, and I have a microphone right here. Come on up, let’s have… If you don’t mind saying your name and your question to our panel.

Sara Biyabani: Sara Biyabani, GridComm CTO. So the question I have, for engineering… We talked about acceleration, so I want to start with FPGA is a [inaudible] processors, you know, they’re part of the screen, right? They’re not the… Well, I mean they could be the star, they could be the diva. So then there’s the processor, right, and you’re not going in the space, competing with x86 or R. So what does that architecture look like? What’s your ideal architecture?

Jennifer Wong: I think we have a pretty good… We do have a processor. So we have a… So for the Versal we have the A72 cores, we a have a dual A72 core in the processor subsystem, in the scalar engine, what we call the scalar engine these two processors, subsystem, you a two A72 cores and then two R5 cores in there. So from a processing standpoint, we do have capability of doing that. And now we added a lot of these architecture that we can accelerate functions. So that is what we think were our niches. So we can partition it where very critical functions, we can put it into the more expensive side. We use a lot of silicon area for acceleration hardware. So we would smartly do that, and say, okay, “What is the important one that we can partition out, put it into hardware acceleration, and leave the processor still running?” So that is what we think where the niche. Everything is on one piece of silicon. So think about it, if you’re doing it outside, you have processors and other components. So, the interface takes a very long time. The key is integration for us. Everywhere, the smaller the footprint, the more integration you do, the more performance you… Both performance and power. So, whenever you got our chip, power is a big deal. So, performance and power both are advantage to integration. And that’s where we think we going in future.

Eva Condron-Wells: Thank you, Jennifer. We have actually three questions just popped up, right all in the same area. So we’re going with that energy, and please, your name and your question.

Sylvita: Hi, I’m Sylvita. First of all, thank you to Xilinx for hosting this event. Given all the other Girl Geek Dinners I’ve been, it’s nice to see somebody on the hardware chip design side kind of take a lead here. So my question is related to that. Because most of the discussion in the big data, enterprise SaaS, or AI has been around the software and the algorithms. The first time I saw a focus on hardware was actually a Startup Grind where one of the presenters talked about , don’t write off the chip guys here, because there’s a next revolution coming where we’re going to see a lot of custom built chips for AI applications. And given that we’re talking about… First of all, this is a Girl Geek Dinner, and given that we’re talking about the fact that we need women to be on an equal footing in the way that the AI is going to evolve, what are some of the ideas you might have for some of the younger folks to continue in this space?

Jayashree Ranga: Mic check? Can you hear me okay?

Jayashree Ranga: I can think of a couple of things, right. Computer architecture, ’cause as I was talking earlier about, hey there’s many architectures that need to… They’re probably waiting to be innovated, right, because we are talking of machine learning in many many domains and stuff. So I do think, as young women who are looking to hey, what do I want to do if I want to enter into hardware, I think having a good grounding in computer architecture principles is going to go a long way. And just learning the hardware aspect, alone, is not going to be sufficient. You also need to understand which domain you’re targeting. So you need to know the end customer that you’re going to be influencing. So you need to learn about the software component, also. So I think, if you want to excel in both sides, as you are getting your early education, having good grounding in both hardware principles as well as software programming principles help you better understand the needs on both sides. But as you go further, and you are looking to specialize, then at that point, it’s an individual style choice as to whether hardware attracts you more or software attracts you more. But I think it’s good to keep your options open.

Eva Condron-Wells: Great. Thank you so much. We’ll take two more questions, and then continue with our networking.

Mung: Okay, and my name is Mung. I’m a software engineer working hardware design company. You already addressed a little bit about that, and I just wonder if you can address a little bit more regarding into FPGA application in machine learning and hardware acceleration to win over ASIC in that field.

VP of FPGA Product Development Jennifer Wong speaking at Xilinx Girl Geek Dinner.

Jennifer Wong: I think the jury is still out, but I think everybody is working very hard. This is a very hard space. And I think everybody is trying a very different route. And Xilinx has Xilinx’s niche. And I think what we give here… What our specialty is, is we are reconfigurable. Aside from being able to partition… That’s definitely a big deal, we can allow software to run on software, hardware fabric to run on past hardware… The bigger part, I think, is the reconfiguring part, which I didn’t talk about earlier. So we talked about many many different workloads today. During the day, the data center can be running one kind of work load, in the evening it’s a different workload. So, what we excel, here, is we allow reconfiguration. So even you can… Within the ACAP, you can run different workloads at different times, with the exact same piece of hardware. So we believe that is a very big niche that we can further leverage in this particular space.

Eva Condron-Wells: Thank you.

Lori Pouquette: I just want to add that, beside the reconfigurability, there’s the costs. So the costs of doing ASIC is becoming prohibitive for many applications. So on higher volume applications it’s still an option. But many of our customers who might’ve classically done ASICS, are moving away from it because it’s just not making financial sense. So there’s that piece of it too.

Eva Condron-Wells: Thank you, Lori. And our final question.

Wolfgang: Hello, my name is Wolfgang. I’m a hardware engineer. And I want to thank you that I have the chance to ask a question even though I am not a Girl Geek. So, my question is about reliability. When we have Ultra high speed computations in applications such as control of industrial processes or autonomous driving, it is not only imperative that those computations are very fast, but the results need to be very reliable. And we don’t always have the option to just [inaudible] processing memory because we have a sensor somewhere, a camera, it sends data to some processor, and that processor has to make a decision and send it to some machine that needs to act and do the right thing. So what are your thoughts about reliability, or maybe a hint, it has to do with something with high speed interfaces and a robustness against big errors, but there are also many other aspects to that.

Eva Condron-Wells: Thank you.

Jennifer Wong: Okay, so reliability has always been a big issue for us. In the past, what we do is we allow [inaudible] reliability and we do things very carefully with our foundry. Because foundry give us some specs that we have to follow through. TSMC is pretty well known in terms of it being rather conservative. So, they give us specs and we obviously negotiate with them. There is a little but of push you can do, because everybody is competing for performance and power. The more you can push the foundry, the more you can gain the advantage. So, this is maybe against what you are asking, but what I’m trying to say here is we do a very [inaudible] balancing act in terms of balance versus reliability. We don’t completely ignore reliability. We go for performance, but we always make sure we can meet our reliability. And we do a very very thorough quality QA in the end. And, also, we have qualification… Pretty substantial qualification… Different time, maybe you can add to it.

Lori Pouquette: Yes. So on the quality side, we have many in markets we serve. So we have commercial grade, we have automotive grade, and we even have an aerospace… So all of those have different levels of quality and reliability qualifications that we go through. So we definitely are very attuned to the upcoming challenges of the technology to be able to respond quickly in those types of situations you’re describing. But we definitely offer a variety, and do quite a bit of different qualifications depending on the markets that we’re serving.

Eva Condron-Wells: All right. Thank you all so much for your very thoughtful questions. And thank you all, our guests, our speakers, panelists, for your insights. We genuinely appreciate you taking the time to share your insights with this team and group. So, that said, we are officially closing our technical talk. But we are not done with our evening yet, and we are happy to stick around and answer more questions. Some of our Xilinx employees are wearing “ask me about” stickers, you’re welcome to engage with them. I will be wearing one too, so find me later. And of course, we have our dessert, and networking, and demos. Thank you all so much. This concludes the technical talk, thank you.

Applause from the girl geeks at Xilinx Girl Geek Dinner after the panel discussion. Thanks for joining us in San Jose!

---

Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at Xilinx and check out open jobs at our trusted partner companies.
• Find more Xilinx Girl Geek Dinner photos from the event – please tag yourselves!
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at Stitch Fix and check out open jobs at our trusted partner companies.
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Hundreds of girl geeks came to Stitch Fix Girl Geek Dinner for the food, drinks, good company and excellent talks from CTO Cathy Polinsky to Principal Software Engineer Erin Dees.

Speakers:
Cathy Polinsky / CTO / Stitch Fix
Emma Colner / Software Engineer / Stitch Fix
Lila Bowker / Product Manager, Engagement / Stitch Fix
Anna Schneider / Data Science Manager / Stitch Fix
Erin Boyle / Data Scientist / Stitch Fix
Bingrui Tang / Senior UX Designer / Stitch Fix
Erin Dees / Principal Software Engineer / Stitch Fix
Angie Chang / CEO & Founder / Girl Geek X
Gretchen DeKnikker / COO / Girl Geek X
Sukrutha Bhadouria / CTO & Co-Founder / Girl Geek X

Transcript of Stitch Fix Girl Geek Dinner – Lightning Talks:

Angie Chang: Hello! Awesome. Thanks everyone for coming out to Stitch Fix tonight. My name’s Angie Chang, I’m the founder of Girl Geek X. It’s been 11 years of hosting weekly dinners at companies around the Silicon Valley in San Francisco. I want to thank you so much for continuing to come back, and also if it’s your first time, I hope you’ve met some amazing people. I learn something new every time, when I come to another one of these events. This is Stitch Fix’s second time hosting this event, and it’s really great seeing them grow, and I’m really excited to hear them speak again.

Gretchen DeKnikker: Hey, I’m Gretchen. How many of you guys are wearing like your Stitch Fix stuff today? Yes. I made sure that I had on all my stuff, and I feel really cute, and I have to say you guys look extra cute. Like you always look cute at the events, but you look extra cute tonight. I can tell. Okay, so how many people is this your first time? Whoa, a lot. Okay. So we actually do these every single week, so make sure you’re on the mailing list, and the next one’s at Cisco, and there’s a few more in the city, Intuit, like there’s a whole bunch of really awesome ones coming up.

Gretchen DeKnikker: We do have a podcast also, so find that on your favorite station. We just did one on uncommon unconventional career journeys, which is a fun topic. But we do mentorship, imposter syndrome, like all of these, so check it out and give us some feedback. Read it, tell us what you’d like to hear about. Tell us how we can be better. And Sukrutha.

Sukrutha Bhadouria: Thanks. Hi everyone, I actually did try wear my Stitch Fix pants, but they don’t fit anymore.

Speaker: We have maternity.

Sukrutha Bhadouria: You do, you have maternity? Okay. I did not get that notification. Thank you. No, but, anyway, the clothes are really cute and fun, and the last time when Stitch Fix sponsored, they spoke about how they used data to correctly identify the styles and the sizing that you need, and it just really blew my mind, how you think, “Oh, I’m just getting clothes in the mail.” But there’s so much thought and engineering and data that goes into it that shouldn’t go unnoticed.

Sukrutha Bhadouria: Speaking of data, we’re also trying to get a sense, so if it’s your first time, how many of you are going to sign up for our mailing list? That’s-

Gretchen DeKnikker: You can decide later. Like we haven’t proven anything, other than that they can get you good food.

Sukrutha Bhadouria: Yeah, but I do want to say it’s really easy to sign up for our mailing list. Go to girlgeek.io, and there’s like one click and you’re in. I do want to say Cathy is my absolute inspiration. We do podcasts, and we also do virtual conferences, and the first time we did a virtual conference she was kind enough to agree to kick it off and do the keynote. And so even that audio and video is available on our website, you can easily access it, and also on our YouTube.

Sukrutha Bhadouria: So enough from me, because there are great talks coming up, and Cathy is going to kick it off. I know Cathy, because she used to be at Salesforce, kicking butt there. And she’s kicking butt here. I’m going to hand it off to you, Cathy. Thank you so much for hosting again.

CTO Cathy Polinsky talks about the company’s product, the people, and partnership at Stitch Fix Girl Geek Dinner.

Cathy Polinsky: Thanks. Thank you guys. Great. Thanks for giving us the opportunity to host. I really appreciate the organization, Geek Girl X, and the mission of really bringing together women from all over the Bay Area to connect, and to network, and to share our experiences. So thank you all for coming here, and especially welcome to people who are coming to their first Geek Girl event.

Cathy Polinsky: How many of you have never used Stitch Fix before? Yay, awesome. Well, you all have gift certificates, and so hopefully you’ll give us a try. But Stitch Fix is an online personal styling service, so we’re disrupting how people can find things that they love, and our mission is to help people feel and look their best. The way that we do this is we pair human stylists with data science to help people discover their looks.

Cathy Polinsky: And so, I mean many of you have tried to buy clothes online, it’s a pretty miserable experience, and what we’re doing is first you fill out a style profile, similar to a dating profile, but about things that you like to wear. We then use dozens of machine learning algorithms to get match scores against all the items in our inventory. We send those match scores to our stylists, and we have over 4,000 stylists who work for us.

Cathy Polinsky: These are employees of ours. We feel really passionate about not replicating the Uber gig economy of contractors, but really these are employees of ours who work from home, and are really passionate about serving our client’s needs. They hand curate a Fix just for you. They pick the clothes, they send it to one of our fulfillment centers. We put it up in a nice box, we send it to your house. You get to try it on at home. You keep what you like, you send back the rest. We pay for shipping both ways.

Cathy Polinsky: The great thing about our model is that we get better and better the more we get to know you, and you share data about what you like, and what you don’t like, and that gets fed back into our model. And so we’re an eight-year-old company, and proud that Fast Company named us one of the most innovative companies last year. Woo! Thank you.

Cathy Polinsky: This is Katrina Lake, our CEO and founder, and I feel really passionate about working for a female led company. She is a really amazing entrepreneur and leader, that I really get so excited every day to get to work with her. It’s a very data driven company, led by a very data driven leader. That really goes into the innovation of what we do.

Cathy Polinsky: We’re eight years old, we’re a newly public company, making over a billion dollars in revenue. We’re profitable. We’ve got business lines for women’s, for men’s, for kids, for plus, maternity. We’re just about to launch our first international company in the UK, so look for the announcement soon. If you know anyone in the UK, let them know once it’s launched and we’d love to have them try out our service.

Cathy Polinsky: I really get excited about our business model, and it’s just very interesting to deal with something that has a tangible product, and a huge operational aspect. I’ve worked at companies like Salesforce and Yahoo and Amazon, and this is just a different aspect of how I’m working with technology, with our tech teams.

Cathy Polinsky: It’s also really amazing what a strong engineering culture we have, and company culture. We have what we call the Stitch Fix OS, it’s our operating system for how our teams work together. And I think it’s really great to be able to work for a company where you can see the values that you have, aligned with how we run ourselves as a company.

Cathy Polinsky: Two things that really stand out to me as a technologist, is one is our value around authenticity. When I started my career as a software engineer all I wanted to do was fit in and look like one of the guys. I didn’t want to stand out as a woman. I wore baggy T-shirts and jeans, and just wanted to be treated as a great software engineer, and to be respected for what I did, rather than stand out as someone who is different.

Cathy Polinsky: There’s a lot of overload that goes in your brain to just try to fit in, to just try to fit in the same box, talk like one of the guys, to play the first person shooter games, or whatever it is that we were doing back then. It is a little exhausting to think about trying to fit in, instead of just getting to work, focus on your work.

Cathy Polinsky: Stitch Fix is a company where we are trying to help people look and feel their best, because we feel like that they can go out and lead more confident lives, and just the feeling you have when you’re wearing that great outfit and being able to be yourself really matters. We also have a mission for our employees to do their best work, and a lot of that comes down to being their authentic self. So not having to feel like they have to double check their email three times, or phrase things to seem less emotional, or more powerful, or whatever codified words that it might mean to kind of fit into that mold.

Cathy Polinsky: And so we have authenticity as one of our Stitch Fix values, of letting everybody be themselves, and thinking about the culture additive that brings to the organization, so that people can focus on their work instead of just trying to fit into the mold.

Cathy Polinsky: Then the other thing that we have a really strong value is around partnership, so it’s a complex business model that we have. First it starts with the merchandise. We own all of the inventory in our system, that we’re selling to our clients, and if we don’t buy the right inventory, and the right quantities, we’ll never serve our client’s needs. Then we’ve got this huge workforce of stylists and making sure that they have all the tools that they need to operate at their job every day.

Cathy Polinsky: Then we have a huge operational aspect of how we get those products to the clients in the right way, and manage all of our inventory and our costs. And then we have this whole business with our website, the style profile, and engaging products that want you to come back to our site every day, so that we can get to know you better and better.

Cathy Polinsky: If you think about all those aspects, there’s … I’m always surprised by how little changes on one side of those have deep impact on other sides. We could make some changes to how many items we send in a Fix, and that could change the inventory allocation that we have available for the next person. And so it is one of the most partnership driven companies that I have worked for, in that you really have to think about not just your own area, but how that could have an impact across the company.

Cathy Polinsky: We thought what’s really interesting about our business is how strong of an EQ we have here at Stitch Fix, and how that has really led to our innovation and success as a company. And that we really strive for that when we’re hiring technologists, so people who can think not just about building something to spec, but really thinking about understanding the business model and how they can work together across different lines. Whether it’s a data scientist working with an engineer, or someone of the design team working with our marketing team.

Cathy Polinsky: And so we’ve got a theme today of partnership that we’d like to share with you. Talking a lot about some of the interesting projects, but really leaning into how you can use that aspect of your skills to really be a big success. I’d say one of the things that I learned about partnership came from a big misstep, I would say, as an early engineering manager. So going back to this feeling of authenticity, you only know what you see, and so being in an organization where I saw mostly male leaders, I tried to emulate a lot of them in my leadership style. It didn’t always feel comfortable, but you just try to do the things that you’ve seen to be successful when you run into tough trouble.

Cathy Polinsky: A few companies ago I was working on a project, and my team was getting pulled in different directions, and I felt like, okay, I’ve seen the way that guys handle this, and they pound on the desk, and they really fight for their teams to make sure that they’re not being jerked around, and that they’re getting the staffing and the support that they need. And so I tried to do the same, and I pounded on the desk, and I yelled in meetings, and I said that this was just really unacceptable for how we could get something done.

Cathy Polinsky: And it didn’t work, and I kind of failed miserably at making the changes that were really needed to work on this project in a way that was getting clarity on the architecture and the designs needed at a large scale. It was like, I don’t understand why this isn’t working, it works for other people, I’ve seen managers do that in the past. But I have to say, I had some self-reflection, it didn’t feel good to yell in meetings. It wasn’t successful and it wasn’t me, and it just made everybody really miserable in the process.

Cathy Polinsky: And so the next time I went to a new company, I was like, okay, never again, I’m not going to try to be someone I’m not, and to try to get through this with anger and yelling. I’d say that I learned a lot in my next role, of really starting to build relationships. So how can I build relationships upfront, build trust, so that when we have difficult situations, instead of it getting to a point of anger it came to a conversation. I found that I really developed my leadership style, because I leaned more into my authentic self and led into building partnerships. Because I think that when you have those partnerships, you can get a lot more done.

Cathy Polinsky: I see that here every day at Stitch Fix and hope you’ll see some great learnings around partnerships that we have here today with some of our speakers. So without further ado, our theme is around technology and partnership, and we’re going to pass it off to Emma. Yeah, Emma, come on up. Where’d she go? There.

Software Engineer Emma Colner gives a talk on “Mind The Gap: How Our Brains Fool Us into Thinking We Understand” at Stitch Fix Girl Geek Dinner.

Emma Colner: I don’t know if you want your wine that’s up here, Cathy. All right. Hi everybody, my name is Emma Colner. I’m an engineer here at Stitch Fix, and I work on expert use systems. That means I build tools for my co-workers that help them do their jobs more efficiently. But in a previous life I was also a former experimental psychologist, so I really enjoy thinking about thinking, and today I’m going to be talking about how experts think, and how that differs from how novices think, and the different implications that can have for when we are collaborating together at work. Oh.

Cathy Polinsky: I stole this.

Emma Colner: Thank you. I only have two hands. So I’m just going to hold this. Okay. Like Cathy was saying, we value partnership a lot at Stitch Fix, and I think it makes a lot of sense to really try and understand how we can bridge this gap between how experts think and how novices think, and what they know. But also it’s a great opportunity for us to harness each individual person’s expertise. In that way we can kind of learn and grow together as a company.

Emma Colner: To start, I’d like you to think about someone that you really admire, someone whose skills and knowledge you really look up to, and someone that you might want to emulate one day. Imagine that this building here on the left is a representation of all that person’s knowledge about a certain topic. Now, if you were to learn how to do what this person does, how would you do it?

Emma Colner: Well, you could try and just copy what you see, but you could also … You’re not even sure of whether the building is going to be structurally sound, so what we don’t see is that this building started out as an idea, and a series of discussions, and lots of back and forth plans before there was even a foundation built. What I’m trying to say is that basically when we look at an expert we don’t see the path that they took to get to where they are.

Emma Colner: We don’t see all of the hypothesis that they tried and tested. You don’t see all the doubt that they experienced, so it’s just important to keep in mind. Part of what makes being a good partner a challenge, is that it’s impossible to truly know the experience of someone else. We can be excellent observers, and we can infer a lot about someone just by looking at them, but we don’t know everything. Part of the difficulty is that as people we naturally just fill in the blanks. When we don’t know something, we infer based on stuff that we already know.

Emma Colner: In some situations this can lead to some false understanding, so to succeed at partnership we need to work hard to bridge that gap between minds. I included this quote here on the right from Domain Driven Design, by Eric Evans, just because I thought it was relevant to what I’m talking about today. I’m not going to be talking much more about it, but I’m still in the middle of reading it, but it was a really good introduction. I recommend it.

Emma Colner: For now, I’m going to be describing three different scenarios that I have experienced in my software engineering career, and what they can teach us about our brain’s natural limitations, so we can become better communicators, problem solvers, and business partners.

Emma Colner: In scenario one I’m a junior software engineer, I’ve just started at Stitch Fix, and I have a mentor who’s a principal engineer. We pair pretty much every day, and half the time I don’t know what he’s talking about. He’s speaking in a different language, and also I don’t even know how to phrase the questions that I’m trying to ask. I don’t know how to phrase the search terms that I want to put in Google to understand what I’m trying to do. So that’s the first scenario.

Emma Colner: Second scenario is, in this scenario I’m the expert. I built something, I built a new feature, I’m trying to get it to work, but there’s a bug that I just can’t figure out. I’ve spent many hours on this, and finally I decide to ask someone for help. But then as soon as I explain the issue to that person, the answer just kind of pops out at me, and the other person didn’t have to say a word.

Emma Colner: And then the third scenario, collaborating with business partners. So let’s say I’m working on a new feature with my business partner. We’ve met and talked about the project several times. We meet each week, seems like we’re all on the same page, but at some point it becomes obvious there’s been some kind of miscommunication, and the project doesn’t end up as we had expected.

Emma Colner: What do all these scenarios have in common? Well, they all demonstrate a gap between what we as experts think we understand and what we actually understand of someone else’s domain. When experiencing cross-functional teams, this difference in expertise can cause friction and lost productivity. I want to advocate for a solution of adopting a beginner’s mindset. So now the neuro scientist is going to come out of me, and I’m going to have a very, very simplified explanation of how learning works in the brain, and how differences in expertise can lead to different outcomes.

Emma Colner: When we learn something for the first time it’s an effortful process that takes a lot of mental resources and attention, so that’s what those red scribbles represent. Over time, as we change from novices to experts, our brains become more efficient as memories consolidate, and unnecessary information is forgotten. The representation of information shifts from the sensory regions to cortical regions of the brain that operate more heuristically and more efficiently. That’s what all the green squiggles are meant to represent.

Emma Colner: So the more knowledge you have on a topic, the more associations you have built up in your mind, and the greater the network of brain areas that are involved while working on a problem. What do I mean about our brains becoming more efficient? I’ll share with you a study called The Development of Expertise in Radiology, and they basically showed that expertise can reduce the complexity of the environment. They did this by showing chest radiographs to novices and expert radiologists and they tracked their eye movements.

Emma Colner: They were told to detect some kind of an anomaly. You can see on the left that … The red represents more time looking at a certain spot, and green is less time. What we see is that novices are just kind of looking all over the place. They don’t know what they’re looking for. They’re spending a lot of time just kind of lost, whereas the experts, it seems that their attention is automatically drawn to the important aspects of the image.

Emma Colner: So that actually, the knowledge that they’ve had over their experience has helped reduce complexity and made the problem easier to deal with. Novices, on the other hand, are using more rudimentary tools. They might take longer, or it might be harder to solve the same problem. Most of the time being an expert works to our advantage, saving us time and energy, but in certain cases, like in the three scenarios I talked about earlier, it can be a handicap.

Emma Colner: Why might that be? Well, the price of expert efficiency is that the scaffolding, or the context surrounding when you first learned something, has been forgotten, probably by the time you’ve become an expert. So just as your brain actively consolidates memories it wants to keep, it forgets most of our daily experience, so over time we only remember the important stuff. Sometimes the only way around a problem is to work through it from the bottom up, starting with basic concepts and building up your understanding, rather than starting from an existing mental model and working down.

Emma Colner: That way we’re forced to think more deliberately, which helps expose weaknesses in our logic, and in other words it helps to just adopt this beginner’s mindset. Circling back to the different scenarios, on the left, that represents my mentor. He was working with a full-fledged Lego set with pieces that all fit together in a sequence that makes sense. And that’s actually like Legos. Whereas I, on the other hand, was working with a bunch of wooden blocks and playing around, trying to stack one idea on top of another, hoping it doesn’t fall down.

Emma Colner: I think what might have happened with my mentor is maybe like he had lost the scaffolding, he had lost the context of when he’d first learned a certain topic that he was trying to explain to me, and so it becomes harder to kind of connect with someone who has such a different skill level.

Emma Colner: And then in scenario two, I was basically describing rubber ducking, which is a debugging method where you basically explain your code to some inanimate object. It doesn’t matter what you explain your code to, but it’s surprisingly helpful in letting you know what it is that you’ve done wrong. The reason it’s so helpful is because you’re forced to approach a problem from a different perspective. You’re building up and filling in the scaffolding that you had lost previously, and that can help us gain some new insights.

Emma Colner: And then in Scenario three, when I describe how I’m working with my business partner, where we didn’t fully connect on our vision, what happened was we both had like a false understanding of the problem and/or the solution. We’d both made some kind of assumption about each other’s work without even thinking about it, because we were both experts in our own domains, and our brains are filling in the details of things that we might not understand fully.

Emma Colner: So, yeah, really, this picture is a joke, but it just demonstrates how easy it is to misinterpret things that might seem really easy. Lastly, I just wanted to return to the skyscraper and reconstruction metaphor for mental models. So while it’s being built up, there’s scaffolding all over the place, allowing workers to place one brick on top of another, but when construction is done, the scaffolding is taken away and all that’s left is a perfect shining tower.

Emma Colner: It can be hard to remember how we arrived at a conclusion once the scaffolding is gone. The next time that you’re collaborating on a project with someone of a different background, remember that you don’t see the path that they took to get to where they are, and it’s often necessary to spend the time to translate, describing one person’s solution in a language the other person understands. Or better yet, coming up with a common language together. Thank you.

Lila Bowker: Thanks Emma. Hi, my name’s Lila, I think I met most of you when trying to sort out the name tag situation up front. I’m just here to make the transitions less awkward, but there’s no guarantee that that’s actually what’s going to happen, so I have notes. Thanks, Emma, for the reminder of how continuous learning and kind of taking a beginner’s mindset is incredibly important as we work with our cross-functional partners.

Lila Bowker: Next up we have Anna Schneider. She’s a manager on our merch algo’s team, and she’s going to talk about how she partners with experts in merchandising to help make our buying better. I’ll give you that. Oh, you have LaCroix. How are you going to switch slides with LaCroix?

Data Science Manager Anna Schneider gives a talk on “Transforming the Way Merchants Find What They Love” at Stitch Fix Girl Geek Dinner.

Anna Schneider: I’m going to put it down, is what I’m going to do. Hi. Yeah, so I’m Anna. As Lila said, I’m a data science manager here. I’m going to talk about a project that I worked on that is very similar to the scenario three that Emma was just talking about, where you’re working with a cross-functional partner, and you think that you’re solving the same problem, and it turns out there’s a whole different kind of problem, and a whole different kind of solution that was needed.

Anna Schneider: So when I say buying better here, I’m not talking about how clients buy better stuff from Stitch Fix, I’m talking about how buyers who work at Stitch Fix buy better stuff to send to clients. So digging in a little bit more, we have a team of people called buyers who are Stitch Fix employees, and their job is to figure out what we should be stocking in the warehouses. That determines the pool of merch that then the stylists can choose from when they’re deciding what to send to a particular client.

Anna Schneider: And upstream from the buyers, the buyers work with vendors to figure out what they should be stocking. So by working with the right vendors and buying the right things from the vendors, the buyers have a huge influence on the end experience that the client’s have, by making sure that we have really good stuff in the warehouses. That’s going to be good no matter who shows up as a client who wants a Fix.

Anna Schneider: If you think about what the buyer’s day would look like, an old school company or at Stitch Fix in the early days, this is what a buyer would do. They would look at a list of things that are on offer from the vendors, and because if you’re lucky to work at a place like Stitch Fix, there’s going to be some performance metrics associated with each one of these items of clothing. And so the buyers would look through a list like this, pick the things near the top of the list, and that’s what we would buy, and that’s what the clients would get sent.

Anna Schneider: This has a number of problems. One of them is that it’s kind of like a standard e-commerce experience for the buyer, so they have to like search through all these lists and hunt and peck, and decide what merch to be carrying. One of the big values of Stitch Fix is that we don’t make our clients go through that experience of doing all the e-commerce shopping themselves, so why are we making our in-house buyers go through that experience?

Anna Schneider: We thought there must be something better. And another more subtle problem with this way of buying, is that by only looking at one single performance metric, you’re only buying for the average client. So let’s dig a little bit into why this problem comes about.

Anna Schneider: Say we have these bunch of clients and we’ve sent them these shirts, and we have some performance metric for each of these. So maybe they all hate the yellow shirt, maybe the gray shirt does really well, and the two green shirts are like, nah, kind of in the middle. If you were a buyer looking at this data, you would think that, oh, the right thing to do is to buy a bunch of the gray shirt, and not buy the others. Seems good. Seems fine.

Anna Schneider: Now, what if we dig into the data and pull out some niche client segment that has different preferences than all the others. The top line shows pretty similar data to before. Those people still like that gray shirt, but the client segment on the bottom likes something else. They like the striped shirt on the end. Now if you’re a buyer looking at this data, the right thing to do is to buy both of these shirts, that way no matter who shows up there’s going to be something that’s really good for them.

Anna Schneider: And this is our mission at Stitch Fix, is to have something that’s going to be good, no matter who shows up. And so it’s really important for us to be enabling the buyers to do the job of buying a really diverse set of merch, that’s going to be really good for a diverse set of clients. So we, on the algorithms team, stepped back and thought about how can we give the buyers this data in order to make better decisions.

Anna Schneider: We thought, well, if giving them one list is bad, what if we give them multiple lists, one for each client segment. So algorithms, and engineering, and merch all collaborated to build a tool that did that. There was multiple lists, so every item of clothing would have how good is it for this client segment, how good is it for this other client segment, and the buyers would go through and choose things from the top of all the lists.

Anna Schneider: This was great. So was it going to be a better experience? We hoped so, but no. It was better at accomplishing the goal of having something that’s good for a lot of different kinds of people, but it made the hunt/peck problem way worse, because they had to dig through a whole separate list for every client segment. They were working harder instead of working smarter to accomplish this goal.

Anna Schneider: And they were like, well, we can do it with a handful of client segments, maybe, but Stitch Fix wants to get to thousands of client segments, millions of client segments, and this was never going to work. Is not a scalable solution. Something really interesting happened as we in Algorithms were talking with the buyers about all of their struggles using this tool. They started seeing some things that should have been obvious, and now that we learned them, it is things that we really care about taking into account.

Anna Schneider: They’re running a real business, and they didn’t give us this feedback in rap battle format, but they almost may as well have. So when they are thinking about what to buy, they have a huge number of other things that they’re keeping in mind, not just this one metric. In addition to having the right diversity across clients, there’s also things like having different price points, having different size and fit preferences.

Anna Schneider: We want to make sure that we really cover all of our bases in those kinds of ways. We’re running a profitable business, as Cathy said. We care about hitting our revenue and margin targets. Buying is something that happens on a seasonal cadence, and so we want to make sure that the stuff we’re buying is going to be good a few months from now. And there’s also other targets around the sort of inventory level management, so are we buying too much and we won’t be able to sell it? Are we buying not enough and there’s going to be missed opportunity?

Anna Schneider: The buyers were thinking about all of these things in their heads while using our tool, and not being able to fully actualize in their role. When we heard all this feedback, we stepped back and thought about, okay, is there a better way that we can be addressing this problem? There’s this performance metric that we want to be maximizing, and there’s all of these constraints that we have on the business. What does that sound like? And we go and like dug around in our algorithms toolbox and said, “Hey, that sounds like constrained optimization.”

Anna Schneider: So we reformulated this as a constrained optimization problem. Our decision variable is the number of units of each item to be buying for each client segment. We choose those units in order to maximize the predicted performance of the whole assortment overall. So we add all that together, as subject to all of those constraints that we talked about on the previous slide. It took a fair bit of work to formulate all of those as equality or inequality constraints, but we were able to get close enough for most of them, through a lot of partnership and talking with merch about like, “Hey, so when you want to meet this margin target what does that really mean?”

Anna Schneider: We were able to get all these formulated. To solve this we’re using a open source Python solver called Pyomo, and it’s been working pretty well for us, even though the documentation is pretty bad. If you’re interested in checking it out yourself, I would recommend googling this blog post by one of our other data scientists who is using Pyomo on a different team within Stitch Fix algorithms. It’s way clearer explanation than like anything else on the Internet.

Anna Schneider: Then in order to make this algorithm available to the buyers, we built a tool that they use to interact with it. So from their perspective, they actually specify values for all of these constraints within the tool, and then press go. And what happens then is that our algorithm, running Pyomo, combines those constraints with a separate algo’s predictions of how well each item is going to perform for each client, and spits back out the recommendations.

Anna Schneider: And so within just a couple of seconds the buyers get the whole assortment recommended returned back to them. This has been a way better experience for everyone involved. So our goal of having something for everyone, this achieves. We’re optimizing over all of the client segments at once. And something that has been really interesting to work on is that it totally changes the contract between the buyers and the algorithms, so the buyers are now responsible for having the right goals for the assortment, entering the right targets, and the algo is responsible for figuring out the best way to achieve those goals.

Anna Schneider: And so it’s almost like the experience of getting a Fix, where as a Stitch Fix client you’ll say like, “Hey, I want a cute dress for a wedding next month,” and the stylist will go off, and the stylist algorithms will go off and figure out the best way to achieve that goal for you. Now, the relationship between the buyers and algorithms is much more like that, instead of the standard old e-commerce experience.

Anna Schneider: Some lessons that I’ve learnt through this that could hopefully be transferable to some other business context, like whatever you’re working on. One is that algorithms are good at tactics, and people are good at strategy. In the old versions of this tool, the people were responsible for the tactics of digging through all the lists, and nothing within the tool was responsible for the strategy. Now we reformulated the tool so that the people are doing the strategy and the algo is doing the tactics, and that’s a much more powerful human-in-the-loop algorithm.

Anna Schneider: One important enabler of allowing people to handle the strategy, is being able to capture that strategy for the tool, and so that’s been the really interesting user experience problem, because sometimes the buyers knew explicitly what their targets were, and sometimes it was all just implicit in their head, and so that’s been a fun partnership experience too, is figuring out a good UX to tease those implicit intentions out of the users, and make them explicit.

Anna Schneider: If we were only working with collect telemetry kind of data, instead of asking like, “Hey, what do you actually want?” it would have taken much longer to figure out what some of these targets were. And last but not least, especially if you’re working in like B2B or enterprise, your users are running a real business, and their jobs are way more complicated than you realize, no matter how much time you spend getting in the weeds with them.

Anna Schneider: So in my experience, at least, it’s really worth getting into the weeds to try to figure out how their processes actually work, and then abstracting it back to something that can achieve their goals even better than they thought and you thought that could be possible. So thanks.

Lila Bowker: Yeah, thanks Anna for explaining how you use data science to make our merch buying process better. I’m going to look at my notes again, sorry. Next up we have a dynamic duo, half of which is already up here. Come on over. All right, so we have product designer, Bing Tang. Oh-oh, there goes my notes. And data scientist, Erin Boyle, and they’re going to explain how we use data to understand client style, and then how we use that to inform stylists to send better Fixes. Take it away, guys.

Data Scientist Erin Boyle and Senior UX Designer Bingrui Tang give a talk on “Partnering on Style” at Stitch Fix Girl Geek Dinner.

Erin Boyle: Thanks, Lila.

Bingrui Tang: Okay, hello everyone. My name is Bing, and I work on the UX design team, [inaudible]. And this is Erin, she works on the data science team and working on AI instruments, and today we’ll be … being the dynamic duo, and talk about communicating style together.

Bingrui Tang: I think Cathy already talk about this a little bit, but at Stitch Fix we really have this human-in-the-loop process. We have algorithm empowering the stylist’s work, where a stylist will actually make the call. And if we actually put our foot into a stylist’s shoes, it is a very challenging work. Imagine you have a client who is there in a 50s Fix, and their profile might haven’t been updated for a while, and their style preference has been changing over time. It is really difficult for a stylist to dig through all the things, and do we really want to empower them to use their creativity to really send a delightful Fix to our clients?

Bingrui Tang: So on the backend side, a lot of our work is really to design for the styling platform, so, as you can see here, on a platform we would combine the client data which has some basic information. Their style preference and their Fix history, as well as how the algorithm is recommending the items of a variety of categories, and decides by going through all the information about a client, they would actually be able to pick the items that they think would fit the client well, which might not always 100% fit with how the algorithm sorted out.

Bingrui Tang: So for us, a big part of our work is to make sure the way we present the client data is really helpful for the stylists, so they can really do their job well. Previously, the way we represent a client’s style is more aesthetic, so for those who have been using Stitch Fix before, when clients sign up they would rate outfits of different styles, and we will translate this summarized data into some different formats, to the platform, so that the stylists will see and be able to understand what the client wants.

Erin Boyle: Okay. Cool, thanks. The data science team that I’m on works on contributing to a broad set of problems, kind of in this category, that rely on having some more nuanced definition of style. So we actually built a new platform to try and support this. Can I ask, has anyone in this room played style Shuffle? Got a few hands. Okay. Style Shuffle is a kind of game-like experience that we released a little over a year ago, where clients can rate items in our inventory, thumbs up or thumbs down.

Erin Boyle: It’s been really fun and engaging for them, people really like giving us data like this. We have more than a billion ratings now. You can imagine that, like Bing said, one way that you can represent someone’s style is with these kind of like limited number of static questions that we used to have in our style profile. But you can imagine that there’s a few limitations to that, like one, it’s just not very much data, so it’s not going to have a lot of nuance, and then like Bing said, it will also get stale over time.

Erin Boyle: So since people can continually play this game, we can keep collecting more information and get kind of richer, and richer, and up-to-date information about our clients. It’ll come as a surprise to no data scientist in this room, that one way you can deal with data of this format is with an algorithm called matrix factorization. If you want to come up with descriptions of a client’s, what we’ll call, latent style preferences, one thing you can do is you can treat the ratings coming out of this system as kind of sparse observations in this user item matrix, where every row is a user, and every column is an item.

Erin Boyle: You can decompose this matrix into a lower rank co-representation that is composed of the product of this lower dimensional user embedding matrix, and lower dimensional item embedding matrix. When you’ve done this, what you have learnt, you’re learning the coefficients in these two matrices on the right. What you’ve learned are what we call latent factors, that describe client preference. So the kind of columns in this user matrix and the rows in this item matrix are going to describe … they’re going to be kind of like hidden underlying variables that describe a big sources of variance in the preferences that we see from our clients.

Erin Boyle: If you want to get a prediction back out for any of these user item pairs that we haven’t actually observed. If you want to get back out a prediction of how well a user might like some particular item, all you need to do in this framing is simply take a dot product of this user and item vector to recover a kind of score for that pair.

Erin Boyle: In reality, we might actually expand on this, and use a more nuanced algorithm, but I think this is a good kind of framing to understand it. What can we do? So one thing you can do once you’ve trained an algorithm like this, is you can simply pull out the top recommendations for any client. Here I’ve sampled five real clients form our clientele, and I’ve also sampled some of the top items, given this prediction task for these clients, and you can see that you really uncover using this algorithm, like a range of different aesthetics. You can see that these people are all of different style.

Erin Boyle: That’s one thing that we can provide to technical and business partners like Bing, to paint a picture of our clients. But we are interested in also providing something else, so just like from the style profile, you have this set of user features that comes out. We’re also interested in whether the user representation, like just this user vector on the left here, was kind of in and of itself useful and interesting, and whether we could use it as a feature that we could communicate through language as well.

Erin Boyle: One way you can do that is to really think about this kind of style space that you’ve created when you run this algorithm. So let’s say we ran this algorithm with just three latent factors like we’ve done here, just three columns in the user matrix, that means that once we’ve learnt coefficients for this user, they have a location in this kind of XYZ space. They have an address. We’re curious, does that address itself kind of convey information?

Erin Boyle: One way you can dig into that and try and come up with a language to describe this user representation from this algorithm, is you can actually look at these axes, and you can see what they represent. We’ve found these latent factors that predict client preference, but what do they mean? Like what’s going on? One thing you can do, an item that is good for this user … Since our prediction task is done by taking a dot product, an item that is good for this user is going to be an item that is near this user in this style space, and an item that is bad for them is going to be far away.

Erin Boyle: One thing you can do, is you can look at how the items change across some particular axis, to come up with what this axis is encoding stylistically. When we do this for this one X axis, you can see that this is sample of items on the left, and they have this kind of boho style to them. And this is a sample of items on the right, and maybe you would call them more preppy.

Erin Boyle: And you can then go through and do this for the major, what we would call, principle components of this style space in order to really explain what this user’s representation means. There’s a couple little technical details if you want to do this in real life. I actually have a blog post on it, if you want to dig into that.

Erin Boyle: But basically what this gets us to, is that our team can kind of partner with people like Bing, to provide these intermediate data products, that they can plug into whatever their domain is within the company. One thing might be actual recommendations. One might be the client’s representation. Maybe on merch they might care about the style’s representation instead. What we try to do is just provide people with whatever products are going to be useful for their application.

Bingrui Tang: Thank you. Thank you, Erin. I know it’s very magical, isn’t it? I remember when I first got on the project, I was like, “Ooh, look at that.” So, yeah, that’s come back to what I was talking about, designing the styling platform. Once I wrapped my head around this project and going through many meetings with our cross-functional partners, there’s two big questions sitting in front of us.

Bingrui Tang: One is, how to make sense of latent style? Because, as Erin just said, it is a very mathematical model. But we want to make sure it is something that somebody who doesn’t really know statistics, or doesn’t really know much about math, could still understand it in some way. And more importantly, we want to make sure that it could stylists actually make decisions, and with confidence.

Bingrui Tang: So for these two questions we actually conducted two different studies and I will just quickly go through them, and share with you guys. The first one, as Erin just said, and she actually showed these two clusters of clothes too, I think most people would see, “Oh, these two are different styles.” But it’s very hard to say why, or how, or what, or exactly, so this is also how we started the study, because, as a human, when we see a bunch of things sitting together with similarities, we just have this natural tendency to give it a name, a label, a theme, or something like that, so that was how we started.

Bingrui Tang: Let’s use this, two extreme maxis as example. The first moment we saw it, we said, “Oh, the left-hand side looks very classic. The right-hand side look trendy.” Yeah, something like that. But because we thought this is coming from the mathematical model, we should really think about it very objectively. So we say, what is the most visible objective factor? Let’s say, the left-hand side is very less skin exposure, it’s much more covered, and the right-hand side looks more skin exposure, what about that?

Bingrui Tang: And then we show it to the stylists and also other people on the team, and then there has been a lot of voice raised, because everybody’s like, “Yeah, there is skin exposure, but the print seems different, there is seasonal implications. The color scheme is different. The fit is different. There is different levels of embellishment. Which one is the most important? How do we kind of …” Then there’s a lot of debate around that.

Bingrui Tang: At that point, the team and everybody, we actually started stepping back a little bit and think about what is style in reality? This is one quote from the movie The Devil Wears Prada–I still remember when that quote come out in a movie–but I think essentially it is talking about what style was really manifested over time, from the cultural and the historical influence. For us, it is really important to recognize and embrace that instead of trying to invent some new way of talking about style overnight.

Bingrui Tang: At the end we actually come back to our original idea, and actually showed it to stylists, and see how they reacted to it. Do they recognize these words as representative of these clusters? And seems like the old way, the classic and trendy, actually worked the best.

Bingrui Tang: For the second study, I think Erin also showed this image before, now that we know we can get a sample set of items that seems to be into the client’s preference, and we can also get a relevant location of the client preference in this spacial model, how can we display the client’s style representation in the styling platform so that the stylist can actually understand it within a short amount of time?

Bingrui Tang: The goal for us is to find the right way to display each client’s style, to help stylists actually style a Fix. So, again, oops, let’s use this image as an example. Imagine this is client 123, that’s her style preference sample. So we tried a few options again. The first one, we said images only, here’s what she likes, and you can interpret it as much as you want. And we were not feeling quite 100% confident with it.

Bingrui Tang: And then the second option is the more mathematical one, we say, “Her style is 53% trendy, and 28% boho.” And we definitely know it is a little bit too extreme, so we also pulled back a little bit, and look at this, the happy medium. So we would say, “Her style is very trendy, and a little bit boho.” And now we have three options later, we actually ran a quick study, and we are surprised.

Bingrui Tang: We compared a bunch of different factors, from how accurate the stylists are able to find what the client likes and figure out what the client dislike, and also how fast they are.

Lila Bowker: Try that.

Bingrui Tang: Okay. Oh, thank you. And also how the stylists feel about using this feature as an experience, and seems like the image only one actually won all of them. And from the result, seems like, at least during the experiment period, we learned that displaying the images only has the best overall outcome.

Bingrui Tang: So when we look back at the long journey, we realized at the beginning we were trying to display as much information as possible and tried to put massive amount of data in front of stylists, and then we pulled back a little bit, and then we realized we can actually let the information speak for itself, instead of we try to add more things on top of it. Which reminds me of a quote from the very famous designer, Dieter Rams. Her work inspired a lot of Apple products, and he once said, “The good design is as little design as possible.”

Bingrui Tang: I think that really speaks … the philosophy’s really representing how we work here.

Erin Boyle: Awesome. So I guess I’ll add to that, is that … Ooh, hold that thought.

Lila Bowker: Keep talking. Keep talking. Just pretend there’s the lady behind the curtain over here. Oh, no.

Erin Boyle: This collaboration was really interesting for me, too, because A, it was really interesting to learn that it’s hard to summarize aesthetics with language, like I think of language as being very rich, also, but in this case it wasn’t quite up to the task. Our brains are really good at processing images, so that was a good thing for us to learn kind of broadly, even outside of your use case.

Erin Boyle: And then also, I still got the benefit of having language that experts had applied to this space for cases where you really need language, like you can’t always show a collection of images for every kind of use case, and so we do still have language that we’ve gotten from a bunch of the work that Bing’s team did. So, this is an example of one of many technical partnerships that we do here, and thank you for listening.

Bingrui Tang: Thank you all. So I believe most of the teams that are representing here are hiring. Lila will speak more about that, but design team is hiring, so we can talk later. Thank you.

Lila Bowker: Thanks, Bing and Erin. How they takes styles, shuffle data, and turn it into better Fixes for clients is one of the reasons that I joined the company. I love it. Next up I wanted to introduce Erin, she’s a principal engineer on our Fix request team, and she’s going to talk about the importance of giving feedback, and some strategies for how you can give feedback more regularly. Go for it.

Erin Dees: Hi, friends. Is it on?

Lila Bowker: Try this one.

Principal Software Engineer Erin Dees gives a talk on “Dossiers of Awesome: One Way to Help Folks Get the Recognition They Deserve” at Stitch Fix Girl Geek Dinner.

Erin Dees: Hi friends, how are you all? One more talk on the theme of partnership before we all get some networking time. The focus here now is going to kind of narrow in on the personal partnership, because as strong women we are told that we need to lift one another up. But how? Now part of the answer lies in helping our peers get the recognition and the visibility that they’ve earned.

Erin Dees: It’s about giving meaningful feedback that lifts them up, and also helps you. So this talk will be … The title, Dossiers of Awesome, is just a way to frame these habits and practices. It is not a complete solution to how to lift one another up. It is not a universal recipe. It’s an idea. What I’m really hoping with sharing it, is I get to hear your ideas afterwards. So let’s talk during networking time.

Erin Dees: My name is Erin. I joined Stitch Fix about two months ago as a principal engineer. These are my goats, it’s the day I brought them home in my Kia.

Lila Bowker: I’m so sorry, we’re having … Try this one. I don’t know, I’ve mixed up which one was the good one.

Erin Dees: Can everybody hear me now? Is this better?

Lila Bowker: Is that better?

Erin Dees: Okay.

Erin Dees: Thank you, friend, sorry about that. All right. You didn’t miss anything. Something, something goats. All right. Okay. I really enjoy working on a big thorny systems engineering problems, the kind where you have to reach out across teams, learn from industry, learn from one another, and then package that learning up and bring it back to your team. That’s my jam, like sharing knowledge like that. That’s why I write programming books. That’s why I coach athletes how to race walk faster. It’s not why I have goats, but they are very good listeners.

Erin Dees: This talk came out of a conversation that I had with a former colleague, Liz Abinante, a few years ago. We were talking about this series of stories we’d read, prominent women leaving their posts in the tech industry because of being passed over for opportunities, and being harassed. I said to Liz, “How do I get better at observing, at noticing? I would hate to think that something like this happened to somebody on my team, and I didn’t even see it. How do I catch this happening?”

Erin Dees: And Liz said, “People aren’t going to harass in front of an audience. Right? They’re not going to harass in front of witnesses. So, that’s not the way. If you want to help your teammates, that’s not really a great way to do it.” And I said, “Well, how do I help my teammates? Half my teammates are women, how do I … I like them, I like working with them, how do I hang onto them?”

Erin Dees: And Liz said, “Help them get the visibility and the recognition that they have earned.” The conversation branched out from there, we brainstormed a lot. But I want to pause here for a second, and say that even though this idea came out of a difficult situation and a tough conversation, it’s going to apply in a lot of different scenarios, and I also am aware that a lot of the people in this room are already doing a lot of emotional labor for their teams. The last thing I want to do is give you all more homework.

Erin Dees: I really want to talk about this in terms that will help you in your careers as well. How many of us are in a job where we are expected to give feedback on our peers regularly? Right. A lot of us, right. How many people kind of dread that time of year when you’ve got to go write a bunch of peer performance reports, right? It’s exhausting. It takes forever to write, and by the time you’ve done your fourth or fifth one, it’s hard to come up with something that is unique that could only apply to that engineer.

Erin Dees: Like, “Well, you’ve built some great products for us, and keep learning more advanced Ruby skills, I guess.” Like I mean, right, that could apply to everybody. So how do we get feedback that helps that specific person get better in their career? And that’s what we’re going to talk about. So in order to achieve this goal, whatever process we adopt should be lightweight, because we’re all busy and we’re just not going to do it if it’s too much of a burden on our time.

Erin Dees: It should be something that we keep up with in little increments throughout the year, instead of having a big deadline dropped on us. And again, it should be actionable, it should give our peers information that they can use to grow their career. One idea had been sitting right in front of me this whole time, which is an engineering journal. Now, I had started keeping this a few months prior, because we had formed a brand new team, and there was so much learning for all of us, that we really had to capture what we’d learned and what we’d done in some way, and my solution was to keep a journal.

Erin Dees: I tried to do it every day, ended up doing about 50% of days, and here’s something like what that looked like. 4:30 p.m. every day right before the end of the workday, a blank window pops up on my desktop, it’s time to write in your journal. And I spent just a few seconds, literally just a few seconds typing in a couple of bullet points. I implemented this feature. I fixed some tests. It’s okay to get snarky, it’s your own journal. Nobody’s going to read it. And the twist here then, was to realize that there’s no reason I couldn’t put anybody else’s accomplishments in here.

Erin Dees: Sometimes my journals talked about stuff I’d done with my teammates, but it was time to get systematic about it, and use hashtags and stuff. And so that’s what I started doing. So in addition to what I’d worked on, I might add a couple of bullet points that a teammate worked on, and tagged them with a hashtag, which comes in handy later. So then if you’re in a culture that does sort of quarterly feedback cycles, when it comes time to do this, I can click on that person’s tag in my journaling software.

Erin Dees: This is totally real data you all. I did not just type a bunch of this stuff at 1:00 a.m. in my hotel room last night. So then you can either copy and paste, or kind of paraphrase, but you start with a blank document, and you can paste all this data in here. And you start to notice as you do this that these items kind of fall into patterns. These few items seem like they’re about incident response. These couple of items here seem like they’re about tech leadership, and so you can group them. You can start moving them around and you can add some headings.

Erin Dees: Now, then what you do with this information depends a lot about your feedback culture. If you’re in a place where you’re expected to write your own review, for starters, a self-review, you can give your peers the ammo, the raw material that they can use to write their self-review. So here’s what that might look like, you can compose an email, and if your manager is someone who’s supportive, write it to them and Cc your friend or your peer. Well, I hope they might be your friend too. And this now tells them a story. If they’ve been waiting for a great opportunity to write a promotion pitch for this engineer, you’ve just given them all this ammo.

Erin Dees: But the audience is also your peer, because if they’re going to be writing their self-review, it’s going to be a bunch of stuff that they may not remember in the moment that they accomplished that quarter. That said, though, do please let the manager know what they’ve done as well, because there’s a lot of cultural pressure on us not to brag, and we should fix that too. But if this is all stuff that happened, this isn’t bragging. It’s data.

Erin Dees: It’s a good idea to share it with … again, if you have the supportive manager, and with your peer. So how to make this feedback actionable, so that somebody can act on it and grow their career? One way to do this is to work these data points into a story, so it’s not just data, it’s a narrative. What this looks like, for example, if you start noticing this person developing or showing an aptitude and interest in tech leadership, is to call that out, and say, “Hey, maybe it’s time to start handing this engineer larger projects and have them run bigger initiatives. They seem to have a knack for it.”

Erin Dees: So I want to pause, just one second, and say that we’re at a women’s conference, and we’re talking through this originally through the lens of a conversation about women in tech, but there are lots of teammates that we have that are dealing with other marginalized identities, some multiple marginalized identities. So I want us to keep in mind all of our teammates who might be marginalized along one or more axes when we think about lifting one another up.

Erin Dees: That is one way that we can have an impact on our careers and our peer’s careers, is lifting them up. But again, this helps us do our jobs better. If we’re expected to give feedback, and we can do something that gives better and high quality feedback more quickly with less overhead, that helps us too. That makes us recognized for giving good feedback that really helps people, and that’s the impact I’m hoping that you all will have, no matter how you all choose to do it.

Erin Dees: So I’m really grateful to Girl Geek and Stitch Fix for putting together this event. Lila, this has been amazing. I love it. I have loved every talk I’ve watched. The other presenters, you all are amazing. I’m really grateful to Liz Abinante for this original conversation, and Lila and Miriam for helping me, appropriately enough, with meaningful actionable feedback about this very presentation.

Erin Dees: As we prepare to head into networking time, I want to come back to that cultural expectation that we don’t brag, and I want to chip away at that just a little bit tonight. I’d like to invite you all, as you’re introducing yourself to other people, to say something that you’re awesome at, work related, non work related, doesn’t matter. Lead off with something you’re skilled at. And if that’s one notch too extroverted for the comfort level tonight, maybe ask the person what they’re awesome at. Just know that they might turn around and ask you, be prepared for that, it’s okay.

Erin Dees: As we were brainstorming for how to lead into networking time, as a group of presenters, Lila brought up these two articles about the importance of owning your awesome, of owning what you’ve built. I really want to embody that spirit tonight, if at all we possibly can. It’s been hugely inspiring to be in this room full of awesome badass women in tech. I’m really grateful to be up here. I hope you all have a great rest of the event, and cheers.

Erin Boyle: Thanks, Erin. That was amazing, thanks, Erin.

Audience Member: I have a question about the buying process, like particularly more like the supply chain issues you might face. How do you, if you end up with an extra load of clothes that don’t fit anybody, how do you steer away from trying to push product on people just to sell it, and what do you do with your unsold inventory?

Anna Schneider: Yeah. There are several algorithms that help with that, unsurprisingly. So, yeah, we have clearance algorithms that figure out, hey, this thing isn’t performing very well, and then sometimes we will get rid of it through donating it, so that’s one common way that low performing stuff will leave our inventory. Although when we’re doing that, we do want to make sure that we’re not getting rid of stuff that’s like bad for a lot of people, but like really good for someone. We do want to keep that niche stuff around.

Anna Schneider: So with the client’s expectation, like I was talking about, like that’s like something that we’re always trying to figure out how to get better at.

Cathy Polinsky: But I think one of the things that I really appreciate about Stitch Fix is that we separate out the merchandising team who buys the product from the stylist team that sends the product to the right people. So we have intentionally created this firewall between the two, so the stylists are never incented to send bad product out to their clients.

Cathy Polinsky: They don’t know, “Hey, we’ve got a lot of those lime-green shoes out there that aren’t selling, can you just send it out to people.” Instead, we really try to make sure that the stylists are incented to really keep their clients happy, and if we buy bad product, we eat the cost ourselves, and learn from it for the next time, to make sure that we don’t make the same mistake over and over again.

Cathy Polinsky: It is this interesting system where sometimes actually getting rid of bad product helps everybody else up, just making sure that that bad product is not inadvertently getting into Fixes. And so we’ve seen these times where we have changed the dynamics in the model, of how often we’ve gotten rid of bad product, and we’ve now learned that it’s not a good idea to hold onto it, but it’s much better for us to get fresh stuff in. Was a good question, though.

Lila Bowker: Nice. All right. I’m getting my cardio in, you guys, hang on. There you go.

Audience Member: Hi, I’ve a question with respect to recommendations. How do you deal with surprise, because you can learn someone’s style, but often when someone really, really likes something, it might be because it’s a little bit outside their comfort zone, and it’s one item. Do you like work that in by humans or algorithms?

Erin Boyle: Yeah. That’s a great question. And I should say that the exact answer to that might change, depending on what context you’re talking about. So, probably you’re asking about in someone’s Fix, and in that case a lot of that would be done by the human stylist. We do have algorithms that try to think about assortment, but we mostly rely on stylists, or … Yeah? Yeah, stylists would definitely be injecting a lot of that.

Erin Boyle: The other thing I’ll say is another place where you might use recommendations, I mean we have recommendations everywhere, but another place you might use them is like in the stylish level itself. Of course there are recommendations that are fed back to the client, and certainly there is some like assortment logic and experiments and stuff that have gone into like injecting surprise into that experience too.

Cathy Polinsky: But we try to get employees to style, as well as the stylist, and so I try to style a couple Fixes a month, and occasionally I’ll get someone that says, “Stop sending me skinny jeans,” and the recommendations only knows that this client buys skinny jeans, and so the stylist has to think of like, “Well, maybe I’ll send her a boyfriend jean, or a boot cut jean.”

Cathy Polinsky: Our algorithms have no idea what they’re going to want next, based on their previous purchases, but we’ll have to use this … we like to call it the blend of humans and machines. The art and science of what we do is sometimes there is this kind of stylistic creativity that goes into generating a Fix. It’s kind of fun.

Erin Boyle: Well said.

Bingrui Tang: I was also going to add on that, because sometimes there will be clients say, “Oh, I really don’t wear dresses,” and then all of a sudden they’ll say, “Oh, I’m going to a wedding, then I need a dress.” So it’s very important for us, when we design the system, we really keep in mind this kind of flexibility, and people’s preference will change, either occasionally or over time, and we really want to recognize that. I think that’s a really good question, and that’s definitely the fun part of the work, is trying to juggle them both. Yeah.

Lila Bowker: All right, let’s start here.

Audience Member: The question is about the buying process in the old traditional way. One of the reason they’re using that method is because probably the cost optimization, the more you buy, the cheaper it is. But buying optimized by diversity using algorithm, how you deal with the cost optimization to the supplier?

Stitch Fix girl geeks: Cathy Polinsky, Erin Boyle, Bingrui Tang, Anna Schneider and Emma Colner answering audience questions at Stitch Fix Girl Geek Dinner.

Anna Schneider: Yeah, so in the constraints that the buyers will put into the problem, one of them is exactly around that. We know how much we’re going to buy it at wholesale, and we know what our margins will be because of that, and often the algorithm will say like, “Hey, this one has really good margin, buy even deeper into it.” So if that’s something that we want to be … Yeah, yeah, it’s something that’s just rolled into all of our other data.

Anna Schneider: That’s something that we … Margin isn’t the only thing we care about by any means, there’s all these other metrics about making the experience really right for the clients, and so that’s what we are … currently it’s formulated, where we’re trying to give the best experience to the clients as possible, constrained by having a profitable business. Yeah. So the client is really first.

Audience Member: Sure. One of the things I’ve always struggled with with these try before you buy services, is that the things I like aesthetically don’t always match the things that look good on me. I’m curious what you guys think about that, and how you are imagining how the Stitch Fix product addresses that challenge?

Erin Boyle: Yeah. I mean that’s a tough question. I think there is kind of a fine line between aesthetic and fit. I mean even in this latent style data, which we think of as being largely a style thing, like people are rating whether they like an item and it’s a very visual thing, they’re not necessarily seeing the size of that item that they would actually buy, or the can’t tell the inseam or whatever.

Erin Boyle: And yet we do see fit preferences coming out of that data, like we can figure out some fit preferences too. Yeah, I mean I think we certainly try to collect information on both style and fit in the style profile, and through other means, and then certainly the client has a conversation ongoing with the stylist, where they give the stylist feedback on the details of what does and doesn’t work for them.

Erin Boyle: And then those kind of like subtler pieces of feedback that really take a human to interpret can be acted on by that stylist.

Cathy Polinsky: Anecdotally, I see this. I have some clients that have an Instagram feed, and I’m like, “Oh, they liked this item, we have it in inventory, I’m going to send it to them, even though the match score is really low for the product.” And I send it to them and they hate it, but I’m like, “You said you wanted it.” But there is this notion of what they like stylistically, versus what is good on them, and so I’m really impressed with how these match scores that we have factor that into the recommendations.

Cathy Polinsky: And it’s not like, “I don’t want that navy blazer, I want a navy blazer that looks good on me.” And so how can we take the style things that people are sending us, and pair that with other factors, like fit, to find the exact right thing for you. It’s a hard problem, and that’s why it’s so much of a pain for people to try to shop online and scroll through lists and lists of jeans, to figure out what’s going to fit them.

Cathy Polinsky: So I think that this is a different model that we can use, that can factor in a lot of those different attributes.

Audience Member: I found it really refreshing to see multiple senior female engineers on this panel, and I was wondering what your approach was to sourcing and retaining female engineers. Sort of like what percentages of your teams are female, with regard to engineering specifically? And did that change as Stitch Fix got larger?

Cathy Polinsky: We are really fortunate for the gender representation that we have at Stitch Fix. It helps that we have a female CEO and founder, Katrina Lake, who has started this company. Half of our leadership team is women, more than half of our board is women, and a huge representation of our stylists are women, so if you look at our employee count, which is predominantly stylists, it’s over 80% women who work at Stitch Fix.

Cathy Polinsky: And then our technology organization is really strongly represented. We’re in the high 30s for representation of women in our tech organization. It’s still not 50/50, and still opportunity for us to grow, but compared to every other tech company that I’ve worked at, we are really leading the pack at having a very strong gender diversity in our teams.

Cathy Polinsky: We hadn’t done it through top down quotas or mandates, but really it was generated by teams and managers who cared about this. Of understanding that diverse teams build better products, that we had primarily women as our clients, and to really understand them and understand the products that we’re building, having those diverse teams helps us to make sure that the things that we’re building are really strong and supporting that client base.

Cathy Polinsky: What was great for me to see, as we were scaling out our organization, it’s hard when you’re hiring fast, to make sure that you’re thinking about diversity and all of the different criteria, but we only got better as we scaled. It really came down to the managers who didn’t have that diversity on their teams, were asking the other managers who were doing well, like, “How did you do that?” And, “How can I do that?”

Cathy Polinsky: It came through inclusive language in our job postings, or thinking about sourcing in a different way. We talked about experiences that we had to make sure that we had diverse panels, and then we also look for really product centric engineers at Stitch Fix, and I think that that helps us generate a more diverse pipeline of what an engineer looks like throughout our organization.

Cathy Polinsky: We’re not done. We still really want to hire more architect level senior individual contributors here at Stitch Fix, that’s an area that we don’t have as much diversity. And then we’re also looking to increase our racial diversity, and think that we are really leading the way on gender diversity, but have some more way to go in other aspects of diversity, like racial diversity.

Cathy Polinsky: But I love that we’re a group that cares about this, and we talk about it, and we celebrate where we are doing well, and also understanding that we’re not done.

Erin Dees: May I jump in for a second?

Cathy Polinsky: Yeah.

Erin Dees: I joined pretty recently, and it was one of the things that drew me to Stitch Fix, was seeing that there were so many women in engineering specifically, and then as you look up, senior leadership, in terms of like management, there’s a lot of women in senior management as well. That was something that really shines through, so one reason I think that Stitch Fix has so many women in engineering, is because we’re drawn here. We see like, this is a good place for me. None of us wants to be the only woman on the team, so it’s been awesome.

Cathy Polinsky: Great. Thank you. Glad to have you.

Lila Bowker: Another question over here.

Audience Member: Hi. I have a question on I guess the outlook on how accurate you think your algorithms are becoming over time? Are they becoming more predictive? Do you think that we’ll get to a point where the algorithm can handle both the strategy and the tactics? And if we are at that point, do you think that Stitch Fix would ever move to a model where maybe there is less reliance on human stylists? If yes, why? If not, why?

Erin Boyle: I’ll let Anna speak to the strategy tactics, because that’s definitely your framing. I will say they’re definitely improving all the time. Obviously we can’t give you any quantification of that, but I’ve actually been here almost four years, and it’s shocking how much better we are. You can see it in so many ways. So, yes, they’re improving all the time. I still expect them to improve more, but I don’t expect it to really change anything about the role that people are playing.

Erin Boyle: I think that the role our stylists play is always going to be critical. It’s a critical part of our business model, they play a critical role, and similarly with our merchandising partners, and so many other people, so yeah, I’ll let Anna speak to the strategy and tactics. But that would be my reaction to that.

Anna Schneider: Yeah, one analogy that we’ve started using for the strategy versus tactics, is are we building a self-driving car, or are we building something like Google Maps? I’m mean for a self-driving car that’s just like all about the tactics. It’s like … can be completely automated. There’s no human in the loop there, whereas something like Google Maps, it doesn’t even make sense to think about what that would mean if it was fully automated, because who’s telling it where to go? You need someone giving the instruction.

Anna Schneider: And so there’s always going to be that company leadership and leadership at levels all throughout the company saying, “Where are we going?” And that’s always going to be a human decision, and so we think of it almost more like a scenario exploration engine, where you can say, “Hey, what if we went in this direction? What if we went in this direction?” And then the humans have the opportunity to choose what of all possible futures we want to be going after, and that’s always going to be a human touch.

Cathy Polinsky: So any chess players in the room? Yeah. Was it Kasparov who was the chess player who lost against Deep Blue? So we look at the sense of … At that time it was like this is the end of chess, and the age of the machine, and machines are going to always be better than humans, and what was it? So that was when the first chess game played against a human and won.

Cathy Polinsky: But what’s interesting is after that point, there was a new emerging game that was this freestyle chess. Have you guys heard of freestyle chess? So the freestyle chess, is this idea that it doesn’t matter whether you’re human, or you’re a computer, or a blend of the two, but anybody can play against anyone.

Cathy Polinsky: So they have this competition every year and there’s a freestyle likeness, and the first year that they did this a novice group of chess players won using computers, and they did it in a very novel way, of using a suite of machines to solve the problem. It was this really interesting thing, it was the blend of humans and machines. Humans that had an interesting approach for how to solve the problem, but they were backed by computers.

Cathy Polinsky: And so we use this likeness of what Stitch Fix does. It’s not computers alone, it’s not stylists alone, but it’s this blend of humans and machines that work together in a novel way, to solve a problem in a unique situation. And so we feel really strongly about this model and how it’s helped us, and sometimes we’ll lean a little bit more on the machine side, sometimes we’ll lean a little bit more on the human side, but regardless of where that line is, we feel like the power of the two together is really a magical thing.

Lila Bowker: Awesome. We do want to leave enough time for folks to chat afterwards and brag about themselves and hear what makes everyone else awesome, so maybe one more question, is that all right with you guys? All right, here you go.

Audience Member: Hi, sorry, one of the last questions. A lot of fashion is traditionally geared for women, so I was wondering if you have seen patterns that are maybe encouraging males to be a little more adventurous in their stylistic choices? Just out of curiosity, you know, I think … I mean, I don’t know, but personally I have brothers, that I would like them to be more adventurous, and it’s hard to get them into the space.

Audience Member: I don’t know if you have encountered patterns or had strategies, or even have some vision for even working with fashion industry to sort of expand, that fashion’s not just for women, but also for people who traditionally are not associated with fashion?

Cathy Polinsky: We started out as a women’s only business, and for the first five and a half years we only had women clients, and then about two, two and a half years ago, we launched our men’s business. It’s been really great to see that business growing and thriving. I’d say, I don’t know if it’s specific to gender or not, but we see some clients that start out and they have a specific sense of what they’re looking for, and as they use a stylist, over time, they get to be more adventuresome. And they might try new things and get some of these serendipitous things in their Fixes that they never would have picked off of a rack before, and try it on.

Cathy Polinsky: You know, we see this with our women clients, but I think we see it a little bit more so with our male clients. And I think that that is just kind of a fun thing about our service, is that something that you might never try on, something that you may see in a Fix, we don’t even show you what’s coming, because we don’t want it to taint your view of the Fix before it arrives at your doorstep. And sometimes there’s something that you never would have like wanted in your Fix, that you try on, and I’ve had this happen to me as well, that I’m like, “Oh, this looks amazing.”

Cathy Polinsky: And so I think that that aspect of discovery is a really amazing part of our service, that it works for those folks that may be stuck in their style, whether you’re guys, stuck in a rut, or even a gal, yourself. And so it’s kind of a fun part of our surprise and delayed-esque model. I hope I answered that.

Lila Bowker: Bing had a client recently that was awesome. Bing, you should tell that story.

Bingrui Tang: Oh, yeah.

Lila Bowker: Remember your client?

Bingrui Tang: Yeah. So I had a client who is probably in his 40s and 50s, and I got his first Fix, and he said, “Oh, yeah, so my style is a little bit conservative, really boring, but I want to make my wife happy, so I really want to stretch it out a little bit.” This is his first Fix, and he such high expectation, so I’m really nervous, and I share it with the team, I say, “Yeah, what do you guys recommend?”

Bingrui Tang: And somebody actually recommended a conversational piece, which is essentially pink flamingo prints. Yeah, and zebra prints, and things like that. So I actually sent a bunch of them to him for the first Fix. He returned all of them. I think I definitely went too far, but he said, “Thank you for all the pieces you sent, I think it’s a little too stretched for me, but I really get idea, and I really want to try it again.”

Bingrui Tang: And he actually immediately scheduled another Fix, and so assigned me as his stylist. So for the second Fix, because now I know where he would be more comfortable with, I actually pulled back a little it, and I think he kept all of the five pieces. Yeah, so I think, back to your point, I really think, yes, I’ve sent many, many pink shirts to male clients, and a lot of them end up keeping them, so I think, yes, the discovery part is really the fun part.

Lila Bowker: Awesome. Well I think we want to open it up so everybody has the time to chat and brag about themselves, and hear what makes everyone else awesome. The quick plug of, of course, we are hiring. If you work at Stitch Fix, can you raise your hand? Everybody who works at Stitch Fix, stand up and raise your hand. Yeah, so if you want to chat more about the roles we have open, aim for one of those human beings that just raised their hands.

Lila Bowker: But I’m excited to get to know more of you better. Enjoy. Yeah, and thanks for coming.

---

Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at Stitch Fix and check out open jobs at our trusted partner companies.
• Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!