Engineering Leadership Perspectives: Kimber Lockhart-One Medical CTO, Jen-Mei Wu-Architect, Arquay Harris-Slack Director of Engineering, Rachael Stedman-Engineering Manager (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Angie Chang, Sukrutha Bhaduoria, Arquay Harris, Jen-Mei Wu, Kimber Lockhart, Rachael Stedman

Girl Geek X founders Angie Chang and Sukrutha Bhadouria with the engineering leadership panel – featuring Slack Director of Engineering Arquay Harris, Indiegogo Software Architect Jen-Mei Wu, One Medical Kimber Lockhart Kimber Lockhart and Lever Engineering Manager Rachael Stedman at a Girl Geek Dinner!

Arquay Harris / Director of Engineering / Slack
Jen-Mei Wu / Software Architect / Indiegogo
Kimber Lockhart / CTO / One Medical
Rachael Stedman / Engineering Manager / Lever
Angie Chang / CEO & Co-founder / Girl Geek X
Sukrutha Bhadouria / CTO & Co-founder / Girl Geek X

Transcript of Engineering Leadership Girl Geek Dinner – Panel:

Angie Chang (Girl Geek X): Hi! My name is Angie Chang, and I am the founder of Bay Area Girl Geek Dinners. Thank you for coming out! It’s our 147th Girl Geek Dinner tonight. I wanted to bring together strong engineering leaders to talk about their careers and journeys.

Sukrutha Bhadouria (Girl Geek X): Hi! I’m Sukrutha. By day I work at Salesforce as an engineering manager, and by night I show up with Angie at Bay Area Girl Geek Dinners. We get a lot of requests when people are on the wait list to see tweets so they can follow the conversation.

Angie Chang (Girl Geek X): I brought this panel together because we see [job] titles all the time like architect, and CTO, and director of engineering, and we don’t know what they mean from company to company, so I wanted to ask Jen-Mei about what an architect at Indiegogo does day-to-day, and broadly, how you got there…

Software Architect Jen-Mei Wu speaking at a Girl Geek Dinner.

Jen-Mei Wu (Indiegogo): My name is Jen-Mei. I’m an architect at Indiegogo. What I do… I’m on the engineering leadership team along with the directors of engineering, and the vice president of engineering, and I don’t manage people but I manage technical direction and technologies that we use.

Jen-Mei Wu (Indiegogo): We have two architects at Indiegogo and I focus both on fronted architect and platform based things like CDMs and the cloud. I’m kind of like a sandwich, and I’m the bread, and the other architect is the filling (he does the backend, which is in Ruby). In the past, I have spent most of my career in management. I was the director of IT and director of engineering for many years, and I started a consulting company which I ran for many years. When I shut down my company and wanted to be a regular worker bee (and not worry about making sales and making sure people get paid), I ended up joining another consultancy that a friend of mine (Sarah Allen) started, and I was director of engineering… and we were acqui-hired by Indiegogo, where I was an engineer, and became a product manager, and now architect. I got here in a non-traditional path.. my major was english and history in college.

Sukrutha Bhadouria (Girl Geek X): We were catching up with each of the panelists backstage. The first time I met Kimber, she was the VP of Engineering, and now she is the CTO [at One Medical]. I’m curious — what is your day like as CTO, and what skill set you acquired along the way that you put to good use now as the CTO?

Kimber Lockhart speaking

CTO Kimber Lockhart talks about her role at One Medical at a Girl Geek Dinner.

Kimber Lockhart (One Medical): CTO is probably the most varied title in all technical roles, and the reason for that is that it can mean different things. At a startup, the CTO can be the first technical person on board, or it can be the technical founder. The CTO can be the most senior person on the engineering team that manages people. Sometimes the CTO doesn’t manage anyone and is the chief technical architect and decision-maker.

Kimber Lockhart (One Medical): I am none of those kinds of CTOs. In the healthcare space especially, often the CTO is the person in charge of technology — which I guess kind of makes sense? So I am in charge of our product management, design, and engineering teams — as well as some of our technical support, IT and security. And before that, I was actually VP of Engineering at One Medical and had the experience of stepping into the CTO role and taking on a bunch of functions that I had never managed before, figuring out how to do that, and then bringing on a very good VP of Engineering to fill the role that I formerly had.

Kimber Lockhart (One Medical): What skills did I develop along the way… I started my career as an entrepreneur, which was an amazing way to learn leadership the hard way, went thru all of the ups-and-downs of running a small company and ultimately ended up selling that company to Box. I joined Box as the 43rd person (12th engineer) and that company I think was about fifteen-hundred or so by the time I left… absolutely explosive growth, and with that growth came the growth of my management career. I had the opportunity to learn to be a manager, with some really great guidance, and ended my career there as a director before I went to One Medical.

Angie Chang (Girl Geek X): Arquay, can you tell us about what it’s like to be a director of engineering at Slack and how you got to that role?

Arquay Harris speaking

Director of Engineering Arquay Harris speaking at a Girl Geek Dinner.

Arquay Harris (Slack): I am a director of engineering at Slack, I’ve been there about a year and a half. What do we do… Slack is a very early stage startup but we’ve had a lot of hyper growth. When I joined the company, I was a senior engineering manager and I had two reports. I had a very small team, and that was totally fine with me — I wanted to work at the company and believed in what they were doing — and very quickly, I got more and more responsibility and after six or eight months, I got promoted to director of engineering.

Arquay Harris (Slack): A lot of what I do day-to-day is essentially building engineering processes, because I come from a much larger company before Slack (about sixty thousand people who worked there, and so there’s a process for everything) and coming into a new engineering organization, a lot of what you do is paving the roads, making it so that you can make and build teams to be efficient and do their best work possible.

Arquay Harris (Slack): How I came to Slack… I also have a pretty non-traditional background, I actually started college as a mathematics major, I was going to be a math teacher, yay math, but I was introduced to visual arts by an after school job and so I learned to do Photoshop and Illustrator and liked that work, so I ended up transferring schools and changing majors to do media arts and design — that’s everything from making movies and doing sound and my concentration which is digital illustration.

Arquay Harris (Slack): So I got into coding because I didn’t like the process of designing something and handing it off to someone else to code, and I had this very analytical math background, so I just taught myself. And then I ended up continuing that learning when I went to grad school, and there, I did even more coding, fine art painting, 3D.

Arquay Harris (Slack): I am a developer and I also have an MFA. And I think I have to continually have that conversation about “I have an MFA but I also code!” I like this idea of form following function, and I think that’s kind of how I came into tech. After grad school, I worked at CNET, a very old media company, and a company called Google, and then Slack.

Sukrutha Bhadouria (Girl Geek X): Rachael, you are next. We definitely want to hear about your journey. You are now an engineering manager, how did you get to this point, and what your day is like.

Rachel Staedman speaking

Engineering Manager Rachael Stedman speaking at a Girl Geek Dinner.

Rachael Stedman (Lever): I’ve been at Lever for almost three and a half years now. I actually joined as an engineer. About two years in is when Nate asked me for the first time whether I wanted to be an engineering manager, and my first response was, ‘nope’! Basically what had happened is, being an early engineer at an early-stage startup (I joined when it was about ten people), you start taking on more and more responsibility as the company was growing. I moved from building features to building features that involved refactoring the entire data schema and running migrations and doing it all without the users noticing — so kind of moving down the stack… and so I was working very closely with the infrastructure team and we were looking to hire an infrastructure manager.

Rachael Stedman (Lever): Hiring managers is a difficult thing to do well, especially at a startup because they have a huge influence on the culture, so you need someone who is very aligned with the companies values and who is also interested in doing the engineering management, and we had been searching for a very long time. In the absence of having a manager, I started taking on those responsibilities — so they asked me if I wanted to do it full-time, for real.

Rachael Stedman (Lever): The first time I said no because I was afraid of missing programming — I enjoyed being an engineering — and I enjoyed the management (that’s why I started doing those related responsibilities) and I was really afraid to let go [of programming]. So I went back to being an individual contributor for a while and then he asked me again three months later and e’d came up with two different paths — he said here, if you want to focus on the technical stuff, here is a really cool technical project that you can focus on but you would have to let go of these things that you have taken on, or you can go down the manager path which I really want you to do.

Rachael Stedman (Lever): I really appreciated him laying out those two options for me, because that made it seem like I was choosing, and not pushed, into management. I considered and asked, ‘if I don’t like management, can I go back?’ He said ‘of course’, so I ended up choosing that, and I’ve been a manager now for over a year. I manage our backend and infrastructure teams.

Rachael Stedman (Lever): What that looks like day-to-day… I’m currently reading Radical Candor, and the way Kim Scott defines what a manager is supposed to do is guide teams to achieve results. So the guidance part is a lot of listening and having conversations with people, and listening to what is being said and not being said, and providing input and asking questions to get people to arrive at their own conclusions. The ‘achieve results’ part relies a lot on communication and finding alignment between company goals and how your team contributes and making sure everyone on your team understands how they contribute, and how their personal goals also align with how they are contributing to the company.

Angie Chang (Girl Geek X): That’s a great segue into our next question… What are some books, resources, or thought leaders, or people to follow on Twitter, that have helped you in your career?

Kimber Lockhart (One Medical): If you get me started on leadership books, we’re going to be going all night. My favorite pick is the book Multipliers — it’s about how do you as a leader make your team better than the sum of its parts, and bigger than the sum of its parts, and more effective, a place that people want to be and achieving their goals. For me, I read it early in my management career and I read it again a few months ago, and I learn something new from it every time. Multipliers.

Sukrutha Bhadouria (Girl Geek X): I heard all of you talk about your path and with that I assume that your goals kept changing every time you got to the next step. How do you continually keep a watch out for the gaps that you have to get to that next new goal, and once you get there, whether it changed or stayed constant, how do you keep an eye out for the next gap? And how do you do this? How have you been doing this, and recommend others to do this?

Arquay Harris (Slack): I try to think about it in terms of, what is the highest aspiration I have for myself? A lot of people think about five year plans, and if I look back at my life five years ago, I probably would not think that I would be in the position that I am in — but what I mean by highest aspiration is — is it to be CEO of a company? Is it to be CTO of a company? Is it to just continue to be Director of Engineering? Knowing that helps me figure out how to chart my career — it’s like the north star.

Arquay Harris (Slack): So for example, if I said ‘I want to one day be CEO of a Fortune 500 company’, I would probably make different career decisions. I might try to get bigger and bigger teams, I might move jobs more often, I might have different goals. Up until very recently, my aspiration for myself, I like Director of Engineering. I like the ability to mentor people on a one-to-one kind of level; I like the human interaction. If I am CTO of a company that has ten thousand engineers, it’s probably difficult to do that in the way that I want to do that. I think if you had asked me that today, maybe that answer is different.

Arquay Harris (Slack): Understanding that as my experiences change, that that aspiration maybe changes, and maybe I should think differently, maybe I should network, maybe I should do Girl Geek Dinners to get more exposure, and understanding that north star is pretty important.

Rachael Stedman (Lever): So there are a lot of times where I realize later I had no idea what I was looking for, or what my goals should be.

Rachael Stedman (Lever): Before I joined Lever, I spent four months job-searching, and that’s a long time to think about what you want in your next job, and I put together this huge packet and I was reading a lot of books and I was thinking about what I wanted in my next company. I had come up with this check-list of all these things that I was looking for, so I was really interested in learning and growing — and I assumed that meant I needed to join a team with all these senior engineers with ten years of experience on me, and that’s the way that I am going to learn.

Rachael Stedman (Lever): At the time I was really into data visualization, and wanted to do something in the personal fitness or personal health space, and all these things were — these were my goals and what I was looking for — and when Nate contacted me about Lever, and I was in Boston looking for a job, and he said they were in San Francisco and hiring in software — it did not remotely sound like what I was looking for. But he convinced me to talk to Sarah, and Sarah convinced me to come out to San Francisco to visit Lever for a week, to spend in the office getting to know the company and what they were working on and meeting the people on the team, and I did, and I’m really glad I did.

Rachael Stedman (Lever): Even though Lever checked none of the boxes of my four month huge, documented check-list, I really came away from that experience feeling like it was a company with a product and a team I could really get behind, and I joined and have been there ever since. But I don’t think the exercise of thinking thru what I wanted was invaluable — there was a lot I learned about myself in the process.

Rachael Stedman (Lever): What I took away from the process is, you often don’t know what all the opportunities and options are going to be, so you can think about what you are looking for, and go thru that exercise — but also don’t limit yourself when you find something that maybe doesn’t match what you expected.

Jen-Mei Wu (Indiegogo): That’s a really good point. Sometimes you don’t know what all the options are. One of the things that really helped me in my career. It was kind of a different time — I knew more women engineers leaving the industry than coming in so it was really hard to find other people, so support groups and Systers and other organizations were really helpful.

Jen-Mei Wu (Indiegogo): Talking to people is a good idea for support — I think it’s a little better now to be a woman in engineering — talking to your peers, and also, if there is someone who you admire, someone whose job you think you might want, people are often very willing to have an informal interview with you. If you reach out and you say, hey I really want to find out more about being an architect, or a product manager, or a director of engineering — can I just buy you lunch, or can we get coffee or just chat after work — a lot of people are really open to that, and I think that would be really great for people to take advance of that. I think at an event like this, you are going to see all kinds of people, you can see us, and say hey — can we connect on LinkedIn and maybe have a follow up meeting, these are totally things that are open to you at events like this — it is so very helpful.

Kimber Lockhart (One Medical): I occasionally give a workshop on career paths, and thinking about career paths, and one of my very favorite exercises from the workshop is that we draw three very different pictures of things that could happen — it might be like, dream career moments.

Kimber Lockhart (One Medical): So I draw for example, well maybe I’ll quit my job and join a venture capital firm and go interview a bunch of heads of engineering and write a book, wouldn’t that be fun.

Kimber Lockhart (One Medical): My other one is, I’m going to be the CTO of the US and wouldn’t that be very exciting except maybe not right now… The point of the exercise isn’t so much the crazy visions but the part where you look at each of those visions, and say what you can do about this can enhance my career right now.

Kimber Lockhart (One Medical): About writing that book — I thought I wanted to get my ideas out there, and so about a year ago, I decided to make time to start writing essays and start posting on Medium (something anyone can do) and found that it was a wonderful way to grow my career that I haven’t been before. It wasn’t about aiming for what is next level up in the management chain, but what is another dimension that I can add to my career today.

Angie Chang (Girl Geek X): Thank you for all those thoughts. One thing I hear a lot about in the Silicon Valley and in women’s events is the topic of mentorship and sponsorship. What are your thoughts about getting mentored or finding a mentor, and how you approached that?

Kimber Lockhart (One Medical): When I became a director, it was the first time in my career that were some things that just didn’t quite make sense.  So I got some feedback that said I needed to be ‘more strategic’ — has anyone gotten that feedback before? I was like, ‘what does that even mean? Where do I start being ‘more strategic’? So I thought the only way I am going to really find out, was to talk to people who were supposedly strategic, and ask what they are doing to be strategic, and then I can do those things, and I can also be strategic.

Kimber Lockhart (One Medical): This is one of the wonderful parts of being part of the sisterhood of women in technology, is that when I went out to ask people to be introduced to, I got to meet all these badass female, largely, also men, heads of engineering who were ahead of me in their career and could help me figure out that gap to figure out what that ‘strategy’ actually meant.

Kimber Lockhart (One Medical): I’m still in touch with some of them. I’m pretty sure that if I told them that they were my mentor I’m sure they would look at me and say ‘that makes me feel old!’ but we are friends and do meet up and compare challenges. There is nothing like someone who has done it before a couple of times to help you cut thru your thought process and figure out the right next step.

Arquay Harris (Slack): I think that in recent years, I have been able to find mentorship via peers, and there are so many amazing groups like this, and meet ups and that kind of thing, and there are a lot more opportunities for women in tech these days… but to be honest, when I was coming up in my career, I first became a director of engineering at the first tech company I worked at, and there is to be honest a certain amount of privilege to having a mentor because people who are CTOs of VPs of Engineering, and they maybe have two spare hours a week to spend on something that is not work or family and unless you have a connection, they are much more likely to spend time with someone who looks like them, or is the brother of someone they went to Ivy with, and so, quite honestly, those doors were not always open to me.

Arquay Harris (Slack): Now that’s why I try to make that time… There are some people here who work with me at Slack, and I try to always make time for one-on-ones. I go to events. I mentor. I just came back from a big recruiting trip at HBCUs in Atlanta, and I try to make it so that road is a little bit easier for those behind me, but it wasn’t easy, to be honest.

Rachael Stedman (Lever): When people talk about mentorship, the vision that comes to mind is you find a mentor, and you meet with them, and they mentor you over the course of many years. And that sometimes happens, but for me — it has taken many different forms. It may be a one-on-one conversation with a manager who is on a different team cross-functionally and I learn something from that conversation and I take it away. It may be a coffee meeting with an eng manager at another company, and we have one conversation that I really enjoy and take something away from and it’s not necessarily an ongoing thing.

Rachael Stedman (Lever): There are a lot of different shapes and forms of conversations and connections with people that you can learn from and take insights away, and it doesn’t have to be a really long on-going formal relationship.

Jen-Mei Wu (Indiegogo): I’ll echo what everyone said, especially in my own case, it’s much more peers that are helpful than looking up. The other part of that — the question — was about sponsorship. I am not exactly what you mean but I think what you mean is that someone is going to advocate for you and find opportunities for you. A lot of it is expanding your network.

Jen-Mei Wu (Indiegogo): The things they say about getting informational interviews or just going to events like this is really helpful because who you know is really going to help you out — and you don’t know if that person you went out to coffee with is going to help you get that job. I have a friend who did an informational interview at a place, and eventually she ended up becoming the VP of Product there.

Jen-Mei Wu (Indiegogo): A lot of these things lead to other things, but you don’t know. But you can make more opportunities for yourself by meeting more people and taking advantage of networking. And on the other side of that, I feel really fortunate to have the positions that I have had. I feel it’s really important to give back, so I try to think about the things that would have helped me.

Jen-Mei Wu (Indiegogo): Sometimes people are afraid to ask and they are afraid to approach you, so if there are other people at events like this, or at work, and I may offer suggestions in terms of career advice, like for example, the best time to plan for a negotiation is a year before the negotiation — you want to plan ahead and set things up for your negotiation — that’s something we talked about at the women in tech group at Indiegogo recently. Also, I try to look for opportunities — not just the jobs I’m directly involved in hiring for, but also opportunities all around. I may mention to someone who is looking to hire someone, or someone who is thinking about creating a position, that somebody might be really good for that.

Jen-Mei Wu (Indiegogo): Sometimes it’s not about positions but it’s about projects — some projects have a higher profile than other projects do, so it’s good to think about that. Occasionally, I run into a situation where somebody who has very little experience has a job with a lot of responsibilities like a VP position or something like that — and I wonder how did that happen, and it almost always turned out to be they had a connection — that they had a mentor at the company who helped them go to manager, then director and then maybe VP as well. So it’s really important to both help other people out and well as it is to look for those opportunities.

Kimber Lockhart (One Medical): Can I take one more stab at this? I just want to share — there are three things to be an awesome mentee. So if you are out looking for a mentor, there are three things that I highly recommend.

Kimber Lockhart (One Medical): One is — don’t be afraid to reach out, but take on the logistics yourself — do the hard parts (send the available times, and really be on top of setting up the meeting schedules).

Kimber Lockhart (One Medical): The second thing is to listen really closely, and come back with reports of what you tried and how it worked. As a mentor, you would not believe the number of times I’ve had these great conversations with people and I’m so excited to hear about how it goes and nobody ever gets back to me, and I really wonder whether our plan was a good one! I learn a lot too about from the process of people trying things out at their own companies.

Kimber Lockhart (One Medical): The third part, is there are more ways to give back to a mentor than you may immediately think of. I’ve had a mentee offer to feature me in an article she was writing, and she actually got quite a bit of traction on it and it was really cool for me. I’ve had people send people send me leads for good candidates — if you want lots of mentoring, send great candidate leads, this is great (sarcasm)! It’s not a bidirectional exchange — it’s not like I mentor and I expect to get paid in some fashion for that work. I like to help people in their career because it’s the right thing to do and I was there once too, but at the same time — if folks are willing to put in the effort and time to be a great mentee, then it really makes the relationship worth it.

Sukrutha Bhadouria (Girl Geek X): This is all amazing advice. I wanted to say, every Girl Geek Dinner we attend, we are fortunate because our list of role models grows with that. While you talk about mentorship and sponsorship, I am curious, even a peer can be a role model. Are there many or any that have really impacted you?

Arquay Harris (Slack): A specific name of a person?

Rachael Stedman (Lever): I think the benefit of having a peer, or someone who is likely going thru the same type of growth as you, so they are really close to it and you can really help each other. Sometimes it’s just helpful to have someone who is willing to listen and help you talk things out and come to realizations yourself. Early Lever, when we were growing as a company, there were a lot of challenges and ups-and-downs when you are growing a startup, and being able to rely on the people around you to learn and grow from them, there’s a lot you can get out of it — and give, too.

Arquay Harris (Slack): When I was starting up, when I was an IC (individual contributor), there were a lot of technical people who really taught me to do higher-level problem-solving and whom I really wanted to learn from. And as I moved into management, the people who were really influential to me were people who I felt like I basically wanted to emulate. If I went into a meeting with these people, and I felt like wow that was really an important meeting, I really got a lot out of that — how did they do that — how did they structure it and not waste people’s time.

Arquay Harris (Slack): It also helped me because, sometimes in our careers, and I am certainly guilty of this too, you have skill inflation — you think ‘I am the most amazing engineer there ever was’ and you are not — this is just the reality of the situation — I try to look at, if there is someone who is really successful, I try to think am I really as successful at that person, is there something that they are doing that I am not doing. It comes from different directions, and I think for me it really comes down to.. what I try not to do though is to be like…

Arquay Harris (Slack): Do you ever see someone who is in a really amazing dress, and you think ‘that would not look good on me’. You have to also know when to emulate the things that work for you, and to not completely lose sense of yourself because that makes you doubt your own instincts and your own ability to grow. It’s been a hard path but I think now I’m at a pretty solid point where I know who to associate myself with and who to learn from.

Angie Chang (Girl Geek X): There was a mention of ‘hard things’ in the Silicon Valley. Is there a time that you felt that you felt that you might learn tech and the workplace, and what did you do about that?

Arquay Harris (Slack): Oh yeah, I totally left tech. That’s why I had a big-eye reaction. In between my last company and current company, I was semi-retired for three years. I just traveled… I went to Thailand, Portugal, Spain, India. Also, as a tech person, when you get into management, one of the things that is sad about being a manager, you pretty much never code again — I mean there are exceptions, but for the most part, it’s very difficult.

Arquay Harris (Slack): So I just did all the projects! I built a CRUD app in every single technology ever. I took an MIT course on algorithms, Harvard CS 50 course, all the things! And I did that because, yeah you get to point where you are working working and working, you are working so much, you are achieving, and you start to lose sight of the thing that you really love to begin with, and I really wanted to reconnect with that thing. But I also feel that as much as I love Slack, it’s a great company, I sometimes have thoughts like — is this my last tech company? Should I just retire and teach math to seventh graders? These are thought that I have had… It is a thought that I have had.

Jen-Mei Wu (Indiegoogo): Yeah, I’ve had this thought over and over again. As I mentioned, I majored in english and history and I always thought I’d go to grad school in literature or history or something. When I first started in tech, I started in testing — I was a QA (quality assurance) person doing black box testing. I thought I’d pay off some of my debts and then the debts didn’t really decrease as quickly as I was thinking that they would, and other opportunities came up, and I started getting more interesting jobs, and I kinda got sucked back in… and then I ended up working for a company where I started as the manager of IT, and quickly became the director of IT, and then became director of engineering, and during that director of IT time, that’s where most of my PTSD of management came from. We grew the company from about 30 people to more than 300 in less than two years, and while that’s very small compared to what Slack had gone thru, it was really harrowing because to hire 300 people, you hired maybe 600 people because a lot of people didn’t really stick around.

Jen-Mei Wu (Indiegoogo): After that, I was questioning my life choices — ‘do I want to work in tech?’ A lot of my friends were working for non-profits and doing social justice work, and I was noticing that my income is getting higher and it was starting to feel really weird, and so I thought: ‘do I identify more with people that I work with, or people in my community?’ and then I was falling on the side of the community, and then I did actually quit — and I thought I was going to take a little bit of time off and write a novel and completely change stuff up. And then, I’m so bad at coming up with directions for my career, and I realized that there are all these non-profits that needed help, and they kept asking for help, and I kept volunteering, until one of them really really really wanted to pay me — which is how my consulting company got started, and then they had offices all over California… and so I did that for a while, then I started to burn out of that… these are about a lot of times about leaving tech for various reasons.

Jen-Mei Wu (Indiegoogo): I got really sad because I discovered that in the non-profit world there are a lot of people who are really passionate, and I really love that, and there are a lot of non-profits that work their people in really unsustainable ways, and that made me really sad. What brought me back… then I shut down my consulting company. In fact I could have sold it, someone offered to buy it and I refused because they would not guarantee that everyone would keep their jobs. So I found jobs for all my people and shut down my consulting company, and I thought that now I’m really done. Suddenly, there were all these small shops where able to do apps, the things in mobile were really interesting.. I found that even among my social justice friends, people were like ‘wow development is really cool again because you can make APPS!’ Not that you couldn’t make apps before, but now they are apps!

Jen-Mei Wu (Indiegoogo): [Working in tech] became really interesting again too because I started talking to people and started thinking, it isn’t just about what the goal of the company is (the mission) but how you do business. And so I started getting more involved with businesses that were trying to be sustainable, trying to keep people working under 40 hours a week, having a good work/life balance — and Indiegogo is absolutely like that — and really valuing people. That kind of helped rejuvenate me. That’s a long-winded answer. There you go.

Rachael Stedman (Lever): I haven’t considered leaving, but that doesn’t mean that I don’t think are things that could be better about tech. The reason why I am very happy staying in tech and envisioning myself here, is because I have chosen to take this view of, I can learn to help create an environment that I can enjoy working in. When I joined Lever early on, one of the things that I really cared about is being on an engineering team with engineers that very much valued non-technical skills / soft skills / whatever you call those things that you contribute to a team beyond just coding.

Rachael Stedman (Lever): I had seen engineering teams where there were engineers only valued what they could contribute technically, and built an identity around it and didn’t see themselves as social, or wouldn’t invest or value those skills, and you ended up with people on those teams where those skills are really important and they would disproportionately be burdened with the emotional labor, and I didn’t want to work on a team where that was the case.

Rachael Stedman (Lever): Early on [at Lever], I started these conversations that evaluating for these [soft] skills in interviews is really important in addition to technical skills. I want us to recognize and reward people on these teams for contributing in these ways, and I love the team at Lever because the engineers — each of them values what they bring to the team in beyond their technical skills and value and invest in making improvements in their areas. It’s not the kind of engineering team everyone will want to work on, and I don’t have any proof of this, and I believe that it’s why Lever’s engineering team is over 40% women today. We work on hiring software, and hiring is really difficult to do well, but if you can help build a team that you really enjoy working with, you can go and work on any problem that you find important and impactful, and build a team around you that you really enjoy working with — in tech. So that’s what keeps me going and why I don’t consider leaving tech.

Kimber Lockhart (One Medical): I think I could have pretty much — and not nearly as eloquently — said what I just heard. For me, solving hard problems is really, really fun. And I think that’s a lot of what brings people to technology. But turns out, there are a lot of hard problems in the world.

Kimber Lockhart (One Medical): Healthcare! The healthcare system, and how to make that work! How to build great teams! That has some parallels to building software. I find that, especially where I sit in tech — being able to build teams and thinking about really hard problems that are affecting society today — and to also have that grounding in technical problems is such an exciting place to be.

Angie Chang (Girl Geek X): Our final question tonight is, what would you tell your younger self?

Arquay Harris (Slack): I love this question. For me, one of the harder things about being in tech when I was coming up was, I faced a lot of the bigotry of low expectations. People often ask me do face discrimination being a black person and my answer is almost always no, but being a woman — absolutely. People always assuming that I’m not technical, that I don’t know how to code.

Arquay Harris (Slack): So the advice that I would give to my younger self is.. I feel like instead of asking questions or finding a mentor or finding someone who could kind of help me, I would really invest a lot of time in learning things on my own. If I didn’t know how to do a thing, I would buy a book, cram, learn the thing. And if I had just asked for help, I could have gotten a lot further, faster, right? I think I would just give myself a break and say it’s OK — you don’t have to care so much about what people think — find other people who don’t know something and learn together. That’s basically what I would say [to my younger self].

Kimber Lockhart (One Medical): So in my spare time, I train at the circus center. I don’t have a lot of spare time, so not lots of training, but I tried out the flying trapeze probably six years ago, and fell in love. Now I’m afraid of heights! This is one of those terrifying fall in love moments. But what’s cool about the flying trapeze is that when you learn, you’re in safety lines — you are being held by the instructor in safety lines, and you are over a net, like this is not a particularly risky proposition in any way. But the feeling of fear is still there — and it’s still very real!

Kimber Lockhart (One Medical): It’s a really good metaphor of, in general, the feeling of fear is different from an accurate perception of risk in a scenario, I would just have my younger self get a lot better at identifying whether the fear I feel was a feeling, or whether it actually indicated that the thing that I was about to do was risky.

Kimber Lockhart (One Medical): Because ultimately there are a lot of things that are scary but not all that risky, that help land us in really wonderful places in our careers, our personal lives, and I think — I just wish I had known that sooner.

Rachael Stedman (Lever): I think for me, earlier in my career, I felt like I had to do everything — all at once. I wanted to learn all the things about being a great engineer, and I wanted to learn project management, and I just got very… like you try to learn all these things but you can’t actually learn any of them well, it’s too many things, and you’re not spending enough time with any one of them to have it engrained and learn it well enough.

Rachael Stedman (Lever): Some advice that I got at that point in time was to just slow down, you can do everything, just not all at once. Focus on one thing, and learn selective neglect. I think it’s sometimes hard to know what not to do, what not to spend your time on, to say no to things, to stop doing things, and it’s OK to do that. Actually, selectively neglecting things to do creates space to do those things that you do choose to do well — and learn from them. Selective neglect!

Jen-Mei Wu (Indiegogo): I’ve learned a lot from my mistakes in my career, so I think one of the things I’d tell myself is, you can learn a lot from mistakes, but they don’t have to be your mistakes, because it would have been better if I learned from other people’s mistakes. I think it would have been great if earlier in my career I had talked to people — I actually didn’t intend to be an engineer, it weirdly happened — think ahead and talk to people doing jobs that you might want to do, talk to peers, have work / life balance — which is really important.. it may have been a cultural thing that the companies that I worked with expected 60 hours a week which is really unsustainable — and to really think about strategic things. Think about what you want to do, and how each step is going to take you there.

Jen-Mei Wu (Indiegogo): And also, think about the impact beyond yourself. Who would have thought that trying to solve the problem of ride-sharing… could end up creating one of the most evil companies in existence? I mean it’s completely counter-intuitive, and that’s because they didn’t think ahead and think about the ramifications of what they were doing. And also, who thought that the tech industry for all the great things that we’ve done, would have such a dramatic effect on displacement of people, and that’s like one of my really sad things is like how this industry that I’m a part of is creating all these problems and as an industry we are not doing an awesome job of solving some of these problems. I kinda woulda said some of these things to myself, maybe think ahead, think about the things you can have an impact in, and be happy with.

Sukrutha Bhadouria (Girl Geek X): Oh my gosh, awesome. Thank you all! Let’s give them a round of applause, please. I feel like Angie and I have definitely asked everything we wanted to ask. Does anyone in the audience have any questions?

Audience Member: What was the advice that people gave you to be more strategic?

Kimber Lockart (One Medical): Specifically, we were in a hyper growth phase, and I was chartered with more stuff than my team of that size could possibly take on. And so specifically, there were two things — one was to draw out what my organization was going to look like in three months, six months, and a year.

Kimber Lockart (One Medical): So, write out the org charts, show how the teams are going to be structured, what are the parameters of each teams would take on, you know, basic stuff, and things I already had in my head, but things that I wrote down, drew out, and took in front of our VP of Engineering and said ‘this is the plan for my area of the product for the next year’.

Kimber Lockart (One Medical): The other thing about planning in general was to, not allow the PM team (the product managers) drive all of the prioritization, and to come with a clear point of view, so I was that solid, second part of the planning — because really, the further you go up in the engineering leadership, the closer product management and engineering frankly get, and to show that I really had the skill set, I just wasn’t showing it or articulating it that was particularly useful.

Audience Member: What was the best advice that really helped you in your career did you receive, and what actionable steps did you take to implement that advice?

Rachael Stedman (Lever): Early in my career, I read a lot of articles so I didn’t really read a lot of books. My manager said ‘you need to read full books’. I did take the advice to heart and it was the only New Year’s resolution that I really followed thru on, and resolved to read a book every week of the year. I did it, and I now love reading books, and read thirty to fifty books in a year, and I learn a lot.

Rachael Stedman (Lever): The thing about reading full books is that you get a lot more in depth on topics instead of just skimming articles on the internet (which I still do a lot of). You can read books for fun, but a lot of books that I read I’m hoping to get actionable insights on. So I use Evernote and I have a Notebook called “Content Consumption” and everything I read — whether it’s watching a video or listening to a podcast, reading an article or a book — I take notes on it, and it forces me to actually think about what I’m reading.

Rachael Stedman (Lever): A lot of times, you read for confirmation — you’re reading an article for the joy of confirmation. And sometimes reading things that are actually changing your mind, or introducing new topics, is like another approach.. When articles or books introduce new complex topics that take a long time to sit, it’s like a new paradigm or fundamentally changes your perspective or worldview, those can take a longer time to process and actually requires sitting and thinking.

Rachael Stedman (Lever): Another thing that I’ve started to do that I’ve considered ‘advanced reading’ is reading three or four books on the same topic, all at once. If you read different books on the same topic, you can start making your own connections between the books, and you can make up your own connections between books.

Kimber Lockhart (One Medical): I’m totally joining your book club, that sounds awesome. My best advice was that you need to learn to lead as you. So there is a bunch of different styles of leadership out there. There is advice in these books that conflict with each other. There are mentors and even great leaders who lead in wildly different ways, and what is the best for you?

Kimber Lockhart (One Medical): Turns out you get to pick, but I got the advice that I should pick in a rather informed way, and to experiment! So every time I read something in one of these books that I thought might be a good idea, to take it out, try it out, and write down what happened.

Kimber Lockhart (One Medical): You are almost launching this set of experiments on yourself and your team, based on the input that you’re getting from these sources. The big thing is to break down what you tried and how it turned out. And I go back to some of these early in my career, and it was really foundational in figuring out how I was going to be as a leader.

Kimber Lockhart (One Medical): I think if I tried to lead like the VP of Engineering at my company at the time, or like one of the CEOs that I saw in the media, I wouldn’t still be in technology leadership. It was really important for to lead in a way that felt authentic to me.

Audience Member: Have you had experience with building a family, and assuming our biological role as being mothers while holding a full-time career in tech, and actually doing technical engineering work or management work, if you had such experiences, what is the pushback that you felt or acceptance at the company or team that you are working with — and what’s your experience dealing with this?

Kimber Lockhart (One Medical): I don’t yet have kids but this is something that definitely comes up in the dialogue frequently in my family. My husband is also a CTO at a small startup.

Kimber Lockhart (One Medical): One of the neatest things about being in the role that I have today is the opportunity to help people figure out what works for them. So earlier in my career, someone showed up and was pregnant one day. It was really good to work with her to understand what was going to make the most sense for her life and in our organization, and it really helped form my early impressions around ‘what really should be flexible with work, and what can we do to make it work with people who are really good,” and in leadership roles you have to make decisions about what makes the most sense for your company.

Kimber Lockhart (One Medical): I was so lucky to have mentorship from someone who reported to me in figuring out how I wanted to think about supporting my team members who were in the toughest years of starting families — it was really impactful.

Audience Member: Hi, my name is Heather. I have some questions about politics in the workplace, and if you have any advice on how to navigate that? I know that starting my career when I was twenty something, that was definitely a challenge.

Jen-Mei Wu (Indiegogo): What kind of politics are you talking about?

Audience Member: Like office politics… Sometimes change can be hard, so you have to navigate who to go to to get decisions made, things like that.

Jen-Mei Wu (Indiegogo): It’s different at different companies. Sometimes it’s easier to make change when you have other people who want to make change with you.

Jen-Mei Wu (Indiegogo): A lot of companies create employee groups and like at Indiegogo (a lot of initiatives come out of engineering for some reason), but we started a diversity and inclusion group, that has started to influence some policies at the company. It’s a group that some engineers started, and then other people joined from other groups, and then we had executives and founders join as well. It became a place where everyone could talk and get things out, and turns out there is a lot of commonality and things that everyone could agree on and move forward.

Kimber Lockhart (One Medical): Navigating a company can be pretty difficult, whether it’s because of the chaos that happens with hyper growth, or whether it’s because of particularly political situations.

Kimber Lockhart (One Medical): I find that I do best when I can play the newbie card. Or the ‘oh, I didn’t know things worked like that, thanks for letting me know!’ and being the person that’s always positive and solutions oriented and always willing to pair with people to make good action has been the best strategy I’ve had to get thru both the chaos moments and the politics moments.

Kimber Lockhart (One Medical): That being said, I think politics can be really, really destructive in an organization, and if this is something that is shaping where you are now, there are companies with varying levels of politics and varying kinds of politics, and I encourage you to take a look at what’s out there — maybe there is someplace that is a less political situation that may be a better fit for you.

Arquay Harris (Slack): Every / most companies have varying degrees of politics. Sometimes politics aren’t even a negative thing — you have some situations where you have an inner circle because you have a group of founders who are invariably tight-knit. A couple of things I use to navigate this…

Arquay Harris (Slack): One is, be pretty intentional with who you align yourself. The same way that negativity is contagious, so is positivity. If there is someone who just inherently hates everything and everyone, maybe it’s them! And they’ve alienated everyone else, and they made a beeline for you because you just joined the company and you’re instantly their best friend because you just joined the company. The other thing is surrounding yourself with people you respect and get things done.

Arquay Harris (Slack): I try very hard to do is: being very intentional about the things I do and the battles that I fight… I’m not going to ‘die on every hill’ is how I think the expression goes… You have to think: I have this goal in mind, I want my company to be successful, I want my team to be successful, whatever that is, and focusing on that goal. How can I have the highest impact with the decisions that I make? And part of that is strategizing, because if you are that person who gets things done..

Arquay Harris (Slack):I have this thing that I always say — “no beefs” — I don’t beef with people. Meaning: people don’t come to work wondering ‘how do I really ruin Arquay’s day?’ Nobody does that! Give everyone the benefit of the doubt. Assume good intent. And if there are conflicts, really try to focus on the goal — we’re all a team, how can we work together? And absent of that, if you have people who are really unreasonable, maybe it’s not the company for you. But if you are going into a situation with those things in mind, I think it generally goes works out a little bit better.

Arquay Harris (Slack):I don’t like people pulling me into the fray — particularly as a woman, and a woman of color, I don’t have the opportunity to get really… I could talk a lot about ABWs — does anyone know about this? No? What, no one’s heard that expression? Someone raised their hand in the back — thank you! When people come at me with this really negative, hot, intense, angry politicky thing — I don’t engage in that, intentionally, and I try to stay really focused in what I am trying to do, and that generally works out a little bit better for me.

Sukrutha Bhadouria (Girl Geek X): I want to hear more about ABWs for sure… but I think we are going to wrap up tonight. And for the lady who asked about navigating about thru having a family, reach out to Angie and I. We have met so many women thru all the girl geek dinners. And we know people who have been really successful.

Sukrutha Bhadouria (Girl Geek X): There’s one person who Angie introduced me to who is a dear friend of hers, and is now a good friend of mine as well. The first time I met her, she was pregnant and she was showing the app that she had built during her maternity leave. It was amazing to stay in touch with her while her career progress while she raised her two children. Thank you to everyone for coming — and to the panelists! Thank you!

Angie Chang (Girl Geek X): Thank you! Please stay and hang out with us, chat, grab some beverages, grab some food, meet each other. I always like to encourage you to not leave until you’ve met at least one to three new friends. Hope you get to stay and meet the panelists and each other, and hope to see you at another Girl Geek Dinner!

Like what you see here?

Does your company want to sponsor a Girl Geek Dinner in 2021? Talk to us!

Girl Geek X Branch Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Mada Seghete / Co-Founder & Head of Marketing / Branch
Zeesha Currimbhoy / Director, Engineering / Branch
Deepikaa Subramaniam / Senior Software Engineer / Branch
Javeria Khan / Systems Engineer / Branch
Ann Massoud / Strategic Partner Growth / Branch

Transcript of Branch Girl Geek Dinner – Lightning Talks & Panel:

Mada Seghete: I’m Mada. I’m one of the founders of Branch, and I’m incredibly excited for our panel tonight. We have three amazing engineers and myself, a quasi-engineer. I used to be an engineer, and now, I kind of … I don’t do engineering anymore, but I’ll tell you about my story from engineering to where I am today. Yeah.

Mada Seghete: We’re going to have four talks and then a Q&A session and then some more mingling after. We start with Zeesha, who’s our director of engineering, and she’s incredible. Can’t wait for you to hear her story.

Zeesha Currimbhoy speaking

Director of Engineering Zeesha Currimbhoy speaking at Branch Girl Geek Dinner in Redwood City, California.

Zeesha Currimbhoy: Great. Can everyone hear me okay at the back? Thumbs up if you can hear me. Great. Thank you. I’m Zeesha. I’m one of the directors of engineering here at Branch. Who am I? I thought I’ll start off with a little blurb about who I am.

Zeesha Currimbhoy: On a more professional level, I’m currently a director at Branch where I run part of the engineering team. I’m responsible for the product backend team. I am running a search and discovery initiative where we’re basically building a pretty cool search product. I’m also responsible for part of the most of the data science organization. I’ve been at Branch over two years now, worn multiple hats just like you would at a startup, been in several roles, worked with a lot of different people. I think Branch has given me everything that I’ve been looking for so far.

Zeesha Currimbhoy: On a more personal note, I’m a daughter. I’m a wife. I’m a philanthropist by heart. I’m a mother of two adorable children, Sarah who’s four, that little one there, and Rehan who is a little over a year old. Fun fact, I had both of my kids while working at startup, so if ever anyone’s curious or is trying, just planning to start a family and is in engineering or is in a startup, feel free to come chat with me later. We can discuss everything around that.

Zeesha Currimbhoy: Last night, I was basically thinking about what I’m going to talk about, and I was reflecting on my career, and I said, “You know what, what’s kind of gotten me here? What’s made me successful?” The thing that resonated the most about this, about my journey and the thing I wanted to share with all of you is I truly believe that the thing that stands between each one of us, everyone in this room, me and every human, the thing that stands between us and success is really the fear of failure.

Zeesha Currimbhoy: The fear of failure prevents us from taking risks. It prevents us from exploring new opportunities. When I talk about failure, it’s something as simple as being scared to raise your hand in a room full of men and being able to ask that question that’s been nagging you right through because, well, you fear sounding stupid.

Zeesha Currimbhoy: How many of us have been in that situation? I most definitely have. I don’t know how many of you all have, but it’s the most common thing ever, is the fear of perception, of how people perceive us, holds us back from actually being ourselves and actually being successful.

Zeesha Currimbhoy: A little bit about my background, I was working in Evernote a long time ago, probably eight years ago and was incredibly successful there. I used to run … I was the technical leader, very comfortable in my role, and then all of a sudden, I decided I’m done with this company. I want to basically work on a product that I can impact many people’s lives. I want to work on something that people can relate to. I want to point to something and say I built that. I interviewed at only one company or a couple of companies, but I decided to take up an offer at Evernote. I was going to join their backend team, which was my experience and expertise.

Zeesha Currimbhoy: Then, a day later, the manager who I was going to report into the calls me up, and says, “You know what, we’re starting this new team. It’s called the data products team. It’s like an AI team, and we want you to be the first engineer on it.” I was like, “Oh my God, I’ve never done that before. Don’t know the space. Never done mobile development. Not worked with any Mac clients or iOS clients.” Very out of my background. I was almost going to say no. I actually said no to him. He called me a day later and said, “You know what, you interviewed really well with us. We think that you have great potential. This is an opportunity for you. Don’t let it go.”

Zeesha Currimbhoy: I thought about that, and I was very afraid to take that risk, right? It’s completely out of my comfort zone. I was going from 100,000-plus-person company to like now, a 50-person company and the first engineer of a team that they’re actually building out a strategic arm around, but I decided to take the opportunity.

Zeesha Currimbhoy: I decided to overcome that fear because I asked myself one simple question. What’s the worst that can happen? What’s the worst that’s going to happen if I take this role? The more I thought about this, I kept asking myself. I said, well, you know what, I can get fired. I can lose my job. So, what? Well, then, I’ll be out of a job. So, what? I’ll find another job, but if I don’t take this, I wouldn’t know what is on the other side, and so I decided to take that. I ended up running their search team. Eventually, I built all of their Mac lines search out right from iOS to Android, and it was an opportunity that I never regret having taken.

Zeesha Currimbhoy: Similarly, a year later from their … A year or two later, the same manager who I reported to who decided to take this risk on me, he got me in a room with the CEO and said, “You know what, we want you to actually be the VP of the augmented intelligence team, the team that we’re betting the whole company on.” I sat there looking at him and said, “You got to be crazy.”

Zeesha Currimbhoy:  It was such an incredible opportunity. I’m pretty sure all of you are thinking, “Well, why would anyone say no to that? That’s such an amazing opportunity,” but what it meant for me was being an engineer, having operated in a leadership role, I would now be responsible for strategic direction, vision, planning, road mapping, responsible for the career of 20 plus people who are all researchers, PhDs, machine learning engineers, a skill that I didn’t have. I could sit there and find 100,000 reasons why I shouldn’t have done that because I just felt that I wasn’t qualified for the job, but someone took that risk on me. Someone put that bet on me. Someone said that, “You have the potential to do this, and you can do it, and we believe in you.” Then, why didn’t I believe myself?

Zeesha Currimbhoy: I decided after many conversations with my manager at that point, he was the best manager I ever, ever, ever reported into, I decided to take that opportunity. The path forward was amazing. It was challenging. It definitely had its share of challenges. There were several products that I led from ground up, came up with … If ever any of you all have used the Mac line, a lot of their intelligent products were things that I had come up with, but behind all of those products, there were several failed attempts. Those are the ones that people don’t realize because they fail, people don’t even know they exist, and then you find something else.

Zeesha Currimbhoy: Behind each one of those, behind each one of those stories was a failure, a story of failure, a story of hard work, a story of determination, a story of grit. I can stand here, and I can tell you that take every single opportunity you get and run with it because if you don’t, you won’t know what’s in the other end. You won’t know what you’re missing out on. What’s the worst that can happen? I can sit here and literally, for every single problem you’re facing, I’ll tell you 100 reasons why it’s not going to work. I’m sure each one of you will come up with 100 reasons why it’s not going to work, but if there’s a single reason, if there’s a single reason that it might work, take it, and don’t look back. You might fail, but you know what, you might succeed. You might learn a lot from it, and it might make you a different person than you are today.

Zeesha Currimbhoy: The golden door is what stands in front of you. What really stands in front of you, what kind of stands is the fine line between you and success is that fear of failure. That’s what’s always helped me in every role that I’ve taken. I’ve worn several hats. You won’t even recognize the opportunity in front of you if you’re afraid, so don’t be afraid. Each one of us is afraid about every single thing. It’s normal. Being scared is very, very normal. I get scared every time I come up here and have to talk to an audience, I get scared. It’s a normal feeling, but you got to overcome your fear, and you got to take that opportunity because if you don’t, you don’t know what’s ahead of you. I’m going to let Deepikaa go next. Deepikaa is an engineer on our product backend team. She runs a lot of our different systems and a lot of works and a lot of our engineers. I’m super excited to hear her story.

Deepikaa Subramaniam speaking

Senior Software Engineer Deepikaa Subramaniam speaking at Branch Girl Geek Dinner.

Deepikaa Subramaniam: Thank you, Zeesha. It’s really weird to use a mic, but hello, everyone. Welcome to Branch. It’s really amazing to see so many women from the tech space gathered here today. I’m Deepikaa. I’m a senior engineer in the product backend team at Branch. Today, I’m going to use my time to actually give a higher level overview of what the product backend team at Branch does, the components we manage, and also, a little bit about the technologies that we actually use in the backend team.

Deepikaa Subramaniam: Branch, on an average, actually, any given day it gets, it receives about seven billion events. When I talk about events, it ranges from anything like a Branch link click or an in-app activity, an app that has partnered to use Branch SDK, so when the app gets installed through the App Store or when the app gets opened or any purchase activity that happens within the app, the partners uses our SDK to send an event to Branch.

Deepikaa Subramaniam: It’s very critical to handle these events real-time because apart from deep linking, Branch is into mobile attribution, so we have to … When we see a conversion event like an install or an open, we have to determine at that point if the user has decided to actually install the app or purchase a particular product in the app by clicking on an ad.

Deepikaa Subramaniam: The product backend team is actually responsible for this entire attribution flow. As you can see, when an event … Let’s say an install event from the App Store comes into our backend service. It immediately gets inserted into our Kafka topic.

Deepikaa Subramaniam: Kafka’s actually a messaging broker. It consists of producers and consumers, so the producers … It has multiple event topics, which has producers and consumers feeding into the topic. Servers can actually feed the event into a … become a producer and insert an event into a particular event topic.

Deepikaa Subramaniam: We can have multiple consumers actually feeding on the topic, so they receive events from the particular topic. Our event processor does a lot of the backend, has a lot of the backend logic. What the event processor does is it actually listens on one of the event topics. It consumes the event, and then, it calls on to all these attribution networks, so Facebook, Google.

Deepikaa Subramaniam: These are all self-attributing networks, so if a partner has integrated with Facebook or Google as their ad network, and when we see and install event, what we do at that point is we call out to Facebook or Google and see and ask them, “Hey, did you see a particular click associated with this user for this particular app?”

Deepikaa Subramaniam: If we get attribution information at that point, we add it to the event. We call it like decorating the event, so we actually add the attribution details to the event. What happens if the partner does not have any of these self-attributing networks integrated, so what we have is a user store.

Deepikaa Subramaniam: For every user, we aggregate based on the app the last clicks that they’ve seen, and all these attribution information gets appended for the user. When a conversion event like install, open, or purchase comes in, we query this user store. The user store is actually … The backend is Aerospike. Aerospike is a NoSQL database. We query our user store to see if there was any event that has happened that we can actually use to decorate this conversion event.

Deepikaa Subramaniam: Once we find a winner from all these attributed networks, then we actually publish the event to downstream to again to Kafka topics. Why we publish these events to other Kafka topics is we have a lot of internal applications that actually consumes data from these Kafka topics. They consume the data, and they run through a lot of analysis. They power our dashboard, and also, we have a service called Webhooks, which the partner actually …

Deepikaa Subramaniam: Webhook is nothing but a partner-hosted API endpoint that they provide to Branch. Branch, when it sees an event, it immediately fires a webhook saying, “Hey, I saw an event. I saw a user interacting with your app, and this is the revenue or this is the impression or ad that they viewed or to do this particular activity.”

Deepikaa Subramaniam: This is kind of our attribution flow on a very higher level. I joined Branch actually four months back. Before Branch, I used to work for a company called Citrix. My background was, there, I used to work with C#, dotnet, and completely Microsoft all. Actually, when I was looking for a job change, most of my … Companies that have resume filtering with Java on it might assume even … didn’t even actually make it through because I refuse to add Java just for the sake of having a language in my resume. Luckily, I ended up in Branch.

Deepikaa Subramaniam: Branch is like one of the best things that happened to me. I’m saying this because a programming language should never be a barrier in our career to get into a company or into any position because it’s something that we can always pick up. The challenges that we face in learning a programming language is like the best kind of challenge we can have because we are learning, and we are growing.

Deepikaa Subramaniam: And I think that’s it. I would like to introduce Javeria. She’s a senior systems engineer at Branch, and next up.

Javeria Khan speaking

Senior System Engineer Javeria Khan speaking at Branch Girl Geek Dinner.

Javeria Khan: I’m Javeria, and I’m a senior system engineer here at Branch. By degree, I’m an electrical engineer, and I started off my career as a hardware engineer. I’m also a chip designer. I later moved to the software domain, and I’ve been now in the software and systems domain for the past five years. What’s cool about what I do here? Systems, in itself, is more about a breadth of knowledge over multiple engineering domains, and it is about depth at any one of them.

Javeria Khan: What that means is we get to work with everything from databases to networking to security to CI and many more. All of those cool products and features that Deepikaa and Zeesha’s teams create, we make sure that they get deployed to the CI and that they keep running smoothly through our monitoring infrastructure and that they have the resources that they need. My team, which is the infrastructure team is also the first line of defense for any infra issues, so we make sure we have the monitoring tools in place to detect and to be able to solve those problems easily.

Javeria Khan: What are the main focuses of my team? My team is, one of our main things we’re responsible for is making sure of our scalability and our reliability. We learned a lot of hard lessons here, so we host all of our infrastructure on a public cloud. Public clouds can have incidents, and they can have outages. Those affect us, as well, where we also go down for a few hours.

Javeria Khan: What we learned there was that we needed to make sure that our infrastructure was resilient enough to such kind of events, so we started a project to make sure that we were truly multi-region and which we redid everything we had done on the infrastructure side over the past two years in a new region. What that provided us with was we’re now able to failover traffic to any active region if any one of them is having an issue.

Javeria Khan: While doing that kind of project, which took about six months in the making, and other similar projects that we do on the infrastructure side, because we have to do a lot of provisioning, we have to put a lot of machines. This sort of racks up your cloud bill.

Javeria Khan: We have to also manage for costs and manageability, so that’s also one of our main goals. Besides that, one of the other things is we have to make sure that our data is fast and accessible. One of the metrics that we do hold ourselves accountable to is latency on the infrastructure side. These are just some cool numbers from our infrastructure side like Deepikaa mentioned earlier.

Javeria Khan: We do process seven billion requests a day. Branch has seen tremendous growth over the past three years where we’ve seen almost a 70% increase year by year. We’ve even seen up to 90% increase over two successive quarters over the last year. We also have three billion user sessions that can come in in a day, 100k requests.

Javeria Khan: Our data pipelines also see 10 terabytes of data per day, so with that tremendous kind of growth, what are the challenges that it brings in? The challenges on the infrastructure side are mainly to do with outgrowing your databases, making sure your apps scale well enough, also putting in a sensible auto scaling policies, and making sure that you’re appropriately provisioned for any kind of high volume traffic events.

Javeria Khan: Two such interesting events I’ll just mention here, we get a lot of these, but on the top left, HBO is one of our customers. This was the season premiere for a very famous, a very popular TV show with dragons and ice walls. When they had their season premiere, on the eve of that premiere, this is the kind of traffic we saw on infrastructure, but we were anticipating it, and we made sure that we were scaled appropriately for it already.

Javeria Khan: The bottom right one is from about two months ago during the FIFA World Cup, and we sort of saw spikes especially during goaling when somebody scored a goal for like some of the popular matches. It’s usually when people were sharing links or sharing links to the app, and people wanted to check scores. Such kinds of events, which are sort of relatable in the real world, we have to make sure that we cater for them. That’s what my team does, so that’s about it. I will now hand it over to Mada who is our amazing co-founder and also the head of marketing. Thank you.

Mada Seghete speaking

Co-Founder and Head of Marketing Mada Seghete speaking at Branch Girl Geek Dinner.

Mada Seghete: Thanks, Javeria. Okay. I don’t have slides. I have one slide. That’s me. The most important parts of my life was getting born, then graduating. I did the engineering, so I was also Computer Engineering, and I was training processor design, but I never did it. I went and worked as a software developer. It’s my first job out of college. Then, fast forward, I ended up starting an app with my co-founders here at Branch. We failed a lot. Then, we started Branch.

Mada Seghete: I think what I wanted to talk about was my journey into my current job, which is both being a founder but also doing marketing. I started being really good at math. When you’re really good at math, you’re kind of pushed towards doing engineering.

Mada Seghete: I remember, I felt this expectation that because I was good at math, that’s what I should do, and I really wanted to be creative. I would draw and paint and do things that were on the more creative side.

Mada Seghete: My mother, in a very Romanian way, would say, “No. You are not good at this. This sucks. You’re good at math. Just stick with math.” I was like, “Oh, God.” I went, and then when I got the scholarship to Cornell, and I remember I wanted to study architecture, and my mother was, “No. You can’t do architecture. You’re good at math. You have to do engineering. You have to the hardest engineering possible,” so I did it. Although she was in Romania, she still had a big influence on me.

Mada Seghete: I started Computer Engineering. I minor in CS, but I did take animation courses, and I actually animated a few movies in Maya, and 3D Studio Max, which is really cool. That was kind of when I started realizing that I really loved doing that more than I loved doing engineering.

Mada Seghete: Then, the story after that was very interesting. I worked in engineering, and then I had an issue with my visa, which meant I had to go to school. After school, I would explore something new, so I explored being a consultant and learning the business side of things.

Mada Seghete: I realized that I hated being a consultant. I was very good. I was actually at the top of my class, but I kind of realized that I love doing things, not telling other people what to do, and sometimes, they would never actually do it. As a strategy consultant, you don’t really do that much.

Mada Seghete: Then, I really wanted to do marketing. There was a practicing in Deloitte that did marketing, but they didn’t want to take me because I always got assigned to the technical projects. I then got the job. A startup came up to me and wanted to hire me, and because I had business experience, they put me in the business development and product because again, I was more technical. I ended up doing that for a while. I learned about being in a company and being in a startup. I decided I want to start my own.

Mada Seghete: Then, when we started working together, it was like, okay, my own company, I can do what I want. I want to do design, and I want to do marketing, so I started doing that from the very early days. I designed our app and did all the marketing for it. Then, when Branch came along, and Branch came from all the failures that we had as app developers, we decided to build Branch to solve all the problems that we had. That’s when we basically started Branch, and I started doing marketing.

Mada Seghete: I think the moral of the story is sometimes, you can be very good at certain things, but they’re not necessary … I actually think that at the end of the day, I’m a much better marketer than I was a software developer because I love it so much. I love promoting things. Marketing is all about numbers. I love numbers, but I can be a lot more creative. It just, it was an interesting journey. I run marketing at Branch, but I don’t have as much experience as probably other people of my age who lead marketing teams because I’ve only had the experiences in product and engineering.

Mada Seghete: The moral of the story is that you can start again. I’ve started again many times, and I was able to get to a point where I do something that I’m good at and I love. Don’t settle, I guess, to doing what you are good at. Try to also go for something that you’re good at but you also love, and believe in yourself.

Mada Seghete: I think the story that Zeesha said earlier … I would still remember the moment when I decided I was going to start my own company. I was at Stanford. I actually went twice. Every time my visa got denied, I went back to school. The first time after my interim began and I went and did the MS&E program, I was part of design school.

Mada Seghete: I was working with all the startups, and I was … This professor who’s also an investor, Michael Dearing, and I was helping him take stuff to his car one night after class. I remember telling him how much I loved working with these startups but that I would never start a company myself because I don’t think I could do it. I mean, who am I to start a company?

Mada Seghete: He stopped, and he like dropped what he had, and he looked at me and he said, “What do you mean, Mada? If you’re not going to do it, who do you think will? Of course, you can start a company.” I was like, “What? Oh, my God. I never thought about that.” That was a moment that I’ll always remember, the moment where something shifted. Then, I just went and worked for a startup. I quit consulting, went and worked for a startup, but it all started in that moment. I think for all of you, all you have to achieve any of your dreams, is kind of what Zeesha said, all you need is to believe that you can do it and to have tenacity and keep going after it and not be … It’s okay to fail.

Mada Seghete: We failed three times before Branch. No one would give us money. No one would talk to us. They keep saying … We tried Fitbit for dogs, photo book printing app, and then a printing SDK. Our name is still Pawprint Labs on some. No one wanted to talk to us. Then, the same team then ended up raising money and building a company, but it’s okay. It’s okay to fail. Just try again, and you will succeed. I promise you.

Mada Seghete: Okay, that’s it. I got to introduce Ann.

Ann Massoud speaking

Strategic Partner Growth Ann Massoud speaking at Branch Girl Geek Dinner.

Ann Massoud: Hi, everyone. I’m Ann Massoud. I’m on the partner growth team at Branch, so you’ve heard from a lot of very technical folks here, but if you have any questions about the sales or business side, feel free to come find me after. We are hiring, but we’re going to do a slight intermission, so feel free to help yourselves with snacks in the kitchen or drinks or anything while we get everyone up here for a panel, so you guys can ask questions.

Ann Massoud: Okay, so we’ve heard some very inspirational stories from some really incredible women. Let’s give them all another round of applause. Perfect. All right, so we wanted to open the floor up for Q&A. We know that everyone’s come here with specific agendas, so we’d love for you to ask any of the women up here anything about their stories, their current roles, their career paths. Basically, anything’s on the floor, so go for it. Okay.

Ann Massoud, Zeesha Currimbhoy. Javeria Khan, Deepikaa Subramaniam, Mada Seghete speaking

Branch girl geeks: Ann Massoud, Zeesha Currimbhoy. Javeria Khan, Deepikaa Subramaniam, and Mada Seghete answering audience questions at Branch Girl Geek Dinner.

Audience Member: What’s the percentage of women in engineering at Branch?

Ann Massoud: The question was, what’s the percentage of women in engineering at Branch?

Deepikaa Subramaniam: Monica probably knows the answer to that.

Monica Cuyong: About 20%.

Javeria Khan:  20%.

Ann Massoud: Perfect. Any other questions from the audience? Okay, over here.

Audience Member: You all kind of talked about your unique journeys. Can you talk about the most challenging leap you made or maybe the most challenging job that you took and why was it the most challenging when you took the change?

Ann Massoud: The question was, can you please talk about the most challenging leap that you took when you made this change for those who had to make different career path decisions?

Zeesha Currimbhoy: I guess you already heard my story on the most challenging thought, but I think for me, the most challenging one was when I moved from an engineer kind of in leadership role to actually running an entire organization with people who were once my peers, now being my direct reports, and people who had years and years more of experience than me, researchers, PhDs, definitely more skilled specialists who now reported into me, and I had to figure out how to provide them the growth opportunities and the mentorship while also planning out the entire roadmap of a strategic arm of the organization. That was by far, and with zero mentorship because I reported directly to the CEO, who had no time. It was definitely by far, the most challenging jump I’ve had to make in terms of different roles.

Zeesha Currimbhoy: I think that in the beginning, like the initial part of that journey was the most challenging because I couldn’t find my bearings. As an engineer, I still very much gravitated to an IC. I wanted to just go and code, and I wanted to fix things, and that was not my role anymore. I still very often found myself just wanting and itching to write the code and getting the credit for it because as a manager, you got to give the credit. It’s your team’s work. You’re responsible for growing the team and getting out of their way as quickly as possible.

Zeesha Currimbhoy: I think the first couple of months in my new role was the most challenging until I figured out how to get the mentorship I needed, and then make sure that I was very open and honest with what I could or could not provide the team. I could not directly tell a machine-learning engineer how to actually build their models because I didn’t have the experience myself. I couldn’t figure out, “Oh, you should use XGBoost versus something else,” and so I figured out how to get them the mentorship that they needed for them to be successful so that they could still respect me as their manager, and I could take care of the other things for them. That was the most challenging thing I had to deal with. Anyone else wants to add?

Javeria Khan: Sure. For me, it was when I switched from hardware into software. I had already been working for three years in hardware design, and I had also actually just gone and done a master’s with specialization in circuit design. When I switched over to software, because all the hardware tools at that point are basically for Windows, I had never really used Linux before, so I guess I started from my first RM and LS commands to actually doing programming for Linux. That was hard also because when you’re switching fields and you have people that are younger than you, less experienced but they’re farther along because they’ve been doing it before, but I guess technically, that was a challenge.

Deepikaa Subramaniam: For me, I think when I started off my career, I actually did an internship at a startup, which got converted as a full-time. As Zeesha mentioned, at that point, it was a great opportunity, but most of the time, I was so afraid because in a startup, it was a very small startup. Every opportunity that came your way was like you had to own the projects, you had … like from scratch design and interact with PMs.

Deepikaa Subramaniam: Since I was starting off my career in that particular company, everything was new, and most of the times, I approached the projects with a tremendous amount of fear, which was actually pretty much like as Zeesha mentioned during her talk, it was like setting myself up for failure, but I had a really great CEO and CTO who encouraged me a lot. They made me take up … They voluntarily made me take up a lot of projects, and that’s where I learned how to interact with PMs, how to own, how to design something from scratch.

Deepikaa Subramaniam: Also, it’s not just designing, right, like as a developer. You have to be responsible for what you do, and also even after you push the project, you have to be responsible for any issues that comes along extra [inaudible]. That was very challenging, but that was the most I learned in my career. I think it’s helped a lot.

Mada Seghete: I think for me, I was similar to Zeesha. When I had to transition from being just an IC at Branch and doing … I was doing all the marketing to hiring a team and learning how to manage people. I didn’t have any mentorship either. I just remember the first person I hired didn’t work out, and it was one of the hardest things I have to do to let the person go after … I think it was very fast. After like three weeks, it was very obvious. I realized that I didn’t know how to hire. I didn’t know what the team needed and kind of figuring that out, and then figuring out in time how to scale the team was probably the hardest thing learning. I guess, that transition to becoming a manager especially in a very high growth business like Branch was at the time.

Ann Massoud: Any other questions from the audience? Oh, perfect, right in the front.

Audience Member: I just started my own company, and one of the challenges.

Ann Massoud: Congratulations.

Audience Member: One of the challenges is mentorship. Zeesha and Mada, you both mentioned being in positions where you didn’t really have anybody to mentor you once you are at the top, so could you tell me a little bit more of where you ended up seeking mentorship when it wasn’t directly above you or anybody who was immediately available to you, and how that works out in terms of building your leadership skills?

Mada Seghete: I can start. There’s this great website called Female Founder Mentor Hours started by one of my friends. They have, I think about 300 women founders at all different stages who we all do at least one hour or two of mentorship a month, so I highly recommend trying that as one. Then, the other one, I think I really looked for mentors, I realized I needed mentorship in being a marketing leader, so I actually went my board, and I admitted that I needed help. They introduced me to a few CMOs of companies that were way further along than Branch.

Mada Seghete: One of them, Megan from MongoDB, is now one of my mentors. I really do run to her. Now, there’s some issue. The board wants me to show some new numbers, and I went. I just had a call with her for 15 minutes. I had her mentor me, but I actually had her come and talk to the whole team a few times and give advice in how they do things. They’re already IPO. They’re a lot further along than we are. I realized that I needed help, and I went asking for it, if that makes sense.

Ann Massoud: Awesome. Over here, question? Yep.

Audience Member: I have a question for Mada. How important is it to have a group of like-minded people along to create a startup and to try out ideas? If you were going to do it alone, can you do it on your own, or is it important to have like-minded people?

Mada Seghete: You can. I have founders, friends. The question is, how important is it to have like-minded group of women or either founders around you to start a company? I personally don’t think I could have done it alone. I think it depends on who you are as a person. I don’t think I could have gone through the hard times on my own. I mean, it was really hard. I wanted to quit sometimes, and I think having my co-founders there to keep me going, I would have never been able to build Branch on my own. I do have other friends who are founders and solo female founders who are successful. I do really think it depends on the business, on the type of business and the type of personality that you have. It’s definitely not impossible. It’s way harder.

Mada Seghete: I invested in and I mentor one of my friends who’s doing a company by herself. Man, she messages me a lot, because, I think, she’s by herself. She doesn’t know where to get advice from, and she has a group of other female founders. It’s not just me. I think she pesters all of us to ask for advice and support. I never needed that.

Mada Seghete: I think in the early days, we relied on each other, so if you do it on your own, make sure you have a very strong support network. If you can have co-founders, I think having at least one person that’s you’re other person … It’s so hard at the beginning when you keep failing, and when you get to our stage now, it’s different. We have amazing other leaders in the organization. We can rely on other people, but in those first two years when it was just us failing over and over again, I would have quit, I think, if it was just me.

Ann Massoud: Karen, I think you had a question.

Karen: [This is] mostly for the engineers in the group — how do you balance company objectives, getting new features out, building stuff with the tech background that you have now versus learning new things because sometimes, it feels like it’s two full-time jobs, staying on top of what’s new, and then expanding your skillset versus actually doing stuff with what you already know?

Ann Massoud: To repeat the question, it’s how do you manage learning new things and still doing your day-to-day?

Deepikaa Subramaniam: I completely agree. Actually, this has been a constant challenge in my career as well. One of the things throughout my career I’ve tried to actually build is, it’s very hard to be motivated to learn something that’s outside your work unless you’re building your own company and constantly put in time to build that and learn through that project, right? What I try to do is I try to find, like if you can build a good rapport with the product managers in your company and try to figure out like something that’s not very high priority project that’s very interesting, and at the same time, it’s not something that aligns with your day-to-day work.

Deepikaa Subramaniam: Here at Branch, when I presented one of the major components that I work with is the Aerospike store and how the services interact with it, but I’m super interested in learning about Kafka and Spark consumers and Apaches, how they all interact and work together. I’m constantly trying to find time and maybe do a POC like figure out if there’s some small project that you can work on, which involves a technology that we don’t know and probably build our expertise on it.

Zeesha Currimbhoy: You want to go?

Javeria Khan: Sure.  Personally, I think I’ve gotten better about this over time. I always have a list of things that I want to look at or read up about. What I do is I try to set aside half an hour every day to sort of go through articles or things that I’ve sort of pinned aside. Also, when it’s not too crazy, and it’s not supposed to be always crazy … You have days where you don’t have that much work to do. You can get more reading in. You can get more exploring stuff in. I think it’s up to you to set those goals and to sort of make lists for yourself and know when to go through them.

Zeesha Currimbhoy: My answer is more a managerial standpoint answer. A, know the technology and know the space. Even if you want to learn something, I would suggest everyone set aside, like Javeria said, set aside some time for yourself. It can be one in the morning when you’re having tea or coffee and just chilling at home. Set aside some time to read through articles that you care about that you’re interested in. Get to know the space. Get to know what you want to do.

Zeesha Currimbhoy: Second, be very open and have a conversation with your manager. It’s very hard to do it yourself. It’s very hard to try to make time for this for yourself. You’ll never find the time. To be honest, you’ll just never because there’s always other things to do. There are always business priorities. Be open and have a conversation with your manager if you’re working about things that you want to learn, areas that you want to start exploring because chances are that there will be something that you will either get to learn by maybe spending some time in another team or maybe there will be a new project that comes along that will need the application in there. I think that the best way to actually get that time is to make your goals and objectives clear.

Zeesha Currimbhoy: Another asset is if you’re a part of a company like Branch where we’re constantly … the way we build technology is we don’t just take the latest, greatest, and then go and play and tinker around with it. We’re trying to build a sustainable company, and so business objectives matter a lot, but we always are aware of the trends in the industry and whenever we’re faced with a difficulty like Kafka’s falling over, we say, “Okay, what else is around? Let’s go investigate. Is there something else that we can use? Let’s play around with it and see if it’s going to solve our problem.” We always evaluate technology in the context of a problem that it’s going to solve, but that’s always an opportunity for something new to be introduced into the stack.

Ann Massoud: Any other questions from the audience? Perfect. Right in the front.

Audience Member: I actually had a question a little bit more about your origin story. You talked about you’ve done a bunch of startups before this, but how did you and your co-founders actually get the idea to do Branch? What was that process like?

Mada Seghete: There’s a story we tell, and then there’s the real story.

Audience Member: Real story.

Mada Seghete: The story we tell is that we started building Branch for our own app, and then, we ended that we saw there was much bigger. The true story is really that we got to a point where we realized that printing wasn’t going to be a big thing, and we had tried. I think Alex, our CEO, is very good at seeing the future in some ways. He was always the first one who would notice that, okay, this is not going to work.

Mada Seghete: We got to a point where I think we were about to graduate from business school, the app was still live. We were selling photo books, but it wasn’t doing that well. Then, we had tried his printing SDK. When we were trying to sell the printing SDK, people would say, “We don’t care about monetizing our app. We care about growth.”

Mada Seghete: One Sunday, Alex was like, “Who has time to brainstorm an idea?” It’s like this printing thing is not working, so I went to his place, and we sat in his kitchen. We started thinking about, what was the biggest problem that we had and what was the one thing that if we solved would have made our lives better?

Mada Seghete: The previous months, I was trying to build for the app. I was trying to build a referral program and a sharing program. If I invite a friend, they should get a free book, and I would get a free book. Then, if I share, if I start a book and I share it to the friend, and they installed the app, they should be taken to the book that I shared with them and then add more photos and then send to print.

Mada Seghete: It was impossible. To me, it was crazy. I kept bugging Alex and being like, “No, there must be a way.” I would go to Stack Overflow myself and be like, “Oh, Alex maybe doesn’t have time for this. I’ll try to figure out the technical side of how to do this.” Same with ads, on Facebook. I would run ads, and I would want to know when someone installed the app which ad that came from. I was crazy. I would actually go to friends that work at Facebook, and I was like, “It’s an ID. Can you send me this ID?” They’re like, “No, we don’t do that.”

Mada Seghete: When we were sitting at the table, I came, I said, “How about this referral thing?” and then he’s like, “Yeah, you’re right. That could be a thing. Let’s figure out if other people have the problem.” We actually called four people that worked at other companies, one that worked at Van Gogh, which is the first app that used Branch, one that worked at GoGoBuy, then one that worked at Zynga, and one another classmate that worked another gaming app. We asked them all, “Hey, was this a problem? Did we just not figure it out because we didn’t know?” and they’re like, “No, this is why …” The guy from Zynga was like, “That’s why Zynga didn’t grow, because referrals were so well on Facebook, and you could just invite a friend and get five gems when they were into the app, but you couldn’t do that with apps because you couldn’t pass any parameters to install.”

Mada Seghete: At that time, I think Alex was like, “We’re onto something.” Then, I think as we started building this, he realized that there was more than just referrals and sharing.

Mada Seghete: These links could be what powers … this idea of deferred deep linking could be used in all other channels, and that’s how it started. It really was from a problem that we faced. I wouldn’t say that we fully started to build it for ourselves. We were thinking. I mean, he had investigated a way to build it, to solve the problem that I kept bringing to him. I was like, I was basically reading growth and trying to grow the app.

Mada Seghete: We did decide that we needed something new. We realized that the business we were in wasn’t going that well and that we needed a new idea, so we probably would have never found Branch if we weren’t like working in the app space for like a year and a half beforehand.

Audience Member: That’s cool.

Ann Massoud: Any other questions from the audience? We have two. Go for it.

Audience Member: I have a comment. If you’re a woman and a mom and you feel a little bit isolated, there’s a fabulous Facebook group called Moms in Technology, and they’re very supportive. I really recommend it if you’re a mom in technology.

Zeesha Currimbhoy:  Yeah, it’s great.

Ann Massoud: Thank you for that.

Mada Seghete: Zeesha also wrote a blog post about being the first mom on Branch that I highly recommend on Medium.

Ann Massoud: You had a question? The one on the side. Yeah. Yeah.

Ann Massoud, Zeesha Currimbhoy. Javeria Khan, Deepikaa Subramaniam, Mada Seghete speaking

Branch girl geeks: Ann Massoud, Zeesha Currimbhoy. Javeria Khan, Deepikaa Subramaniam, and Mada Seghete speaking at Branch Girl Geek Dinner.

Audience Member: I wasn’t sure if you were [inaudible]. Thank you for sharing all your stories today. It was really inspiring. My question is for Mada. You alluded to advanced degrees that you had to pursue, but I’m just wondering if …  The rhetoric that I’m surrounded with today is very much that if there’s something that you want to learn, if there’s someplace that you want to be, you learn the skills, you get there, and you don’t really need a degree to do that, but I was just wondering since you’ve had multiple experiences of advanced degrees, are there any tangible or intangible benefits to kind of going through that process and getting you to where you are today?

Mada Seghete: We would not be where we are today if we hadn’t gone to business school. It’s not because of the things we learned in business school — it’s because of everything else. I think you are … you go and you are surrounded by all these people, and it’s the same with my MS&E degree.

Mada Seghete: I was an engineer and going to MS&E changed me into being able to think more businesslike. I met different people. I think in business school, it was very much like the type of people that were there were very different. There was this idea that you could do anything, and I don’t think I’d learn … I mean, maybe I did probably learn from classes because most classes were a case study with an entrepreneur from there who came and talked about their failure, their successes. They kind of gave me this belief that I could do something.

Mada Seghete: I think all of us had very different stories. For me, I don’t think I had the belief that I could start something myself. I kind of believed it, but I also needed a team, and I found Alex and Mike. Alex came from doing semiconductors. I think he really learned how to be like a business leader. He had a mind shift change in a way. Mike came from Minnesota where he was just working for 3M as an engineer, and I don’t think he wanted to come to Silicon Valley. Alex and Mike didn’t really have a network. I had.

Mada Seghete: I was in BD, and I knew some people. A lot of the original deals came from me because I had the strongest network, but they didn’t have that. Going to business school helped them build that, and it also gave us this credibility when we went and raised money. It did open doors. When we’re doing the Fitbit for dogs, people took our email. I mean, most times, they didn’t give us a meeting, but they did like say, and sometimes, we did take a meeting, and they were like, “You guys are smart guys. Work on something else.” They did, and we’re like, “No.”

Mada Seghete: I mean, we only lasted like maybe three months, the Fitbit for dogs, but even if printing. We would get a lot of meetings because we had a really good app, and people will be like printing sucks. Get out of printing. We’re like, “No, we’re going to do it differently. We’re different,” and then eventually, they were right, but it does.

Mada Seghete: Business school gives you … I think an advanced degree gives you certain things, but you’re right. I mean, it’s not like a lot of the actual tangible things, we could have learned from books, we could have read, but I don’t think you get to meet the type of people you meet. You don’t get the environment, so there’s a lot of intangible. I don’t think we would have been where we are today. Maybe one day, we would have gotten there, but I think it accelerated our path.

Ann Massoud: On the left.

Audience Member: Thank you for sharing your stories. I was curious if you’ve ever been overlooked for a promotion or even just what you said for being a woman.

Ann Massoud: Question was if you’ve ever been overlooked for a promotion just for being a female.

Audience Member: That’s great if you haven’t.

Mada Seghete: I mean, I’m very aggressive. I’m like a little … If I feel I’m overlooked, I just go and fight for it like crazy. I mean, I think it’s possible. I don’t think so. I think I was very lucky, and also, I think my personality. I’m like very aggressive, but yeah, I think I was probably more discriminated around my career for being Romanian and having an accent than I probably … I probably felt more insecure about the fact that I was an immigrant than I felt for being a woman. People didn’t want to be my partner because I was a woman in Cornell because I think they thought I was dumb in computer engineering, but I think that’s probably the most I felt it. I don’t know if you guys have stories.

Javeria Khan: I don’t have a story, but I think the industry is better about it generally. I mean, this is maybe something that happened more frequently about 10 years ago, but in the past five, six years since I’ve been working, if you do what you you’re doing well, I don’t think there’s actually any sensible manager out there that would actually overlook you just for being a female. In most companies, Branch included, obviously, what they’re looking for is talent and what they’re looking for is good workers. Perhaps, I’ve been lucky to be having worked in such good companies that I never felt having experienced those kinds of things simply because of my gender, but I think maybe it’s not as common as we think it is.

Audience Member: One, I’m a hiring manager, and what I see is a lot of female engineers, they don’t ask for the salary, but male engineers are. Just as a general comment, if you have that experience, if you’re good enough for that job, make sure you know your value and you’re asking for the right salary.

Javeria Khan: I agree. We tend to be more shy and introvert than our male counterparts, but that’s something that can fix on our own.

Mada Seghete: I think that’s true. As a manager, and my team is pretty much half-half, men are more likely to ask. I mean, I give promotions even when people don’t ask, but usually, with the women on my team, they come from me versus in the past. I’ve definitely noticed a trend especially with a team that’s pretty half-half.

Deepikaa Subramaniam: Sometimes, I feel like when you’re a [inaudible] engineer, you don’t think about moving forward, progressing in your career. Title doesn’t … That was my attitude in my previous company. Well, how does our title matter? Actually, how do I put forth, like she mentioned, how do I put forth asking for more salary and stuff like that, but I agree with Javeria. That’s something that we should build on ourselves, build by ourselves. It’s not actually an issue with the people that we work with. It’s our issue. I think we should think about it more and fix, act on it.

Zeesha Currimbhoy: I think the main question that was being asked was, how do you know what to ask for? How do you know the amount to ask for? I think that the way is the first most important thing is to know your worth, is to always know there are several sites out there that’ll tell you what does a senior engineer make, or how much is someone going to make, but I also think that making sure that you’re trying to work at one of those good companies that’s fair.

Zeesha Currimbhoy: It’s harder to know which are the good companies that are fair, but one of the things that I encourage a lot of people that I mentor to do is to speak with enough people.

Zeesha Currimbhoy: When you go on the on-sites, speak with enough people, speak with the panel to make sure that you get a sense of what the culture of the company is as well. You should be able to sense that this is a company that I want to work to that respects individuals, that respect people. Through your questions, kind of try to get a sense of that.

Zeesha Currimbhoy: I think that the other thing is also, in an interview, when you’re interviewing, it’s fair to ask, what is the band that you have in mind for this position? That’s one way for you to ask upfront, which is before you disclose the number you want, is try to get a sense of what is the band that is being offered for that position.

Ann Massoud: I think a good way to vet companies is to look for companies that have diversity inclusion working groups, different employee resource groups, and that’s a great way to know if the company respects individuals like Zeesha had mentioned.

Audience Member: Can I just add a word? If you’re in product management, there is a Women in Product group. It’s global. It’s not just San Francisco Bay Area. People have seen demand from all over, from countries, Turkey, U.S., UK, everybody asking questions, mentoring, salary related, how they are switching from one job to the other, so it’s a great resource group. It’s on Facebook. You can join, and it’s very well structured through. The founders are also VCs, former product managers from Facebook and so on.

Audience Member: The other thing I wanted to add on the engineering side is there’s a group called Systers. It used to be very popular before when I was in engineering. It’s run by the Anita Borg Institute as well. It’s one of their original arms. It’s a community, again, a community thing run by women for women in engineering. It’s so old. I think it’s probably from the late ’90s. It’s still very active. It’s very simple mailing list. When I went from engineering to business school, I was struggling with that transition, and I posted, and I got a lot of help from the women. I would say, you have all the portals, but it never is enough. How much you need, you do your own research and that one on one is where you will get the confidence to actually negotiate as well. There are women out there who will do mock interviews, who will help you do mock salary negotiations because that’s where the rubber hits the road.

Ann Massoud: Perfect. This has been a very amazingly passionate crowd, so thank you very much. We only have time, unfortunately, for one more question, so hope it’s a good one. No pressure. Perfect.

Audience Member: This is sort of a different angle. It’s actually a follow-up question to the one I asked Mada, but I actually want to ask you, Ann, because you said you’re on the sales side.

Ann Massoud: Yes.

Audience Member: I was actually curious, like can you give us examples of who are the profile, who are your customers or partners that you go after, and then, what is the value prop? What is the key thing that makes them sign up for your service?

Ann Massoud: I’d need like an hour with you, but essentially, for the … The key customers that we go after is essentially anyone with an app. I’m on the more strategic enterprise side, so we’re going after apps that have very high MAU, so that’s monthly active users, so people that are constantly going to their app on a monthly basis are typically the people that I would personally target. In terms of our differentiators or why someone would work with us, essentially, we’re going to help them with their mobile growth, so being able to drive people into the app, give them a very seamless user experience, and then most importantly, being able to tell these enterprise companies who these people are, how they’re interacting with the brand, where they originated, how they continued to interact with these marketing channels, and then ultimately, what led them to convert.

Audience Member: Thanks, Ann.

Ann Massoud: No problem. All right, everyone. Well, thank you so much for coming again. We’re super happy to have you. Another round of applause. Again, if you have any questions for any of us, please feel free to come back and ask away. Branch is hiring, so we have tons of open reqs at If you have any specific functions that you’re interested in, I’m sure anyone up here would be happy to talk to you about them. Have a good night. Enjoy the rest of the snacks in the kitchen, and I think there’s some drinks left. Good night, everyone.

Mada Seghete: Thanks for coming.

Ann Massoud: Thank you.

Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Sumo Logic Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Sumo Logic girl geek speakers: Shea Kelly (VP People), Bret Scofield (UX Research Team Lead), Riya Singh (Senior Software Engineer & Team Lead), Stacy Kornluebke (Training & Documentation Manager), and Jen Brown (Compliance & Data Protection Officer) on September 10, 2018 at Sumo Logic’s Redwood City headquarters in California.

Shea Kelly / VP, People / Sumo Logic
Bret Scofield / UX Research Team Lead / Sumo Logic
Riya Singh / Senior Software Engineer & Team Lead / Sumo Logic
Stacy Kornluebke / Training & Documentation Manager / Sumo Logic
Jen Brown / Compliance & Data Protection Officer / Sumo Logic

Transcript of Sumo Logic Girl Geek Dinner – Lightning Talks:

Shea Kelly: Is that working? Am I on? Welcome, everybody. Happy Monday, or as we say in my household, and everyone knows this here, happy day. How many people watch Sponge Bob? Then, you know what I’m doing. Listen. I really, I want to reiterate what Angie said. First and foremost, we are actually just thrilled and delighted that you’re here, and I mean that. We have been talking about doing an event like this for a while, but it’s always so backed up, the schedule like when can we get and do one?

Shea Kelly: We’re thrilled to do it for a couple of reasons. One is probably the obvious — we’re a growing business. We’re very deliberate and intent about wanting to continue to expand and diversify our company with rock star talent, and so, at a minimum, we want to expose and have people get greater awareness of Sumo, who we are, what we do, but obviously, the biggest thing, I’ll be honest, I think we have joy today in having some of our fabulous team members, who are all smiling at me now, share some of what we do.

Shea Kelly speaking at Sumo Logic Girl Geek Dinner.

Shea Kelly: You can get some insight into, aside from Sumo, I think it’s an industry that we’re in that’s pretty darn exciting and interesting, and we hope you get some good things from it. It’s a big week for us because this actually is the week of our second annual user conference. We have some 600 people registered to be at a hotel in Burlingame later this week. We don’t do anything small, so we thought in the same week, let’s have a board meeting, and let’s do a Girl Geek Dinner and just six other things. It’s like, but we like to go big or go home as they say, so again, welcome. I do want to also tell you, in case you didn’t notice, when you leave later, there’s a gift bag for you up front with a little bit of what we would call Sumo schwag. You cannot leave without a Sumo squishy. In the bag is a Sumo squishy for those … So many people are like, “Are we going to get a squishy?” You’re getting a squishy. They are so popular. Go on a campus, and it’s like they look at you, go, “Hi. What are your jobs? Can I have a squishy?” It’s like, “You don’t want to work for me. You just want a squishy.”

Shea Kelly: Okay. I am only going to take a few minutes here before the main speakers. I am not the main speaker. Take a few minutes. Just tee up a little bit about Sumo Logic, who it is that we are, what we do. I don’t know how many folks are aware of us, so I’m just going to do a quick welcome and intro there. The welcome, I think, is done. Then, we’re going to shift to our presenter. Bret is going to focus as we’ve talked about the whole, how we enable customers to be super successful in using our product is, we’re coming at it obviously from different ways — we come at in terms of the user experience and the design and how do we think about what customers need, and how they use the product, and how do we build that into the product, which then leads to Riya, who is going to talk to us about from an engineering and development perspective, how do we do that? How do we continue through the process? It is literally non-stop, and you’ll see that and hear that in what we talk about today. How do we infuse that into our development of the product?

Shea Kelly: The third piece is learning. Stacy is going to talk about the tools and all the things that customers need, and whether it’s docs and it’s training and it’s certification. They need to be enabled, so they’ll use the product, right? That’s pretty obvious. Oh, lights are out. Okay, so we’re into a groovy mood here. Okay, this is good. No, I like this. Okay, thanks. We do it right here. Then, the last piece, Jen’s going to speak to down here at the end is security and privacy. I mean, it is obviously, you can’t pick up a newspaper or see anything or think about your own data, personal data, but companies are migrating, as we’re going to talk a lot about, their data to the cloud. That has with it inherent, oh my gosh, how do I make sure that it is secure, it’s private, and that we have all the compliance and regulatory things in place? Jen’s going to talk a bit about that. Then, at the end, we’re going to open it up for discussion. I’ve already had a few questions that have been raised that will feed into the discussion later. Into that mix, we’re going to add the lovely Mary Ann O’Brien who’s sitting here who leads a big team for us in sales, and she can also bring a bit of perspective in terms of how we go out and speak to customers, okay? You can always throw a question out in between. Otherwise, I think what we’ll do is save a lot of it for the end. Sound good? Happy Monday? We’re still good? All right, you’ve had food. I hope it was good.

Shea Kelly: Again, I’m just going to go through this quickly, but if you think about and say, “Well, what does Sumo Logic do?” what we would say our tagline is, is Sumo Logic is a machine data analytics service that helps companies to build, run, and secure their modern applications. You say, “Okay, what does that mean? What does that really translate into?” If you think about the path over the years, the transformation from, I guess what I call the old school with data centers and on-prem solutions where data actually is stored and housed, and the transformation, as you know, this digital transformation we’re living in and will continue is to the public cloud.

Shea Kelly: What the natural outcome of that is that we’re building a different kind of software, and it is a software that is always on. It’s a software that, in many cases, or these modern applications that result from that are customer-facing. They are revenue-generating. Those are new aspects of something. It’s a 24-hour if you think about it in terms of availability. Take out your phone. Think of the things that you take for granted now, and all of that data has to somehow, in the private cloud, in the process, be built, run, and secured in that manner.

Shea Kelly: To us, that leads to three things, and I hope this translates. If not, sometimes, the … I don’t know if the white on white is looking good here, so the very first thing is obviously the cloud. Data smog rating, you know this. It’s going to continue too. It’s going in a rapid phase, and it’s going across industries and verticals into some of the sectors that before, would have been hesitant to, into banking and in insurance. You’ve got that migration happening pretty quickly and pretty voluminously, if that is a word.

Shea Kelly: What that second part of that is it leads to this notion of continuous delivery. It’s always on, and that’s where it translates to the humans, to the people who are building and running and securing these apps because they’re not doing it the way they used to do it before. Now, if you think about it, and you think about the old way of developing software, the old … What do they call? Waterfall, so you would have an idea. You’d put it in development. It would go through a whole bunch of steps, and at some point later, something would spit out on a disk, and you’d take it somewhere. That’s not what happens anymore, right?

Shea Kelly: here’s a different way of architecting these modern apps, and so, what has to happen and has to follow that is we have to have a supporting mechanism for these teams who do it. For us, we say, okay, if it’s all going in the cloud, and it’s continuous delivery, it’s always on, that’s where Sumo Logic comes in. What we do is we provide a product that’s called continuous intelligence. What that does is it gives these teams who are having to architect those modern apps in a different way and who have to manage and do things in a different way as teams, it gives them an analytics product that helps them to again, build, run, and secure those apps in a way that meets these modern needs.

Shea Kelly: Just quickly, we are about now eight years in business. I think our founder might still be over here, so I’m going to have him give a wave. Christian Beedgen. Should I tell the story of how Sumo came to be? I won’t do that, but anyway, it’s a dog. It’s not Sumo. Catch me for the story. Anyway, so, Christian is our founder. Bruno’s still here. He’s our founding VP of product and strategy, and so, eight years in business, we are now some 1,600 customers strong… 50,000 users I think… so it is proliferating pretty quickly. We have, I think as a lot of SaaS models do what you’d call a land and expand. We get in on a use case, and then over time, they say, “Well, it works so well here. We need to expand this,” and it easily can expand across the organization.

Shea Kelly: We are serving companies, like I say, in every vertical, in every industry. If they have a modern application, they have got to have a way to analyze and think about that data as they build, run, and secure it. We have customers from Anheuser-Busch to Pinterest to … Who are some of the others? As I’m thinking about it, JetBlue or Betfair, Hearst, and so, again, these are also global organizations, in many cases. For us, it’s, like I say, eight years strong. We’re some $250 million in investment, so we were no longer a startup. We’re a late-stage private company, I would call us, funded by some of the best and brightest in the Valley, and so we’ve got a ways to go on what my last point to be here is this journey. We’re just getting started in some ways. I mean, this proliferation of data, if you think about it, I am going to step in here just third from the right. If you think about it, that’s 2018.

Shea Kelly: Machine data by 2020 is predicted to account for 40% of all data created. If you think about it, this machine data, which is what we focus on with our application service, is growing at such an exponential rate. Customers and companies have got to find a way to analyze that data. This is the background. Again, if you just think about it, it’s digital transformation, software as a service, companies are going digital, modern applications always on, and through that process, which is what leads to our discussion today, is this question of, how do we make sure, because customers may have different needs and how they do these things, that we are listening to them and looking at all of their needs and figuring out how we build that into the product ultimately through how do we do security and compliance around it, okay? With that, I am going to turn it off to the real presenters. I’m going to start here with Bret. Here’s our group. Here you go.

Bret Scofield speaking at Sumo Logic Girl Geek Dinner.

Bret Scofield: Hello. Good? Okay, perfect. I’m Brett Scofield. I lead UX research at Sumo Logic. UX research is a relatively new discipline, so I wanted to talk a little bit about what that actually means. Design’s been around for a while, and I think product development has also been around for a while. UX research is really a combination of … or it works very closely with those two, so a lot of times, what we’re doing is getting feedback from our customers and from our non-customers on ideas. Sometimes, these ideas are really big ones. Should we build this feature? Is there a place for this new product? That type of thing. Sometimes, it’s more granular. We’ve decided, yes, we’re going ahead with building a metrics feature, and now, we need to get into the nitty-gritty of what exactly that looks like and how our customers are going to use that type of thing. UX research, I converse with our customers pretty much all the time. I advocate for them heavily, so I’m always telling their stories to our engineering, product management, designers, just really emphasizing like, “Hey, our customers are running into so much pain with this. Can we please, please change something?” et cetera.

Bret Scofield: Then, on a personal level, obviously, I enjoy Boomerangs. This is with my work wife. Hi, Rebecca. Then, I run a lot. I’m a marathoner, so I’d love to chat about all those things. I’m on the team, I’m the food contrarian for all things, so people are always like, “Hey, have you tried this awesome new like hybrid taco thing?” I’m always like, “That sounds like it sucks.” That’s usually my role. Then, I wanted to get into a little bit about why I chose Sumo, why I’m working here.

Bret Scofield: First off, we have a lot of really interesting gnarly problems to solve, and that’s great. You start off Monday morning, like today, you really dive into those problems. You’re working with engineers. You’re working with security people. You’re working with a bunch of people who are really, really smart and have a lot of things to say. That’s amazing, and that’s great to work on those problems, but then, our team does a really good job of balancing that with a little bit of levity. We go to lunch with the UX team. We talk about tacos. We talk about horoscopes. We talk about whatever else. That gives you enough breathing room and enough space to go back in in the afternoon and to like really dig into those problems again.

Bret Scofield: I also really appreciate that there is a huge customer focus throughout the organization. That’s why we’re giving this talk. We’re talking today about how the customer influences everything that we do at Sumo Logic, and so, it wasn’t hard to pull that together with all these other women because everyone in the organization really thinks about the customer and keeps them at the forefront of the decisions that they’re making.

Bret Scofield: Okay, so today, I’m going to be talking through the product development process, and I’m going to be talking through the two halves of it, which here are running concurrently. Instead of sequentially, instead of having all of this discovery happen before delivery happens, we have both of these processes going all the time at Sumo Logic. The first part of this, the discovery bit is really answering those like super-high level questions. Should we build this thing? Is there a market for this thing? That type of, it’s exploratory, it’s really like pie in the sky sort of stuff.

Bret Scofield: Then, after we’ve figured something out from there, we’ve determined, yeah, there is space for this, this is something that our customers would actually want and would use, then those findings are given to delivery focused teams. These teams have an idea. Now, they’re like, “OK, we need to build a metrics functionality or we need to build this type of functionality,” and so they need to dive into more of the specifics of that. They’re figuring out, “OK, with our customers, how exactly are they going to do what we want them to do in this? Is it a dropdown? Is it a visualization?” They get into the meat and bones of that.

Bret Scofield: What I’m going to talk about is how our customers drive both of those processes. With the discovery processes mentioned, these are the really big high-level questions, and the major question that we’re trying to answer here is, are we building the right thing? Is there space for this thing that we want to do? One of the recent … Well, not recent. One of the things that I’ve been working on for a long time at Sumo logic has been our personas. Many of you are familiar with personas. These personas are not real, but they are an amalgamation of our customers and their mental models.

Bret Scofield: The reason that we went through the exercise of creating personas is really because we want to give the entire organization a common vocabulary. Melinda has a certain mindset and approach to the product, so when she goes into Sumo Logic, she’s likely trying to accomplish a specific set of things. She has a certain familiarity with a product. She goes in in a certain frequency. Andre, completely different. He uses Sumo Logic for different things, goes in with a little bit more uncertainty, all these types of things.

Bret Scofield: When we refer to our personas, I can say, “Hey. Stacy in documentation, I’m dealing with an Andre.” She immediately knows what that means, and she knows what sort of pains he’s likely to be feeling, where he’s coming from, all that type of stuff. The process of creating our personas, we attacked this in, first, a qualitative way. We met with a bunch of our customers on site, watched them do work. We talked to them about their educational background, their career trajectory, what are the things that are painful for them, et cetera.

Bret Scofield: Then, after that, we aggregated all the data, came out with three personas. Then, during a hackathon, we went through the process of deriving quantitatively what our personas look like. Sumo Logic, as Shea mentioned, has a ton of data on what our customers are doing in the product. We analyzed it during this hackathon. We ran a k-means clustering algorithm on all that data, and we derived the key use cases. Those key use cases actually mapped really well to the personas that we had derived qualitatively. There’s both a quantitative and a qualitative backing for these. Then, they’ve permeated the organization. I think almost everyone at Sumo Logic knows who Kathy and Andre and Melinda are, so.

Bret Scofield: Then, as mentioned, so this was a discovery project. As mentioned, the discovery stuff often influences the delivery things, so now, in a lot of the delivery projects that are going on, we define at the very beginning who is this specific feature for? How do we expect an Andre to approach this? How do we expect a Melinda to approach it? How would it be different? Those types of things.

Bret Scofield: Then, I want to talk a little bit more about the delivery process. As mentioned, the delivery process is when that sort of overarching thing is defined. We know we need to build this specific thing, but the question that we’re seeking the answer here is, are we actually building that thing right? Are we doing the things that need to be done so that a customer can actually do the thing that they want to do in here? With delivery specific things, we start at the very broadest level, and then we narrow in.

Bret Scofield: One of the things that we commonly do here is participatory design. We have quite a good relationship with internal users of Sumo Logic, so our customer success team uses Sumo Logic heavily. They also interface with our customers. Sales engineering, similar case. We bring them in. We also bring in our own engineers. They love dogfooding Sumo Logic, and they generally like working with us, so we bring them in. We do design exercises where it’s pretty much, we start with a blank canvas, and we have them draw, what is your 10-star experience? What would this future look like in an ideal world?

Bret Scofield: We’re actually, as Shea mentioned, we’re holding our user conference on Wednesday and Thursday of this week, and we’re doing a large-scale participatory design with a bunch of our customers. We validated that there’s a specific feature that we want to build, and so we’ve made kind of a Lego kit for all of our customers to build their ideal version of this. It’s a super fun thing, and it’s really great because it gets buy-in from a lot of people. They feel like they are a part of this thing.

Bret Scofield: After that, the researchers and the designers will aggregate a lot of the feedback, and they’ll put it together, and they’ll start actually working with the pixels. This looks a little washed out, but they’ll start building designs and prototypes. Once those are in a solid enough state, we’ll put those in front of our customers. This is actually one of our customers in Australia. He’s expressing disappointment, which is … that sometimes happens with research. He’s a little bit bummed about one of the things that we had sort of neglected to redesign. We were launching this redesign, and there were some areas where we’re like, “Oh, well de-prioritize those. It won’t be a big deal.” Then, we found out from a ton of our customers, “No, it actually is a really big deal, and they are really upset, they want us to redesign this.”

Bret Scofield: This changed the trajectory of the project. We stopped, and we said, “Okay. We actually do need to redesign these few screens and make this a cohesive polished experience for our customers.” Then, after something has been designed and released to all of our customers, we measure. We work very closely with the product management team and then with our engineers to instrument the log data so that we can see what people are doing with this new feature. Are they exiting Sumo Logic right after? Are they continuing with their workflow? What exactly is being done with this new thing?

Bret Scofield: This is great because it allows us to measure success, so if we can see a lot of engagement with this, we’re pretty happy. It also sort of allows us to jump off with more qualitative research. If we see, for instance, that everyone’s exiting Sumo Logic after using this new thing, then maybe we should delve into that. We should hear some stories. We should figure out what’s going on and make some adjustments. We’re using the Google HEART framework actually to drive which sort of metrics we’re tracking for this.

Bret Scofield: Yeah, to sum everything up, I just talked through how the customer feedback drives these two halves of the product development process at Sumo Logic, so we have the delivery process and the discovery process. Both of those are heavily influenced by the thoughts and feedback that come from our customers. Then, I’m going to hand this over to Riya for her perspective from engineering.

Riya Singh speaking at Sumo Logic Girl Geek Dinner.

Riya Singh: Thank you. Do I use this too? Okay, cool. Lights are bright. Hi. Hi, everyone. How’s everyone doing? My name is Riya. My name is Riya. I’m in the engineering development team. I’m a team lead of the data engine team. Quick show of hands, how many engineers out here? Software engineering. That’s one-third of the room maybe. Okay, cool. Feel free to connect with me on LinkedIn if you have more questions after my talk, and I’m happy to answer more questions.

Riya Singh: The theme of today’s talk is, how does customer feedback impact different product areas? I’m going to give you the engineering perspective or how we are listening to our customers. A little bit of background about myself, I joined Sumo about four years ago. I started as a software engineer. Today, I’m the team lead of data engine team. My team is about five people, and we mostly work on streaming systems at Sumo, so anything that requires real-time, low latency applications like live tail, live dashboards, alerting, and, so on.

Riya Singh: I’m also a dance fitness instructor, so my mornings are all work, work, work. Nights are all dance, dance, dance, so it’s been a lot of fun. Sometimes, I’d work. I love Sumo because I can combine both my passions. I do all kind of flash mobs at work and all those fun stuff at work, but the main reason I’ve been with Sumo for years is because I’ve seen many companies in which engineering is given a project and said, “Okay, these are your requirements. Go and build it.” Very few times, we try to understand why we are building it or if you’re building the right thing or why is this important? Sometimes, I feel engineering does not get that perspective. In Sumo, it’s ground up, right? As soon as you’re trying to decide what to build next, we’ll ask the right questions. The advantage that we have is that we are our customer zero, so we are using our product as much as anybody else. You kind of understand why this is important, and that perspective really makes you make good products.

Riya Singh: Let’s talk about customer focus development. Some things are very strategic, right? Sometimes, you want to build products ahead of the market, right? We want to build things that nobody else is building. We want to be innovative. We want to be strategic. There are a lot of projects, which come through deciding to be innovative, so that aside, there are other things that we do when we get customer feedback from various channels. We work very closely with Bret and her team, so as soon as she’s going to a customer and having her discussions, we are reading back the reports come back in, and we try to figure out, “Okay, where are the different things we’re developing incoming?” and build something to help that customer.

Riya Singh: Two very nice things that we have in Sumo, I’ll just describe them a little bit of detail. This is my kind of favorite place to hang out. It’s the ideas portal. We have a public ideas portal wherein customers can go and mark the ideas or features that they want to see in our product. It’s very nice because you can see where it was created, how many words are there, and what’s the progress. That’s a very nice way to find out what is important to our customers, and we can build the right thing. That gives us a lot of perspective.

Riya Singh: The other place, which I really, really like as well is we have our own public Sumo Dojo, a Slack channel where all of our customers or most of our customers are there. That is nice in a way because it’s very real-time, right? They’re asking a question, and sometimes, they’re helping themselves or we are answering some questions. It adds that human connection to the conversation when it’s real time, right? That is one very nice place where we get our ideas from. Yeah, so as I said, we talked to field. We have all these channels, and we try to get feedback and build the right things.

Riya Singh: Cool. Shifting gears a little bit, one thing in engineering for running a SaaS service is very important is that we can’t just build and forget about it. The system has to run. It has to run 24/7. It has to run at 100% availability, which is easier said than done. A lot of elbow grease, which goes on in trying to make sure that the service scales, and it works under different load conditions. Engineering realizes that it is very important that the current service work, right, so we spend a lot of time in trying to make sure that our service are reliable. Some ways how we do it, we use our own product to monitor our own services, so this is an example of an outage dashboard. We’re trying to monitor different lag latencies. There is a group called the IRC, incident response coordinator, so in case our metric here does not look good, we declare it as an outage. All the teams are involved in trying to resolve it as soon as possible.

Riya Singh: When I joined Sumo, we were about 80 people. We had maybe 10 micro services. Today, we have 500+ and I think more than 50+ micro services. Being able to monitor and run the service at scale is a challenge, and we have learned so much through just trying to run this well. We have learned that things that work when you’re a small company does not scale when you’re a bigger company. You have to automate as much as possible. Anytime you’re adding more humans to it, there is more chances of errors coming through, so the human side of scaling does not translate to the skill. We spend a lot of time in trying to automate things so that if it in case something goes down, we can recover from it as soon as possible without human interaction.

Riya Singh: Despite all the works, sometimes outages happen, and you move on, but one thing very nice about Sumo and its culture is that our focus is very clear. We are one with our customers, so in case an outage happens, we all gather together in an outage war room and try to help each other to resolve it as soon as possible. You’ll see people leaving the meetings, leaving any presentation and coming and helping. It was very nice for me to see when I joined Sumo that everybody’s so helpful here. If an outage happens, we do post mortems within a day. At this point, any new feature development stops, and we try to work through the action items that have come through the outage post mortem.

Riya Singh: Our policy is no repeat offenders, so in case we find the root cause and it has happened once, we will find the ways to ask the five whys and fix it, but you want to make sure it doesn’t happen again, so no repeat offenders is our metric to see how well we are doing.

Riya Singh: Okay. Talking about outages kind of makes my head hurt too because I’ve been to, well, more than a couple of them, but we’re getting better.

Riya Singh: Let me just shift things a little bit and talk about this newest initiative that I’ve been leading at Sumo. I’m leading a new team at Sumo called the Quick Wins team. As you know, we have grown so fast that today, we have 50,000 users and 1,600 customers, right? They’re very engaged customers, so they’re constantly asking us for new things that they want to see in the product. Sometimes, a team is not tasked to do those small things because they’re already working on the next big initiative, right? These smaller things or UX feedback somehow doesn’t get prioritized. I’m sure you guys have noticed that too in your companies that the smaller things are sometimes skipping through.

Riya Singh: We started a new team called the Quick Wins team, and a secret sauce is the sriracha because a little goes a long way, and so we’re trying to make all these small, small changes, which makes our product better over time. It’s a cross-functional team, so we have … It’s a small team, one UI, one UX, one backend, one docs, one PM. One nice thing about this is we are growth hackers.

Riya Singh: We are not tied to any particular section of the product, so it’s not like somebody who knows search cannot do dashboards or somebody who knows dashboards cannot look at collection. We are training people and empowering them to be able to make progress in any part of the product, removing any kind of silos that exist.

Riya Singh: We’re trying to make the product easier to use for Andre, for Melinda, and all the personas that we have. The aim is to improve the NPS, which is net promoter score, which is a metric showing how much our customers like our product. That’s it for mine. Stacy shall talk about learning.

Stacy Kornluebke speaking at Sumo Logic Girl Geek Dinner.

Stacy Kornluebke: Hi. I’m Stacy Kornluebke in case any of you here looked out at the agenda and wondered how to pronounce that. It’s a frequent question. I am the manager of training and documentation at Sumo Logic, which if it’s a lot for you to say, we just kind of reduced it down to the learn team. I am also the mother of three wonderful boys. I have replaced my couch three times in case anyone’s wondering. People say to me, “Wow, that must be a lot of work.” I say, “No, it’s a lot of fun, but I do miss my furniture,” so I am an avid fan of the arts. I am kind of infamous for dragging my friends two and a half miles down Manhattan to see Gustav Klimt’s The Woman in Gold. If you don’t feel like making that trek, you can just watch a very wonderful movie with Ryan Reynolds, and you’ll see the painting a lot.

Stacy Kornluebke: I’m also a voracious reader. I have 80 out of my goal for 100 books this year. If you find me later for networking opportunities, do not ask me what I’ve read. Once you reach this level of quantity, there’s very little quality. I like a lot of steampunk and vampire and werewolf books. It gets me through the night, but in all honesty, what I really like to do is make users confident. My whole goal is to give them the materials they need so that they feel that they are able to use the product the way they want to.

Stacy Kornluebke: People often ask me what’s special about Sumo. I did leave Cisco to come here, and I did it for a number of wonderful reasons. One of the top reasons is that we have very technical and sophisticated users. These people do a lot with their day, and they’re pretty knowledgeable to begin with, so they’re fun to write for. They’ve got a good background. They’re also, interestingly enough, highly collaborative.

Stacy Kornluebke: One of the things that I don’t think we’d addressed enough is how cool Sumo users really are. I watched them in a training class helping each other, and I don’t mean people from their own company. They were answering questions for people for other companies. They’re very proud to know the product, and they share it well.

Stacy Kornluebke: I happen to be a member of the customer service team, and it’s a really nice place to be, but everyone at the company has a huge drive towards making customers successful. I can go to Bret frequently and ask how things would look or what’s going to happen or what’s the next design. I can often come to Riya and say, “Hey, what’s the Quick Wins team doing this week?” so I never feel shut out of what we’re doing, and I feel that it’s always highly based on customer feedback. They work very closely with them. Jen’s giving me the face, but I can always come to her for compliance and security reasons. She’s really a very great resource if you need to understand GDPR.

Stacy Kornluebke: Sumo Logic also has a great sense of fun. What you’re looking at, in case you’re worried, is a suitcase full of 70 Sumos. The squishies that you’re going to receive tonight are extremely popular with our students. We didn’t get them shipped in time, so in addition to other interesting things I have done with my life, I took 70 Sumos in a bag through JFK, and TSA did not ask me one question.

Stacy Kornluebke: Think the last thing that I want to talk a little bit more about is that when I came here, we had this great phrase about how Sumo Logic was going to be democratizing data. At first, I was trying to imagine, what did that really look like? The more I learned about the product, the more people shared information with me, the more I understood the importance of having information at your fingertips. A lot of information usually gets siloed off, and in my world, when I get siloed off, I don’t really know as much about the customer as I think I do. My whole goal is to make sure that they learn about the product.

Stacy Kornluebke: All right, so if everything’s perfect, and you came here, and you loved it, what’s the big challenge with Sumo? What have you been doing for the last 18 months? Christian’s giving me the face, so I think he’s wondering too. To be perfectly honest, there’s a lot of challenge in the world of learning. The world has kind of gone from a directed approach. “Hey, I’m going to tell you what to do. These are the five steps to use this toaster,” to an open approach.

Stacy Kornluebke: There’s Stack Overflow, there’s GitHub, there’s just Googling it, right? Everywhere you go, you try and find the answer to your question. If you want people to use your product correctly, you need to make sure that you’re answering their questions because if you’re just telling them how you think it’s being used, they’re going to stop coming to you, and they’re going to go someplace where they can find the answer. That may not be a good answer, and it may not be a professional answer, but it’s giving them what they need. Users absolutely have the right to expect that you’re going to help them accomplish the task they want with your product.

Stacy Kornluebke: What do you know about your users? The first thing to kind of try and answer their questions involves understanding your users, and so, we’ve talked to you through two presentations about Melindas and Andres. We’ve talked to you from a design perspective and from a development perspective, but from my perspective, they need to know about the product. Melinda’s on it. She comes to Illuminate. I have a million Melindas. They tell me that the docs are like the Bible. They love it. They go into it every day. That’s because they use the product every day. Our product has a query language, and for her, it’s a second language. She knows what to type, what to find, where to go.

Stacy Kornluebke: Andre is a less frequent user of our product. I don’t know if you’ve ever tried to speak a foreign language that you only use once a month, but as you can imagine, trying to look things up, hoping that you got the right translation, the right inflection, that you understand the subtleties of the language, it’s really not going to happen if you pull out one of these dictionaries. It’s also not going to work quite so well with Google Translate, so you want to be able to get advice and understand what do these terms actually mean and how can I accomplish my task.

Stacy Kornluebke: If you’re trying to understand how to grow your knowledge in any part of your product, use the data you have already. Usually, Google Analytics is quite popular. Everybody uses it. No one knows what to do with the data. I suggest that you get the data, and understand a few things about what you want people to do with that information. For us, we were extremely excited to have people coming in, but then, we had to spend a lot of time figuring out, are they spending enough time on page views, do we want them to leave this page and find other pages, or do we want them to exit? Google will tell you what people are doing. You need to understand whether those behaviors are helping or hurting them.

Stacy Kornluebke: We also were very fortunate to have marketing reach out and give us information on SEO because a high number of our users Google. We needed to know whether or not we were obeying search algorithm rules. We also use net promoter scores. I don’t know … How many people are using NPS? Does anyone want to … Okay, so I see we’re talking to maybe 10 people. All right, so, the way NPS works is that it’s a ten … Excellent. No. You’re using my argument against me. I like it. I frequently do that to other people in the audience, so I deserved it.

Stacy Kornluebke: Net promoter score. On a basis of one to 10, a nine and a 10, those people are going to recommend you. Anyone below a nine isn’t as thrilled about you as you think they are. While they’re using you, they’re not excited about you. Hopefully, when they give you that rating, they’re going to tell you something. If they say, “Great product, but the docs suck,” I am the one who will find you and say, “Hey, what can we do to make it better?” but in these scores, that information does give you a sense of whether or not they’re truly satisfied. If only a nine or 10 matters, then you need to step it up and make sure you’re not making a seven or an eight.

Stacy Kornluebke: We also have this cool direct feedback mechanism in the docs, so down here where you see, was this article helpful? Yes, no. Leave feedback. I know that doesn’t seem that revolutionary, but we get feedback twice a week to our information based on this from customers. That’s a high level of participation. People come in. They feel very comfortable. They can say yes or no, but frequently, they can say, “Hey, I was expecting it to do this.” So long as you’re responsive to that, people will come back to you.

Stacy Kornluebke: We were also fortunate because we’d spend so much time making data visualizations. We could use our own tools, which was pretty awesome and a great kind of group project, and then again, I have to thank the UX team for giving us their customer journey because we could see the pain points people had with the product and try and align them to whether or not they were having a problem reaching information, training, documentation. Was it just an education issue, or was it something that was more of a product problem, and we were not going to get doc around it? It was going to have to be fixed.

Stacy Kornluebke: I speak a lot to search because 60% of our users and specifically, our struggling users, are coming to us for this reason, but if you are trying to figure out how to reach more users, find how they find you. If that’s through your Twitter feed, if that’s through your Slack channel, make sure that you know how your most struggling users are coming to you because if you aren’t speaking their language and you’re attempting to force them to use another tool, good luck. I don’t think anybody wants to learn one more thing. They want to find it the way they normally find it.

Stacy Kornluebke: Also, I recommend using your own tools. I know I’ve said it. I know you’ve seen all of us pull a dashboard showing how we use our product with our data, but what this allowed us to do was to take our certifications, take our data, and show it to sales. We pulled in customer success to make a meaningful dashboard. It became a group project. By involving more of the company, you’re going to get more people starting to understand what customers are struggling with and how they can help them. Frequently, our salespeople come in and they see who is certified and who isn’t and whether that makes a difference in whether people buy or renew. A certification is a great way to learn about our product, and it’s one indicator of whether or not people understand and feel confident with it.

Stacy Kornluebke: Okay, so how do I get answers to them? Well, everyone’s going to have to come up with a different way of doing this. For us, we found that a number of users desperately wanted some way to interact with a human being. They wanted immediate answers, and whether that was a Slack channel or that was an in-person training class, they were happy. Just bring me somebody who can answer my questions. The first thing we did with certification and training was we started holding what we called Cert Jams. We took the user classes that we hold at Illuminate. We came to cities where there were at least two, 300 Sumo Logic users. We said, “Hey, there’s a free class if you come today. Come in. Learn. Get a t-shirt.” Jane and Kerry were helping me out today, so somewhere around here, they’re wearing their t-shirts. Yes, so that, believe it or not, was a big thing for people.

Stacy Kornluebke: Quick answers to questions online as I’ve mentioned before. I know SEO is kind of this dirty word in the engineering world. It’s what marketing people do, and we shouldn’t touch it, but if you don’t, you don’t get the results you want. Then, just quick answers in real time, so if you can’t talk to a person, if you’re out on Anchorage and you’re using our product, you can still get on Slack. Anyone can find and ask a question.

Stacy Kornluebke: Here’s a picture of our Cert Jams. This is kind of cool for people. They’re really happy. We call this the t-shirt shot at the end. When they get their certification, they get their shirt. For every city that we go to, we try and take at least one of these. We’re doing at least 25 this year, and we hope to do 30 next year, but this is really a great networking opportunity for anybody who uses our product, and it’s a chance for people in our company to kind of get to know our users in a non-sales context.

Stacy Kornluebke: These are all the mistakes that we made with SEO. Please don’t make our mistakes. Please make new ones.

Stacy Kornluebke: We, first of all, did not know how search engines worked, so we had to use the SEM rush report to understand things like the algorithm will punish you for dashes and for underscores instead of dashes. Good to know. Our tool was automatically generating underscores all the time for us to take out those spaces, so we had to change the default setting, and magically, our documentation was popping to the top for our search results. Good to know.

Stacy Kornluebke: Use their terms. One of the things that I also find with learning materials, with documentation is we frequently try and force people into an exact definition of exactly what we’re doing, which is great, but it’s not the term people are searching for. A classic example of this is we talked very specifically to two-factor authentication because there were two factors when a lot of people were looking for MFA. We do not have the right to try and teach them the difference between multi-factor and two-factor. We can take them to the two-factor page, and then, they can see the difference between that and multi-factor, but please just get them to the information that they want. Use the term they use even if it’s a little hacky. They want to find their stuff.

Stacy Kornluebke: Then, the last bit is please avoid stubs. I’m just going to give you this bit of an SEO advice. If it’s under 200 words, your search engine hates you. I know there’s this great trend to try and bring down documentation to short little stubs, easy-to-read pages. Please bind up all those little stubs into a single page. You user will thank you because they don’t want to click through 60 pages, and your search engine will thank you because then, it will think it’s a real document. You can also make your own SEO mistakes. I recommend it. Keep having fun, but get people what they need.

Stacy Kornluebke: We also created an in-product learn tab. What we discovered was that people just want to go to the product, find what they need to do to learn, and go through stuff. Well, that’s great, but I have a 30-video library, and there’s no easy way to stick that all in the product. What we did was we had people from the UX team help us out with a design, and engineering helped us implement just kind of a short five-video series. We used APIs from our knowledge base to pull in the tutorials, and then, we provided quick links to other parts of the product. Now, this is really useful to a lot of people. They come in. They watch one quick video. They review the tutorials, and they’re done. It’s not a long-term thing. It’s a help in onboarding, but it made a big difference.

Stacy Kornluebke: You also sometimes have to accept change. As wedded as I am to SEO and I like forums, and I want information to be found, and we’re going to be the next Stack Overflow, a community where you have a timed response where people have SLAs and they’ll get back to you even within a day is not as cool as it used to be. People like Slack. Slack offers real-time response to questions. It makes people happy. They’re not really concerned if the next person can Google it. They’re getting the answer they need. Being where your users are is kind of what you need to provide.

Stacy Kornluebke: Also, just remember this is a process, so use any data that you have, CSAT, NPS, Google Analytics. Whatever you’ve got, start trying to understand your users. Please work with any group that you have that has similar customer based mindsets, so if you’ve got a UX team, if you’ve got a customer success team, if you’ve got a field full of salespeople that desperately want to help customers, reach out to them. Don’t hesitate to ask a couple of questions to do your job better.

Stacy Kornluebke: I also recommend that you join some grassroots movements. I like Write the Docs. It’s a bunch of people that take a more hacker-based approach to writing documentation, but it’s also just a user conference where people come who work with problems every day. We can talk about our struggles with trying to use analytics data. Talk to your users directly. I don’t know, depending upon your corporate culture, how open people are to that, but Sumo’s really never barred us from reaching out and saying, “Hey, how could we make things better?” if it’s just sitting silently on a call with someone who is working with the customer, that’s good too. Understand what people are really saying about what you’re offering and what you’re teaching about the product.

Stacy Kornluebke:  I want to say thank you. I know I gave you a to-do list, but that’s kind of what I do. Now, we’re going to hand you over to Jen who’s going to talk to you about security and privacy.

Jen Brown speaking at Sumo Logic Girl Geek Dinner.

Jen Brown: I’m Jen Brown. I am DPO here at Sumo Logic. As you see, I control everything security and privacy as far as GRC goes. That means I get to work with all of the external auditors and do all that fun internal audits policies. I’m putting you to sleep already, so yes. I’m also a contributor to Dark Reading. If any of you aren’t aware of what Dark Reading is, it’s a really great resource for technology news, so check it out if you haven’t and don’t know about it. I’ve been in this space for over 20 years. I’m a grandmother. You can see. I’m blessed to be booed. That’s what they call me, grandma to Lucy and Will, and then Max is in the middle there. He’s over here sleeping. He’s our security mascot. Happy to be at Sumo. I’ve been here for about two years now, was here for about six months as a consultant, liked it, so I came on. We kind of did a try before you buy. They made sure they liked me. I made sure I liked them. Luckily, it all worked out.

Jen Brown: Our group is broken into three different groups. We’ve got the Security Operations Center, so we’ve got a manager for that who’s building that out. We actually are just beginning that journey. As I said, I do compliance, all our external audits. You can see we have PCI, SOC 2, ISO, CSA Star, HIPAA, Privacy, and then we also do risk management. We’re also looking or going towards FedRAMP certification, so a lot of fun to be had, and then DevSecOps. We’ve got an engineer that we poached from finance, which sounds really strange, but our last two hires have been from the finance groups. I have to be really careful when I’m walking around that group. I have to kind of skirt it, so nothing hits me, but he’s helping us to automate everything possible. I’m going to show you one of the things he built for us to save a lot of time and make our customers much, much more happy.

Jen Brown: What we’ve got here is we’ve got a self-service portal. When I first came onto Sumo, what would happen is if a customer needed something like our PCI AOC, which is an attestation of compliance, the salesperson would go in. They’d enter a JIRA ticket. It would come to us. We’d have to make sure there’s an NDA in place. There was this back and forth. Depending on how busy we were, if I had audits, whatever, sometimes, this could take three to four weeks to get a document. Doesn’t equal happy customers at all.

Jen Brown: What Mike built for us … Let me just pull this up real quick. Hey, Brian. Is Brian here? I was going to take you through the portal, but this will be good enough. There’s an NDA on the front of it, which really helps us reduce that time of making … We don’t have to go and make sure there’s an NDA with a customer before we let them see this. They agree to it, and this is what they’re going to see. They’re going to see that they can pick from any of these documents here to get sent to them. If they’re our customer, so if they’re in Salesforce or they’re a customer or prospect in Salesforce, they’re going to get those documents right away. I mean, instant. Just no more two to three weeks. They get it right away.

Jen Brown: If they’re not in Salesforce, there’s going to be a little bit of investigation that goes on. Jane who works with us back there, she’s going to go to the sales team, and she’s going to find out like, should this person who’s requesting this actually get these documents? This has made our customers really, really happy. It has cut down a lot of work for us too, which of course, makes us really happy. The other thing we’ve done that I was going to show you is we put our DSR portal on here. Anybody working with GDPR? We put our data subject request portal on this as well, so it’s really automated a lot of what we’re doing.

Jen Brown: All right, so, as you can see, just in a quarter alone, we had 546 from customers and 616 requests from prospects. That’s a lot of JIRA tickets we didn’t have to deal with, and again, a lot of happy customers. It’s decreased the time, as I’ve said. It enables us to be more transparent with our customers because we’re able to put more documentation out there, and we’re always trying to find new things that we can put out there. At first, it was just the attestations and certifications, and now, we’ve got our pen test results out there. We’re always trying to provide more.

Jen Brown: The other thing on that portal that I wasn’t able to show you because it got cut off is we’ve got a place where customers can come to us and say, “You’re not compliant with fill-in-the-blank.” I mean, there’s so many laws and regulations out there. They’re able to tell us what it is they need us to be compliant to. There’s the German privacy law. There’s the New York CFR, so on and so forth, so we’re able to get that from them, which I think I’ve got a minute to teach.

Jen Brown: Yes, so we’re able to, instead of just throwing a dart and trying to figure out what maybe customers want, we’re able to hear from them. We’re hearing from sales all day long. We love sales, but it’s better to hear it from the customers and really see what it is that they want. Just based again on a quarter’s worth of data, we found out that 11 customers really need it. I’m not even going to try to … Maybe you can pronounce it, Christian. It’s a German privacy law. They need us to go get compliant with that. New York has got a new financial services law on the book. We’ve got 11 customers who need us to look at that.

Jen Brown: Australia has got our IRAP. We’ve got another 11 customers who are asking for that. Then, GLBA, we’ve got 10. There’s some others that people are asking for, but we’re able to really see what people are looking for us to be able to show and demonstrate compliance against. That’s really helped us to build that roadmap instead of just guessing because there’s hundreds you can choose from.

Jen Brown: All right, so that was mine. Mine was quick, sweet, and short, but we’re always trying to automate what we’re doing in security, so that way, when we build our roadmap, we’re building it correctly and adding more and more and more every time that we do. Our customer input influences all our decisions at Sumo Logic. I mean, it’s just a really, really big point. Customers are so important. That includes how we design, engineer, like end-to-end. It’s amazing. We share a common goal and partner closely with our customers, impact cross-functionally. I mean, as the women said, it is really end-to-end. This reflects the core value of what we … We’re in it with our customers. All right.

Shea Kelly: All right. Thank you, Jen. Everybody still with us? Happy Monday still? Okay, okay, okay. Good, good, good. We wanted to set some time aside. Is that too loud? I feel like I’m talking really loud. Set some time aside at the end for some questions, so I thought we’re going to scooch … How about if we scoot our chairs over? Does that work? Is this not working anyway? While we’re doing that, I actually want to shout some thank yous out, so first and foremost to Angie, and to our team Stacy and Bronwyn for the recording and all the help to get this recorded for us, so we’ve got it in perpetuity.

Shea Kelly: Second, if she’s back there, I want to shout out to Tori Lee. Is Tori here? All this food, beverage, everything, Tori is magnificent. She does this for us every day, and I was thrilled she was able to do it for all of us here today. Obviously, I want to thank our panel, our fabulous panel. I want to thank all of you for coming. We’re going to shift to a few questions, and we’ll see. We have mics we can hand out and around, so if anyone has the questions, just put a hand up, and we can bring a microphone to you, or you can shout it and we can probably just repeat it. Okay. Max is out. Yes, we are very dog-friendly. At any one point, what do we have? Like 50 here a day? Yes, that’d be great. Yes. Thank you. Is he up? We’re just going to adjust lights a little bit here. Did somebody have a question?

Audience Member: Yes. Right here.

Shea Kelly: Oh, I’m so sorry. Okay.

Audience Member: First of all, thank you. The set of the presentation is one of the best I’ve seen because you touched on real everyday life in a company versus how you, being a woman, makes a difference through your job. No. Thank you. I mean, I got from each of the presentation something that I could relate to but that brings the question to Bret and Riya. You both … and actually, to the learning lady.

Shea Kelly: She had to leave, just so you know.

Audience Member: Yeah. All of you, your presentation talked about the product management but there’s no product management talking, and each of you … Obviously, I’m in product management, so I’m concerned, but you are doing things that almost, throughout my career, part, all of it would fall under my responsibility. Can you talk about the product management dynamics in the company?

Shea Kelly: You want to repeat over there so …

Jen Brown: Louder.

Shea Kelly: Do you guys have mic? Can you turn it on?

Jen Brown: Product management dynamics. He’s Bruno. He’s one of our co-founders.

Bruno Kurtic: Hi. Could somebody-

Riya Singh: Hello.

Bruno Kurtic: Hi. Could you repeat the question because nobody behind here could hear it?

Shea Kelly: This is Bruno Kurtic.

Bruno Kurtic: Here. I’m going to give you the mic. In fact, we’d throw with this thing around.

Audience Member: I’ll shorten the question. First of all, the compliments still hold. One of the best set of presentation in Geek Girl Dinner ever because it touched on real life topics. Instead of what being a woman in tech means, you talked about what doing my job as a professional means, which I can … and happily more to relate to, but back to you. Many of the responsibilities that I as a product manager or head of product view as my responsibility were amazingly well-described by people that their title is not product management. Can you describe the dynamics of product management in the company interacting with UX, with engineering, with security?

Bruno Kurtic: Sure. We have a really easy job in product management. We just sit around and all these guys do everything else. That’s how it works here. No. Just to be serious, we don’t actually talk about ourselves as product management or engineering. We actually call ourselves product development, and all of these people here are in product development, bar one, who has a big input into product development. We run very integrated teams, right? Everybody has their primary responsibility, product, for strategy and requirements, user experience for design, engineering for architecture, things like that.

Bruno Kurtic: Ultimately, we basically break teams up into small units that have cross-functional team members who do the things that are necessary for that unit to succeed. Usually, that’s rallied around specific customer outcomes, so when we build things at Sumo, we don’t build features. We build outcomes. We focus ourselves around whose life are we going to make easier if we do X, Y, and Z. That’s why when you talk to everybody across the company, you’ll hear a lot of things that are relevant to other cross-functional topics. We usually don’t have very stringent decision-making that you get to decide this, you get to decide that. The team decides. They do the best that they can. That’s why.

Audience Member: Thanks. Great. Thanks.

Riya Singh answers a question from the audience at Sumo Logic Girl Geek Dinner.

Riya Singh: I’ll just add a little bit more to it. Firstly, we’re missing one of our PMs — She is on a maternity leave, so that’s why there’s one missing function here. I work very closely with Lavinia and all the other product management as well. I think when we said that engineering takes in customer input, obviously, the vision comes from the product management, right? It’s not me you or it’s just her or one person. It’s all together, right? Customer experience is important to us. Especially with things-

Audience Member: It was a compliment because you said product management, so I think I was wondering you have no job for the product manager. It was a compliment because there was no dedicated speaker as a product manager, but you spoke product management. It was a compliment. That’s why I was asking.

Riya Singh: Yeah. I hope Lavinia will be back soon, and we can talk more. Yeah. More questions, guys?

Audience Member: Yeah, I got it. First, thank you for the wonderful presentation and the hospitality. You talked about that collaborative environment, for example, when there is a problem, you go to the war room, you drop everything, and then you solve the problem. My question is, how does this culture scale when the number of projects, applications, and customers grow?

Riya Singh: More war rooms. Every room is a war room. The concept of war room was particular to outages. The definition of an outage is that it shouldn’t happen too often. It’s not that you’re spending all the time in the war room, right? We are working very hard to make sure that these outages don’t happen over time. As I said before, no repeat offenders, right? It may sound stressful when you are in this war room, an outage situation, but it’s not frequent. The frequency is just going down with time. Does that answer your question?

Audience Member: I was stressing the example, the war room, but on a larger scale, the idea that there’s cross-functionality, collaboration, I think that’s a recent change in culture. How does that scale?

Riya Singh: If the team is larger, how do we make sure this cross collaboration happens?

Shea Kelly: Maybe Jen, you …

Jen Brown: Yeah, so communication. We’re always working together. He thinks it’s a laser. Sorry, he’s very excited. Sorry. We work really closely together. We make sure that there are people who are on point for when things like outages happen, items like that. I mean, even though we’re growing, I don’t know if it’s just unique to Sumo, but we just haven’t seem to have that problem yet. I mean, Christian?

Christian Beedgen: We’re always… It’s a divide and conquer sort of strategy. We make room for more teams. We make more teams, so we keep them cross-functional. Then, over time, as we need to grow, we needed to grow more leaders as well, and then there’s sometimes an additional layer of cross-functional leadership discussion, so to tie it all up, it’s a tried matrix, I think.

Jen Brown: A good example of that is when I first started in security, we had a security engineer, we had me. Now, we have DevSecOps. We’ve got the SOC engineer, so we are growing and adding more teams and functionality like Christian said. Does that answer better? You look like … Okay.

Audience Member: Who are your direct competitors, and then how you differentiate yourselves from them?

MaryAnn O’Brien: All right, so good question. When I came to Sumo, I actually have a lot of friends that work at one of our direct competitors. As I started to research … Actually, to be quite honest, I had never heard of Sumo Logic when a recruiter had reached out, so I had an opportunity to research and do my own level of understanding in terms of the company itself. I’ll just tell you one of the primary competitors that we’ll see especially on the enterprise side is Splunk, if you’re familiar with Splunk. Another … I lead a mid-market sales team, and one of the actually big competitors that we see a lot is open source, so if you’re familiar with ELK, they’re actually … We run into them quite a bit more actually, just as equally, probably about 50/50. In my particular segment is ELK, as well as Splunk are the two primary.

MaryAnn O’Brien: Overall, that’s mainly on our log side, but we also have a unified logs and metric strategy, which includes metrics, so every so often, we will also see some monitoring and metrics type of competitors like Datadog as an example. Does that help?

Shea Kelly: Anybody else?

Audience Member: It’s a really big microphone. Yeah. All right, it’s awesome. I always want to pause a microphone. Thank you for the presentation in [inaudible]. It’s really good. My question is that, listen to your customers, and since you have so many customers, I’m pretty sure there’s a lot of feature, there’s a lot of requests, a lot of I wonder if some want that. Based on experience, what do you find the best way to prioritize and how do you choose what feature or what new things to work on first?

Riya Singh: That’s very in the PM to answer that question about prioritization. Bruno, you want to take that?

Bruno Kurtic: It’s a loaded question, but we start with the overall strategy. What is the true north? What are we trying to achieve as a company? I oftentimes tell people that strategy’s knowing what not to do. It’s not knowing what to do. We start with the strategy. We align those strategies with customer outcomes. We focus on certain set of things as few as we can. It’s not always easy, right? You always try to kind of do less, but you end up doing a lot more than you probably can, and you should. We try to really tightly scope what problems are we trying to solve.

Bruno Kurtic: We try to sort of align on what is the minimum amount of work we need to put in to produce something that actually changes the outcomes for the customer. We really like to work agile and deliver products out to customers. We’ve built a very sophisticated way to surface new capabilities in production to individual customers even though we’re a multi-tenant SaaS service, so we have very fine-grained controls over what we deliver to customers. We did that because we wanted to be able to give customers things early, get feedback, iterate, iterate, iterate, improve, and that’s how we develop. It has to be aligned with strategy. It has to be aligned with positive customer outcomes. It has to be in scope of certain time horizon, usually six to nine months, and then, you just fill the backlog, and you burn down.

Audience Member: Thank you.

Shea Kelly: Any other question?

Audience Member: I heard the mention of customer success. I was just wondering if you also have a customer support team. Is it one thing for you guys? How do you deal with like 24 by seven support or service?

Riya Singh: Our learning lady is missing. We do have customer success and customer support. The way I understand it, so we have our support portal where customers can put in support requests. Customer support is mostly trying to solve their immediate use cases or reporting bugs or something is not working, helping them troubleshoot. That’s customer support. Customer success is a little more broader. They’re trying to make solutions to help our customers. They’re actively trying to find out why this customer is not giving us a good score or how is he using the product, and how can we increase usage adoption within their accounts? They’re effectively trying to make them more powerful in using of the product.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Scaling Sustainably: Girl Geek X AppLovin Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

AppLovin girl geeks: Katie Jansen, Alice Guillaume, and Helen Wu speaking about scaling sustainably at the AppLovin Girl Geek Dinner.

Katie Jansen / CMO / AppLovin
Alice Guillaume / Director, Marketing / AppLovin
Helen Wu / Director, Growth Partnerships / AppLovin
Swetha Anbarasan / Network Engineer / AppLovin
Laura Pfister / Software Engineer / AppLovin
Anusha Ramesh / Principal Software Engineer / AppLovin
Sonal Gupta / Principal Software Engineer / AppLovin

Transcript of AppLovin Girl Geek Dinner – Panels:

Katie Jansen: I’m Katie, I’m the CMO here at AppLovin, and we just wanted to thanks to Girl Geek X. This is awesome. We haven’t really done anything like this before, so this is our first time, and they’ll probably ask you for feedback at some point so feel free to give it to us. This is a new thing for us. I would also like to thank Brianna. Brianna, where are you? Brianna is the one, and Marissa who is probably not here right now, she’s out front, they really put all this together, they got all the food, which was outstanding, and the color coded name tags, and they were down here putting the seats here today all on their own, so I just wanna give a big shout out for all your hard work, thank you so much.

Katie Jansen: If you don’t know a lot about AppLovin or even if you do, real quickly, we’re a comprehensive platform that connects app developers of all sizes to about two billion global users each month. We have a growth monetization and publishing services that really focus on giving game developers the expertise and insights they need to grow their business. Whoops. We’re only about six years old, and we, I think we have about 160 employees, 165, and up until recently we were self-funded. So we’re actually pretty small compared to a lot of the companies around here, but we’ve really experienced hyper growth in the past few months, even just this year, you could say. We have about seven offices worldwide and our valuation is a little north of two billion. So we have a very few amount of people doing a lot of work and producing a lot of great things, well beyond just the revenue we’re bringing in.

Katie Jansen: Today we thought we could partner with Girl Geek to give you a real unique perspective on how to adapt when you’re in a fast growing company, because we have a very strong perspective on that, and we grow our own roles kind of within that company. Today I have Helen and Alice, and they’re gonna kinda start things off with me. We’re gonna get the business perspective on that, and then next we’re gonna move over to engineering and ops and really get a more technical perspective on how we scale and how we’re adaptable. And oh, I’m sorry, there was an agenda as well. Panel one and panel two, and then we can eat and drink some more. Alright, so let’s start things off with you guys just telling me a little bit about what you do here at AppLovin and about yourself.

Alice Guillaume: Hi everyone. This is really great turnout, I’m super excited to be here. A little bit about myself, my name is Alice. I’m the Director of Marketing here at AppLovin. I’ve been here for four years. My journey here started out on growth and as the business has scaled I’ve taken on creative services and most recently, I built out that team to be over 30 people, and it’s currently one of the largest teams in the business.

Helen Wu speaking at AppLovin Girl Geek Dinner.

Helen Wu: I’m Helen. I’m the Director of Growth Partnerships at AppLovin. I joined in 2013, and since then, have grown and trained our global teams who are responsible for helping our customers achieve their performance goals. And a little fun fact about me, I used to attend these Geek Girl Dinners like six years ago, so in 2012, I was in the audience like you guys learning the stories and experiences of other geek girls, so I’m very excited to be here today.

Katie Jansen: I didn’t know that. You saved that for this. So you guys both come from a different background. It’s not like you were in this industry when you first came to AppLovin, so what drew you to AppLovin, and what do you enjoy most about being here?

Alice Guillaume: Why don’t you…?

Helen Wu: Okay cool. So I joined because I wanted to be around the smart and hardworking people in this company. I wanted to learn how to build a business, and before I joined I was at a large bank. I did internet … I did research on internet trends, and I learned that mobile advertising was gonna be growing at a tremendous rate and we were just at the very beginning of this long term growth trajectory. So in 2013, AppLovin was, had already been operating for about a year and a half. They had invested heavily into building the platform and product, and was starting to make their first business hires.

Katie Jansen: I remember interviewing with a couple people at the company and just being so inspired and in awe of how smart and passionate they were. I was like, I just wanna be around these people all the time and work with them and learn from them, and you know, because I knew where mobile advertising was headed, I really believed in our CEO Adam’s business vision of us being this growth engine for app developers. So I knew there was gonna be endless opportunities to learn and to grow with the business. So I joined and I haven’t looked back since.

Katie Jansen: Alice?

Alice Guillaume: My journey’s similar but a little bit different in a sense that I had also come from a very structured background. I had grown up in management consulting, banking, sales and training, that’s what I thought I wanted to do with my life, and until I reached a point where I realized I needed something more. I wanted to work in a place where I could make an impact, where I could bring ideas and promote change. I needed that in my career.

Alice Guillaume: I did not know very much about mobile advertising to be totally honest, and I took a chance and I took a risk to interview at AppLovin, and I was nervous because it was a deviation from a very structured route, and actually taking that risk was probably one of the best decisions I’ve made.

Alice Guillaume: And two things drew me to AppLovin — one, and I think this echoes what Helen said is the people, and two is the learning opportunity. So for me when I was interviewing, I realized that everyone was so smart, and so much smarter than who, than I am, but they were down to earth enough to wanna teach me. So I thought that was really cool. Two was that leadership really cared about my growth, and they were invested in helping me get to where I needed to be. That was also something that, for me, made a big impact on my career here.

Alice Guillaume: And three was the mentality about work. People here are really passionate about making things better and not doing things just for the sake of doing them. If something is menial, or something is low value, let’s find a way to like automate that. And that really spoke to me because it demonstrated the company’s value in innovation and change. So all of those things really inspired me. I wanted to be around those people, and I wanted to be a part of that race.

Katie Jansen: And the fourth reason is Helen, right? Didn’t you guys go to school together?

Alice Guillaume: Yes. Fun fact about me and Helen. Helen and I went to college together and she was my big sister there, and that’s how I learned about AppLovin. And she was gonna kill it when she joined.

Katie Jansen: Right. And is there anything surprising you guys have learned since being here?

Helen Wu: Yeah absolutely. So I mean I think it’s just learning that I would be constantly stretched and challenged all the time. I mean before I joined, again, I was at a large bank where the work environment and culture was very different. Even though the financial markets were changing every single day, I felt like the mindset and the processes were very routine and very stagnant. So I wanted to be in a place where I felt like I could add value, where I could learn something new every single day.

Helen Wu: The most important thing I learned over the last five years is the value of being fast and flexible, which is staying in tune with changing market opportunities, being ready to action, adapt, and re-adapt. And I think the speed and mindset with which we operate is really the key to our tremendous business growth that we’ve had since 2012.

Helen Wu: Also, you know I think everyone in this company has the ability to wear a lot of hats to help out in ways that are beyond what’s just being asked of them. So I think really people, I think people here are very, very keen to like help the business but they’re also very invested in helping each other. So you know I think hiring people who can add to this culture of continuing pushing each other, you know, challenging each other to ask intelligent questions, to take risks, and to do better than their last is really important for continuing to grow our business.

Katie Jansen: So you guys have grown from being individual contributors to being managers now. You both manage teams. Alice as you mentioned, Alice is actually on my team, and then you have one of the biggest teams at the company right now, and you have a big team too. What do you guys typically look for when you’re hiring, because lately it feels like we’re always hiring. So what do you look for? How do you kind of figure that out in an interview? What are some tips you can give to these guys about that?

Alice Guillaume: Definitely. So I’m very passionate about hiring and recruiting. The resume is important, but for me it’s really the human and the psychological aspect of who you are. When I interview candidates there are two main things that I care about.

Alice Guillaume: The first one is ability to learn. So you will hear throughout the theme of our panel that the only constant is change, and that’s, I think, a core thing that has drive our company to be so successful today is constantly evolving.

Alice Guillaume: To be able to move that fast, we need to hire people who are open to learning, who are open to self improving, and who are receptive to knowledge and feedback. For example, when our team ramped up from 15 to over 30, that doesn’t happen over night. That’s a collaborative effort of everybody on the team from the individual contributors to the leads, and that really requires that openness and heart to be flexible.

Alice Guillaume: The second thing I care about is grit and passion. So I think that speaks to the first one is be receptive, be open to learning, and two is apply that in your day to day and be able to put in the amount of work that it takes, and you need to have the passion to be able to wanna do that. Those are things for me. You wanna add anything, Helen?

Helen Wu: Yeah definitely. I totally agree with everything Alice said. I think we spend a lot of time thinking about hiring and who we hire and why we hire. I also wanna add that I look for candidates who have really deep comfort with data, simply because we have a lot of it, which is readily available and waiting for someone to take, extract meaning out of it. So our growth analysts do the really important work of analyzing the data, building models that can inform us of, you know, how we can drive better performance for our customers.

Helen Wu: The second thing I look for is some love of solving puzzles of any kind. So even if a candidate doesn’t have quantitative or technical background, you know, I look for any indication that they might be, for example like a chess master, or an avid player of strategy based board games, anything that suggests that they’re a strategic problem solver who can look multiple steps ahead and formulate a plan. And these kinds of things demonstrate to me that someone possesses deep problem solving skills, which we need in our business to continue to grow the way we have.

Katie Jansen: And so what about, okay let’s someone’s gotten hired, they’re on your team, so let’s fast forward a little bit. How do you guys as managers coach and manage, you know, keep your staff motivated? Again we’re small, we’re scrappy. How are you able to kinda get in there and work with them in way that’s not micromanaging but also helping them move forward? Especially when we change goals all the time, right, or we’re flexible or we fail fast, right?

Alice Guillaume: Yeah, go ahead.

Helen Wu: I think it’s a really good point, and I think in addressing staff motivation, I think it’s important to also discuss the culture and the space in which we work. So we have a very open office concept where analysts, managers, and VPs all sit next to each other, and they’re sitting next to other functional teams. We encourage a lot of questions to be asked openly, a lot of ideas to be raised, and so problem solving, learning, and teaching happens organically at our desks and out in the open, and it happens in every sort of of like direction you can imagine between levels of the organization and across different functions. I think those are really great opportunities for people to learn and to grow all the time.

Helen Wu: And then, on my team we do regular check ins between managers and the employees where I like to take our employees out to coffee, and it usually starts with asking questions like “How are you doing,” “What’s going well for you,” “What do you need help with,” and these are important opportunities for an employee to share their ideas and questions that they have, or goals, and get support from their manager on how get there. And I really like that these are outside of the office, like over coffee, because it’s a more casual environment where people can connect on a more personal level and dive deeper into the discussion.

Alice Guillaume: This is a really good question. I think there are a lot of things a manager can do in the day to day to help foster this, but for me the fundamental value, and I don’t take any credit for this ’cause someone else taught me this, is empathy. So for me it’s about recognizing that every single member of your team is human. So for me it’s spend time with them.

Alice Guillaume:  I do walk around on my team and other teams, probably ’cause I just like talking to people a lot, but I like getting to know them, getting to know then in a work environment, but also outside of work. As a manager that also helps me connect the dots when I see opportunities for people who can help each other out or would just be a good fit to talk to one another, whether it’s a mentorship relationship or not. So that’s one thing.

Alice Guillaume: Another thing is with my direct reports, don’t be afraid to talk about things that are not work related. I think building that relationship is really important to having tougher conversations too. That openness is super key. That last thing I would say is have your EQ feelers out. So how is the team feeling?

Alice Guillaume: I have a team of 30 people, it can be hard sometimes to know how everybody is feeling, but there are ways to kind of learn that, whether it’s through your own direct managers or just through day to day talking to people and being sensitive to that.

Alice Guillaume: Like if something’s going or the team’s feeling stressed out, can we take a break, can we go for a coffee, can we catch up, can we lighten the mood a bit. I’ll crack a joke, I’ll play some songs or something like that. Or if you have a life event going on in the team, like celebrate that. Their win is the team’s win, but most importantly it’s winning everyone over as people.

Katie Jansen: Yeah, I mean I think what I heard both of you guys say is it’s about the individual, and it’s about connecting in different environments. I will say having worked at other companies before, AppLovin is very unique in that we actually have a very low amount of meetings here.

Katie Jansen: As the CMO, I can easily go sometimes with only one or two meetings in a day. I’m not in back to back meetings all day, so that actually gives us the time to connect with individuals outside of your team and on your team, we really take the open office concept, like why have it if you’re not gonna actually leverage it, so we’ll just do impromptu meetings or conversations that’ll just start up right at the desks versus going into conference rooms and having meetings.

Katie Jansen: For me, having been at so many other, not a ton I guess, but at a few other companies, the concept of not being in back to back to back meetings and actually to connect with your team and other teams in a real environment, and also leveraging things like Slack and you know the different tools that we have is what makes a big difference too, which is what you said.

Katie Jansen: And what about mentors? How do you guys leverage mentors on the team? Do you leverage mentors on the team? Do you think it’s valuable?

Alice Guillaume: Absolutely, it’s valuable. So my team it depends, I focus on what the purpose of the mentorship is. So an example could be if I have a new manager, I want them to have a resource where they can talk to that person more candidly than they probably could with me to help them grow. In that case what we’ve done on my team is we’ve sourced externally through networks of our leadership team, to provide that person with a mentor, and we’ve seen that has been really effective.

Alice Guillaume: An alternative example could be just pairing a junior person and a senior person together on the team. That use case is more internally driven, so you know, making sure this person has a support on the team that they can learn from them about their day to day jobs as well. So absolutely mentorships are important. I had mentors as well. They’re your greatest advocate, they can help honestly point out areas where you can improve on more candidly, and it’s just nice to have someone to talk to sometimes.

Helen Wu: Yeah, definitely everything Alice said I agree with too. In thinking about how I think about mentorship, I think it’s also worth discussing like how I define it. I think mentorship is also a two way partnership where both parties can get a lot out of that interaction. Not only is it a way for the mentee to share their ideas and goals and get support and guidance from an experienced person, but it’s also a great opportunity for the mentor to teach, to pass on knowledge, and to get perspective on what’s happening at other levels of the organization.

Helen Wu: I think this is actually one of the very critical interactions between an employee and their own manager, so we invest a lot into this, definitely. And then as far as more long term mentorship that we have in the team, it’s most commonly gonna be when a new hire joins, we try to pair them with a more experienced analyst who can teach them about the role, about the business, and that’s a really great opportunity for the experienced analyst to practice their leaderships skills and to help develop the team.

Katie Jansen: So this is probably a harder question, but we have small teams here, we’re a small company, and so outside of this like, okay this is a new hire that’s coming in and they’re getting paired with someone senior, how do you help people find mentors? What are some practical ways? ’Cause I feel like quite honestly I go to these events and everyone’s like “Let’s find a mentor, it’s really important, and that all is organic. Thank you.” And I don’t think that … I mean I think that’s great, and I guess it should be organic, but there’s gotta be ways to kinda facilitate that process. So have you employed that at all, or what have you guys done?

Alice Guillaume: So I’ve tried … That’s a really good question. It can be tricky. I’ve employed both assigning on my team, so I look at kind of like what the needs are and I have paired people together, but I also allow them the flexibility to kind of figure out if it works for them. I tend to think if it’s more natural it’ll work a little bit better, but at least I’m providing an opportunity to that person to try it. The other thing I’ve tried is just observing. So there are natural kind of relationships that form through time just kind of, that’s just how humans are, and if I see that happening then I can have that conversations with both parties and see if that would make sense as a match. So I’ve tried both ways. I will say the one where I match it has probably has less chances of working than the organic one, but because there’s not always an organic fit, I still have to go try.

Katie Jansen: Do you have anything to add, Helen?

Helen Wu: Yeah, I think it’s definitely tricky to you know try to pair employees with new mentors. So I think the check-ins with people to understand where do they need help, how things are going for them is really helpful for understanding what they need. On the business growth side we … The nature of our work is very collaborative too, so I think there’s a lot of opportunities to build partnerships that are cross function. So for instance if two people, you know, are both familiar with a specific account or specific type of game we can pair them together to share their own unique experience and perspective, and so we build these partnerships based on business opportunities that are coming up too. And then, oftentimes, this is when we can pair an employee with a more experienced member of the team, and this person can serve as a mentor.

Katie Jansen: I wasn’t gonna say this, but I’m gonna, probably like a year and a half ago Adam, who’s our CEO, told me “Go mentor Helen,” which you might have suspected since I was like “Hello, Helen. You’re not on my team. Would you like to get coffee with me” randomly once a month at this scheduled time? But I’ll be honest. It probably took like about a year, ’cause for a while it felt maybe a little awkward and forced, but I noticed in the past probably five to six months, I’m like okay like this feels like a real, and we’re both getting something out of it to your point mentor-mentee relationship, it took a while sometimes I will say with the pairings, but you can hit your stride.

Katie Jansen: And then I have a mentor that I met, it wasn’t at this group. It was at Women in Wireless, that’s not their name now and I don’t remember their name to be honest with you. They rebranded. But it was a Women in Wireless event, and I was on a panel about mentors, and the EVP of revenue over there was just so impressive to me, and so I afterwards sent her a thank you note for being on the panel, and then two weeks later followed up with coffee, and now she’s definitely my mentor, and it’s been probably three years. And she’s not in the same vertical as me. She’s in finance and rev ops, which is very different, but I still learn quite a bit from her, but it was almost like, I guess, I was courting her or something. But she was open to it so it worked out okay.

Alice Guillaume: Sometimes you have to go get it.

Katie Jansen: Yeah I know. I think at that point I realized I don’t have a mentor, maybe I need one. So that’s all we had today, and I think we wanna open it up if there’s any questions from the audience. Yes, go.

Audience Member: I have a question for Alice.

Katie Jansen: Am I supposed to give her the mic?

Audience Member:  I can talk loudly.

Katie Jansen: Alright. Do it.

Audience Member: I was just curious so your title is director of marketing, which is fairly straightforward, of creative services. What does that include like more on a day to day basis?

Alice Guillaume: Definitely. And what’s your name?

Audience Member: Jessica.

Alice Guillaume: Jessica, okay. Good question. So yes I’m director of marketing, but I lead the creative services team. So what we do is we make video graphic and playable ads for our customers, and that is marketing because we are helping them market their apps and trying to help them sell their product. My day to day is, I manage a couple of amazing teams of designers and game developers, and we focus on market research, story boarding, producing, QA-ing, and looking up the results.

Katie Jansen: Yeah and that team rolls into my team as the chief marketing officer.

Audience Member: Alright hi everyone, my name is Regina and thank you for this panel. I’m actually in marketing, and that’s actually one of the reasons I came. I was like “Oh they’re talking about marketing, yes!” But my question is this, because there’s been a lot of call about diversity and inclusion, and I’m like this is great, but especially because you were talking about hiring, what I find is is that when I’m interviewing a lot of companies aren’t really looking at transferable skills. Like if you’re trying to get more diverse candidates then you can’t really expect that everybody is cut from the same cloth, and love that you know yeah you came from banking and …

Katie Jansen: I didn’t come from banking. No banking here.

Audience Member: Yeah yeah yeah, insurance and teaching, and now I’m in tech. So I’m just wondering as people who have done that pivot successfully and have stayed, because I’ve been at a few companies, just like what is your advice to that because there’s a lot of talk about diversity but then I’m like “What do you mean I’m not a fit?” So there’s a little bit of frustration there.

Katie Jansen: I actually I have, we are hiring a director of content right now, and I have … Maybe yes, we’re still in the interview process. But there was a candidate who I would say, and you know Lewis here has been part of the interview process is not what was described up here. She is not from tech, she’s actually from nonprofit, and so I really had to challenge myself and really focus on what are the skills that this job needs, does it really need a tech background, and I don’t think it does, actually. I think that if she’s really good at the brand, and she’s smart, he or she, they can learn the content. So this person is midway through, and I’m not sure if this individual is gonna work out one way or another, but I move …

Katie Jansen: Okay yes, you. You’re here. But I’ve moved her through the process and she’s done quite well because she does understand like she can run a persona project, she understands how to go through and do a customer journey project, and those are the skills that I need. She can learn the content, essentially, and so that’s what I’m trying … And Alice is reporting to me, I have had clear conversations before where I will say I am not talking about skin color. I am not talking about gender.I am talking about this person is different than we normally hire, and I don’t wanna talk about team fit. Do they fit the company? ’Cause when you say team fit that is actually just who people in the team wanna hang out with in my opinion, and that’s uncomfortable to say, but it’s kinda the truth. So how can we …

Katie Jansen: And so now we have more people on the team that are, you know, commuting from an hour and a half away, or they have kids that are in high school, and that isn’t really what the make-up of our team used to be. We have someone who just started who is a former professor. Like it is different and I’m really trying to push the team to do that ’cause I think we will be better and we will push the envelope more if we can start to do that. Did you guys have anything you wanted to add? Nope. Okay. Right there.

Audience Member: My name is Aurora, I also work in gaming and I’m curious about what games you guys are playing.

Katie Jansen: I don’t play games. That’s the truth. Honest truth. But they do, so go for it. I have two kids so I’m busy. I’m super busy.

Alice Guillaume: I just have a dog, so I can play games. Video games? Console games? Mobile games?

Audience Member: Mobile.

Alice Guillaume: I spent $500 on Cooking Fever. I finished the game. Task management, resource management, things where I feel like I’m accomplishing things.

Aurora: What was the name of the game, sorry?

Alice Guillaume: Cooking Fever. Look it up.

Katie Jansen: She’s like, I’m steering clear of that game. I don’t wanna spend $500.

Helen Wu: I do play games a lot, so I usually will download like the top free games, like the top 10 to 20 just to understand what’s out there in the marketplace. We do work with 90% of the top gaming developers out there in the ecosystem. So that’s part of the market research and intelligence that we do. And then beyond that I’m really addicted to word games right now, just like can you find all the words in this, like, jumble, and like improve your vocabulary and train your brain, and beyond that, I love playing board games. We actually do board game night like every two weeks or so at this office, which our creative team and game developers started, and it’s, we do it like on a Friday, we order pizza, we drink some beers and it’s just like everyone having a great time.

Katie Jansen: I play Wordscapes. I do play a few games, but I’m nothing like you guys. And I do insist we play board games and do puzzles with the kids, it’s really fun. There was a question back there.

Audience Member: I’m the back, so I’ll stand up. I’m Jessica and I work at Lime, and to Helen’s point I belong to a private organization of Rubik’s cube solvers. So we’re kind of going through hyper growth as well, so as a small company, what is something that you value that you’ve seen large companies lose along the way that you think is not worth sacrificing?

Helen Wu: That’s a really great question. I think as a still-small organization we really value, just the people that we spend time with. You know my greatest moments of joy are just my colleagues like challenging me to figure out, like, you know, how can we deep dive into this operational change together and how can we figure it out, and like how can we find a better way to get this solution, or let’s just geek out together about the data and about the product, and so I think it’s, you know, people coming together to solve the difficult challenges of the business and investing and spending time to help each other grow and learn along the way. I think it’s so important to hire people who can add to this culture because you know that’s, I think that’s what’s gotten us to where we are over the last five years, and I think at a larger company, I hope we never lose that part of our culture, is just like people geeking out together about like data and product and how to do it best.

Alice Guillaume: That’s a really good question, Jessica, so adding onto that I would say for me it’s remembering to keep things nimble. So our team has grown really large, and that means we need to change too. So some of the processes that we had in place don’t work when you’re at a larger scale, and you know, for me, I don’t wanna get in the way of really talented people, too, who can run with things. So taking a moment, even if things are going at 500 miles per hour, just stop and think about what still needs to exist in the structure, what can we break, and what we can recreate. I think that’s really important because it doesn’t make sense to keep things the same way if the team is no longer the same way anymore.

Katie Jansen: And we just did that yesterday. Alice and I sat down and we said some of these things aren’t working and some of these things are, and we haven’t rolled it out to those of you who are on analysis teams, so I won’t announce it here. But we are moving and changing, and I would just add, I think, which really you hit on in some ways, is being okay with failure. Fail, and this whole, fail fast thing, but I’ve been at this company for six years now, so what they have done so great is be able to say this isn’t working, now let’s move. And maybe not even, it’s not maybe even a pivot, but like let’s change, let’s scrap this, let’s go forward, like this isn’t working, we’re not gonna hang out here and try to make it work forever.

Katie Jansen: Now, admittedly, being self-funded that’s a lot easier to do when you don’t have a board. So I do wanna say that with some context because I think that we benefited from that quite a bit, but being able to fail and just move quickly has been pretty big. And not having a lot of meetings. I will reinforce that. Every time when you start to grow, all these meetings get going, and we’ve done a good job of keeping them to a minimum, and that’s one of the things Alice and I just did. We looked at the meetings she had, and we’re gonna start cutting some of the meetings ’cause we started noticing there was people in too many meetings and did they really need to be. I think ,ready, do you wanna go to the next panel? Yes you do.

Katie Jansen: Okay in the interest of time we’re gonna go ahead and get started, but feel free to get up and get a drink and move around as you want to. So now we’re gonna move more into focusing on tech, so this is engineering and ops here that you see represented here. This team works out of our Palo Alto office. So our business team mostly is here in San Francisco, and then Palo Alto is finance, ops, engineering, HR, some of HR is here, we split. You know how HR is. So this is gonna be about embracing change through actually building systems, so that actual adaptability we talked about on more of like a, you know, broad base and this is actually on systems itself. So this is Sonal, Anusha, Laura, and Swetha, and why don’t you guys just tell the audience a little bit about yourself. Take it away.

Sonal Gupta: Hi, I’m Sonal. Can you guys hear me? I work as a principal software engineer at AppLovin, and I’ve been working here for a little more than five years now. Anusha and I actually started a week apart from each other. So the infrastructure team, our main goal is to ensure high performance and scalability in our system. Essentially we wanna make sure that our ads are served in the most efficient and sort of fast way possible such that we are not only, the response times are as minimum as they could be, but we’re also not using as many resources and servers. So yeah that’s what my team does, and I work as part of the team and then some other things that I work on are revenue tracking infrastructure, among other things.

Katie Jansen: Anusha?

Anusha Ramesh: Hi, I’m Anusha, I’m also a principal software engineer at AppLovin. Like Sonal said, we’ve both been here about five years and two months now. So it’s been a while. And I actually work on the platform team. Funny enough all four of us actually work on different teams, so you’re gonna get a slightly different aspect on our engineering side, and the platform team, what we do is we make sure that all the data that we get from internal sources, external sources, get into the right databases, the right locations in the right time so that everybody in engineering, business side, everybody can actually use it. So some of the systems I work on are things like Pub/Sub system and custom counting frameworks.

Laura Pfister: Hi, I’m Laura, and I’m a software engineer on the ad server optimizer team. I’ve worked for AppLovin for almost three years now, and on the optimizer team, we strive to answer, essentially, two questions related to ad serving. So whenever we get an ad request we’re trying to determine, one, whether or not we wanna show an ad and spend the time and resources that it takes to do that, and then two, what ad do we wanna show and what valuation is that ad placed at. And so we’re actually making use of the data that Anusha’s team and the platform team give us to answer these questions.

Swetha Anbarasa: And hi my name is Swetha, and I work as a network engineer as part of the operations team at AppLovin. So we provide the backbone for all the stuff engineering does. So primarily my responsibilities include designing and configuring and maintaining the production and development networks at AppLovin, and also help with like maintaining our data centers as well as optimizing our network routes. I’m also responsible for capacity planning as our businesses grow.

Katie Jansen: Yeah so obviously we are processing a lot of data and doing a lot as you’ve heard here, and as I mentioned early on, we do that with just 160 people right now worldwide.

Katie Jansen: Let’s get real specific and talk about how and when do we decide to automate. You heard the panel before this talking about how that’s been a real advantage for us.

Katie Jansen: And then really what’s the best way to automate. If you guys could give some examples around tools, I think that would really help the audience here too to, like, get real specific. So you wanna take it away, Swetha?

Swetha Anbarasa: Yeah.

Katie Jansen: Okay.

Swetha Anbarasa: When did we decided to automate was a point where we needed to really scale. When the business started, we were able to you know manage everything with a few servers in a rack, managed or something, and it was all working fine. But as our businesses grew, we had to scale very quickly and we had increasing demands.

Swetha Anbarasa: From a networking perspective, our main concern was to find a vendor that actually has a development driven product. With the major players in the industry at that point, it was kind of a challenge to find a vendor that was actually development driven for our networking, so we had to do a little bit of research, and after some evaluation, we ended up using Cumulus Linux.

Swetha Anbarasa: Cumulus Linux is an open box solution. So you could actually install … It’s a switching software, so you could actually install the software on a bare model Linux machine and start working with it. So some of the advantages for going with that sort of a vendor is you can use the software to automate very easily, and we chose Chef as our automation tool. Chef is based on Ruby, so it was easy for us to write code in, as well as our server infrastructure also uses Chef.

Swetha Anbarasa: So networking and our servers together, everything is in one place in Chef, and we have like one repository where all of us contribute and change stuff, so it was easy for us to manage. And another advantage is, they were able to provide us with virtual images, so we were actually able to simulate a data center before we actually invest money and like buy stuff and actually deploy. A lot of it was being tested before we could deploy, so that was giving us a big advantage.

Swetha Anbarasa: Because we had limited resources, automation was actually playing a very big role, and we had our reasons to choose Chef and a few of the open networking platforms. And we also use Juniper routers which have been automated using Python scripts, and we integrated with RunDec to run a job in case we wanna do any sort of configuration changes or upgrades. For a small company, I think we’ve done a lot of automation, and we’re able to scale quickly with limited resources because of that.

Katie Jansen: Okay. And then how about on the engineering side? Maybe Sonal and Laura, what was it like, can you give some examples of automating some processes?

Sonal Gupta: The ops team automated the process of bringing up servers, and now engineering team had to make sure our services could also be deployed on all the servers that were being brought up, without a lot of intervention from us, and one of the big things we pride on in engineering is how agile our development is.

Sonal Gupta talks about automating processes at AppLovin Girl Geek Dinner.

Sonal Gupta: We push to prod every single day, and when I originally joined and even two years after I had been working, we used to push multiple times a day. So a single, any code change that you make could affect all the 60 billion requests that we are serving. For me my first year I was always very nervous when I was pushing that button to sort of deploy to production, ’cause there was no-

Laura Pfister: So nervous.

Sonal Gupta: Yeah. But it’s, I mean, ’cause we didn’t have a lot of structure around how to push and how to make sure that the changes we were making were very limited. So we tried to build an A/B testing sort of test control framework that would ensure that any new changes we push had very minimal impact and we could ensure that once it was verified, the changes were assessed, only then was it pushed to production. And all of these changes, we couldn’t like hack them, do Eiffel statements, or you know basic sequel queries, ’cause they were all prone to errors. So we had to like come together sort of build a structure that was away from human error, we could validate all the changes we were making, and we could triage and sort of back track the issues, and how we push something, when we push something, and how do we fix it.

Laura Pfister: Right and to add to Sonal’s point, so the A/B testing framework that we put into place in order to kind of combat all these issues is really really important to the productivity of the optimizer team. So without this, we were kind of able to A/B test maybe one or two features in like a week or something. The process was slow and we might have to wait for one feature to be clear just because we just didn’t have the bandwidth to do a lot of different things and once. And then it was hard for us to analyze the how one feature might impact the entire system, and it might be hard to pinpoint exactly which feature might be causing an issue, and we might need to even have reverted the entire all of production to kind of backtrack that problem. So with this new framework in place, we’ve been able to really really increase productivity and then just the overall safeness of our system so we don’t have as much of that fear when we’re pushing, we can really control exactly how much of our traffic is diverted into one feature, and really pinpoint and analyze the impact at each and every feature. So we can get a lot of features out and do it really quickly, and that’s really made us much more productive, much faster, and it’s just really a lot safer.

Laura Pfister: So I think that’s one of the biggest things that have allowed us to scale a lot of optimizer features and also infrastructure features really, really quickly as we’ve grown.

Katie Jansen: So switching gears just a little bit then, so we talked about how we automate processes, but what about how we grow and scale with those, have you had to change processes over time? Have you had to scrap them completely and start over, and what does that look like and how easy is that to do?

Anusha Ramesh: Like I mentioned when I did my introduction, I work on a custom counting framework as part of my daily jobs to say, but way back in 2014, if you go all the way back then we were a pretty small company still, we were starting to grow pretty rapidly at that time, and all of our data, all of the answers that we were trying to get from our data, we would use SQL queries. That’s what everybody does, you query your database and you get your answers and you look at ’em and you analyze. And as we grew and grew as the year went by, our data got bigger and bigger and bigger, and suddenly these SQL queries took an hour to come back with data. An hour and a half. They’d hit database connection issues, and networking issues, and so at the end of it, we were like, okay, well, this doesn’t quite work. It’s not gonna scale as fast as we are growing.

Anusha Ramesh: We actually took that entire system, redesigned it, re-implemented it, and pushed out a brand new custom counting framework that provides the answers and the same results that we wanted out of those SQL queries in ten minutes. We can do a query on that database, get results in ten minutes. And this entire thing because of how fast we move as a engineering department, we had it in from design to production in three to four months, which is pretty quick in my opinion, and then, of course, years go by. We keep growing, growing, growing, and with all the alerting and monitoring that we have in place, we realized that we’re soon gonna hit a limit there too. So what do we do then? Alright, scrap the whole thing. And we literally went down back to the square one, ’cause I had to redesign the whole system. Redesigned it, re-implemented it, re-pushed it to production three months later in 2016, and knock on wood, that is the system we have in place today and it will continue to go. And I bug ops about it all the time, but we have nice alerting and monitoring systems that ops has set up for us that keep our systems in place.

Laura Pfister: Another thing to go with the scalability, so as data has increased, the business has increased, and logic would follow that campaigns and ad inventory have also increased. With this increase, the ad selection process has also kind of gotten bogged down at times. There’s more ads to go through, so every single thing that you do in that part of the system is gonna start to be slower. This is a gradual increase, and, you know, it didn’t really impact things, but sometimes you wanna scale things, or sometimes scalability is about kind of dealing with making things larger, but in this case we wanted to actually add campaign breadth, a variety of additional campaigns to serve on a different component of our system called real time bidding, and in this system we are bidding on other ads, ad request from other systems, and in order to do this, we need to meet a certain millisecond threshold. When we tried to add more campaigns in this system, we found that we were no longer meeting this millisecond threshold. We were looking at the data, kinda seeing like okay nothing’s happening here so we’ve gotta do some analysis. We used a tool called a flame graph to actually analyze the CPU usage of our ad server and see where things were taking the longest amount of time.

Laura Pfister: We looked, and sure enough it was in our ad selection process. We were able to use this to see exactly where we were spending the most time, and then we were able to analyze this and go through and figure out where we could make cuts, where we could figure out how to do the exact same thing, get the same result and produce the same ad that we would in the previous system, but do it faster. Through this process, we were actually able to increase speed as much as three times faster in some cases, and drop CPU load a lot. It was really a great success and just another example of how we were able to scale things quickly, and I mean the entire process was like a few weeks, maybe.

Katie Jansen: How do you decide that? How do you decide when to just scrap it all together like you were talking about, or keep working on it and kinda push through? ’Cause it sounds like, especially in your example, Anusha, it was almost like broke and we were in trouble. Do we have something that’s in place that you know lets us know about that ahead of time, or is there some kind of meeting or team situation that occurs for you guys to decide, hey this is looking like it might need to be changed, we should invest more? What does that look like?

Anusha Ramesh: Specifically for this, well not even specifically, all of our systems are actually, we have a common tools package which platform keeps up to date, and most of our services plug into this system and use our tools to send metrics, send stats up to Graphite, which is one of our monitoring tools. And Graphite is amazing. If you ever see our office in Palo Alto you’ll see just TVs full of graphs everywhere, monitoring the most, like every little thing you could think of.

Katie Jansen: I like that you think that’s amazing.

Anusha Ramesh: They’re really cool-looking graphs, though. We have a ton of TVs up everywhere.

Sonal Gupta: I think it’s interesting when people walk by they see all these graphs, they don’t know anything what they’re about.

Anusha Ramesh: [crosstalk] … tiny little spikes somewhere of one thing going wrong, and suddenly you’re like oh wait, that’s wrong, we need to figure out what went wrong there.

Katie Jansen: To be fair, we use those in our videos and all sorts of things for marketing.

Anusha Ramesh: [inaudible] promotional videos. That’s one of the big things that tells us when a system is coming to a limit. We have stats on our databases, are we hitting some size limit where we can’t grow any further, are we hitting a CPU limit, can we not process things a little faster, are we hitting a backup somewhere where files are just sitting around just waiting to be processed and we don’t know. And if you don’t know about that, then that’s a problem, because we are gonna be behind. Our reporting pages aren’t gonna show the right thing, business side won’t see the right data until maybe hours ahead and then they’ll come to us and be like “Where’s the data?” And we have to be like “Uh, let’s go find it.” But all of these tools in place, Graphite, Zabbix, Chef, all of these things help you automate and figure out these alerts that might happen, and for the counting framework, that was the biggest thing that actually told us this is not gonna work. We hit a threshold that we had that wasn’t full but not there, like an 80% threshold, decided to do some things to minimize it for the short term, and then decide a long term solution.

Anusha Ramesh: I think that’s one thing in engineering that we do a lot, is we do a short term solution that is like for five, one week, two weeks, let’s fix the problem immediately so we continue to run, and then figure out a long term process, a long term solution that we can push out in a month, month and a half that will actually fix the entire solution and we can go for another year, or two, or three, or however long it may be, with the system.

Sonal Gupta: One of things I’d like to add to that is so one of the very first and sort of the most drastic scrapping of things we did was our ad server used to be in PHP, and a lot of our other components were in PHP as well. While we were debugging an issue with real time bidding and sort of why aren’t we bidding fast enough, why aren’t we winning as much, we realized it was actually the choice of language that was the bottleneck for our system. We decided, I mean we could’ve continued to patch and sort of improve the PHP system, make it faster, sort of find some wins here and there, but we realized that it was sort of avoiding the unavoidable, ’cause at some point we would have to move away from that system. We took a step back and we decided, okay what can we do, and we decided to come up with a C++ ad server solution. I mean it was a sort of a big undertaking ’cause most people on the team didn’t even know C++, but knew how powerful the language was, so within a few months we all decided to learn C++ and had a working ad server solution that not only supported our in-network ad serving but also our real time bidding solution.

Sonal Gupta: And over the year, then we migrated all our other services away from PHP to Java, or C++, and this is like something that’s very fun. One of our first intern projects was to move a service away from PHP to Java, and the intern finished it in two months. That’s sort of the trend at AppLovin. We try to get, if there’s a need and we realize there is a problem, we don’t shy away from completely scrapping the system, and more importantly we try to sort of learn quickly, everybody grows together, and try to get a solution out as soon as possible, and not a hacky solution but something that will work long time. I mean we’re currently still using our C++ ad server, and everybody on the team, we are always looking to make changes even if they’re like small changes to the two string function so we can get like little wins here and there and continue to optimize our system.

Katie Jansen: Yeah. That actually brings me to our final question, which is what about looking forward? You’re building this stuff now, how do you future-proof it? Can you future-proof anything? Maybe you could each give me your thoughts on that and then we’ll open it up for Q&A.

Swetha Anbarasa: Sure. think yes, we can future-proof things to an extent. So from the ops perspective, the way we try to plan is we don’t just have 2X or 3X capacity. We actually wanna have at least 10 times the current load. That’s the capacity that we look at, ’cause we’ve actually run into instances where that really helped us and we really needed the bandwidth and the servers to help. I know Anusha agrees, ’cause we’ve run into a lot of issues where we had 40 gig of bandwidth and it was actually, we never used it that much, but suddenly one day we needed it, it was during the holidays.

Anusha Ramesh: It’s always [crosstalk]-

Swetha Anbarasa: Yeah it’s always during the holidays that we need it, and we actually had a 40 gig bandwidth between U.S. east and west, and we were able to do the data transfer quickly without anyone even noticing or waking up anybody or like disturbing them from their holidays. So that’s how we plan, and I think one of the important or like key concepts about being future-proof is to have excellent monitoring, and for the size of our company, I feel like the monitoring we have is really good.

Swetha Anbarasa: We use Graphite. So Graphite is beautiful, colorful, so it’s numbers in graphs. So you’re not looking actually at tables, you’re not looking at boring data, you’re actually looking at graphs, and spikes, and dips, and it’s all like time and data, so it’s easy for us to understand, and you could actually add any component to Graphite and start tracking it even before you understand why we are tracking it. You know maybe a couple of months later you run into an issue, and you go back to the one thing that you were tracking, and you look at it, and you are like “Okay, thank god I actually have these stats.” So Graphite keeps collecting, and it works with Zabbix and also holds SNMP data from networking devices. So it’s sort of like one place where we can look at a lot of our stats and attributes and data to monitor.

Swetha Anbarasa: Apart from that we also work with, we also have like Google Cloud Compute, so we have systems where we can just spin up several servers if we need it and then tear them down once we are done. So that also helps us to be future-proof to some extent, and also maybe like from the networking side we also look for vulnerabilities that keep coming, we proactively patch our systems, do upgrades, be on the lookout for anything that’s going on, and kind of have everything up to date so we don’t have to worry about when something actually hits us. Yeah.

Katie Jansen: Laura, you wanna … Final thoughts?

Laura Pfister: Yeah, so when I’m working on something it’d say it’s probably a little bit more granular, I would say for one thing the monitoring that we have in our system definitely makes it really easy to see any small changes. So if there is an issues we’re gonna catch it faster, and that’s really, really important. From more a designing things for future proof perspective, I’d say, since I’m adding a lot of new features, a lot of new functionality, whenever I’m looking to add new things I try to look not just for, will this last like a week, but designing it in such a way that I’m trying to think what else might I use this for. So that’s just kind of an important way that I approach pretty much all the code that I write.

Katie Jansen: Anusha?

Anusha Ramesh: So like Swetha said, we definitely have the ops, like if they have 10X capacity, platform will use it. At some point in time we will 100% use it. If it’s just an emergency data from one side of the country to the other because something went wrong, we will use it.

Anusha Ramesh: I know I bug ops every single day for random stuff, and I’m glad that they allow me to do that. But as for future-proofing, a lot of our systems they’re all repeating the same thing, monitoring, monitoring, monitoring, because that is the biggest thing you can do. And our Java services that we use, we all have one base set of tools that we plug into every single Java service, and it makes every single developer, they don’t have to know how this tool is written in Java, they just use it and they know that as soon as they use it, five minutes later it’s gonna end up in Graphite, it’s gonna, you can pull up a dashboard on it, you can see a graph of it, and it’s done, and it’s there for you to see forever. Until you stop writing to it and it goes away.

Anusha Ramesh: But you at least have the ability to do that, and that’s one of the things that is really nice is it just exists and people use it. Funny enough, this monitoring tool, we actually were trying to figure out some issue, like we were trying to test a new database, something was going wrong, we realized that it’s actually our tool that writes to Graphite that was not working. We took it around, we redid it, and a day later we actually got 200X improvement out of it just by rewriting the whole thing, and we went from about 10,000 writes a minute to two and a half million theoretically writes a minute.

Anusha Ramesh: And it was like, of course, that’s all theoretical numbers and it’s more like a million writes a minute, but in the end it’s, like it went out to production, like it went out into our code system. Nobody knows that it actually got changed on the back end, but they’ll use it. They’ll see an improvement, and it’s just kinda there. And that’s the cool thing about having these systems that make it future-proof. They exist, you use it, things show up in graphs, it works. And you have to think about if you’re gonna do those million writes a minute, can your system handle it? And that’s future-proofing. And that’s kinda what we try to do at least a little bit.

Katie Jansen: Any final thoughts?

Sonal Gupta: I’m gonna add something that Anusha told us earlier in the day, that even with the issue the monitoring system had we still only needed two servers for getting all the metrics from all our different Java services. So the revenue tracking infrastructure that I work on, ’cause it’s such a tricky component, and that is where we track our revenue, any changes I make could potentially make us lose events, make us lose our revenue. So when I started, I integrated with the monitoring service, ’cause the platform team had done just such an amazing job giving it to us.

Sonal Gupta: One of the best things is when you look at our data, it follows a very certain path, and if there’s any variance in it, you will be able to tell there’s an issue. So that’s one of the reasons why monitoring becomes so important ’cause we can see every little detail of our system and how it changes. So for specifically, for example, for the revenue tracking infrastructure I can see how many revenues events I’m tracking every single hour, minute, day, and if there’s a change, I can, we can determine sooner than later that there was a change, and it was usually correlate to a certain push or some sort of event that happened and we can either go ahead and revert the change or sort of push a fix out quickly. So, yeah.

Katie Jansen: Cool. Alright in the interest of time here I’m gonna open it up to Q&A. Maybe take one or two, and then ’cause I know I believe there’s dessert. Yes, there’s dessert and drinks. I wouldn’t wanna promise dessert if it wasn’t there. Any questions? Yes, right there in the middle.

Audience Member: Hi, I’m Kayce.

Katie Jansen: Hi, Kayce.

Audience Member: First of all it’s been so fun to listen to you guys talk. You’re so passionate about what you do. This has been really engaging to hear all of you talk. I guess my one question is what’s the one thing that you wish you could do faster ’cause you’re going at such a big speed? How do you communicate together? I mean you’re all on four different teams so how do you make sure you aren’t messing up each others’ stuff and what’s the one thing you just wish you could do faster?

Katie Jansen: That’s a good question.

Anusha Ramesh: Thank you for your question. One of the things about not messing each others’ stuff up, is we are on four different teams so we do kind of work on separate projects, and we do, at the very beginning back in like 2013, when we started, we were using GitHub, Algin, all that good stuff for code version control and things, but we didn’t have a good code review process in place, and over the time as we grow from just maybe two or one engineer in a team, to four, five, eight engineers in a team, we’ve put a code review process in place, we’ve put in a structure so that we actually don’t step on each others’ toes. We have branches, we have to push the master, and then push from master to production, and do all these things, basically red tape if you wanna call it, before we can actually push it out. But even at that point, with all of these processes, we still push code every single day to production. We still push out our stuff. It’s because we have a good, I guess we are all used to it now, and it’s such a good process that we have in place that it doesn’t seem, it’s streamlined. That’s the word I was looking for. It’s a streamlined process that actually works for us.

Sonal Gupta: And I think we are still a very small team, so really we all have a lot of different responsibilities and roles we play every day, and we are small enough that they’re very separate from each other even if you’re part of the same team. So there’s a lot of communication and as you had mentioned, because you know facilitating sort of the open office structure, whenever we have questions or we wanna discuss something, we just go to the person’s desk, it doesn’t have to be like an elaborate meeting, and so really, like, if there’s something you’re working on and you think somebody else might have done it before you can just go ask them. It’s as easy as that.

Katie Jansen: Yeah I mean I’m obviously not on your team, but I do talk to, you know, the VP of engineering quite often when I have, which is rare by the way.A lot of times VPs of engineering don’t wanna talk to the CMO and they are annoyed by us, and that’s not the case here [at AppLovin].  So I would say – or you know to the engineers themselves – I would say communication is really key, and just different types of communications, whether it’s taking the time to talk to someone, or we really leverage Slack. I can’t say enough how much we leverage Slack. We have multiple channels, we will do direct messages, however we want to use it, and it’s been pretty awesome.

Katie Jansen: And then we all use Asana, I would say too. They use Asana, and then they made my whole team use Asana, and they didn’t like it but they like it now. It took a while. I actually try to sometimes mirror some of the processes they’re using, because I think that it helps. Obviously I’m not gonna do Graphite and those things, but like the communication processes, trying to do similar things helps us communicate better as cross functional teams across the company too. A question back there, yeah?

Audience Member: Hi my name is [inaudible]. So great you guys are so flexible, when the systems just not working you change it. My question is when you change the database, you know scrap it, how is decision made? How do you be able to push that idea?

Swetha Anbarasa: I think the decisions are mostly a combined discussion. Like Katie said we don’t have that many meetings, so in case something like this comes up, we would do a discussion where we pull in like the VP of ops, engineering and then the concerned people in the team teams that are involved, and sort of like white board it, get everybody’s opinion, write down some stuff that, you know our top technologies that we wanna use, and then kinda get everybody’s opinion. And again we would like discuss over Slack in a particular channel and kinda like arrive at a conclusion. So it’ll be more of a discussion with the team’s concerned.

Sonal Gupta: And I think just when everybody wants the best for the product and the company, it’s very easy to come to a quick conclusion.

Katie Jansen: Yeah, we’re all on the same page.

Sonal Gupta: Yeah.

Katie Jansen: Alright, one more question. Go for it.

Audience Member: So as the SDK provider, you deal with a lot of customers and face a lot of issues. How do you track the issues that are reported in and do you have analytics on them to figure out like where to fix things?

Sonal Gupta: Actually with have a separate product team, Jimmy’s team, that sort of does that for us. So one of the amazing things about engineering is that because of our product team they take care of sort of triaging and determining the issues before they even get to us. So by the time an issue comes to us, it’s probably something that is important. And you know you work with the SDK team a lot.

Laura Pfister: Yeah, I’ve done a lot of those kind of ad server, sever side settings, and yeah I only get the ones that we’re certain are a problem immediately. So having that kind of filter above us makes it so that we’re able to get those out really, really quickly and prioritize them as much as possible.

Katie Jansen: Yeah, and the product team, while they are business people, have some engineering and like background, so they can triage pretty well too. So I think that kind of having a team that’s in the middle there makes a big difference. Alright, I’m gonna wrap it. We are available to answer more questions, but I’m gonna let everyone go get drinks and dessert. Thank you very much for coming.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Connie Fong / VP, Marketing /
Sheila Lirio Marcelo / Founder, Chairwoman & CEO /
Abbey Stauffer / Director, Product Management /
Lauren Lee / Director, Product Management /
Rita Chow / Principal iOS Engineer /

Transcript of Girl Geek Dinner – Lightning Talks & Panel:

Connie Fong: Welcome everyone. Is my mic on? Good. Thank you so much for taking time out of your busy schedules to be here with us tonight. Hopefully you’ve had some great food, you’ve had some fun, and now you’re ready for some food for thought. If anyone knows me I’m always thinking about food, but if you don’t know me at all I just want to do a quick introduction.

Connie Fong: My name is Connie Young Fong. I will be your emcee for this evening and I currently head up the customer marketing engagement group at I am also here to warn you that there will be a couple of gratuitous photos of our kids so be prepared for that and so why I don’t just get that party started. I am in the thick of back to school right now. This is Evan, my middle child. If you can’t see what he says, he says when he grows up he wants to be a dad. My social caption says, “He picked the second hardest job.”

Connie Fong: Moms, you know what I’m talking about. I want to give dads a lot of credit, at least one so, but the last time I checked we get the hormones, we get the weight gain, we go through labor and delivery so I’m a little biased, but in all seriousness being a parent is really, really difficult; taking care of your kids, taking care of your pets, taking care of your home. Dare I say making time to take care of yourself? It’s all really, really hard. That might be the understatement for the year. With or without help, it is really, really difficult, and since 2007 has really been the leading company to take on a lot of these care challenges, not just for families but also for caregivers and for companies and if you think about it, this has implications on our culture, within our society and also economies at large.

Connie Fong: This evening I’m very excited. We have a panel of amazing speakers tonight lined up to give you some perspective on how we manage two-sided marketplace and also share a little bit of insight into their personal journey. I just have one favor to ask of you before we start, is that we will save time at the end. Please make sure to remember your questions and we will have a more formal Q&A at the end of all the presentations.

Connie Fong: Many of us are here because we’ve been inspired by a woman named Sheila Lirio Marcelo. She is the founder, chairwoman, and CEO of and we’re very excited to welcome her tonight to be the first speaker within our panel.

Sheila Lirio Marcelo: Thanks, Connie. Super excited to be here. I have to tell you. Lately, I do a lot of public speaking. My team is always asking me, “Am I little nervous getting up here?” I am tonight. I think it depends on how much I drank, if I got enough sleep, depending on PMS, sorry we’re in a group of women, and whether I’m hormonal. That was my answer to them tonight, sorry guys, a little TMI. But one of the reasons I often now, when I public speak, I often accept speeches to actually speak at places with more men because I feel like oftentimes when I’m speaking to women where I’m preaching to the choir kind of nod their heads and said, “Yeah, I already know what you’re talking about. I go to that those women’s event and I know what you’re saying.”

Sheila Lirio Marcelo speaking at Girl Geek Dinner.

Sheila Lirio Marcelo: But when they asked me to come tonight, I was really, really excited because the challenges we have in technology Girl Geek, yes. It’s tough. There’s a lot of challenges that we face and I’m super excited to be in a room of super talented motivated women despite the challenges of the things that we face.

Sheila Lirio Marcelo: Today, I’m just going to talk about tonight on what drives each and every one of us and really breaking out into sort of purpose-driven life careers and what’s important and hopefully I can share a little bit about my story to each and everyone.

Sheila Lirio Marcelo: First off, I grew up in the Philippines, born and raised. It’s interesting, a lot of people don’t know it’s one of the countries with the narrowest gender gap. In the world economic forum reports actually of the top 10 up there with the Scandinavian countries with the narrowest gender gap, specifically in Asia which is interesting. It’s across the globe in Asia it’s … I went to Japan one year and I was speaking in front of women and many of them had worked in the Asian Development Bank and said it’s so amazing to be in a group of Filipinas because they act as role models.

Sheila Lirio Marcelo: I sometimes peel the onion that and thought about why is that because even pre-colonial times in the Philippines women were allowed to be priestesses, to play leadership roles, and they actually were also allowed to own property as sort of part of our culture and I only learned that recently when I went to college and decided I was going to come to the United States to go to Mount Holyoke and really study feminism. Because prior to coming to this country, I never really encountered biases which was really strange for me growing up in Asia. That actually, in the United States is really where I started to encounter biases especially as a female entrepreneur in technology.

Sheila Lirio Marcelo: Today, I thought I’d share with you something I think about that I think we all need to learn together, is that true journeys of strengthen resilience are actually built upon believing in ourselves and each other.

Sheila Lirio Marcelo: Oftentimes, we have role models in front of us and we think they’re heroes. We look up at them. What I always say it’s actually the people right next to you and the true authentic stories that make meaningful differences in our lives that inspire us. As women leaders in the workplace we can and must write our own stories and share them with each other so that we can lead authentically with purpose.

Sheila Lirio Marcelo: I always ask myself these three questions who, what, and how. It sounds pretty basic. I break it up to say who has influenced you? Many times we turn to our mentors. What impact do you want to have in the world? I meet with a lot of young people asking, “How did you end up at What made you decide to start something like that?” Because there are very few companies that are mission-driven or purpose-driven.

Sheila Lirio Marcelo: I’ll talk a little bit about that and how do you stand for what you stand for, which is always tougher. The first question, who has influenced you and what lessons have you learned from them? I actually trace all my influences back to my beginnings and I try and take a little bit of stories of leaders that I run into to incorporate in my life as I met them along the way.

Sheila Lirio Marcelo: First, my parents. What’s interesting about us is that ,not surprisingly, one thing is that I have a type of mom, being raised Asian. I’m getting a lot of nodding. Yes, of course being Filipina, there’s a lot of role models of female presidents in government, a lot of female CEOs in the Philippines and nothing like … She always had to dream of sending her six kids to the United States for college and also pursuing professions because my parents were entrepreneurs.

Sheila Lirio Marcelo: Being Asian, we had designated professions. There were supposed to be the doctor, the dentist, the engineer, the lawyer, but God forbid no one should ever become an entrepreneur. My dad was actually a teddy bear dad. What did I mean by that? He’s the kind of dad who actually never minded ironing our shirts, taking care of us. He is a phenomenal cook. He’s also the kind of dad who would stand at the window or at the door. My parents live with me, to the point of driving away and he would stand there and wave until he couldn’t see me anymore. He’s that kind of dad. You could imagine that these were anti-stereo types of what we’re very familiar with, with gender and those are the parents that raised me.

Sheila Lirio Marcelo: But one of the best gifts that my parents gave me in influencing my life and answering that who is that, we came to the United States in the 1970s and I lived in Houston, Texas, for a few years and I then I forgot the entire language of Tagalog. I understood certain things, certain foods that were my favorite, but I completely stopped speaking the language because I came here at such a young age. My parents then decided that they wanted to raise us back in the Philippines and proceeded to send my older siblings to an American boarding school in the Philippines, but decided to send — I’m the fifth child and the sixth — my youngest brother to a province in the Philippines, a tiny little town that my parents were from so that I could learn the language all over again.

Sheila Lirio Marcelo: You could imagine a girl at 9 years old from Houston, Texas, saying “y’all,” going back to a provincial school in the Philippines, a very local school and being asked to stand every day to read a book in Tagalog in front of all these kids and how hard that was and embarrassing that was.

Sheila Lirio Marcelo: But why was that one of the best gifts in my life that year? Because my parents actually taught me the value of coconut. Like that is really strange, Sheila. Why? Because in the elementary school that we went to, every week we had to clear out all the chairs in the desk and each child was asked to actually help clean. I was responsible for cleaning the floors. I had to get down on my hands and knees and I learned to fall in love with the coconut because that would prevent me from getting down on my hands and knees because the coconut husk had a brush on it that made me sashay so that, I think my mic is still on, where I would literary do this and I’m really good at cleaning floors, really good now.

Sheila Lirio Marcelo: But what it actually taught me was not just hard work, but a sense of humility, responsibility and learning and also being so proximate to all these kids from all walks of life that I played on the streets with that year and to learn the language all over again. That probably was one of the most difficult years in my life other than getting pregnant in college and giving birth at a young age but that was really, really hard and that proximity that my parents taught me in a sense of identity of being Filipino was one of the most influential things in my life that drove purpose in my life.

Sheila Lirio Marcelo: Lessons growing up in the Philippines helped me create my own path, which is especially important after I got pregnant in college, as I mentioned. I started to veer from my parents’ plans. I wasn’t going to follow that designated profession, unfortunately. Tough for my tiger mom. And to think about it, my Catholic parents were very, very upset when I got pregnant between my sophomore and junior year in college and decided to get married and keep the baby.

Sheila Lirio Marcelo: My husband and I were pretty much on our own. My parents weren’t speaking to me. They didn’t expect sending me to women’s college would result in my being a young mother. They thought that men were not allowed on the campus at Mount Holyoke College. Lo and behold they were very surprised. Lo and behold, we have 26-year-old today who inspires us every day and I’ve been married 27 years, as of Friday.

Sheila Lirio Marcelo: During that time, my husband’s parents were also deceased. We didn’t really have a lot of access to resources. I just had friends from college who visit me recently and we caught up and I remember I disappeared my senior year of college in the sense that I was so focused on raising the baby with our son and we were struggling and we were poor and we just didn’t have a lot of help. That’s really inspired me later to start because I realized I wasn’t alone. But as our careers were taking off, Ron and I found ourselves struggling to balance work and family, really felt that pain.

Sheila Lirio Marcelo: And then fast forward when I was in grad school, another surprise pregnancy. Adam who is now 18, lovely gift, I call him. During that time, I decided after HBS that I would join an internet company, and again we needed help because the hours were so demanding that I asked my parents to come from the Philippines at this point. They were talking to me. They wanted to be a part of their grandchildren’s life. They came and then my father had a heart attack while he was carrying baby Adam up the stairs.

Sheila Lirio Marcelo: My father is alive today because I said he still waves to me from the window. He’s all healthy, but that was a big struggle for us because the whole point of my parents coming to the United States was actually to help care for baby Adam, and I found myself at 29 years old stuck between child care and senior care and I was also getting catapulted in my career at a young age to join a management team at Upromise, helping family save money for college and I didn’t have great care.

Sheila Lirio Marcelo: It was really hard and I was going home working at a technology company but using the Yellow Pages to look for care. Something really didn’t add up, which really led to the next question. When I decided to start my own business, I had to ask myself, what impact did I want to have on the world despite all the difficulty and challenges that I’ve faced so far. The second question is what is that impact?

Sheila Lirio Marcelo: Have you ever heard the expression, women hold up half the sky? Yes? Great book. I think that’s not the whole story, though because I actually think women don’t just hold up half the sky, we hold up the whole economy. I think that’s factual. I’m not just making a statement to be controversial. I think it’s actually factual. That’s why we’re so focused on improving the lives of women at

Sheila Lirio Marcelo: If women and men worked equally, from the McKinsey study, the worldwide GDP would grow by $28 trillion or 26% by 2025. Apparently, that’s the size of the combined US and China GDP, if they were just equal. The single biggest obstacle to women’s equal workforce participation across the globe is balancing work and family responsibilities.

Sheila Lirio Marcelo: This is where then I found my authentic itself. Not only did I go through my difficulty, but lo and behold, a year after we started, my mother pulled me aside and said, “Did you know that the Philippines is the largest exporter of care around the world?” It suddenly started to add up, why I’ve had so many friends throughout my career, at this point I was still young, coming up to me saying, “I love adobo. I love pancit.” They would say, repeat all these Filipino words to me because some of them were actually raised by Filipino nannies.

Sheila Lirio Marcelo: As a working mother, I know the challenge to try and make everything work. The need of care is massive and it’s growing. It touches everyone when we think about the demand for care. I often describe the care economy is this: that if you think about young children, 90% of a child’s brain, 90% of it, compared to an adult brain is developed between the ages of 0 to 5.

Sheila Lirio Marcelo: We outsource a lot of this now from many dual income families because it’s about 70% dual income. If we’re outsourcing care, do we just think about physical care or should we actually care deeply about how that brain is developing in terms of the quality of care.

Sheila Lirio Marcelo: I remember once, I was speaking at a conference at Milken, a good friend of mine who was the head of finance in the Philippines, I didn’t know he was in the audience, so after I spoke he came up to me and said, “Sheila, I never even knew that. I better start paying attention to who I hire to take care of my children.”

Sheila Lirio Marcelo: People don’t pay attention to that fact that it actually drives and how much debate is there around how much we should invest in education because it impacts the competitiveness of our economy and overall society that if children aren’t learning a certain number of words by a certain age, it impacts them socially, economically, increases incarceration, a lot of these things impact our society and that comes down to care. It’s not merely just a soft issue.

Sheila Lirio Marcelo: Now let me fast forward it. If you’re thinking about the work environment, right now care is one of the most expensive items in the budget, up there with mortgage and rent. Many of you pay for childcare, some of you who are here tonight and you know what that cost like when you have to outsource it.

Sheila Lirio Marcelo: If you want to pay for great care, you got to work, right? You need great work to actually pay for great care. There’s sort of this codependency. Even in our middle period of our life where we’re working a majority of our lives, we also need care to work and vice versa.

Sheila Lirio Marcelo: Now, let me turn to the end of life, the economic argument for that. We know and we read it all the time in our newspapers, that the key driver of the, health, of the budget deficit is healthcare. We also know that what’s the key driver of healthcare cost. It’s actually end of life. We also know that most people want to age in place at home, about 90%, according to AARP.

Sheila Lirio Marcelo: If you want to age in place but you don’t have great care, why then are we surprised that the readmission rates are so high to hospitals and that’s what drives our healthcare cost.

Sheila Lirio Marcelo: Something so basic of not having great quality care for the elderly is also impacting our economy. I can go on and on, from early childhood and the competitiveness of our society and how we educate them and the quality of care to the work environment, to all the way to end of life and if somebody ever came up to me and said care was a soft issue that has no economic impact, I would look at them and be like, “How do you think about those things?” The reality is it does. Clearly, there’s a growing demand.

Sheila Lirio Marcelo: You’re sitting here in the audience saying, I didn’t have care tonight, right? How many of you have kids? Great. How many of you have nieces and nephews? How many of you have been kids? If you’re sitting here wondering, doesn’t ever apply to me. I don’t ever have to think about care. There is a senior tsunami coming and it will impact you, in case you hadn’t thought about it yet. When caregivers to go work, we all can go to work.

Sheila Lirio Marcelo: Which actually then turns to the issue about the supply of care. When my mom said it’s about the Philippines being the number one exporter of caregivers around the world, how are we also thinking about the supply care? On average, in the United States, caregivers are paid $9 an hour. Golf caddies are paid $17 an hour and that doesn’t include tips. The difference between what we value in caring our children to those that carry our golf clubs. Some of my male friends are so upset with me that I tweet about that, I write about that, and they said, “That’s really unfair that you describe it in that way.” I said, “How else can I describe it?” For people to open their eyes and say, “We have to think about the sustainability and the livable wage that is required for the care force so that we have a real care infrastructure that supports our entire society.

Sheila Lirio Marcelo: Now that we’ve talked about these things, we have a certain set of responsibilities to families and caregivers that we serve and to also to women around the world which leads me to the third question, I’m onto the third question. How do you stand by what you stand for?

Sheila Lirio Marcelo: Through the years, I’ve learned that the same old thinking will lead the same old results. I’m going to repeat this twice because I thought this is an interesting quote, “When you change the way you look at things, the things you look at change.” “When you change the way you look at things, the things you look at change.”

Sheila Lirio Marcelo: Most people know is a place to find child care or caregiving jobs. We use technology, certainly, to address universal problem and provide a better more efficient solution than the Yellow Pages or word of mouth. But what happens when we improve the efficiency? Is the problem solved? Are we strengthening families? Are we supporting women in the workforce?

Sheila Lirio Marcelo: When we zoom out, it was clear we could do more as a company, beyond making profit. We ask ourselves, do we stand by what we stand for and we kept building Care@ Work and now companies can provide family care benefits like emergency backup care so that we can have people show up to work and not stress out about it come to events like this.

Sheila Lirio Marcelo: HomePay, household tax and payroll services so that caregivers can be paid legally and be paid above minimum wage. Then they get access because as we know in the future work the gig economy is struggling. We need to make sure that they’ve got a social net access to social security and Medicare benefits and unemployment insurance.

Sheila Lirio Marcelo: Then we also launched Benefits. We provide access to caregivers to have pooled portable benefits, access to healthcare, workers compensation that I mentioned, savings account, budgeting tools, a lot of things, because where we’re headed with the future of work, there aren’t the institutions anymore for gig workers that provides them the access to a social net.

Sheila Lirio Marcelo: But we can’t do this alone. I always say building a care infrastructure takes a village and there’s a lot that we’re doing, partnering. As an example, we’re training refugees in Europe to be caregivers but we’re not doing that by ourselves. We’re partnering with International Rescue Commission and Rockefeller Foundation. We’ve launched the Care Institute as a 501(c)(3 )to train caregivers around the country and hopefully the globe and we’ve done that we’ve AARP, as well as Boston Children’s Hospital and I can go on. So many different things that we believe is a responsibility because it’s more than just actually creating profit and leading life with a purpose. There are always moments where what we value is challenged.

Sheila Lirio Marcelo: Quick story, it was really hard, and this isn’t about my personal politics, but in 2016 certainly the result of the election was shocking. I was at Javits. I was very emotional at night. It took me about a week to speak in front of the company. There were a lot of issues going on the company. We have a lot of women in the company disappointed that we did not have a female president as a result.

Sheila Lirio Marcelo: I only shared this story because I have to set aside my personal from running a public company. It took me four days to get in front of the company and stand, and the way that I spoke to them, I stood one side and I said, “You know what, this is Sheila upset.” I was in tears. I couldn’t even hold it back. It took me literally four days before to even come up with the words to speak in front of the company. I stood over here and then I said, “This is Sheila as the public CEO telling you guys you cannot blog about how upset you are because we serve the entire country in the world and people are watching us.” We have to set our personal stuff aside and that’s very difficult to do.

Sheila Lirio Marcelo: But the reason I’m sharing this story with you is because two weeks later, I had to go and have a dinner with Ivanka Trump. After all that emotion that I had to go through, because she decided she was going to carry the torch around childcare and childcare tax credits. Guess what? I’m the CEO, the largest platform for care in the country. I had to go to that dinner and represent what’s right for families and caregivers, setting aside anything. I actually found in that dinner was my own biases that was at fault, and I was being too judgmental. I share that to say that sometimes we really have to push ourselves to figure out and set our personal over what’s important.

Sheila Lirio Marcelo: Then two weeks later, we were also challenged with immigration ban. Then I had to ask myself, “Okay, I feel like a ping pong ball. I feel like I’m depressed over here. I got to go pull it together for this dinner, and then I’m going to sign a letter and want to be the first CEOs to sign a letter on immigration ban, just when I’m trying to develop a relationship with this administration?” I mean, what the hell. I mean it’s tough figuring out what you stand for. It’s challenging in leadership. It’s not easy but constantly questioning the values. I’m still proud that we signed that letter and also really supported DACA and the children that were split from their parents just so much.

Sheila Lirio Marcelo: Look, in closing, I’m up here talking about values. I’m up here trying to share my own personal story and people always ask me what makes you tick? I get a little emotional because at the end of the day it’s actually just being human. It’s about the people sitting next to you and listening to their stories, giving them that certain level of respect because everyone has a story. Everybody, not just me. Those were the true stories that should inspire you because that’s often those people challenges and it doesn’t matter what level of success you’ve achieved.

Sheila Lirio Marcelo: I have the same equal difficulties and challenges that everybody has because it’s about being human and so it’s those authentic stories that we should really rely on and turn to each other so that when we learn about those journeys, we ask ourselves, who, what, and how. Thank you for having me. I’m super excited for you guys to meet our female leaders and I’m looking forward to answering any of your questions. Thank you.

Connie Fong: Thank you so much, Sheila. It feels really good to work for a mission-based organization where it truly is founded on love. It truly is founded on caring for people and that is always good. The other thing that’s always good is actually figuring out how to make some money and to actually be profitable. Next up, we actually have Dominique Baillet. She is our Senior Director who heads up our growth and product development team, and the one thing that’s fun about Dominique is she didn’t show gratuitous photos of her daughter in her interview. She literally brought her daughter to the interview at three months. Suffice it to say she got the job. We’re very excited that she’s here with us tonight.

Dominique Baillet: Thanks, Connie, and hi everyone. It’s not easy to follow Sheila but I’ll do my best. Today, I’m here to talk about how to monetize online marketplaces, which is really important to building a sustainable business. This is something that I’ve thought about every day for the last five years.

Dominique Baillet speaking at Girl Geek Dinner.

Dominique Baillet: Today, as the head of where I’m responsible for bringing more members onto the platform and having them become subscribing members with us. Before that I led business operations for growth and new markets at NerdWallet, which is an online marketplace matching consumers with financial products. This is something that I’ve thought a lot about.

Dominique Baillet: First, it’s important to set the stage. We are living in the golden age of consumer technology. The stock market has been booming. There’s been a lot of investor funding going around and we as consumers have benefited from all of this easy money and it’s been great. I didn’t even know how delightful products could be until Snapchat came around and showed me that I could take a picture of my daughter, airbrush her, put on these funny glasses and freckles and I would fall in love all over again. Or how about, I don’t know if you guys remember the time when Uber was offering rides across the city for $2 or when Instacart was first giving out free grocery delivery, all of that consumer delight was funded by this easy money and these companies grew a lot because of it. But it hasn’t always been sustainable to grow in that way and there’s actually a downside to this unfettered growth.

Dominique Baillet: Case in point, are of you guys MoviePass subscribers? Anyone? Are you guys still MoviePass subscribers? You are? Okay. That’s great. They need you. Yes, well what an amazing product and value proposition, $10 a month to see as many movies as you want, it was incredible. They vastly exceeded their user growth expectations, but they really couldn’t continue to service that. While MoviePass isn’t quite dead, obviously, it is dying.

Dominique Baillet: The lesson here is that while we are so focused on growth that it’s also important to balance the idea of how to actually make money. You don’t have to optimize for making money but you absolutely have to have a plan for it before this easy money dries up.

Dominique Baillet: In a one-sided marketplace, growth and monetization are a little bit more, go a little bit hand in hand. You make a product, you sell the product, you sell more of that product, you make more money. It feels like you grow, you make money, great.

Dominique Baillet: A two-sided marketplace is a little bit more complex. Not only do you need to actually grow both sides of the marketplace, buyer and the seller side, but there’s even a question of what your product even is. Is your product the platform? Is it your sellers? Is it your seller’s products? Is it access to the buyers or is it the underlying user behavior data? Only once when you figure that out can you actually think about how to make money.

Dominique Baillet: If you currently work at a marketplace business or if you’re thinking about joining a marketplace business, there’s three simple questions you need to understand to fund the basis of a monetization plan: who pays, what do they pay for and what do you give away for free.

Dominique Baillet: First, in terms of who pays, you’ve basically got three options: you’ve got the buyer, you’ve got the seller and you’ve got a third party. The key considerations you really need to think about is, who benefits the most from this marketplace and who is willing and can actually afford to pay.

Dominique Baillet: The next question is, what do they actually pay for? If you think about it, a marketplace is still fundamentally a place where someone is trying to buy something or find someone, and so as a marketplace, you really need to think about where you add the most value in that, in creating value for your end users and that can be represented by a nice funnel here.

Dominique Baillet: As a marketplace, you can think about is the value you’re adding actually at the top of the funnel creating access. Think about when Etsy started. It enabled these micro-entrepreneurs to actually open these store fronts, giving access to buyers everywhere of crocheted products. Or think about LinkedIn and the access it created of enabling recruiters to post a job that would go anywhere. That would be creating value in terms of access. The mid funnel is all about leads. Is your marketplace actually set up to curate a set of products or curate a set of love interests and is your value really in the vetting process.

Dominique Baillet: At the bottom of the funnel is transactions. Is your marketplace actually set up to shepherd someone to actually make that sale or make that match? Is that the differential value you’re actually creating? These are things to think about when you think about what are you going to actually ask these payers to pay for. Then lastly is the question of what do you give away for free? Free is really what you when you’re trying to acquire users or when you need the marketplace itself to work. Everything outside of that is really a distraction when you think about what you’re giving away for free.

Dominique Baillet: To bring this to light, I wanted to give a few examples of how these three questions are really interrelated. At, we monetize primarily on the buyer side or the family, so families like mine. When I was looking for a nanny for my daughter, Greta, I had two not great options. One, very expensive, I could go to a nanny agency, spend thousands of dollars. The other option was incredibly time consuming. I would have to source a number of high quality caregivers and vet them myself and interview and do all of that. actually created a ton of value in saving me both time and money.

Dominique Baillet: On the other side of the marketplace, the supply side would be the caregivers. Caregivers are in job searching mode, don’t have a job and maybe have less disposable income if they’re looking for a job, and then also job seeking markets, and typically the job seeker doesn’t actually pay. At, it’s really the families, that buy side.

Dominique Baillet: Then what families, what I was looking for, was really access. When I was thinking about finding a nanny, it wasn’t like an Uber driver where anyone could do. I really needed to have a good personal connection. Having access to a base of high quality caregivers that I could really figure out who is best for me and my family, that’s really what I was paying for. I was paying for access to that market.

Dominique Baillet: In terms of what’s free, as I mentioned, is really only successful if we have the best and the most caregivers. Therefore, we made the whole caregiver experience free. It’s free to set up a profile. It’s free to get a job and really what we’re trying to do is get as many of these great caregivers onto our platform. We don’t want to create any friction there.

Dominique Baillet: We also need the families, the payer side, to have confidence in the platform and to understand that inventory we say we have is true. We also make it free to post a job and free to search the caregiver so you really understand that is what it says it is.

Dominique Baillet: NerdWallet is a different marketplace, has a different strategy, and as I mentioned, NerdWallet matches consumers with financial products. In terms of their evaluation of who pays, that one’s is pretty easy. It’s the sell side who pays. Those are the financial institutions like Chase or Citi or Bank of America. They have a lot of money that they put aside toward customer acquisition and are always looking for new channels to make a sale, to find a customer. They have the money. Meanwhile, the buy side of that marketplace, consumers, like you or I, we can really go almost anywhere to get a credit card these days. We’d never pay for that service.

Dominique Baillet: In terms of what the financial institutions pay for, it’s really at that bottom of the funnel, it’s that transaction. For NerdWallet, it’s credit card business. We would monetize on the transaction and the NerdWallet experience from providing advice, to tools, to making the credit card application process really easy, was all targeted toward helping consumer get the product they want, helping the seller make the sale. Then finally in terms of what’s free, like I said, people can go anywhere to get credit cards, so why would they actually come to NerdWallet? In order for that marketplace to really work, we had to have the consumers. Therefore, NerdWallet invested a ton in great content, in tools, in order to build that trust with users.

Dominique Baillet: I know it’s hard to go through three examples but for the sake of comprehensiveness, I want to talk quickly about Facebook, which I’m sure everyone here is quite familiar with. In terms of who pays for Facebook, it’s really third party ads. No shocker there. Marketers are paying for access and leads in the form of eyeballs and click-throughs and in terms of what’s free, well think about what is it take for that marketplace to work, it really requires having very detailed user information. The product is the social medial platform we’re all quite familiar with.

Dominique Baillet: Facebook also invests in a number of marketing tools, for marketers to get the most use out of that ad platform. The lesson here is these three questions are interrelated and every business has a different business model where you really need to think about who pays, what for, and what’s free.

Dominique Baillet: If I’ve exposed for you in this conversation that two-sided marketplaces are complex, I hope that you also feel a sense of excitement in that complexity. It’s really a game of chess, not checkers and there’s a number of strategic choices that can be made.

Dominique Baillet: If you find yourself building a marketplace business, just remember that growth is important, but make sure you have a plan for making money. Consider who pays, ensure you’re offering them differential value and that they can actually afford it. Know what you’re asking people to pay for and make sure that you’re aligning the value you offer with the value you create.

Dominique Baillet: Lastly, have a plan for what’s free because nothing in life is free. Make sure whatever you are giving for free actually reinforces your marketplace. Thank you very much.

Connie Fong: Nothing in life is free. That’s the second understatement for the evening. Next up, we actually have Abbey Stauffer. She is our Director of Product Management, and she actually manages all of the matches within our site experience. She truly is the matchmaker on our site. Here, she’s also going to give a talk on the art and science of matchmaking.

Abbey Stauffer speaking at Girl Geek Dinner.

Abbey Stauffer: That’s me. I have an important question for you guys. Does anyone here ever feel like Netflix knows you? Yeah? I’m not the only one? The first time I watched Gilmore Girls, I was like, “Oh my God. Thank you, somebody knows me.” Someone can see into my soul by recommending the Gilmore Girls. I love this show and it came to me as a Netflix recommendation. Now, of course that didn’t happen by accident and I think probably being here in Silicon Valley, most people understand that Netflix has invested really heavily in their matching technology. They put millions of dollars and many, many years of work into making the recommendation engine what it is today. Guess what? Their work will never be done. They will never have a recommendation algorithm that is complete.

Abbey Stauffer: I think that’s fascinating. They take hundreds of millions of data points in order to match me up with this wonderful show. They’ve actually even invested further than just their own team. They famously declared the Netflix prize, which was a million dollar prize for anyone who could improve the algorithm by just 10%. That’s pretty crazy and somebody won that, but we’ll come back and talk more about that later. Netflix, they take a ton of data and they give you a recommendation. That in itself is a pretty complex and challenging thing. But in this equation, only I have to like Gilmore Girls. Gilmore Girls doesn’t have to like me back, right? Warner Bros. doesn’t have to want me specifically as a viewer.

Abbey Stauffer: That is the challenge of a two-sided marketplace. When you’re serving the needs of two users, you’re matching challenge gets ever more complex, and that’s what I get to work on every day with We are stewards of not only the families that we serve, but also the caregivers and guess what? The needs on the caregiving side are just as complex as they are for the family.

Abbey Stauffer: While I love this match and I think it was perfect for me, at we get to work on a two-sided marketplace, two-sided match, and I’d love to share with you more about what I do day-to-day in that matching process but also share some other matching methodologies and challenges that I see all around us.

Abbey Stauffer: Matching challenges are not just an online thing. They exist in the offline world. They exist online. They’re old school. They’re new school. Think about organ patients waiting for a donor or medical residents waiting for a hospital match or even the entire college admissions process.

Abbey Stauffer: Now, all of these things, they are matching problems, and they haven’t moved entirely online, but they do leverage matching methodologies. We won’t go into the specific algorithms today but each one has kind of a unique way or unique ways that they leverage algorithms. I encourage you to check up on that. Certainly here in Silicon Valley, we have matching challenges abound. We have full industries that have popped up around getting people to a great match. Looking for love, a whole industry around that. I can’t even keep up with it. Every time I turn around there’s a different app. Looking for a ride. This is super interesting, actually, Lyft has an engineering blog where they give you a ton of information about what they’re working on, things that they’ve tried, challenges that they’ve experienced and they’re really forthcoming with what they’ve done so you can learn a ton from reading their engineering blog and others.

Abbey Stauffer: When they started out, the matching challenge was relatively straightforward, although I’m sure if anyone here works at Lyft, you’d probably disagree with because it took it probably a ton of work to get there, but they were matching one rider with one driver, right? Take a rider, match them with a driver who is nearby and available. Not that hard, right?

Abbey Stauffer: Well, the company was not satisfied with just that match and just with the status quo of their business. They thought, how do we make the service even cheaper for our users? How do we grow our ridership? They started to think about how they could add more people to the ride, how they can add more riders into the match, and this is where things I think got really interesting with their matching challenge.

Abbey Stauffer: This, as you can probably tell, is what precipitated their Lyft Line product when they have multiple riders sharing one driver. At first, they had an algorithm that was serving one additional rider, so two riders to one driver. The algorithm that they had came up with four different, that’s three, four different route permutations, then had to declare the winner for and match up the rider and the driver. That started to go well. Users were responding to it. It was a service that was really taking off. Being in Silicon Valley and being an ambitious company, they didn’t stop there. They added a third rider and a fourth rider, and as they got to that fourth rider, the algorithm was then catapulted from giving them four permutations of a route to 1700. That was their challenge, right, to narrow that down come up with an optimized match from 1700 possible route permutations.

Abbey Stauffer: Now, if that makes any of you feel like you’ve been watching Westworld for too long to think about how to solve that product challenge with an algorithm, then I’m totally with you. I’ve been there. I live that on the regular in the matching world, but don’t worry, there are a number of what I would call foundational things that you can think about if you’re getting started in matching.

Abbey Stauffer: Many of you probably already worked on matching challenges or maybe you’re interested in getting into it and so I’d like to spend the rest of the time talking about what I think are four key things to think about. I need a drink of water. Okay.

Abbey Stauffer: Probably unsurprising to all of you, but you will need data in order to match up users. Data can take all sorts of different forms. Of course, you’ll have your user data, the data that they give you into their profile. This is the most straightforward kind of data to collect. But you can also look at behavior data.

Abbey Stauffer: What the user does and what they tell you is often very different. My favorite example of this is

Abbey Stauffer: Does anyone follow’s findings that they’ve released in the media kind of like Okcupid did, as well? Okay, well, a number of years ago, they took a look at their data and they saw conservative men want to date conservative women. It makes sense, or at least according to the dating profile. Then when they took a look at the user behavior, they found that the conservative men were actually spending a curious amount of time looking at liberal women’s profiles.

Abbey Stauffer: Now, that is a fascinating product challenge right there. I have no idea what they did with it. I have no idea if they put it into the algorithm or if they just decided not to touch that with a 10-foot pole, but I would have love to be there for that product conversation to think about how you pair the data of what the user does with what the user tells you. Oops. Okay.

Abbey Stauffer: Triangulation data. What I mean by this is really looking for external data sources. This can be really useful for those of you that work in more nascent products or new businesses that don’t yet have scale with your data. You can also look at triangulating data from users that are similar to the user that you’re interested in solving for.

Abbey Stauffer: Then finally, success data. This is often overlooked at the start of a matching project. Your algorithm is not going to teach itself, people. Well, it will eventually, but only if you’re giving it success data. You have to give it the information about when it makes a successful or an unsuccessful match. You have to train it and coach it and coax it until it is matching on its own.

Abbey Stauffer: You need to have a metric of what success actually is and that’s a lot easier said than done in many cases. If you don’t own the full fulfillment of your product, if you hand it off to a third party or if your users decide to take things offline and not complete on your platform, then guess what, you’re not going to have that success metric.

Abbey Stauffer: What do you there? You could think about intermediate success metrics or indicators of a good match or you could align as a company on a different success metric and that will probably take quite a lot of internal conversation to agree upon something like that. Just a few types of data. There’s many more that can serve your matching needs.

Abbey Stauffer: Unfortunately, that data could be collecting dust if you do not have a strong data engineering group or data warehousing group. Think of it as a sort of hierarchy of needs, the pyramid about needing food and shelter before you can have self-actualization. The same could be said for data. Okay, first you need the data, you have to capture it, but if you don’t store it in the right way, if you don’t structure it in the right way, that makes it consumable for an algorithm. It’s just going to be collecting dust and maybe your dear data analyst will be able to get some insights for you, but you won’t be able to use it in your matching methodology. Having a strong data engineering team, data warehousing team is key.

Abbey Stauffer: Okay, this brings me to my next point, thought partnership. We’ve covered the science of matching and now we move on to the art. Thought partnership is a key part of being a product manager, right? You have to have relationships with all of the different groups in your company and you have to engage them and get thought partnership from all of them in order to arrive at a good product solution.

Abbey Stauffer: A couple of examples here, who you need to partner with will very greatly by your product and what you’re trying to achieve, but a couple of examples. Your business analytics partner can do a lot to help you keep grounded in sound methodologies and keep you honest if you’re trying to extract a takeaway from some data that’s not actually true, to keep you honest there. And engineering can help you think about building an algorithm that will scale with your product, because, hopefully your product is poised for explosive growth and your algorithm will need to grow with that.

Abbey Stauffer: They can also help you think about building your algorithm for speed to make sure that you are actually delivering the recommendation to the user as fast as you need to. The Netflix Prize that I told you about before, they did award a winner. It was a team of a few people and they got their money, but Netflix never implemented the algorithm because the engineering costs do so was so high. That just underscores the need for both thought partnership and the strong data engineering foundation because having a huge fancy algorithm is not always easy to actually implement and scale with your company.

Abbey Stauffer: User empathy, last puzzle piece. Having a feedback with loop with your users to make sure that they agree that the match is successful is hugely important, and Lyft, we’ll go back to that example. They’ve done a great job at this. They took the time to actually incorporate a feedback loop into their product.

Abbey Stauffer: What they’re hearing from users was, even though the match that they had in the multi-rider match scenario was getting them to the final destination quickly, they hated backtracking, that was the main takeaway. “Please, get me to my destination quickly, but do not make me backtrack and God forbid, don’t do it on a highway.”

Abbey Stauffer: What they found, really, was that users valued having a direct path more than they did, having the fastest path. What a fascinating thing for them to find. Then their task became, how do we bring that back into our algorithm as a success metric and they never would have gotten that if they didn’t have a feedback loop that built into their product or if they didn’t take the time to actually sit down and talk to their customers.

Abbey Stauffer: Obviously, a lot of art and science goes into matching. I love my job because I get to work on that problem every day. This is my favorite match of all. I was a part of a nanny share for my oldest son. It involved five adults, two kids, two pets, two households and a ton of logistics, and somehow it all came together for a really wonderful match that helped me go back to work and really get my career off the ground. This is what I keep in the back of my mind when I work at, is driving towards scaling this kind of match for all of our 25 million members. I hope, I wish that upon all of you to as you work on matching within your own products. Thank you.

Connie Fong: Thank you so much, Abbey. I’m so thrown because I’m so glad she threw up that image of Gilmore Girls, because I looked at that picture of Alexis and I just finished binge watching Handmaid’s Tale so I’m totally blown and forgot that she was in Gilmore Girls. But enough about me binge watching TV and food.

Connie Fong: I’m very excited to introduce our next speaker. Her name is Lauren Chan Lee. Before I do that, I always have to throw in a “Go Bears!” because I found she was a Stanford grad. I’m a Cal grad, so you know. Not that we’re competitive or anything in the Bay Area. But Lauren, she is our Director of Product Management. She focuses on mobile trust and safety products and she will lead the conversation on how to build trust within our marketplace.

Lauren Chan Lee speaking at Girl Geek Dinner.

Lauren Chan Lee: Thanks, Connie. Connie, at the beginning of this evening promised you lots of gratuitous pictures of children so I’m just going to get mine out of the way right away. Here are my kids. Thank you for awing on cue. I love you guys. I show you this picture because they look very cute here. They’re giving each other the spontaneous hug. It’s all great, but sometimes they’re also really annoying. The thing is though, they’re my kids so I love them and regardless of how they’re behaving, I couldn’t imagine my life without them. That really underscores the predicament that I and millions of other families face every day, which is something that Sheila talked about. How do I find someone that I can trust to take care of my kids?

Lauren Chan Lee: The internet has made it so easy for us to find everything online from directions to a restaurant to even finding a babysitter on the internet. But the only problem is, how do I really know who I’m talking to is who I think it is.

Lauren Chan Lee: If the fake news scandal this year has taught us anything, it’s that on the internet, nobody knows you’re a dog or even a Russian operative. I’m here to tell you that trust is the key ingredient. If I can’t build trust with my babysitter, then it doesn’t matter what matching algorithms Abby is throwing at me, there’s not going to be a deal that’s happening.

Lauren Chan Lee: Trust is what’s going to get your users comfortable with buying your product. Trust is going to be what gets them to convert faster and trust is going to be what’s gets them to be your customer for longer.

Lauren Chan Lee: Tonight, we’re going to talk about my framework that I use to think about how to build trust, starting from internal to external mechanisms that you can use. This is really not meant to be an exhaustive list of all the things that you can do. We only have time tonight to talk about four things, but there’s certainly a lot more ways that you can build trust. And it’s also not meant to be prescriptive because some will work better in some cases than others. I’ll try to highlight that.

Lauren Chan Lee: Let’s get started on how we think about building trust. The first way that’s really, really common is rating and review systems. Recently, I was looking for a new patch for my backpack. I came across this pink geometric fox on Etsy and it’s $5, it’s super cute, best of all when I scrolled down the page, I could see that it had over 2000 positive reviews for the seller.

Lauren Chan Lee: With such great social proof, it was really easy for me to make my decision, which was, add to cart. Now, I’m not going to be the dead horse when it comes to rating and reviews because I know that you guys have all seen it before. If you’ve ever shopped on Amazon or eBay, if you’ve ever ridden an Uber or Lyft, you’ve probably read reviews, you’ve probably also left reviews.

Lauren Chan Lee: Reviews are really great from a product standpoint because they tap into the wisdom of your user-base, people who are in your marketplace and they also scale really well. Best of all, they create stickiness with your supply because if I’m a seller and I have over 2000 positive reviews, you can bet that I’m going to think twice before I leave your marketplace.

Lauren Chan Lee: Another common way for a marketplace is to build trust is through guarantees. A few years ago, I went to Coachella and a few of my friends decided to join at the last minute. They had to buy their tickets from someone on Craigslist. I think you know how this story is going to end. It’s like a Lifetime after-school special. We’re having a great time that weekend. We found an awesome house nearby the venue. We’ve donned our festival best and we walked up to the gates. What happens? Err, that’s right, they get denied at the gate.

Lauren Chan Lee: It’s just such a devastating feeling when you’ve traveled in to go to this marquee event, you’re really excited to see your favorite bands, and then you don’t get in. That’s exactly why StubHub has a FanProtect guarantee, because they know that when you’re buying expensive tickets, the last thing that you want to have on your mind is, am I going to get in or not? By having this guarantee in place, they’re able to transfer the trust so that you don’t have to trust the seller on the other side of the transaction, you can trust the platform instead.

Lauren Chan Lee: Guarantees are great when the platform is intermediating the transaction, when it’s a high dollar transaction, or when it’s a very rare item. But what happens if your marketplace is not selling a thing but it’s actually selling a service through a person?

Lauren Chan Lee: This is where verifications come into play. If you’ve ever had to answer a question like, which of these addresses have you lived at in the last seven years? Then actually you have had your identity verified. Here’s how it works. Let’s say that I’m listing a house on Airbnb, and this is a gorgeous house in Napa. Unfortunately it’s not actually mine, but if I were to put myself in the shoes of this homeowner, I know that I would only want to rent it to somebody who’s trustworthy, who has provided their identification, and that’s something that I can choose as a host on Airbnb.

Lauren Chan Lee: There’s so much technology developing right now in this area and the cutting edge is AI and machine learning. Here’s how it works, as a guest I actually take a picture of the front and back of my ID. I take a selfie and voila, the magical machine learning algorithms tell Airbnb the chances that I am who I say I am that the ID is real or if it’s been doctored in any way that the face in my selfie matches the face on my ID. That makes me as a host really comfortable knowing that the platform has done this verification on my behalf.

Lauren Chan Lee: Last but not least, we come to certifications. This is something that’s external to your marketplace. If I think back to the summer of my junior year of high school or the summer of SAT, I can still vividly recall every day after summer school, riding the MUNI to go to my summer job and I would open up my Cracking the SAT book or flip through my SAT flashcards, and all of that effort was done in the hopes that I would score really high on SAT and be able to get into the college of my choice.

Lauren Chan Lee: Many of you may be able to relate to that, so the SATs are actually a way that colleges use to be able to compare high school students, apples to apples across the country, regardless of what school they went to or what classes they have taken. It’s an external standard.

Lauren Chan Lee: Similarly, Upwork has created their own test and certification so that when you’re on their marketplace for freelancers, you can easily search across and see if people meet the needs that you’re looking for.

Lauren Chan Lee: Now that I’ve been giving all these talks, I think that I need a website. I’m looking for a graphic designer who can help me do that. I’ve come across Rose R’s profile, and I can quickly scan down and see that actually she scored below average on principles of graphics design. Unfortunately, it doesn’t look like she has the skills that I need, but I can learn that quickly and move on to look at other candidates who might be a better fit. Certifications are a great way to build trust because you’re able to look at these external standards across different marketplaces and things.

Lauren Chan Lee: Today, we talked about a couple of different ways, starting by looking within your marketplace to the power of your own users with things like ratings and review systems, then going to channel the power of your marketplace itself with guarantees and verifications, and finally looking external to your marketplace with things like certifications.

Lauren Chan Lee: My call to action for you guys is, as you’re going back to your day jobs tomorrow, think about how you can build trust strategically by thinking internally and externally.

Lauren Chan Lee: If I haven’t convinced you yet, I have one final example. It’s Zappos.

Lauren Chan Lee: In 1999, they launched with almost no revenue in a very crowded space. Think about all the places online that you can buy shoes, or offline. Over the course of 10 years, they managed to build hockey stick growth until in 2009 they reached over a billion dollars in revenue and sold to Amazon.

Lauren Chan Lee: What was the secret to their success, thank you. You’re making me work for it. Trust, that’s right, trust is what customers knew that Zappos stood for. You could return a pair of shoes even up to a year after purchase. They even took a customer service call that lasted for 10 hours. Customers knew that they could trust Zappos with anything, practically even their life.

Lauren Chan Lee: If you’re a fan of hockey stick growth, if you want to monetize and match, and do all these good things, I urge you to think about, how can I build trust. Thank you.

Connie Fong: Thank you, Lauren. Our last speaker this evening is Rita Chow. We’re very lucky to have her because she has a great experience from startup companies to working to large organizations like Apple. We’re very fortunate that she’s here tonight to share with us what it’s like to be a female engineer.

Rita Chow speaking at Girl Geek Dinner.

Rita Chow: Thank you, Connie. Hi. I’m a mom of two sons who drive me crazy. I’ve been a software engineer for about 19 years. When I was asked to speak at this event, I thought about when I’ve been in the audience at events like these and what I usually appreciate most is hearing about personal journeys, struggles and challenges. Tonight, I’ll share with you my journey, which started in China Town, San Francisco where I was born and raised.

Rita Chow: I grew up with the older generation that wanted a son to pass on the family name. You just get a sense that somehow boys were more special. Growing up, Chinese New Year’s was my favorite time of the year because we receive our envelopes filled with lucky money to bring us good fortune.

Rita Chow: I remember one time when I was eight or so, my brother got more lucky money than I did from a close family friend. I asked my dad about it and he joked that it was because he was a boy. I thought, “What? That’s not fair. Gender wage gap already?” That’s when I decided I wanted to be a feminist when I grow up, but as it turns out the family friend was my brother’s godmother, but that feeling of wanting things to be equal for boys and girls stuck with me.

Rita Chow: When I first got to college, I had no idea what I want to do or study, but I loved math. Some friends suggested I give computer science a try. I did and loved it. In my engineering class, there were not very many women. Today, UC Davis said there are almost 25% women in computer science, but when I was in school, it felt like it was less than half of that. But this wasn’t something I cared about, my focus was just completing my degree.

Rita Chow: When I started working, I was typically the only female engineer or maybe one of two. Being outnumbered didn’t bother me, but some comments started to. There was a male engineer who would say things like, “Hey, I heard you get special treatment from the boss because you’re a girl,” or “I heard that you don’t get yelled at because you’re a girl.”

Rita Chow: Seriously, when I hear this kind of comments, I try to see where they are coming from, but part of me feel defensive. If it was so easy being a girl in a male-dominated field, why aren’t there more of us, and more importantly why do we as women still feel differently? Even though I felt defensive, I was determined never to let people’s biases stop me from doing what I love.

Rita Chow: Sometimes biases have been less obvious. I remember there was a meeting, mostly men. The guy running the meeting picked my female co-worker to take notes. I didn’t really think too much about it at that time because I would have offered to take notes if needed, but my co-worker was upset. She felt he was being sexist by picking her instead of any of the guys.

Rita Chow: Today, this stood out to me when I was taking a women’s leadership class and we had a case study similar to this situation. The purpose of the study was to point out that everyone has biases and they wanted us to think about how we would handle the situation when a male colleague picks you, a woman, to take notes when there’s a room full of men.

Rita Chow: After this case study, I realized why my co-worker was upset. One suggestion given was to say, “I’ll takes notes this time, but maybe someone else can do it next time.” Fortunately, these situations didn’t happen too often for me personally, but I guess, like in college my focus was just to do my work well and enjoy what I do. More than gender bias or treatment, I’ve personally found one of the biggest challenges of being a woman in tech is balancing work and life.

Rita Chow: After I had my first son, I was very sad to return back to work after maternity leave. I just started getting the hang of taking care of a newborn and I was going to miss my son very much. When I returned, I was exhausted all the time because I was still waking up in the middle of the night taking care of my son, feeding, pumping, changing diapers.

Rita Chow: Before I knew it, he was old enough to ask me to play with him and then mommy guilt hit me. Putting down my computer was an emotional tug of war between wanting to finish my work and taking care of my son. I thought, “Should I stop working or should I find a part time job?”

Rita Chow: Well, I did neither, but I started to try to make some rules for myself. For example, I only work near my kids so that I can drop them off and pick them up or be ready to get them in case of emergency. I told myself I should not feel guilty for leaving work to pick up my son or bring them to appointments or take time off to go to their fieldtrips. When I get home I would not do work until I was there for my kids first.

Rita Chow: Another common challenge I hear from parents is finding affordable quality caregiver or daycare for moms to return back to work or during school breaks. Some decide to change to part-time, some would leave the workforce and some stay but are very stressed. In this regard, I was lucky because my mom was nearby and free to look after my sons until they were about three, but a lot of people don’t have this luxury.

Rita Chow: Even so, when they were approaching 3, I also had to start planning for preschool, school, and care during school breaks. It can be stressful because you want what’s best for them. It’s expensive and it takes a lot of time and effort because this challenge is so common and relatable, I was very drawn to’s mission to help people find that best care in the easiest way possible. Maybe if finding care for your loved ones is made easier, less women in tech would need to leave the workforce.

Rita Chow: From my experience as a female software engineer, things that kept me here were, not letting people’s biases stop me from doing what I love, keeping a work life balance, and knowing I am not alone in struggling to do what’s best for me and my family. Thank you for listening to my story.

Connie Fong: I love that final thought. Don’t let anyone stop you from doing what you love. I really want to thank you for your attention tonight. I know we’re running a little bit over, but we’d like to open the floor for some Q&A for the panelist, if you can please come back up to the stage. Who wants to go first?

Audience Member: First of all, this is so fantastic. I’ve been through a lot of these challenges. Thank you. It struck me when you’re talking, Sheila you talked about the gig economy and that you guys are a unique place where you’ve been thinking about the gig economy for years and some of these other companies. This is an open-ended question, but are there any other insights about where you think the future of work is going and was it really important to support massive workforce [inaudible].

Sheila Lirio Marcelo: I think … Is this working? I think there’s a lot that we can do. What’s interesting and we’re sensitive about when we talk about it on the branding side when we debate, we don’t really label a lot of our efforts as a gig economy effort because if you think about caregivers, we’re trying to be advocates to professionalize caregiving and sometimes gig suggests that we’re not valuing and respecting the profession that they’re entering into by calling caregiver a gig worker, but the reality is gig worker should be respected in general whether they’re part-time or full-time.

Sheila Lirio Marcelo: We are in the space even though we don’t label ourselves that way and the way we’ve been thinking about it is if we could be at the forefront of developing products and platforms overall to create more stickiness because if you think about it, Dominique talked about this, right?

Sheila Lirio Marcelo speaking on a panel at Girl Geek Dinner.

Sheila Lirio Marcelo: It’s how do you define products but also addresses the same business strategy and if there’s ways to monetize that, but our way of monetizing it is that, as she described, if we’re attracting caregivers for free and we provide them benefits and we’re investing in it, it’s not only the right moral thing to do, but it’s actually the right business investment that we need to make.

Sheila Lirio Marcelo: The way we think about it at, whether it’s the gig economy, whether it’s the trust, or whatever it is that we’re investing in, we’re always asking ourself is it true to our values and balance that with, is it the right business decision to make, as well. That’s what makes us unique is that we can actually bring the two together.

Audience Member: Thank you.

Audience Member: To start, yes … Is it on? Oh. To start, thank you guys for sharing. All of your stories are awesome, but one of my questions, and I don’t know who can answer this, but as far as your guys’ growth being relatively new, what are some of the biggest challenges that you guys are facing and scaling when it comes to adding on new service providers and users of the platform. Like what are some of the biggest challenges that you guys are looking to overcome right now?

Sheila Lirio Marcelo: We have the head of growth, Dominique.

Dominique Baillet: Yeah. I mean there are so many things that I can think about, but I think fundamentally what it takes for our company to be successful is to always be the best place for caregivers to find a job because families come to our platform if we have the caregivers. But one of our inherent challenges is actually that we have … We are constantly trying to make sure that we actually match supply and demand.

Dominique Baillet: In different markets or in different geographies, we might have more families relative to the number of caregivers we have and then other geographies we might have more caregivers relative to the other families.

Dominique Baillet: We have to balance supply and demand and we have to do it at a regional level. Always figuring out how to manage those dynamics in the right way, I think, is one of the challenges, while still creating really a great experience for caregivers and to also create a great experience for families, because like I mentioned the families are the ones that are paying, but they’re paying because we have the caregivers. I think that’s one of the inherent challenges of a two-sided marketplace, but absolutely that’s one of the things we’re always trying to figure out.

Sheila Lirio Marcelo: Let me add to that. I think it’s hard to build marketplaces without patience. We kind of want to jump to that balance right away, so for the first five years, our focus was really more on density than it was on quality because when we polled the users in the early days other than the 8 and a half by 11 sheet that you got at your local church that you peeled off that little ear with the phone number or the YWCA or YMCA or your next door neighbor, there was really wasn’t a lot of places to go for care because the classified ads were going and maybe there’s Craigslist, if again, as Lauren pointed out, trust is really important.

Sheila Lirio Marcelo: Back then, we really prioritized choice and density and we knew the plight. When you post a job you get 200, sometimes a 1000 results if you lived in New York City. Back then that was great because when you’re looking for care anybody would with a pulse was better than not. You didn’t have a lot of access to care.

Sheila Lirio Marcelo: Nowadays, with smartphones and what Uber and Airbnb and Lyft and everybody’s really reset customer expectations that it’s about quality, it’s about the algorithms, it’s about the match that Abbey talked about, and that’s changing. And that becomes a challenge with also organizational shifts around priorities, because you’ve been focused so much on acquisition free, just get volume, volume, volume, but now you really have to invest in the product overall. It’s a shift also culturally on how we think about things.

Connie Fong: I think I’ll suggest, Dominique talks about the complexity of the two-sided market place, and then the complexity actually becomes exponentially more complex, because if you think about the caregivers who provide care, there’s a very wide spectrum of people who do that.

Connie Fong: We have a lot of college-aged care providers who are very different than your experienced care providers, so you have two very disparate sub-segments within just the supply side. And similarly on the family side, people who need childcare, their needs might be very different than people who need care for their parents.

Connie Fong: You can imagine two-sided being complex in and of itself, but when we think about from a marketing perspective, understanding the right message to the right person at the right time, the audience has become exponentially more complex within each side of the market place.

Audience Member: Thank you.

Audience Member: Thank you all for the amazing talks, by the way. The thing that really struck me was you’re all women in prominent leadership positions with kids at home and I recently took on a very demanding leadership role at my job and I’m wondering what tactics you’ve put into place to help draw the boundaries between your work and life balance that might help you maintain a better balance and a healthier life.

Sheila Lirio Marcelo: Rita?

Rita Chow: As I mentioned, I started to make rules for myself. I think it is very understandable that family is very important, and I think usually at the work, in the office people will understand that.

Rita Chow: I would make rules for myself life, like I said, when I go home, it’s hard for me to put down my computer, but I want to see my kids grow up, so I will take care of them first before I would do my work.

Lauren Chan Lee: I’ll take a stab at this one, as well. I think for me there’s three things. One is just straight up quoting Sheryl Sandberg, “Your partner has to be your partner.” For me, having a husband who is supportive is a really key thing and we really try to balance our home responsibilities.

Lauren Chan Lee: The second, like Rita said, is setting boundaries. One thing that I always make time for is, I always pick up my kids, and that’s just one thing that pretty much always stays the same except when I come to Girl Geek Dinner.

Lauren Chan Lee: The third thing for me, is you really have to like pick and choose where you want to have like your perfection and quality bar versus the things that you want to let go. One of the areas where I don’t maybe hold myself to as high of a standard as other people is just in terms of like the meals we eat. We eat a lot of repeat meals and left-overs throughout the week. I can’t cook every night, that’s just not going to happen in our house. We also have somebody come in and help out with like cleaning on a periodic basis.

Lauren Chan Lee: There are things where you just have to find help. It’s a village and so you figure out which things you’re going to value and do yourself versus you can find help on to help you do it.

Sheila Lirio Marcelo: I’m a very great, professional muffin buyer that we take to school. They’re happy that they bring muffins to school.

Connie Fong: I think women in leadership positions, we are here because we have high expectations of ourselves and I love that. I would never say lower your expectations. I remember this conversation I had with a roommate and she was like, “Connie, the sooner you realize you can’t have it all, the happier you will be.”

Connie Fong: That was like the most depressing conversation that I had with her after Sex in the City, but basically, I didn’t want that to be the path that I wanted to take, but I think the balance is, it’s fine. Have those high expectations, that’s great, but I think to balance that is also be kind to yourself and forgive yourself when things don’t happen the way that you think they’re going to happen and it’s like my house is crazy.

Connie Fong: I have three young kids under 7 and is my house perfectly clean? No, I just take out my contacts and call it a day, like I will forgive myself on that, but you know, you do what you need to do to sort of do the best that you can and have your high expectations, but really just be kind to yourself.

Audience Member: Again, thank you for doing this.

Sheila Lirio Marcelo: Thank you for being here.

Audience Member: I’m a co-founder of the leading internet care platform in Turkey, so we went through some of the same challenges you did, so it’s amazing to hear story. My question is about growth.

Audience Member: When did you, when was the tipping point for This probably goes back a few years for you guys. How did you realize you’d get there in terms of growth? How did international expansion have an effect on your growth and for future growth, how do you see international expansion having an effect?

Sheila Lirio Marcelo: You know, our co-founder Dave Krupinski is here. I would love to turn it to you next. Yeah, I’ll let you answer it first. He’s trying to hide there in the shadows and I noticed him.

Dave Krupinski: Thanks so much for the question. I think the first part of your question was when did we experience sort of significant growth in our history?

Audience Member: Right.

Dave Krupinski: I often tell this anecdote, but I think it was at the time of the financial crisis in 2008, because while other companies were pulling back on marketing and really taking a very conservative approach to spending, we had the opportunity because we were a young startup, recently funded, to really do some experimentation with various marketing channels, especially mass marketing channels like television. We were able to produce some low cost, low budget ads but get out there buying remnant ads through Google TV’s program at the time and really begin to explore what it’s like to run a national mass market TV campaign, optimize the creative, optimize the experience when someone comes to the site, and that gave us the confidence I think and also sort of set us up for a national scale.

Audience Member: Thank you.

Sheila Lirio Marcelo: At the time, to add to that, there’s a lot of things that we do that people say that’s crazy because in 2010, being a very young startup, people thought why would you spend any money on television? That’s only for big companies and profitable companies. But we were always very experimental. We would try, test a small budget, figure it out, but we had a different thesis and that we call today the McDonald Principle. At that time, I had this thought that if in fact, I was wondering, why are all these kids going to McDonald’s and yanking their parents to go to McDonald’s when it’s unhealthy food. Something was working with virality with children, something was also working that was drawing them in to McDonald’s, right? What was it? It was that …

Dave Kopetsky: Happy Meal.

Sheila Lirio Marcelo: Happy Meal and that feeling of like, satisfaction. When we started doing TV ads, we had a rule of thumb which was everything that we do, and Connie knows it, I repeat it all the time is, is it memorable? Is it memorable? Is it memorable? Is it memorable and then the second thing is we’re going to go market and market and market, how great a service this is to kids. Because, it’s a simple message and kids get it, on a TV ad, and, in fact, anecdotally we hear it all the time. If you haven’t heard the story whereas you’ll go out to party or you’re having dinner and you’re saying, “Hey, we’re going out on a date tonight,” and the kids will pipe up and say, “Why don’t you go to” We hear it all the time.

Sheila Lirio Marcelo: The only reason I raise examples like that is actually tipping points are all about experimentation and testing different theories and studying analogous industries and understanding the behavioral psychology, how do you grow certain businesses? We’re constantly asking ourselves that.

Sheila Lirio Marcelo: The move to quality, I said to the team recently, was when we definite it, how does Airbnb define quality now? It’s actually putting shampoos, folding the towel a certain way, tucking the bed. If we think about an Uber car, bottle of water, a little candy, the driver being super nice and saying, “Everything good?” Is the air quality good? You feel like you’re in a limo. When are you getting one of those? But, look at that definition of quality, it’s actually now defined in an offline way.

Sheila Lirio Marcelo: Constantly, we look at analogies in how we run our business and try different things and never always just assume what the rule of thumb for how we run our company should just apply. If we did that, we would have never, to Dave’s point, create an opportunity, the tipping point that built the brand from the get-go. Take risks, experiment and don’t have anyone tell you, “Well, this is the way we just do things.” I just … I would say, “Well, why?”

Audience Member: I lost my train of thought as I was raising my hand.

Audience Member: Here, pass it over. I’m sorry.

Audience Member: That’s fine. Oh, no, no. Go ahead.

Audience Member: Thank you for this. This is amazing. I look up to Sheila so much as a Filipino-American in tech so thank you for that and everyone is just amazing, senior leaders. I’m releasing a book on augmented and virtual reality, that’s the space that I work in, in March of next year. I’m currently a part of Oculus Launch Pad working on an app to release some multiplatforms. Shameless plug. I’m not trying to work here but I think you have an amazing company.

Audience Member: My question is like, thinking about outside of values of diversity inclusion, Rita, you talked about what keeps you here. For many junior women and other senior women that I’ve talked with, if they’re not founding a company within the AR and VR space, we constantly talk to each other about, “Well, what’s the culture like at the company? What are the values? Why would I work there? How would I be challenged?” These are questions I keep in mind because I decided to stop angel investing and blockchain on the side and potentially take another gig. But I’m overwhelmed.

Audience Member: I think with Silicon Valley being the buffet of options of like you can work at Apple, you can work at Google and it’s … I struggle with paradox of choice and I think looking at senior management and leadership, people like Sheila, like, “Oh yeah. I would totally work for someone who I can identify with or that looks like me.” What are the … I guess like top three values about what choices you’ve made in your career and outside of working here at that you think you could impart on us here, whether we are junior software engineers, people who are thinking about working at a company and why, just largely besides being purpose-driven, how you are challenged and why you stayed, choose to stay because I think, picking a company is very challenging when you are a senior woman engineer, like everyone you’re getting hit up all of the time. I’m wondering what made you choose here other than, say work life balance and that keeps you challenged. That’s my question.

Sheila Lirio Marcelo: Very competitive to hire each and every one of these leaders and I think this is a great question. I’m going to turn it over to Abbey first.

Abbey Stauffer: Prior to a couple of jobs ago, I worked almost my entire career in education in some way. I was never actually an educator but I worked in the tangential education space. I worked for Kaplan. I worked for an ed tech company that delivered coaching to college students and what kept me in that industry was the people were just so wonderful because everyone was grounded in a shared mission of wanting to help students and it wasn’t just a mission that they just emblazoned on the wall.

Abbey Stauffer: People really lived it and that really helped me feel energized in my work in hard times and in good times. I would say that was like looking for mission-driven work was what grounded me at the education point of my career but then as I moved out of that industry and into other industries I looked for that common thread because even if I am not working in education, I stepped into finance for a few years and now I’m here at Care, there has to be a common thread of really good people who are all truly committed to the mission that we’re working on, not just a motto on their T-shirt, but something that people really embody, that’s what’s really grounded my career.

Sheila Lirio Marcelo: Lauren, anything else?

Lauren Chan Lee: Sure. Well, I joined Care in January and before joining Care I was at StubHub for about eight years and when I was looking at what my next step was, I was looking at a lot of marketplace companies because that was a clear fit with my skill set.

Lauren Chan Lee: You actually nailed some of the key things that brought me here. One was being in Silicon Valley at a company with a woman founder and CEO is a huge draw. It’s not something you see every day.

Lauren Chan Lee: The other component for me, especially being a working mom, is that I’ve always been very passionate about how we empower women. When I was at StubHub, I was one of the founding members of our women’s group there and I was the president for a couple of years.

Lauren Chan Lee: This was a passion area of mine and I saw an opportunity to align that mission with the work that I would do every day at Care and it’s great to know that my work is actually helping both sides of the marketplace, women being empowered as working families as well as caregivers. Those were two really major components of my decision to join.

Sheila Lirio Marcelo: Connie? I thought it’s such a great question for everybody to answer.

Connie Fong: Yes, what was interesting to me when I joined this company is how altruistic everyone is. I don’t know if you’ve read my background, but all of my career before coming to Care was in omnichannel retail and I had worked for Sephora for several years and the Williams-Sonoma, Inc. company.

Connie Fong: For me, I’ll be straight up. I wanted to work for the number one business in the space that I wanted to be in. From a marketing perspective, Sephora was clearly number one in the space. From a care perspective, was clearly number one in the space.

Connie Fong: When I thought about also what was important to me, I majored in psychology and business and so for me, I wanted to be in a space where there was always irrational demand for the product. At Sephora, you’re dealing with human vanities. There’s really like uncapped potential there, and in the care space, I mean you really can’t put a price on the care for your children and how important that is. That was another, I was looking at the love space, whether it was dating, whether it was weddings, I mean, I was very strategic into thinking about what type company would make sense for me.

Connie Fong: I wanted to be number one. I wanted to have an irrational demand for the product, and for me, having three kids, like literally, the logistics and like the practicalities of finding something that would work from … I was commuting over 3 hours a day and that was just not sustainable for me. That was one logistical element that was very important. But figure out what your passion points are and figure out who the best is at it and that was my guiding principle.

Sheila Lirio Marcelo: Rita, do you want to add anything more than what you shared in your talk?

Rita Chow: It’s just mostly location. The thing I am working on is very interesting and being able to have a work life balance. I guess it’s what I’ve really said earlier.

Sheila Lirio Marcelo: That was great. Dominique?

Dominique Baillet: Yeah. I’ve actually made a lot of career moves in my life. I’ve had a number of different professional jobs, when I was earlier in my career and where I am now, I’ve always made choices based on where do I think I’m going to learn the most and where I’m going to grow the most.

Dominique Baillet: What’s interesting is that what I need to learn and grow has changed as I’ve gotten older. When I was … Earlier in my career, it was all about skill development. Where can I learn transferable skills? Where can I learn the most from mentors about like how to actually do something?

Dominique Baillet: A certain point in your career, you actually check the competency box and then you migrate over into a territory of, now you just need to be really confident and you need to be able to walk into a room, command that room and there’s a different level of skill there.

Dominique Baillet: Actually, like many people up here have said, one of the reasons I joined Care is, I’d gotten to a point in my career where I could check the competency boxes, have the degrees and all of that, but I was in environments where when I looked above, I didn’t really see examples of leaders that felt like, that’s the type of leader I can be. I found myself feeling like, wow, in order to continue to rise, I really need to change my style. I really need to do something different and it felt uncomfortable. It felt like that one is going to be hard for me to do in an authentic way.

Dominique Baillet: Coming to Care actually and being able to learn from Sheila and seeing, not only Sheila, but other executives in more senior positions than me and being like, “Yeah, I get them and I can get there with the style I have or with the skill I have and there’s other things I need to learn,” but still feeling like it was possible, and for me that was really important to continue to get that next level of confidence to truly believe that with what I have I can get there, and that frankly was just a lot harder if I was in environments where I couldn’t look above and see examples of leaders like me.

Sheila Lirio Marcelo: I’ll just add to this, it’s so competitive in the Bay Area in lots of different areas but even more so here, you get recruiters are pinging you the time. There’s just so many opportunities, they’re pitching the next startup and I think … or company, some great companies.

Sheila Lirio Marcelo: I think the thing that I’ve been focused on in my career is long-term relationships.

Sheila Lirio Marcelo: I probably interviewed once and updated my resume from one of my first jobs and I haven’t since because I just kept moving from company to company, following leaders that I believed in that actually gave me opportunities and continue to help grow me and believe in me, because that’s difficult to replicate.

Sheila Lirio Marcelo: There will be plenty of startups, there will be plenty of sexy new technology, there will be plenty of great, great opportunities, recruiters will always pitch you. If you follow the opportunity in the pitch, sometimes you luck out. It’s going to be great and then you could retire young, which there’s a lot of potential of that in the Bay Area, but then there’s also what the journey of life, which is who do you want to be around.

Sheila Lirio Marcelo: There’s definitely purpose, but I think, there’s also just the richness of where we spend most of our time, majority of our time in our lives, and so, if in fact, you enjoy the people that you work with and you found that tribe, I’m always encouraging people to say try and stick with that tribe, move from company to company and there’ve been times when I’m completely fine when somebody says, “Look, we’ve worked together. I’m going to leave for a little bit” and we’ve had people boomerang back or we work two or three companies later together. It might not be the next one.

Sheila Lirio Marcelo: It’s just something to think about as you look at opportunities, is to actually look at the people in your life. Who is helping continue to sponsor and help you grow and catapult you to opportunities because they know you well and that’s really want to do in your career, would be my just a small piece of advice.

Connie Fong: Thank you so much for all of your questions. I know we’re running a little bit over time but we’re actually going to stay for a little bit so you can definitely come find us to ask questions. We really want to thank you for your attention, your engagement, your questions this evening. It was a great opportunity for to be here tonight and thank you again so much for taking the time.

Sheila Lirio Marcelo: Can I just say, thank you for the entire Exploratorium staff. Thank you so much. Thank you to Erin and the entire crew in white shirts who just made this beautiful. I mean we get to up here represent, but thank you so much. You guys made this happen. Thank you.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Girl Geek X Quantcast Lightning Talks & Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Quantcast girl geeks: Esther Hsu (Staff Software Engineer) on machine learning in real-time bidding; and Malvika Mathur (Senior Software Engineer) on transitioning from Microsoft corporation to a 700-person startup like Quantcast; and Brittni Gustaf (Senior Software Engineer) on prototyping customer solutions at an internal Quantcast hackathon at Quantcast Girl Geek Dinner in San Francisco, California.

Dorothy Tse / VP, Product / Quantcast
Esther Hsu / Staff Software Engineer / Quantcast
Malvika Mathur / Senior Software Engineer / Quantcast
Brittni Gustaf / Senior Software Engineer / Quantcast
Somer Simpson / Head of Product Management – Measure / Quantcast
Disha Gosalia / VP, Service Operations & Support / Quantcast

Transcript of Girl Geek X Quantcast Dinner – Lightning Talks:

Dorothy Tse: We are so thrilled to have, hosting all of you tonight for Quantcast’s first Girl Geek Dinner. My name is Dorothy Tse and I lead product management for our Advertising Solutions here. Hopefully, you had a chance to mingle a little bit and do a little bit more networking. I certainly did. It’s just so fantastic to get to know some of you here and your like-minded interest as women all together in a single room. So it’s really fantastic.

Dorothy Tse: We have a number of exciting talks to share with you today. And up first is Esther Hsu, talking about delivering ads with machine learning. And then we’ll have Malvika Mathur, speaking about what her experience has been like transitioning from a corporation to a startup. And then we’ll have Brittni Gustaf, talking about what it’s like to hack into the customer experience, and then followed by Somer Simpson to talk about how a small team can impact an entire industry. And this is the GDPR piece. And then we have last but not least, Disha Gosalia, speaking about her experiences of how she navigated her career as a shy engineer.We’ll close with some Q&A for all of the speakers and continue with more networking and drinks, if you’re still up for it. All right.

Dorothy Tse: One of the things I’d like to do first is to give you a quick intro on Quantcast. Here at Quantcast we believe that AI will fundamentally change every company, industry, and customer experience. Well, that’s not something to fear because in the 21st century AI is more designed around how to compliment and boost human learning rather than replace it. We’re on a mission, as a company to help brands grow in this AI era. The way that we do this is we help brands and marketers make sense of all the data that’s out there, to understand and make smarter decisions faster. We started in 2006 as a company, as an audience measurement platform, helping online publishers understand their audiences as well as their web traffic — this is called our Quantcast Measure product.

Dorothy Tse: And over the years, our technology and our business has grown and we now measure over a 100 million web destinations across the globe and some of those web destinations include what you see here on the screen. Our technology track things such as site visits, keyword searches, content categories, just to name a few. We process over 30 petabytes of data in a single day. That’s kind of hard to quantify so it’s trying figure out what is a good way to give a visualization of that — that’s essentially, think of 600 million four-drawer file cabinets filled with content. It’s a lot of data, but many people say they have a lot of data as well.

Dorothy Tse speaking at Quantcast Girl Geek Dinner.

Dorothy Tse: What we’re particularly good at is our ability to make sense of this data. We’ve been working on Q. It is the world’s largest AI audience behavior platform for the open internet. We utilize the measure platform data to help us drive up predictive models as well as AI optimization to score audiences in real time. As a result of that, there are several capabilities that we provide as a suite of solutions for brands and marketers to help them with growing their business.

Dorothy Tse: First, we offer real-time audience insights. And these help to uncover who our marketers’ target customers are as well as what motivates them and how to influence them. We also provide predictive targeting and our predictive targeting allows us to target the right audiences at any point in their user journey, even before they’re in market. And then we also have comprehensive measurement. This is our ability to, in real-time, share audience level and campaign level insights that inform optimization decisions and decisions about how they want to better market to their target customers.

Dorothy Tse: This is really exciting stuff for our 700-plus employee base here. We’re a global company. We span over 10 different countries and have 20-plus offices across the globe. We are hiring, so definitely talk to us at any point. First up is to thank you so much for listening and again thank you all for attending today’s Girl Geek Dinner. First up is Esther Hsu. Thank you.

Esther Hsu speaking at Quantcast Girl Geek Dinner.

Esther Hsu: Hi, everybody. My name is Esther and I am a staff software engineer on the targeting team here Quantcast. Today, I’m going to talk to you about how we how we use machine learning here at Quantcast specifically in the context of our ad targeting product. This will not be a lesson in machine learning and I’m sorry if you were expecting that.

Esther Hsu: What do we use machine learning for? We use to reach a specific audience. Each of our clients is running a display advertising campaign and they all have a specific goal in mind. That goal can be anything from raising brand awareness or driving certain actions. Like clicks or purchasing products or anything like that. We use machine learning to help our clients target the right people at the right time in order to accomplish that goal.

Esther Hsu: For you to understand what it actually means for us to target people, you need to understand what real-time bidding is. Real-time bidding or RTB is a mechanism to deliver ads by auction on an impression by impression basis — impression meaning, like, ad view. With real-time bidding, we can target people at an individual level and we can also buy impressions at individual level. Hopefully this illustration I’m going to go through is going to clarify what that actually means.

Esther Hsu: Let’s say you’re this little gray stick figure and you go to a website. A lot of things actually happen before you see an ad on website. And the first thing that happens is that a request is fired to an inventory supplier. And what this inventory supplier is is, there are several inventory suppliers out there, some examples are Google, AppNexus, PubMatic, but what happens there is basically a milliseconds-long auction, where the inventory supplier asks all of their bidding partners, how much are you willing to pay to show this person an ad? So what happens is they would send out bid requests to all their bidding partners, requesting a bid. And Quantcast is one of those people doing the bidding.

Esther Hsu: This a blind auction. Meaning that none of the participants know what the other people are bidding, but everybody sends back their bids and just like any other auction, the highest bid wins. Then the inventory supplier will then choose that ad and that’s the ad that you end up seeing. I know all of you were surprised that this happens because it happens in literally 10 milliseconds so you probably have no idea.

Esther Hsu: How do we actually use machine learning to do this well? The part that machine learning has in this is that we need machine learning to understand how valuable each one of these bid opportunities are. We use machine learning to basically come up with an optimal price. What this means is that we train machine learning models and we use them in real time to understand how we should price each bid.

Esther Hsu: What does this model training actually look like and what are these models trained off of? Like I said before, all of our clients have a specific goal in mind with their advertising campaigns. And usually in the most common case, they are trying to drive a certain action, which, a lot of times, was represented by site visit or a certain page. For example, a shoe supplier or shoe company might want to drive shoe purchases, in which case they would choose the thank you page or the shopping carts, which indicates that someone actually bought the shoe or expressed interest in the shoe. Where an insurance company might choose the request a quote page and in any case the client will tag the page and in that way we can then label our data.

Esther Hsu: Our data, like Dorothy was saying, is coming from our Measure network, which is made up of more than a 100 million sites, and from that, we have this really rich data set of user behavior and really interesting things, that we can actually then label as converters and non-converters or the baseline. And then this is very simplified, but then we run a supervised learning algorithm and we produce a model. And that model will then tell us what does someone who converts actually look like.

Esther Hsu: We process about 30 petabytes of data a day. A lot of that is because of model training. We built infrastructure to train thousands of models a day, process again lots of petabytes of data. And that way we have up to date models for all of our clients at all times.

Esther Hsu: Now I’m going to go over an example what a model actually looks like. We train a lot of different models, but this is just like a very old example with a curated set of features. But basically this is an old model that we trained for an online dating service, who was a client of ours. You can see that the green coefficients correspond to features that mean that you’re more likely to be interested in an online dating service and red ones mean you’re less likely to.

Esther Hsu: You can see that a lot of these are actually very intuitive and make a lot of sense. For example, you’re looking for online dating, you’re probably going to register for online dating. And if you’re looking for baby care, you’re probably too sleep deprived or too busy to care about dating. But something more a little less intuitive like, fantasy sports, does that mean you’re single? I don’t know. And if you like books, maybe you would rather meet people in different ways. I don’t know. But The point is … Thank you.

Esther Hsu: The point is that even if you’re an expert in your product or your market, machine learning is going to pick up on all these signals, event that no one would normally be able to find. And normally, models have millions of features. This is just like a very curated set.

Esther Hsu: How do we use this model in real-time bidding? When we get this bid request from the inventory supplier, we have to retrieve the user data that we have for this particular bid request, futurize it, and then basically score it against the model. And from that, we can calculate a basically a number that tells us how likely is it that this person is going to be a converter or someone who’s of interest to our client. And then based on that, we can calculate a bid. And I realize that’s very simplified, but on very, very simplified terms, the bid is calculated from both the score and also several different control signals that we have, which indicate how much budget the client needs to spend and things like that. But very simply, if you’re more valuable, they’ll bid higher. And also this entire process again, happens in less that 10 milliseconds. And we do this for about a million bid requests per second. That’s kind of like an overview of what happens right now.

Esther Hsu: What are we working on in general or what are we continuing to work on? Scalability, obviously, always an issue for any engineer. How do we make sure that we can do this for more clients, more data, more complex models or just more bids? And then also ad tech is a very dynamic industry. It’s relatively young. Things that our clients care about from one year might be different the next and because of that we have to adapt quickly. We have to always be updating our models to be optimizing for the things that our clients care about. And even besides that, if our clients care about multiple things, how do we make it so that we can optimize for different goals, balance those against all the constraints that we have as well. And that’s it.

Dorothy Tse: Thank you Esther. That was a great primer on machine learning at Quantcast. So next up we have Malvika Mathur, who’s going to talk to us about her experiences moving from a large company to a smaller one.

Malvika Mathur speaking at Quantcast Girl Geek Dinner.

Malvika Mathur: Hi guys. Before I start off, I just want to say I’m really nervous so if you don’t get me, that’s not my fault. All right. So I mean working in the tech industry is hard. Like long working hours, keeping up with the latest technology, all that weight that you gain from eating the free food. I mean I did.

Malvika Mathur: Hi, I’m Malvika Mathur, and I’m a senior software engineer here in the Data Platform team at Quantcast. And today, I’m sort of going to talk to you about my journey of transitioning from a big company corporation to a start up. And I’m hoping by the end of this talk, you guys can take away some pointers on what you can do to evaluate the right work environment for you and that can be even within the same company that you’re at right now or somewhere else.

Malvika Mathur: Where was I before this? I joined Quantcast January of 2017, but before that I was working in Microsoft for five years in the India headquarters at Hyderabad. And I joined as a 21-year old, right out of college. And I was like, “Damn it! That’s it. I’m done. I’ve accomplished everything that I need to.” So happy with myself, but the 21-year old in me was really naïve as well. So the first years with Microsoft were really great. They had this program where they give you the opportunity to go between … to go in different teams and different business units and sort of get a feel of what it is to work in these different roles. I got the opportunity to work as a developer. I got the opportunity to work as a tester. I got the opportunity to work as a program manager. But then I decided to work as a developer, continuing that because I really like problem solving.

Malvika Mathur: I joined my new team and I’m there a few months and then my then-manager comes by he was like, “Hey, we have this project. Would you like to join?” I’m like, “Yeah. Sure. Secret project. Why not?” So the task for us was sort of like reinvent the entire calibration process for the entire company. Like, “Okay. How do we do that?” And the other thing that we had to do was we had to deliver this in a really, really short time. That meant for the next three months, we were working nights and weekends and everything. It was super exhausting. But the good thing about that was that is forced me to have like a really steep learning curve.

Malvika Mathur: For the next three months, I was working with great engineers. I was working on the latest cloud technology that Microsoft had to offer. It’s like, “Awesome. This is great.” But then, early 2016, my husband and I decided to relocate to San Francisco. I was like, “Well, okay. Microsoft has offices here. There are teams here. I’m just going to stick and go to one of those teams.” I was in talks with recruiters and figuring what I need to do next and then I decided to talk to one of my mentors and he asked me something really important. Something I never thought I’d ask myself.

Malvika Mathur: He asked me, “Why do you want to stay?” I was like, “Why is that even a question? I mean like it’s my dream company. The pay is great. All my friends are here. I like the work. Why would I want to move?” But then he asked me again, “Why do you want to stay?” And I thought about it. It turns out the answers for both these questions are not the same. I thought about what I’ve done so far in Microsoft. I thought about if I move to a team here, what would it mean for me?

Malvika Mathur: And I realize that it’s going to sort of slow down my growth trajectory, and it’s something that’s really important to me. I mean it’s great to be learning new technologies, but I realize that as a developer, that’s not all I wanted to do. I don’t want to just go in and write code. I want to do something more. Contribute more in the work that I do. Suddenly, life out of Microsoft sort of became an option. Since I was moving to the Bay Area, working at a start up was suddenly on my shortlist.

Malvika Mathur: I started looking for jobs. And looking for jobs is hard — it is exhausting. And I realize that subconsciously that was one of the reasons I didn’t want to move out — I was in a stable job, I was comfortable, I have my friends around. I don’t want to move because of that. But in the whole process of not looking for a new job, I ended up ignoring the whole process of what’s right for me and my career at that point. So I started to evaluate that and I started to sort of like give that a lot of focus when I was interviewing in all these companies.

Malvika Mathur: They’re asking me questions, but I also made sure that I was asking these guys the right questions as well. Because I wasn’t that girl anymore who joined a big company, who was excited with any project. I wanted to sort of do more things. And I wanted to make sure that wherever I went I got those things. Whenever I go and talk to these people, I started checking on like, “Hey, what’s your technology stack, am I going to learn something out of that?” Right?

Malvika Mathur: What are the sort of projects the team is working on right now? What are the projects they’re going to work on later? What’s a big problem that the team is trying to solve for the company or the industry that they work in? And as I started asking these questions, I realized that I am sort of leaning towards working in a start up environment. I think that’s something that’s really important. Whenever you’re trying to find a place that you want to work at, it’s really important to sort of know what challenges you and what excites you to work there. And that’s how I ended up at Quantcast.

Malvika Mathur: I joined Quantcast January of 2017. The last year and a half has been a rollercoaster. I like roller coasters, but it’s one of those Six Flags Magic Mountain types. Well, initially when I joined, I had this really bad habit of just comparing everything that’s done here with how I used to do it in Microsoft. I do that sometimes still, but I try not to as much. And the more I compare, I realize that there’s a pattern. And the pattern is that in these big companies, particularly like what I was doing in Microsoft, work is more divided. Responsibilities are divided. Teams are more siloed. You know exactly what you’re supposed to do. When I go in as a developer, it would be like, “Hey, these are your things to do today.” And you just do that and walk out and that’s it. But here, I was involved in stuff from ground zero. Like I was there at the conceptualization of ideas and while we’re building the feature or while we’re doing the system. And I’ll see it through. And then in the end, I’ll be responsible for taking care of it when it’s in our production.

Malvika Mathur: Another thing was the technology stack. Microsoft was all .NET, here it’s all open source. I mean the first day I walked in, these guys gave me a MacBook. I had never worked on a MacBook before. My first few weeks here were so frustrating. And then after that, they’re just like, “Hey, we have some systems here that are written in Ruby, Java, Python. You own them now.” Right. It was challenging, scary, challenging, but in a good way. So while I was ramping up and figuring out all these differences, I realize that the biggest takeaway is that it doesn’t matter how a company operates or what technology stack they have. The biggest thing that matters is your appetite for learning and where you can get that in a work environment.

Malvika Mathur: In different stages of our career, we have different needs. And it’s really important to cater to those needs. When I started off, everything inside in me, I didn’t care, it’s big company, awesome. But then as I grew up, I realize that I wanted something more specific. I want to do certain things and I tried to find the right fit for me.

Malvika Mathur: I think one thing all of us should do when you go back today is try to figure out why you are where you are and what would help you make the right career move in the way that you want to go to. And that’s something that could be within the company that you’re at right now or outside. Thank you.

Dorothy Tse: Thank you Malvika. That’s fantastic. I’m sure there’s a few of us in the room who can resonate with a story like that, going from a large company, a small one, doing all the comparisons. Pros and cons. So thank you for that. So next up, we have we have Brittni Gustaf. She’s going to speak to us about hacking the customer experience.

Brittni Gustaf speaking at Quantcast Girl Geek Dinner.

Brittni Gustaf: All right. Hello. So I’m Brittni Gustaf and I’m a senior software engineer here at Quantcast. And I’m on the Measure team. So you guys have been hearing a lot about advertising, but I’m in the other side of the company. So I’ve been here for just over four years now and since I’ve started, we have lot of changes on the Measure team, especially on how we go about creating products and features, how we design them and then how we actually implement them. And it’s improved for a lot since that time. So I’m going to kind of get in to why, like how we’ve made those improvements and, yeah, bring my leader’s knowledge to you guys, I guess.

Brittni Gustaf: Okay. Measure has kind of been neglected since I started. We’re not the side of the company that brings in the money. We provide the data that provides … the side that brings them the money, but it’s really hard to quantify features that we’re doing it and for it to actually having an impact on your company or not. Because we are focusing so much on what will make the company grow, Measure kind of got passed to the side. We didn’t have a lot of people looking into what features we should have to continue to improve Measure. What we did instead is we kind of came up with what we thought people would want. We didn’t really ask them very much what they wanted, we just kind of, we’re like, “This would be cool and probably would help. Let’s create it.” As you can probably guess, that didn’t work super well.

Brittni Gustaf: We spent a lot of engineering time creating products that no one actually really wanted. And that was pretty disappointing when you spend all this time as an engineer and you’re like, “Wow, I made this.” And it was like, “Wow, no one wants that.” You’re like, “Okay.” The company, soon after I started was like, “All right. We’re doing this wrong. We need to change something up.” We did a reorg. And I remember the day after we announced that we had to reorg really well, we had our full day meeting with our new leader of Measure. His name is Sam. And one of the things that stuck out most to me during that meeting was he started telling a story about his acquaintance that he had previously. Don’t quote this to him by the way, I am doing this from my memory, from a long time ago, but I think I have it pretty well because it was so kind of terrifying to me.

Brittni Gustaf: What he told us was like, he had an acquaintance and what this acquaintance did was he went and created, spent the least amount of effort he could to create a prototype and then went out to find customers for his prototype, selling it as a product. And he would be like, “Here. Look at this awesome product we have.” And the customer will be like, “Wow. That’s really cool. It would be really awesome if it had this feature.” And he’d be like, “Okay.” He would spend the bare minimum amount of time implementing that feature into his prototype and then he’d keep going back to clients and being like, “Look at this awesome product we have.” Until he got enough investment into his product to then actually create the product. And I was like … While he’s telling me the story, I’m just like, “What is happening? Like this is super sketchy, are we going to be lying to our clients here and telling them we have products we don’t actually have?” Well, he quickly assured me that that was not what our goal was, but that we should have a client-first sort of approach to things, where we create, we spend minimal effort, create a prototype, show it to the client and get feedback before we waste all of this time on it. And that was this mindset that led us to the Measure Hackathon.

Brittni Gustaf: How the Measure Hackathon works and how it’s totally different from other hackathons is that we would actually get all of our … Well, we try to get a diverse and key clients into the actual office and we just brought them in and in the morning, we spend three hours with them, just asking them questions about what they do in their job and how could it be improved. And then trying to come up with ideas for how our software could improve it. So after these three hours working with them, you’ll probably recognize a lot of these from one of our earlier slides because they’re the big ones, right? After this three hours, we actually … they went and got on a bus and went to go do fun clienty stuff and the engineers got stuck in the office for 24 hours to try and put this idea into an actual functioning-ish prototype. That is at least demo-able.

Brittni Gustaf: This creates two different experiences. So yeah, the clients come here. They’re like, “Oh, here are my problems. All right. Cool. You guys work on that and we’re going to go up and get literally, wined and dined and party it up until 24 hours later.” Which they’ll come back and then hopefully we have solved all of their problems. On the other hand, you have the engineering experience. Not quite as glamorous, as you can see. You get a really creative with what kind of seating you’re going to sit in. I love this because it’s like, how many different seats can you try, but you need to be comfortable for 24 hours and it takes a lot of work to be comfortable for 24 hours programming. And then you also gain a very unhealthy dependency on caffeine so that you can function throughout the entire time.

Brittni Gustaf: All right. I don’t know who I’m kidding. We all love hackathons and we all know it. Luckily for us, the company literally butters us up and they give us tons of stuff while we’re doing our hackathon. They’re like, “We love you guys for working. Here, have all your favorite things.” And I feel like they literally catered this food to me. Like sushi, they literally bring in sushi chefs and they make sushi for us. They give us acai bowls, which are my favorite thing in the world. Pizza, it’s like amazing. And they also give us tons of other stuff, like swag. I was going to wear my sweatshirt because it’s really nice, but it’s way too hot. And we get massages.

Brittni Gustaf: One of the best things that we get is that you really get to know people who work in your organization that you don’t really work with. It’s a lot of bonding when you’re trying to solve these problems really quickly and you’re all working in the same code base for 24 hours on the same thing.

Brittni Gustaf: We also like to take breaks, keep the brain lubricated. We’ve gone midnight drinking, which is always a lot of fun. So the client wins, they get wined and dined. The engineers win, the company’s trying to butter us up a ton, and the company wins, as well, because we’ve had a lot of really successful features come out of this.

Brittni Gustaf: Here we have something that came out of our first Measure hackathon, actually. Oh, I forgot to tell you how you win. So the clients, they get a hypothetical amount of money that they can spend on each prototype, so each project, and whoever gets the most money, we actually try and turn into a functioning feature on the website. This is our first one. And it’s one of the most successful products now at Quantcast. People really like it. Yeah. And then if you guys are interested in learning more about how you should prototype things, be customer first, you can check out our blog post. We have the last two years up there and it has some cool videos that give you a full feel of the entire thing. Not 24 hours on, just five minutes, but yeah. Thank you.

Dorothy Tse: Thank you so much, Brittni. I can attest to the comparison of the hackathons here at Quantcast to the hackathons at Facebook. And I’d much prefer the hackathons at Quantcast. So up next, we’ve got Somer Simpson to talk to us about how a small team can impact the entire industry.

Somer Simpson: Thank you. You guys still doing good?

Audience Member: Yeah.

Somer Simpson speaking at Quantcast Girl Geek Dinner.

Somer Simpson: All right. Awesome. So we have a pretty significant success at Quantcast. We’ve been on a lot of industry news over the past, it’s only been two months since May. And that’s like the marketing story. That’s like the cleaned up version. What I wanted to do is kind of give you guys the story behind the story and that’s really the interesting part. And also because I’m like super proud of my team because they seriously seriously kicked ass with a really really complicated problem. We’re going to talk about GDPR. Raise your hand if you’ve heard GDPR. Excellent. That makes my job easier. Cool. And I hope you really paid attention, as well, to the first presentations because it gave you a really nice clean overview of what the ad tech sort of industry looks like. So you kind of know the area that we’re going to have to play with.

Somer Simpson: So what is GDPR? I’ll give you the short version. So it is the General Data Protection Regulation. I have other words that I plug in for those those letters occasionally, but that works. So anyway, this is a law that was passed by EU regulators a couple of years ago. It went into effect on May 25th of this year, which was an incredibly fun day for me. It’s actually an addition to a previous law called the ePrivacy Directive that was passed a number of years ago. And basically, all together what it does is it says that companies who access users’ devices and they set cookies and they collect data on individuals and they process that data, they have to have consent from users to be able to do that.

Somer Simpson: When ePrivacy Directive was passed, there was no sort of like definition of what does that consent mean. Like what do we actually have to do? So it’s open to interpretation. All across Europe you see all these banners that pop up on everybody’s sites that say things like, “If you continue navigating the site … “ And often it’s like really tiny, down in the corner. “If you continue navigating the site, that means you consent to us using your data for any reason that we want and we’re not going to tell you why or how. And by the way, you’re first-born is ours too.”

Somer Simpson: What GDPR did, when they realized what was happening, it actually says, “Okay guys. All right. So first, here’s some rules around what consent means. It has to be unambiguous. You actually have to have somebody like click and take an action that says yes or no, I can send.” Everybody in ad tech, just about anybody on the web sets cookies. Everybody does some level of tracking. You go to a website now, everybody’s got plugin, browser plugins that show you just how many cookies are being set. We’re being tracked a lot.

Somer Simpson: If you can imagine, if every single one of those companies individually had to ask a user if they can consented or not, now when you go to a website, you’ve got 50 pop-ups happening in front of you, asking you for your individual consent. That wasn’t going to work.

Somer Simpson: We knew that disruption was going to be inevitable. But this is tied to revenue. Disruption is not an option, but neither is business as usual. We have to respect consumer privacy. We’re all consumers. We value our data, we value our privacy, and it’s important that the companies we work for and the companies that you work with do that as well. We set out on a project to deal with how we were going to deal with GDPR because we had significant business in the EU and it was important to us, being a privacy-first company to begin with, but to also address this and stay up to date with the clients.

Somer Simpson: We have been working with IAB Europe, Interactive Advertising Bureau, IAB Europe. They have had this working group going for a number of years. That started out mostly with the lawyers talking and trying to debate and understand and figure out what their thoughts were on it. And we kept getting closer and closer down to the wire of May 25th. And then last minute, they pull the engineers in and they’re like, “Hey guys, we need a technical solution for this. You’ve got three months. Go.” Yeah. That was mildly entertaining. We knew that we needed a solution, if that’s the solution to every problem.

Somer Simpson: What we did was we, at Quantcast made a bet on an industry solution. This was the only way that we were going to be able to prevent major fragmentation in the marketplace and still, as an ecosystem, be able to work together and at the same time, not just be compliant with the law, but actually really respect consumer privacy and listen to what their preferences were and actually honored those signals. A number of companies work together. A lot of competitors. Not only were we trying to solve a common problem that we all had, but everybody was coming to the table with their own agenda. It was a lot like herding cats. When I got pulled into these conversations, I’m a little blunt. I have a little problem with patience sometimes.

Somer Simpson: I went through two of these calls, actually three. They were happening weekly. Same set of people and we just talked about the same thing every single week over and over again and never made any progress. We had four potential solutions that had been proposed and we were basically debating like the most … The most ridiculous minutiae of each one. And trying to figure out which one we were going to do. Everybody got impatient. We’re like, “Okay. Fine. This Friday we’re going to put it to a vote.” And I’m like, “No. None of these work. They’re all awful for some reason.”

Somer Simpson: I went back to my team — and, which Brittni was on our team — and pulled everybody together and was like, “Okay. Here’s the situation.” And I explained everything, pulled our chief privacy officer in so she can answer the legal questions because I don’t have a legal degree and we basically had three days to come up with a better solution than what had already been proposed. And then bring that back to the group and hopefully they buy the idea and then we go from there.

Somer Simpson: This is the team. We had chief privacy officer. We had one incredibly busy designer because they always are. We had an engineering lead, who also had four other teams that he was having to manage at the time and then we had four engineers, who also had other work that they were responsible for and none of them had a legal degree, but had to still be able to understand and operate on that level. We had … Well, the first day was me talking so really three days. But four days to get … Until a group vote was going to happen on this proposed solution.

Somer Simpson: We had one moment of inspiration where one of our chief engineers who we talked to about the problem was out jogging one day and just had this moment of inspiration and came back in and he’s like, “I have an idea.” We pulled everybody together. We had three days to figure this thing out and then an hour to visit the idea and basically change the future of everyone in this working group.

Somer Simpson: The way we approach this, and these are the … It’s kind of the things that I think were what really drove our success, other than the fact that we had a incredible team of super, super smart people. We went consumer vote first. The problem with the working group is they were so tied up in their own agendas and figuring out what they wanted. They were forgetting that it’s all about consumer privacy. And that’s what we had to solve first. We did that, realized none of them were doing discovery and talking to consumers or even talking to very many publishers.

Somer Simpson: I very quickly went out, picked up the phone, started calling people, getting input, and and came up with a framework of what was being asked for. We added the context of that so we all got a crash course in the law and what our interpretation of it was, and then we understood the unique sort of positions of the other companies because they’re the stakeholders, you got to convince them. And then we prototyped this thing in just a number of days. Just so that we could have a proof of concept. Because talking about something and the architecture of something is not quite as valuable and strong as actually showing someone that this will work.

Somer Simpson: That’s what we did. We present it to the team and it was like dead silence and then the leader of the group said, “Do you guys want to vote?” We’re like, “Sure.” And then we all voted and ours won by landslide. I think we had one person who voted against it. The dude from the Daily Mail.

Somer Simpson: What we created was this industry-wide standard, and which allows all the system to talk to each other in the same language. It’s open source, which is something that we absolutely demanded because we didn’t want to get into arguments over who owned what IP. It was publisher-centric and it was consumer-centric.

Somer Simpson: The outcome: today, a little over two months since the 25th, we have a new industry. Consent management platforms. This is this whole new thing. We’ve got a 113 that have launched on the on the IUD framework, which is what we ended up calling this thing. We’ve got 400 registered vendors and growing every single day. You are a part of this framework and talking to each other and sharing consent. 19% of the top 10,000 US and UK sites now have an IAB compliant solution in place on their site. 45.3% of the tools, actually they have some sort of CMP in place, 45% of those are IAB compliant and Quantcast has 69% market share of all of those consent solutions in market. That’s it.

Dorothy Tse: Thank you so much, Somer. That was awesome. Somer and her team have such great impact on this organization in ways that I wasn’t aware. I didn’t realize what happened. And the biggest impact for me is that I was able to hire one of my most senior recent hires because of the leadership industry impact that Somer and her team did. So they … he was very aware of the work that we were doing and liked it so much that he joined the company. Thank you for that.

Dorothy Tse: Next, but not least, we have Disha Gosalia, speaking to us about her experiences navigating being a shy engineer. Thanks.

Disha Gosalia speaking at Quantcast Girl Geek Dinner.

Disha Gosalia: All right. Last talk and given the topic of my talk, I should start from a point of vulnerability or I can’t wait for this to get over so as you guys so we can all get back to our mingling.

Disha Gosalia: I run customer support and operations here at Quantcast. And why do I qualify and why am I here talking about this? Growing up in India, when you’re somebody who’s a straight A student, or academically focused, you’re kind of placed at a pedestal and you always make your parents proud and so it doesn’t matter if you’re a loner out there. I never realized that I was a shy, loner kid.

Disha Gosalia: Imagine my surprise when, after I completed my software engineering degree, Computer Science, and went for my first job as a software engineer, in my first half-yearly performance review, when your boss goes through all the great 10 things you did. But that one area of development that you always think about.

Disha Gosalia: He actually asked me, “So are you an introvert? I never see you walking around the desk of your colleagues or chatting up with them and you actually don’t even talk much in team meetings.” And I’m like, “Hmm. Am I supposed to talk much in team meetings? Well, I’m new. Should I not be listening more?” But that was honestly the first time I realized that my personality didn’t have a part to play in my career.

Disha Gosalia: Fast forward several years. Now, as I parent really sensitive kids who are often called shy and quiet, I grapple with this thought on a daily basis — like how do I raise confident young adults who can accept themselves as what they are but at the same time also has this growth mindset. And so today, I’m going to go through some of my learnings as I’ve navigated my career and hopefully as I share my story, you guys can pick up some tidbits here.

Disha Gosalia: One of my first experiences when I became a new manager, I attended a new manager leadership training. And the instructor actually asked me and actually the class to write down your word cloud. What it meant was what are qualities that you look in a leader that you want yourself emulate. And when I wrote that down, how this helped me is I kind of became sure sure of what I wanted to be, where I want to go. And I stopped actually feeling bad about traits that I saw in other people that I didn’t actually have. And so I think this helped me because the first step for me was to understand what I wanted to be and then everything else became easier. I just had to go get it.

Disha Gosalia: As Gandhi says, “You need to be the change you want to be, but then you need to understand what that change is.” Before I talk about personality inventory, I will share this story. There was a academic incident that was a big learning point for me.

Disha Gosalia: I was in a really big meeting with my colleagues, my boss was there, my boss’s boss was there, and we were discussing this solution, an implementation solution, a complex solution and the person presenting the solution kept going on and on and I didn’t necessarily agree with that idea, but being who I was, I decided not to really call her out in front of everybody and just decided to kind of go one on one later and talk to her about why I thought this was not a great idea. When I did that, she actually accused of being indecisive.

Disha Gosalia: She said, “Why did you agree with me in the first place?” And I was really taken aback. I’m like, “Really? Did I even agree with you?”

Disha Gosalia: It actually gave me a couple of sleepless nights. And at that point what I didn’t realize, which I realized a little later, was that it wasn’t that she was accusing me, it was that my lack of speaking up or lack of objection in the meeting was actually taken as agreement by her, and it was only because we have had different ways of processing information.

Disha Gosalia: Fast forward in the same manager training, they made me take this Myers-Briggs personality test or there’s the Myers-Briggs, the Enneagram kind of same type of personality test and my original thought with these was, these kind of pigeon hole you into specific categories and it’s like, “Do I have to choose between being a compassionate person, like Mother Theresa or being a leader like Martin Luther King. Why can’t I be both?” But being a good student that was, I went with the flow and what I understood was this wasn’t labeling me in a particular bracket, but it was really understanding how I communicated and how can I become a better communicator with my co-workers and team mates and kind of others in my circle of influence. But that’s what it is. That’s basically all this personality inventory is.

Disha Gosalia: Going back to that example, this person, the way she processed was she would talk and think while she’s talking while how I process was like think and then talk. Like I would have these long awkward pauses but she would keep going on and on, and what I realized in actually going through this process was I need to just find a pause and then ask clarifying questions and that’s kind of how to better communicate with her.

Disha Gosalia: Now to contrast that is the growth mindset. I read this really great statement that’s made a big impact on me about this contrast theory. The growth mindset actually tells you that, do you accept yourself the way you are or do you actually try to be more, more than what you are and constantly evolve and constantly grow?

Disha Gosalia: Bear with me for a minute. I want to actually give you guys an example that I read that, again, made a lot of sense to me. And this is about the metal industry and how do they rate the hardness of metals. They rate them from a scale of 1 to 10 from a hardness perspective. A diamond is a 10 and a tin is one. A copper is a three. Tin is the softest and copper is three. Now tin and copper are not found in the same vicinity at all. They’re like found in a completely different vicinity. Somebody decided to take tin and copper and combine them. You would actually think that that would be an average so its hardness would be a two, but no, combining tin and copper gives you bronze, which is a six.

Disha Gosalia: This is what happens, and it’s called the contrast theory. It creates this unique magical combination. And that’s how personality traits are. I mean you could be way over here as an introverted shy person or you could be way over here. Aggressive, type A sales guy. I work with sales guys a lot in this job so I can pull on that a little bit. But if you combine and while you are right here, try to get a little bit of this side, you can be unstoppable. You could be an engineer surrounded by a lot of shy engineers. Try to get more communicator, public speaking skills or even skills to make other people feel special and it will just be going places.

Disha Gosalia: This is … what Sheryl Sandberg says in her book Lean In, always sit at table. Don’t take side seats. And it was really important for somebody like me, who had a very soft voice when I was in meetings and if I wanted to say something, I would think and by the time, sometimes the time’s already gone to speak. But when you’re in that center stage, people can actually see your body language that you want to say and can actually give you a way in. I started showing up in some important meetings where I would know there are a lot of people before time so I could get the right seat.

Disha Gosalia: What this also did was when somebody disagreed with you, they actually had to look in your eyes and do that. I hate conflict. I don’t like that one bit, but when I think of some of the biggest innovative solutions, the breakthroughs I’ve been part of. They’ve usually been through a lot of intense intervention, conflict, and I’ve learned to put myself in those situations. Put your ideas out there, let it be beaten up and you will learn something through it.

Disha Gosalia: Beth Comstock was a leader I truly admire. She’s the ex Vice Chair and CMO at GE, where I was previously before Quantcast, said this: “Conflict is a primary engine of creativity and innovation.”

Disha Gosalia: And I’ve learned to accept that, however hard that is. Kind of let that in once in awhile. One other principle that I grew up with was, you do your karma and don’t worry about the results. Other way of actually putting that is you can actually outwork anybody else, you can out prepare anybody else and that’s kind of what I try to do. I try to be double prepared and triple prepared when I know it’s kind of my chance to do things that are uncomfortable.

Disha Gosalia: I use to get really flustered when I would be put in a position by someone or in a spot by someone where I have to give quick responses or make decisions quickly. And what I learned … It was actually a mentor of mine who helped me through this and coach me through this is, you know it’s okay to ask for more time.

Disha Gosalia: It’s okay to say that, “I’m going to need 24 hours. I need to sleep through this. I need to think through this,” and there’s no shame in doing that. Don’t let anybody else put you on the spot and make you give answers that you’re not ready to give.

Disha Gosalia: Let’s bring it all together. Find out what you want and just go for that. Always take a seat at the table, not the side seats. Always be prepared, but if you’re not, there’s no shame in asking for more time. Find a Yang to your Yin.

Disha Gosalia: This is something again, I’ve done when I hosted large events or large meetings, find somebody who is … who can compliment your quiet type of personality. Somebody who’s upbeat and funny and loud. It just makes things easier, and I don’t go to a social gathering where there are too many strangers without my husband who was a talker. There’s like no awkward silent moments. But he sometimes forgets when he’s talking that I’m even around. But that’s a different topic.

Disha Gosalia: Lastly, I think if you remember that all human beings are really at core, alike. And we all like to be respected and we want to perform in our jobs and want to be heard and and listened to. So I think when you remember that, I think everything else is just smaller. That’s it. Thank you.

Dorothy Tse: Thank you so much Disha for sharing your story. So right now it’s time for a Q&A from all of our speakers. Any topics you may want to ask any of us. So if you’re interested and have some questions, you can come over to my right side here and ask a question and all of our speakers will come up and answer any questions that you may have. Thank you.

Quantcast girl geeks answering audience questions at Quantcast Girl Geek Dinner.

Audience Member: This is kind of just general to everybody, but as a female in tech and in engineering and in product, what do you feel like is your biggest struggle and how do you succeed in this role when we are kind of overwhelmed with males in our community?

Somer Simpson: It’s funny when I first stood up here. I mean I’ve been in tech since 1994 and I’m looking out in the audience and I’m like, “You know what, finally I’m looking at what tech should look like.” But having been in tech that long, I learned a long time ago to just not differentiate. Not even admit or acknowledge that there’s a difference and just be myself and speak my mind and be a part of the conversation. Just don’t take no for an answer.

Disha Gosalia: Yeah. I mean I’ll just add in. I think Somer’s really right, what I’ll add is also you know, I’ve always had good women role models, who helped me like when I got first child. How to navigate that and just kind of go through things. So it’s important to obviously not see yourself as different from a man, but then we are different. So definitely try to find somebody that you can follow and who’s ready to like guide you through some life changes.

Esther Hsu: I will admit that it actually took me awhile to realize what a problem it was for women nowadays, and once I did, it was actually looking pretty discouraging. Like you notice all these differences from you to all the people around you and you kind of automatically see it as a detriment. And I think, for me, what made the biggest difference was just having mentors and people who I really look up to — men or women — who really point out all my strengths, and I’ve realized that all my strengths are the things that made me different. As cliche as that sound. It’s like when I started it’s like I hate hearing that too. But it’s like it’s so true. Everything that makes me different that people might see as feminine qualities are what make me a better engineer and a better communicator and a better leader.

Brittni Gustaf: So the thing that probably held me back to the beginning was like the imposter syndrome. I’m sure you guys have heard of it. And I really struggled with that at the beginning. I still struggle with it sometimes now, but at the beginning it was so bad.

Brittni Gustaf: It took me a really long time to realize that there are a lot of people who have very strong opinions and they voice them as fact — but it’s not.

Brittni Gustaf: You sit there and I always at the beginning, I was like, “I’m just completely wrong.” Like, “I don’t think this is right at all, but obviously this guy knows what he’s talking about. He’s so sure of himself.” It took me a really long time to realize that if I’m confident on something, you have to actually bring it up. And then a lot of the times, there are other people in the room who will also be like, “Yeah.”

Malvika Mathur: I feel like whatever I want to say has already been said. But I’ve caught myself in situations where I’m the only female in a team of people or in a meeting and I realized that nothing is bigger than logic. If you have solid points and if you know exactly what you’re saying, it doesn’t matter what gender you are or whatever else. It just like you have a good point and a good point always wins. That’s about it. Just note that. Sorry.

Dorothy Tse: I didn’t realize you were all going around. But one thing I will add is that as a female leader, I try to embrace it. I embrace the fact that we’re bringing a different perspective in a very male-dominated industry and that is an asset to a company. The different ideas that come from a woman’s brain and the types of perspectives that are brought sometimes are very unique and different. So I encourage myself and certainly others to think about just embracing that diversity.

Audience Member: Awesome. Thank you ladies.

Somer Simpson: Just one other thing to add. I think part of my sort of struggling in the journey was, I was fighting more to be queer in the work environment. So being a woman in the work environment kind of like took a side stage.

Audience Member: All right. Thank you ladies. You guys are all amazing.

Audience Member: Hi. I’m Sheryn and I’m a co-founder of a startup. We’re also only females. So it’s really great to see you guys up here. I think all of your stories complimented each other and it’s very nice. It’s a novel of stories that’s set in front of us. My question’s more specific in terms of the hackathon. I’m a UX designer, researcher, and when you talked about the hackathon, it seemed very developer focused. So I was wondering if that’s part of the culture here that the designers are also part of the hackathon or is it very engineer focused? Because you keep saying the users first and we’re the ones that were kind of super obsessed. We made our whole lives about the users. So how does that work here?

Brittni Gustaf: Yeah. I should have probably clarified that better. So it’s not just engineers, it’s definitely product managers and UX as well and all of the designers and we actually sometimes, we like get disappointed if you don’t have the designer on your team in a hackathon because having a designer is like a huge asset because things that look nice and work well for the user, tend to win really well even in your simple prototype.

Brittni Gustaf: So, yeah, they’re a huge portion of it and also a huge portion here at Quantcast at working with the product managers to make sure the design is what customers can understand. And we’ve learned that the hard way because we use to have tests where we would have people run through our stuff and that was just so painful to be like, “Just scroll down. What do you want? It’s just down a little bit.”

Brittni Gustaf: And like people are trying to get to certain paid and they’re clicking everything but the button they should be clicking. So yeah, that was the struggle we had and we’ve become a lot better at that by having both product and design create clickable prototypes and then have the user use it and then get feedback and then make improvements. Which has been really awesome and it’s really improved our products so far.

Audience Member: Thank you.

Somer Simpson: I was going to say that worked so well in the hackathons that we’ve actually reorged our groups to have dedicated teams to each product, that’s made up of a product manager, engineers and assigned UX person and a product marketing manager.

Audience Member: Hey there. The question I have is more specific to Somer’s story, but if it makes you think of stories that you want to share because of what I asked, go for it. My question was about your decision to say, “Wow! All these ideas suck. I’m going to come up with my own and present it and hope for the best.” What was going through your mind when you made that move? What other steps did you take to increase the chances that they’d go for it?

Somer Simpson: The options that were on the table, at the surface, all of them were great ideas. But once you scratch, pull off the surface, they all had problems. For instance, one idea was a centralized registry to store people’s permissions, but that would be one company building a massive database that might hold the trillions of records necessary to do it but all that data would be in the hands of one company. That was bad. And then we had one that was like this pure, what they called daisy bit, which was we just pass this information around.

Somer Simpson: What we ended up doing was we took kind of like the best of the solutions that fit everybody’s needs and were like not quite so controversial and created what we initially called a hybrid solution, but I mean it wasn’t completely my decision. I mean I walked back into the room with the team that’s like, “All these ideas suck.” But it was the team that actually really got together in understanding, in breaking down each solution, took the best out of each and came up with the right thing that ended up working well for everyone.

Somer Simpson: Brittni, you want to give your side of it?

Brittni Gustaf: I think that the other side of things is that it’s really important to bring in other people and get outside perspectives. That was one of the things that disappointed me most about the GDPR implementation is that IAB and all of these people who are meeting for so long, trying to come up with a solution to this. And all it took was pulling in more people with more ideas to be able to get the best one. But we spent so much time not getting there because we weren’t pulling in everybody needed and getting all the different diverse perspectives to be able to come up with the idea that was best.

Brittni Gustaf: I feel like we got a later start than we should have because if it had been … If the correct people had been pulled in sooner then we would have not have such a stressful time trying to get this done before the law was in place.

Audience Member: All right. Thank you so much.

Dorothy Tse: Ladies, thank you so much — and some gentlemen too. Thank you so much for attending the Girl Geek Dinner and we just want to emphasize also that at Quantcast, our greatest asset is our employees and there’s a bunch of folks around the room that are wearing Quantcast clothes as well as all of us up here and we would love to talk to you further about Quantcast. Thank you – we are hiring!

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Breaking All The Rules & Finding Your Own Way: Girl Geek X Guidewire Panel (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

Guidewire girl geeks: Priscilla Hung (Chief Operating Officer), Eileen Maier (Chief Business Officer), Lerk-Ling Chang (VP of Strategic Ventures), Sandia Ren (VP of Professional Services), Roopal Shah (VP of Go To Market Delivery) at the Girl Geek X Dinner held at Guidewire’s offices in Foster City, California.

Roopal Shah / VP, Marketing / Guidewire
Priscilla Hung / COO / Guidewire
Eileen Maier / CBO / Guidewire
Lerk-Ling Chang / VP of Strategic Ventures / Guidewire
Sandia Ren / VP, Professional Services / Guidewire

Transcript of Guidewire Girl Geek Dinner – Panel:

Roopal Shah: All right. Welcome everybody. We are so glad that you guys are here. My name is Roopal Shah. I’m the Vice President for Go To Market Delivery, which essentially is product marketing, sales enablement, business planning, and operations for our Go To Market functions. I’ve been with Guidewire for eight years. That hopefully tells you a little bit about what an awesome place this is to work at. Without further ado I would love to have each one of these ladies introduce themselves, tell us a little bit about what your role is, who you are, and how long you’ve been at Guidewire.

Priscilla Hung: Welcome, everybody, to Guidewire. My name is Priscilla Hung. I’m the Chief Operating Officer of the company. I have been with Guidewire for 13 years and two months. In Silicon Valley, that’s 1,300 years, as you know. I’ve been working in the Valley for about 30 years. This is, by far, the best company I’ve ever worked for. It shows because I’ve been here forever.

Priscilla Hung: My responsibility from a day-to-day basis, basically, is to make sure that the operation work, by design, but my direct responsibilities today include corporate strategy. That includes all the business development, market strategy, M&A, partners, product marketing, Roopal and Eileen’s team, and whatever that entails, including film marketing and definition of the market and product development. What else?

Eileen Maier: Film.

Priscilla Hung: Yeah. Yeah. I did alliances.

Eileen Maier: Influencing.

Priscilla Hung: Influence, like all the IT, IS, and security, cloud operations, customer success. I think that’s it. I really love the company. I love everybody here. I’m so delighted to see so many old faces here. I’m looking forward to have some conversation with you tonight.

Eileen Maier: Hi, everybody. My name is Eileen Maier. I am the chief business officer at Guidewire. I’ve been here 13 years. Today is my 13th anniversary! <clapping> I know! My role is chief business officer; I work for Priscilla. First things first, I do whatever she wants me to, but I’m privileged to lead the team that is actually the voice of the market. In understanding, you saw one of our customers talking about their market needs and what is it that they need to run their business. I have a team that is responsible for really understanding where those market needs are and to translate those into business opportunities for Guidewire. Where do we see a market opportunity by serving that customer’s needs?

Eileen Maier: Then, they work very closely with another member of Priscilla’s team, the product team, to realize that, to make it into something that we can bring to market. Then, I’m also privileged to have Roopal on my team because then it’s her team that does the Go To Market Delivery, so how do we translate that product or that solution into something that our sales people can sell so we can grow our business?

Eileen Maier: It’s been exciting and a wonderful journey to be with Guidewire. This is a relatively new position. I’ve been in it for about a year. I think that that’s something that I’d loved the chance to talk to you about, of transitions within your own company and how you can grow your career without having to change where you work. Okay. Over to you.

Lerk-Ling Chang: Hello, everyone. Can guys hear me? Is my mic working?

Roopal Shah: Yes.

Lerk-Ling Chang: Okay. Awesome. Okay. I’m Lerk-Ling Chang. I’m VP of Strategic Ventures here at Guidewire. My focus is on helping the company grow through acquisitions, through partnerships, and through venture investments. I’ve been with the company now for 16 years. That’s 14 years more than what I thought I would be here, so it does speak to the character of the company and what we love about the company. Encourage you guys to talk to the folks here who are wearing Guidewire t-shirts. Feel free to ask them why are you here and would love to share that with you.

Lerk-Ling Chang: I joined the company when the company was really small, 12 people, the company had just landed the first customer. Had just raised a first round of funding but didn’t have a product yet. They needed a product manager. I worked in one of the six co-founders at Ariba, a previous company where person that I worked at as well. He reached out to me and said, “Hey, would you like to join this startup of mine?” I was kind of in the middle of a transition. I ended up joining. Ken and I product managed the first product. Actually, I worked with Sandia on that one. A couple years later, led up the development of a second product, PolicyCenter. It’s been an exciting journey.

Lerk-Ling Chang: Now, I’m focused primarily on acquisitions, partnerships. Then, in the last few months, I’ve picked up the lead for our venture investments.

Sandia Ren: Hi, everyone. Is my mic on?

Roopal Shah: Mm-hmm (negative). No.

Sandia Ren: Hi. I’m Sandia Ren, and I’m a vice president on our professional services team. We’re the team that goes out to customer sites and helps them implement our product and use our products. We’re the ones who get to travel. I’ve been with Guidewire for fifteen and a half years, so a little less than Lerk-Ling. I actually started as a software engineer. Yes, she was my product manager. I wrote unit tests for our very first product. Now, I’m Vice President of Professional Services. I would love to tell you about that journey. I’ve been very grateful to Guidewire for the opportunities that I’ve had.

Sandia Ren: These days, I look after our specialized consulting teams. These are the teams that work on our products outside of our core systems. This includes our data and analytics products, our digital products, underwriting management, and then competencies like upgrades and testing and infrastructure, all the stuff that people don’t like to think about so that’s in there, the specialized umbrella.

Sandia Ren speaking at Guidewire Girl Geek Dinner.

Sandia Ren: Then, I also look after what we call the Guidewire Services Center. These are our delivery centers around the world that have lots of teams of consultants who help our customers implement and deliver our products remotely. We have four offices now. Our largest is in Dublin, Ireland. Then, we have a team in Pennsylvania. It’s our US team. Then, we also have a team in Madrid, Spain. Then, we just opened an office in Malaysia.

Sandia Ren: For full disclosure, I just got off the plane from Malaysia. If it sounds like I can’t get my words out, that’s my excuse for tonight but I do certainly feel very privileged that it’s my job to travel around and meet just exceptional people around the world who all share our values and are committed to the same goals and our customers. Thank you so much for coming. It’s really exciting to be here.

Roopal Shah: Okay. Awesome. Thank you for that.

Roopal Shah:Let’s start with our first question and this is to all of you. I would love to get your perspective on insurance, specifically there’s a lot of talk about insure tech and what a pivotal time this is right now. What are your thoughts on that? Do you think it’s a pivotal time and what sort of skill sets do you think insurers and insurer tech companies are looking for right now? Anybody want to take that?

Eileen Maier: I can’t speak unless you give me your mic.

Roopal Shah: There you go.

Eileen Maier: I now have control. To rephrase the question, is this an exciting time in insurance? I would say, “Absolutely.” We serve the insurance industry. We are an insurer tech. Within the walls of Guidewire, I’m just so excited because every day, I get to talk about innovation.

Eileen Maier: When I started 13 years ago, today, anniversary, innovation really meant can I improve my business processes? Can I innovate on how I do my business and run my core systems better? That’s still true today. You can see the benefits from that video that we looked at not too long ago but innovation is transforming the industry. When you look around a disruption, it is absolutely impacting the insurance industry. It’s disrupting the way that they sell their products because people are expecting a mobile experience. They really don’t want to buy insurance through a broker. They actually don’t really even understand it. People are looking for insurance actually more associated with the service.

Eileen Maier: I’m sure everybody here is an Uber user. Not too long ago, I got an email in my inbox from an Uber saying, “Hey, with your app, you can rent a car now.” Actually I’m going to rent the car. I’m not even going to think about insurance because insurance is bundled in with that service. We’re thinking every day about what’s disrupting and transforming the industry because we don’t want to deliver products that they just need today. Certainly, we’re doing that. We want to think about what are the products that they need to deliver in the future?

Eileen Maier: I think that if, going back to the second part of your question, what are insurers looking for? They’re looking for creativity. They’re looking for innovation themselves. They’re looking for ways to do things differently because they really don’t want to just continue to do what they’re doing today.

Eileen Maier: Okay. Anybody else want to take a swing at that?

Lerk-Ling Chang: Yeah. I can chime in. I think people can hear me. Insurance is really changing really quickly. In the last five years, it’s probably changed … If you talk to anybody in the industry, they feel like the change has really come at a much faster pace than ever before.

Lerk-Ling Chang: It’s really driven by three things. Eileen talked about some of the things that are coming from consumers. You and I, we all want different things from our insurance companies than we ever did before but it’s also coming from the insurance companies themselves.

Lerk-Ling Chang: I remember 15 years ago, going onto the first customer visit and seeing people still using mainframe systems. It’s shocking but that’s actually a very common system that people still use. When insurance companies are still using that, they can’t actually deliver the kind of service they need to. It’s not because they don’t want to. They’re just stopped by doing that. In that sense, it’s really an exciting time for companies like Guidewire, who have solutions that can help insurance companies provide better service to their customers. That’s been a lot of also additional investment into the industry that then, at least … In the P&C, Property and Casualty technology space, in last year, for example, there’s been a billion dollars of venture funding that have gone into the space. That’s been a lot of startups. We’ve been the beneficiary of that, being one of the leaders here.

Lerk-Ling Chang: The industry has seen a lot of change and insurers themselves are looking for new solutions. It’s one of the best times to be in the industry right now.

Sandia Ren: I thought I’d address your question about what I personally think about insurance. I’ll be honest. When I started 15 years ago, I just followed good people to Guidewire. I didn’t really think too much about the domain or even the product that we were working on, but over the last 15 and a half years, I’ve definitely developed a true, very strong appreciation for what insurers try to do. It’s been amazing to see it evolve over the years but certainly we’ve seen a lot more when it comes to natural disasters.

Sandia Ren: I’m from Houston. We went through Harvey last year. That really hit home when I was hearing my neighbors and that was all that the talk was about in the neighborhood. It was claims and all these terms that I understood. When being in professional services, what I love about it is I get to go out and I get to meet with our customers and understand their business goals. Their business goals are about how to help people like you and me when we’re in our biggest time of need. That’s pretty awesome.

Sandia Ren: In the beginning, we used to say, “Well, insurance isn’t glamorous,” and whatever. We didn’t talk about it too much, but actually I think insurance is awesome. It’s really meaningful. I just wanted to share that perspective because I don’t think it’s something that you really realize until you think about it. At least, it was for me.

Eileen Maier: I actually wouldn’t mind building on that because it’s also the industry itself has changed. One of the more recent acquisitions, the most recent acquisition that Guidewire made was with a company called Cyence. It really is representative of how the insurance industry has shifted because 13 years ago, there wasn’t the need for something called cyber insurance or cyber risk but now there certainly is. Innovation and technology has driven, they actually created a new risk for us.

Eileen Maier: This is really incredible because what Cyence does is they have created a data engine that allows them to sweep up massive amounts of data so that they can use algorithms, detect where cyber risk is. This is completely different type of insurance than the property and casualty insurance that you use to insure your car.

Eileen Maier: I think that there’s this dynamic change within the industry itself because what is an insurable risk or what is risk itself is changing, which means the needs of consumers or businesses is changing and the market has to keep pace with that. The times around us ourselves is actually making it a dynamic time for the industry.

Priscilla Hung: Yeah so maybe I just really briefly round out. I think the ladies will touch on a lot of points already, pretty complete but just a few words. I echo Sandia’s sentiment. When I joined Guidewire 2005 and I joined 100% because of the founders. I’ve known four of them from my previous job. I had no idea what insurance industry is. In fact, when Marcus, our CEO co-founder call me and say, “Come join us,” I was like, “Why do I want to work for a company that serve the insurance industry?” All I thought about is I have to write money to them and when I have a crash, that it takes them a long time to pay me.

Priscilla Hung: But it’s a little bit of a learning curve for me but very, very quickly I have completely falling in love with this industry because you got a preview of some of the videos that you’ve seen but genuinely, our customer, the insurance carriers, they are generally full of people that spend a life and their career in making people whole. They are learning every day and trying to respond to the market and is an inflection point because it’s only very recently that all of us spend all our waking moments looking at devices.

Priscilla Hung: This morning on Today … I don’t know whether you guys watch morning television. I live by them. It’s one of the morning news. It says that an average person I believe is an adult. It didn’t say age group. It says that it spent on the average of 11 hours on electronic devices and between the age of 50 and 65, 13 hours. I don’t know how many hours people sleep but imagine that you spend so much time.

Priscilla Hung speaking at Guidewire Girl Geek Dinner.

Priscilla Hung: People don’t talk to each other anymore. My husband and I were texting each other the other day. I was like, “We have to stop this.” What the insurance company is trying to do is they want to understand this pattern. They want to understand how you live now because in the past, older people like me buy insurance, they call an agent and they’re all, “You fill in the form,” but nobody talk to people anymore. Most of you don’t have a landline. It is to response to the market, respond to this really rapid change because of what technology has come to play. I’m really proud to be part of this team and provide enabling technology for these insurance carriers to respond to your needs.

Roopal Shah: Awesome. Is my mic on? Can you guys hear me? Oh! There we go. All right. Okay. Let’s switch topics a little bit and talk about careers. What I would love for each of you to tell us a little bit about your career progression, how you got to here, if you made any calculated moves, if you had sponsors, mentors, just tell us a little bit about your journey to where you are?

Priscilla Hung: Me first? Okay, sure. My career journey. You might expect me to say, “Oh, you know, you must plan, you degree that you study and you plan your steps.” Absolutely negative. In fact, I go completely opposite side. For those of you who were raised by Asian parents, I’m sure that you appreciate what I’m talking about. My parents wanted me to be a doctor or a lawyer or a doctor or a lawyer, but especially I was pretty good at my grades when I was young and whatnot but I’m terrified of blood. My grandmother said, “Don’t be a lawyer because you will have to represent bad people.”

Priscilla Hung: Any case, so those path are gone. I said, “Meeny meeny meeny moe. So, what I do?” I ended up picking an engineering degree. It’s actually by elimination as opposed to plan. If you really asked me what I wanted to study, I want to be artist. I was a trained ballerina. I play pianos. I was a performer. I wanted to be artist. My parents are, “No, no, no, no, no.” You know how it goes.

Priscilla Hung: I got a degree that I really didn’t want to go for. Then, it’s like I go for a job now. I really stumble into … I finished my degree in the East Coast. I came over here because of a boyfriend. Two months later we ended. That was like, “Okay. All right. I need to look for a job.”

Priscilla Hung: It really, completely out of the blue, I got a call from a friend from high school who worked for Oracle. Oracle, at the time, in 1989, was a very tiny company. They were not in the Redwood Shores, up on Belmont Hills and said, “Come work for Oracle.” I was like, “Why to work for a database company?”

Priscilla Hung: I went through an interview that basically, in hindsight, if I were interviewing me when I was back in 1989, there is zero chance I will offer this girl a job because the interviewer asked me … I’m sorry. Am I running out of time? It’s a pretty long story. It basically is saying that, “So what would you like to do? You have an engineering degree. Would you like to be an engineer?” “Absolutely not.” I was like, “Okay. So, what do you want to do? Do want to be marketing?” “What is marketing do?” Show you how it goes. I thought, “Okay. You got good grades. You know, I’m going to hire you.” That’s when I started in 1989 at Oracle.

Priscilla Hung: In fact, I hop many, many jobs in the first 10 years because I actually didn’t really want to be in the discipline. I want to be an artist, remember? I hop around. I didn’t have a compass but I was fortunate in my career in the last 30 years that I ran into two people, both of them are my manager. Both of them saw that I’m someone that could be cultivated. I followed my first manager to three jobs. I finally, many jobs later, I landed at Ariba.

Priscilla Hung: I met someone that really wanted to develop me. He hired a professional mentor for me. The professional mentor was a retired woman executive. She completely changed my life. She completely utterly changed my life by basically putting a program together, told me five things I need to change, including my voice. “Don’t squeak like this. Don’t talk like a girl, so you have to talk with a certain voice.” This is my work voice. The second thing is, you won’t believe this, is that you smile too much. Stop smiling. I was like, “Stop smiling.” It actually took me a long time to really understand why do I need to stop smiling?

Priscilla Hung: The third thing is when you’re in meetings, talk at least once, speak at least once. Don’t go to a meeting and be in complete silence because if you are complete silence, you become irrelevant. Even if you’re repeating what other people are saying, paraphrasing it. Just speak once, and how you dress, how you present yourself. Now, I forgot what the fifth thing is but it’s five things.

Eileen Maier: Write things down.

Priscilla Hung: Yeah. Sure. Yeah. Yeah. Those things really completely changed my life. I would have to say that my career really started to go somewhere after I practice what my mentor told me. Add, after Ariba, four of the founders actually came from Ariba. Is again because of people I join here. The opportunity that the founders created for me. I got an opportunity to talk, do a lot of things that I have absolutely no experience, zero experience. It’s not planned for. It’s not because I’m experienced. It’s all because you have a new problem to solve. I work very, very hard. I focus on what I need to do, drive outcomes and then, one good outcome lead to another good outcome.

Priscilla Hung: Also, in hindsight, because all the randomness, all the different flopping around. My career actually helped my current job today because right now, I have a pretty wide scope of responsibility. Many of those jobs I actually have done in the past, not because I planned it. I would just say that I didn’t go through the traditional way. I didn’t plan but it worked out. I think it’s because of people that honestly is people relationship I built and people that help me along the way.

Eileen Maier: Okay. Are you sure you’re done? I’m going to start my story differently but then I think you’re going to find some similarities between my story and Priscilla’s story. Wasn’t an artist but also wasn’t an engineer. I got an English degree. I remember telling my father that I wanted to move from accounting to English. That was a pretty difficult conversation, at least for me. He took it pretty well, but it was, when you get out of college and you have a Bachelor of Arts in English, you do struggle a little bit to say, “Where am I going to get a job?”

Eileen Maier speaking at Guidewire Girl Geek Dinner.

Eileen Maier: This is where I think this theme of two pieces of advice. Be curious and also be open to luck. I think my parents felt a little bit sorry for me. I’m trolling around, living upstairs. They’re like, “You need a job.” Again, be open to luck. Take advantage of favors.

Eileen Maier: I got an interview with an insurance company. Really, if you’re an English major in Liberal Arts, actually insurance is a place that will consider. That’s how I entered into the insurance industry. Then, the curiosity kicked in because I was curious about, “Well, what it like to be an underwriter?” I had no idea. Learned that trade. Also saw what other opportunities were available for me within that insurance company. It was Liberty Mutual Insurance.

Eileen Maier: I was living in Philadelphia. I got the opportunity to move up to Boston. I took a role in a training organization. Again, within the same organization exploring different opportunities, different interests. I really started to learn something about myself by trying different things, by being curious. Through that, I got an opportunity to go into risk management. I joined a consulting group. That required me to get an MBA. It’s like these things start to build on each other and you follow a path. There’s something that you do when you get your MBA and you’ve worked for an insurance company for 10 years. You quit and you get another job.

Eileen Maier: That’s when I left and I moved to PeopleSoft. This is where I entered into technology but, again, being open to luck, why PeopleSoft? How could they possibly even consider me? It’s because somebody I went to grad school with. I called her up because I just wanted to get some advice from her. I wanted to understand how she made a career transition.

Eileen Maier: By reaching out and having a learning conversation, she actually wagged her finger at me and said, “I know exactly what you need to do.” We all want somebody to say, “I know exactly what you need to do. Give me the answer,” but she was right. She turned me on to this profession I’d never even heard of. It was called sales consulting or sales engineering. It’s where you have this opportunity to help customers, you’re consulting with customers, you’re understanding what is the business problem that they’re trying to solve. Then, you say, “Oh, well. I have just the thing for you.” Then, you go into a product demonstration and you show them how this product that you have can change their life and transform their world. It’s a little bit of performance. It’s a little bit of teaching. It’s a whole lot of consulting. It requires a lot of courage and empathy. I loved that.

Eileen Maier: I worked at PeopleSoft for quite a number of years. We got acquired by Oracle. I went on that journey too but it was within a couple months that I moved out to the West Coast, another similarity, poor guy. Within two months of being out here, my phone rings. It’s somebody again that I used to work with. He had just joined Guidewire. He said, “Eileen, they’re starting this team. It’s really cool. I’ve met the founders. They’re doing something very different. It feels like PeopleSoft,” because that was a bit of a culture, a cult. He said, “But there is one thing you need to know. Mmm, they serve the property and casualty insurance industry.” He’s pitching that to me like you’re going to have to deal with that. I was like, “David, there’s something you don’t know about me.”

Eileen Maier: With Guidewire, it was bringing together 10 years at Liberty Mutual Insurance, a number of years. I won’t give it to you because then you can figure out my age, at PeopleSoft in technology. Then, I joined here in 2005 and worked with Lerk-Ling because it was PolicyCenter, the second product that I was able to go out and start to build the sales consulting organization.

Eileen Maier: Then, with so many of the opportunities, the same thing that Priscilla has spoken about and you’ll hear Lerk-Ling speak about it and Sandia as well is that just in a growing organization, there’s so many things that need to be done. You start building a sales consulting team. Then, you start building a global sales consulting team. Then, you realize we really need a demo team infrastructure so we need to build that. Then, we really need to enable our sellers better as we’re starting to scale the organization so you start to build that.

Eileen Maier: With a mission and a vision, you start to collect really good people around you. I’m so privileged. I’m humbled by the people that I work with. It really is this journey of curiosity but also being open to luck and go into learning conversations, curious to find out what you might hear but also be transparent with what you’re looking for because you might be really surprised. Somebody might be able to wag their finger at you and say, “I know exactly what you need to do.” Anyway, that’s my story.

Lerk-Ling Chang: We haven’t planned this, but my career, it merged there, was not planned. I didn’t set out to work in a software company. In fact, I didn’t even know that was there was such a thing called software product management. Graduating from college, I had an economics degree. Coming out of college, the two positions that people recruited for was investment banking and management consulting. I picked the one that I thought was most interesting, which was investment banking. Did that for three years. Went to business school. Didn’t hear about product management either. Didn’t want to do investment banking, so decided to do finance. Eventually decided that was not my track. When I moved out to California, I talked a business school friend.

Eileen Maier: Was it for a guy?

Lerk-Ling Chang: It was not for a guy. I was already married, so not for a guy but it was through a business school friend who was working for the startup in Mountain View. I had no idea what a software company was, no idea what even development looked like in a professional enterprise. He just said, “Hey! Why don’t you join this software startup? Got this cool health care tech, monitoring health system,” called Health Buddy. It’s like this cool, little system. “They needed someone to help them with product partnerships. Why don’t you come join? You have investment banking. They’re going to need help us figure out how to put deals together.” I was like, “Okay. Great. Let’s go figure it out.” Anyway, I joined. Pretty soon after, I realized the product was not quite ready for any partnerships of any kind. It needed a lot more work in order to support and not any partner of any sort.

Lerk-Ling Chang: Then, I started talking with the product team and said, “Hey, you know, it needs this, this, and this.” There was only one product manager, who was completely overwhelmed with work. He wasn’t going to have any time to do it. I just ended up deciding to write up all the requirements that I thought would be needed for the product. I didn’t know this was product management. I just started writing out learning requirements, step-by-step flow of what I thought someone would need. Then, just started socializing it with the head of the product team. This guy said, “Hey! Do you want a job? Do you want to be our second product manager?” That’s how I fell into product management. I did that for a bit.

Lerk-Ling Chang: Then, got a job at Ariba after that, where I ended up working with Ken Branson, who is one of the six co-founders here. That’s how I end up here at Guidewire. I did product management at Guidewire for, I guess, it’s probably about 12 years. Then, decided to switch out of that role into something completely different, focusing on corporate strategy.

Lerk-Ling Chang: What that means initially was two things, strategic partnerships and then second is acquisitions. It’s been fun doing that because I worked on acquisitions as an investment banker before, but, at that time, you run numbers. You say, “Hey! You can cut cost here. You can add here. You can the increased revenues by 10% 20%,” but you don’t really know what it looks like.

Lerk-Ling Chang: Now, I’m on the other side of the table where we have to go through systematically to understand, “Hey, can we really grow revenues, work with all the different teams around a company to understand how to plan an integration and make sure the acquisition actually comes to fruition?”

Lerk-Ling Chang: I’ve been involved in all of the five acquisitions that we’ve done. It’s been a really interesting experience seeing that. Now, I’ve had the opportunity as part of this to now lead up our venture investments, which are going to be starting out and doing a lot more of.

Lerk-Ling Chang: It mirrors the careers of these two ladies here. It’s not planned at all, taking the opportunity, taking the initiative when you see something that’s a problem that you think you can help fix, taking the initiative to suggest solutions, and then working with people to see if that can actually come to fruition. That has helped quite a bit.

Lerk-Ling Chang: But the other thing, too, is finding people along the way that have helped me. For example, Ken was instrumental in bringing me here but even in my first job in investment banking, I had a senior managing director who I was able to work really closely with. She let me run a bunch of her deals, which is pretty unusual coming out of college.

Lerk-Ling Chang: Then, now I get to work with Priscilla and Eileen. You get different opportunities and you find people who can help support you and give you new things to do.

Sandia Ren: Okay. Going to be the same themes. Tells you something about Guidewire, but I remember looking recently at my high school yearbook. It has the question like, “Where you going to be in 10 or 20 years,” or something like that. I said I was going to be an electrical engineer and I was going to have two kids. That’s totally wrong. I am past 20 years so you can figure out how old I am but I have a three year old, and I’m not an electrical engineer.

Sandia Ren: Certainly my life didn’t quite turn out the way that I thought it would be, but I did start out on that path. I did get a computer science degree out on the East Coast. Then, I came to California. I actually followed the gold rush because that was during the dot-com boom, but I caught the tail end.

Sandia Ren: When I joined the company that I joined, I think the stock price was at 200. When I got the offer, it was 200. When I actually started, it was like 20. I caught the end so I had to work.

Sandia Ren: Anyway, actually and it’s all about people connections, too. The reason that I actually ended up at that company was because of somebody named Charlie Lee, which some of you guys know from the industry. He invented Litecoin and all of that stuff. But Charlie and I were in school together. He said, “Hey, I interviewed for this great company. You should really come and meet with them.”

Sandia Ren: I met with them and my hiring manager turned out to be the future CTO of Guidewire. I worked with him and had fantastic mentors there who taught me how to be an engineer. I think in school, you learn how to code but that’s pretty much it. I had a mentor who taught me how to work with the requirements, not to just build whatever was given to you but take a step back, understand why and make sure that you feel like this is the right way forward. I don’t think that’s something you learn in school. Just the start of learning from many wonderful mentors along the way.

Sandia Ren: Anyways, really, it’s when one door closes, a window opens. As was happening often there, the development at the company that I was at because they had gone from 200 to 20. The team was getting smaller and smaller. They eventually decided to outsource everybody to India. I was told that I had six months left.

Sandia Ren: I kid you not. That very day I had lunch with former co-workers who had left. They said, “Hey, Guidewire is hiring. Are you interested in coming to meet with us?” Sure. It was my old team, my old hiring manager and all of my great mentors who were over there. That’s how I ended up at Guidewire. Like I said, I wrote unit tests for our very first version of our product.

Sandia Ren: Then, I was a software developer for a while. I thought, “Yeah. This is the path I’m going to do,” but that lasted probably just for three or four years. That was when I started thinking, “Do I want to do something different?” I got enough…it was starting to feel a little routine, so thought maybe a different industry, maybe just needed a change. I actually started looking outside of Guidewire.

Sandia Ren: Then, I remember what pretty much changed my life, it was an email. Again, full disclosure. I am a huge Red Sox fan, being from Boston, so huge Red Sox fan. The year was 2004 and our Liberty Mutual had just gone live with our first product. Our head of professional services said, “They’re so excited. They’re as excited about the go live as the Red Sox winning the World Series for the first time in 89 years,” which is impossible but that’s what he said.

Sandia Ren: A light bulb went off in my head that, “Hey., maybe I can stay at Guidewire but do something different here. Maybe I can move into the services team and it would also give me the opportunity to move back to Boston,” which is what I had been hoping to do.

Sandia Ren: At that time, we were still pretty small. It was 2005. I went to Ken, and I said, “I have this really crazy idea.” I told him about it. Actually, I was working on PolicyCenter then. He said, “You know, it’s not a bad idea. You built PolicyCenter, so you can go implement PolicyCenter when we sell it.”

Sandia Ren: I learned really quickly how hard that was. Lots of lessons learned there, but he was really open to it. Again, just I think that is just a leadership style that I want to emulate. It’s being open to people’s ideas. I really appreciate it he didn’t say, “You know, that’s crazy.”

Sandia Ren: That really kicked things off. I found that consulting professional services worked really well for me because I really like to understand the business side. But I still got to use my technical skills in helping our customers come up with solutions that would work for them.

Sandia Ren: Then, from there on just within Guidewire, I was open to opportunities. I had managers who had crazy ideas, too, and for whatever reason would let me help them implement it. I think there are many times throughout my career where I was given a role that we could have easily hired for externally and would have found somebody who had been doing it for 10, 15 years to do a really good job at it, but no. They let me have a shot at it. I don’t know why but I’m really appreciative of that. I think it’s a big part of our culture as you’ve probably heard throughout, what the other ladies have said.

Sandia Ren: Now, as a leader, as a manager, that’s what I try to do as well. I’m so grateful for all the different roles that I’ve had because it’s given me a really good perspective of the business. That’s the type of career path that I want to give to my team as well. My team really focuses on growing people, development. I highly encourage transfers. We transfer a lot of people between consulting and product development, even over to sales consulting and education all over, but I think it’s a win-win for both the company and for our team member. That’s my story.

Roopal Shah: Okay. All right. Hopefully, that gives you some insight into how these ladies got to where they are. I’m going to do a quick hit, just because I want to make sure we have time for these guys to ask questions.

Roopal Shah: We’ll start with you, Eileen. You’re known for your presentations. Do you have any tips, tricks, anything to share?

Eileen Maier: Sure. No pressure. I’m going to tag off with something that Sandia said because she said, “Start with why.” If you’re doing a presentation, you’re thinking about a presentation. First, have an answer to that question: why am I giving this presentation? One of the questions that I’d like to challenge people with is what is going to be different in the universe after somebody sits through your presentation? So why?

Eileen Maier: In articulating that why, you really want to define for yourself where is the audience starting from and then where do I want them to end up? Then, I can get into a lot of techniques of how you tell that story but I think the most important thing is to also remember that it’s not about you presenting it. I’ve got this idea. I want this idea to get across. I know why I’m doing it but I’ve got this audience that I need to pitch it to. You’ve got to spend a lot of time thinking about who is my audience? Why are they coming here? Why are they listening to me? What do they want to get out of the presentation? What do they want to get out of this meeting? How are they hoping that their world is going to be different after they listen to me?

Eileen Maier: I think that that’s where a lot of people fall down because people get so in their head about presentations. They’re in their head because they’re thinking about themselves. They’re like, “I’m going to embarrass myself,” or, “I’m going to put myself out there and people are going to be judging me.” Actually, they want you to succeed. They’re going to be judging you if you get up and you actually waste their time.

Eileen Maier: Spend a lot of time thinking about why you want to give the presentation and then spend a lot more time thinking about your audience because you want to craft your message at a place where they can meet you because good communication isn’t just about speaking good words. It’s about speaking words in a way that they’re going to be heard. That’s really, to me, the key of a great presentation.

Roopal Shah: All right. Thank you. Priscilla, this one’s for you. You’re on the board. I know, for a lot of people, that’s a goal, for whether it’s to get to the highest rung on the ladder or to actually make a big difference. Would love to just get your perspective on what it’s like and just share with us.

Priscilla Hung: Yeah. Interesting. I think that when people say, “I would like to sit on boards as a goal,” I find it very interesting because I will always ask the question, “What do you think board members do that you want to sit on boards?”

Priscilla Hung: First of all, let me show you what board members actually do. Board members have fiduciary duty for shareholders and also as an advisor to the management team but primarily as a CEO and a CFO and key executive of the company. They show up in board meetings four times a year typically and a little bit more, if you are participated in some varying committees, like compensation committees or audit committees, so on and so forth. Of course, you get compensated for that.

Priscilla Hung: When people say they want to sit on boards, you have to understand just like why do you want to be in a certain job? You want to know the job description. It is work and it is work that it will be fulfilling to you, if you want to be in that role, like you want to be an advisor and you truly, genuinely want to help the management team.

Priscilla Hung: Of course, board members are a prestigious job but the thing is, if the goal of being on a board is…it’s a more of a fame or ego-driven goal, then you would not be a good board member and people can smell it because you got to be interviewed to get on a board as well and management team will seek out attributes from you or your vibes, then, “Why do you want to sit on their board,” and how do you help them?

Priscilla Hung: I would say that people typically seek you out as board members as a reflection of all the hard work you put into your career. It’s typically is the experience that you built, the reputation you built that people will come to you and say, “Hey, you have worked in all these company. You have these experiences. We have a company that, at this stage, would like, it would be lovely if you can share your experience and guide the principals along.” That’s you how you sit on a board.

Priscilla Hung: It’s very difficult to set that as a goal because it’s not something you apply for but again is if you focus on your work, you drive, focus, excellent work product and I would say that sitting on boards is a reflection of all the hard work you put into your career.

Roopal Shah: Thank you. It’s working. Thank you for that. Okay. Lerk-Ling, so you’ve been here for 16 years. Can you share with us or what it’s like to work for a mid-sized company versus a startup and any perspectives?

Lerk-Ling Chang: Sure. When I joined Guidewire, there were 12 people. It’s actually the second startup I’ve been at. I was at a previous startup that was not quite that successful. In fact, similar story. It was dot-com bust years, that raised $20 million and then effectively let 75% of the staff go within a year of raising that money. It was those crazy times. I’ve seen the whole gamut of startup all the way through the mid-sized companies.

Lerk-Ling Chang: Some of the things to think about if you’re thinking of do you want to work on a startup? Should you work at a startup? A few things, a couple things. As a startup, you definitely get a breadth of experience across many, many things. In the very early days at Guidewire, I was not only the product manager, I was also the UI designer. I was also the product marketing person, also did the sales demos, also did desk manager sometimes, was also the scrum master sometimes in the QA. The list goes on. Y

Lerk-Ling Chang: ou just do whatever you need to because they’re just not that many people and there’s stuff to get done. The customer is waiting for you. You just do whatever you need to get done. Great experience. A lot of breadth and something you get a lot more responsibility but it’s also very exhausting. You can imagine. Which one do you prioritize? It’s fun whereas as the company gets bigger, your job has to get more focused. It frees you up to actually focus on the things that are most important.

Lerk-Ling Chang: For example, in product management, personally I was very glad not to have to do demos anymore when Eileen joined, for example. I didn’t have to fly to go. It’s too hard to focus if you have so many things to do.

Lerk-Ling Chang: As the company grew larger and you have people who are much, much better at doing demos than I was ever at, I could then focus on being a good product manager. I focus on really understanding what makes the design, what things we should put into the product and how best to do that. That’s one example.

Lerk-Ling Chang: The other example is about, the other thing to think about … Sorry, but this is mid-sized. I talked about the company going downhill, the other startup that I joined. That was, I guess about 20 years ago, but very few people know that actually Guidewire also went through tough times. We actually had a layoff. I can’t remember when that was. You remember that? I don’t remember that. It’s long gone, but very few people know that. Actually, Sandia probably remember that. It was not great. We had to pull the product out from the market, not great.

Lerk-Ling Chang: As a startup, you’re still trying to figure out what the fine line is. You may have gotten one success but to actually get beyond that first few customers, that jump to the next level customers is actually not that easy. Being able to do that successfully is actually a lot of work. It’s a lot of …

Lerk-Ling Chang: I did a lot of things that as a typical product manager in a larger company wouldn’t get to do but it was also incredibly, very stressful. Just those things to think about.

Roopal Shah: Okay. Then, Sandia, I’ve got a question for you. I know one thing that you’re doing is leading this initiative called GROW. Can you tell us a little bit about that and what inspired you to start that?

Sandia Ren: Yeah, sure. GROW stands for a Guidewire Recognizes Our Women. It is an initiative that we have within the professional services consulting team that is focusing on our female colleagues and how to provide better support for them. I think we know the reason for things like this is because we don’t have that many females in the tech industry. Then, you add on IT consulting and then that number really dwindles even more. It’s something that’s been in the back of my mind. Then, certainly, of course, their current events have raised it to the forefront as well, but I have to say it wasn’t really me. Actually, there are a lot of people on the services team, a lot of women on the on the consulting team, that I think we’re sharing the same sentiment.

Sandia Ren: We came together and our focus, again, is I think most importantly to provide a forum so that we can we can understand what challenges women face in consulting and in IT consulting. Then, hopefully figure out things that we can do to make it better. As part of my own research and development, I have gone to a women’s conference earlier this year. I found that just being within that forum and being able to talk to other women like me was a huge help.

Sandia Ren: To be able to hear that one of the most accomplished people in science felt the way that I do, which is a hot mess a lot of the times, was comforting to talk to other women. I see impostor syndrome right in front of me, like it’s yelling at me, but to talk to other women who also feel that way, that, “I’ve no idea of what I’m doing half the time but I’m going to figure it out.” It was really comforting to me.

Sandia Ren: That’s really what we want to do in the beginning is to at least is to provide that forum because I think events like this, it really helps me to be able to talk to other people who are in similar situations and realize that I’m not alone.

Sandia Ren: Those are the things that we want to do with this is to look at women in our consulting team. It’s hard with the travel, especially those who have families and who have kids that we want to be home with and see what we can do to make things better. Then, second, just as important, is to raise awareness.

Sandia Ren: Guidewire has a great culture. We have collegiality as one of our strongest traits but even so, I think there is awareness that can be raised about the challenges that women face. When I’d spoken to my male colleagues about this, they’ve been very receptive. I’m excited to see what we can do with this.

Roopal Shah: Awesome. Final question. Then, I would love to get some questions. Tell us your most embarrassing moment, professionally.

Sandia Ren: I’m glad you qualified that.

Roopal Shah: Who wants to start?

Priscilla Hung: I don’t have any. I can’t think of … Okay. You go. You go.

Eileen Maier: Where do I begin? It’s funny that you ask me the question, Roopal, about presentations because, actually, my most embarrassing moment actually comes from a presentation, so don’t do this. I’m working for PeopleSoft. We’re doing a product launch and we’ve got, I don’t know, five, 600 people at the Hilton up in San Francisco. It’s a big stage and everybody’s in the audience. I’m giving a demo. I’m doing it with my boss. I’m up on stage. I’m up towards the back. If anybody here has ever seen a product demo. Actually in the audience, you really can’t see anything. You have to use big, sweeping motions and so, “Up here, you know, you see blah, blah, blah and if you look down here,” and with one sweep of the arm, I fell right off the back of the stage. I was just gone. I was just gone. I’m all mic’d up. It’s a tuck and roll. I’m under the stage.

Priscilla Hung: Oh, my god!

Eileen Maier: The one thing that did happen that was actually good about this is that it was right before lunch. Now, everybody is now up. They were like, “Uh!” They’re just waiting because … I’ve got the mic on. They just start to hear, “Heh, heh, heh.” I’m giggling under the stage. I’m sure I’m in total shock. I come out from underneath the stage. I pop my head up. I think the really embarrassing part of the story is that, at the time, I had really short, bleached blonde hair. I think that’s really the more embarrassing part but they see this blonde head out on the suit, climb back up on the stage. I don’t even use the stairs. I just climb back up.

Eileen Maier: You don’t know what to do. I’m like, “Well, I guess I’ll just pick up where I left off.” I try to get back into it. I could just tell I wasn’t settled down, the audience wasn’t settled down. If you’ve been on a big stage like that and you’ve just got like these lights in your eyes, you really can’t see anything. I look out into the dark where I know people are. I just look at them. I pause and I say, “I wish I could say that was my most embarrassing moment.” They’re like, “Oh, okay.” I finished out the demo. Then, I think I went back to my hotel room and I crashed for a good eight hours, but I think from that it’s like just keep going. It can try to keep it out of your head and you just get it done. That’s one of mine.

Roopal Shah: Anybody else willing to share?

Lerk-Ling Chang: Mine is somewhat similar. We have an annual user conference. This was probably about 10 years ago. I was doing an intro to new things in PolicyCenter. I’m talking about all the cool stuff that we have been doing the last year, all the accomplishments. Then, talking about one of the biggest news that we had was a new partnership that we had signed that I completely blanked on the name of the partner. Completely, no idea. Silence for two seconds. It felt more like two minutes, right?

Eileen Maier: Mm-hmm (affirmative).

Lerk-Ling Chang speaking at Guidewire Girl Geek Dinner.

Lerk-Ling Chang: Luckily, someone from Guidewire shouted out the name for me, and I just try to go on. Lesson there is, as they say, just go on. We all have that moment. I guess the main thing is you just have to keep practicing and hope that your muscle memory eventually takes over. Mine didn’t, so just keep practicing. Things happen. It’s okay. We all live to tell it.

Eileen Maier: That’s right. You’re still here.

Lerk-Ling Chang: Yes.

Roopal Shah: Anyone else?

Sandia Ren: Mine’s not really funny, so I don’t know if it’s a great way to end, but, so in consulting, you get to meet all sorts of customers with different personalities and different thoughts about the way things should be. Anyway, I was involved, working with a customer. We had gone to do what we call an inception project kickoff. We had done that and then we had finished.

Sandia Ren: Then, what we typically do is we go back periodically to check on how … We do what we call health checks, which is to see how the project is going and check in with the team and check in with the customer and such. I guess either this hadn’t been explained to the big boss in charge or he just didn’t want to spend the money because I traveled there. I showed up. I was sitting there at my desk working away. He comes up to me. He just pretty much hovers over me and says, “What the are you doing here?” I just froze. I didn’t know how to answer that. I stumbled a little bit but eventually was able to say, “I’m doing a health check and these were the benefits of it,” and such.

Sandia Ren: He really didn’t buy into it. That was my last health check at that customer, unfortunately, but I think it was very awkward moment. But, now actually my coworker, who was there with me, he tells everybody as a joke because he thought it was just the most hilarious thing ever.

Sandia Ren: We can laugh about these things years later but I think, at least for me, the lesson learned was that actually I think he was trying to intimidate me. I thankfully recovered and just have confidence in what you’re doing. I got through it.

Priscilla Hung: I really don’t have anything good thing. Maybe just like a little while ago, I forgot what was the fifth thing that my mentor said but now, in hindsight, the whole time of thinking what’s the fifth thing. Now, I recover because there’s actually no fifth thing. It’s the four things.

Roopal Shah speaking at Guidewire Girl Geek Dinner.

Roopal Shah: I would have loved to have things to make. You guys have any questions, concerns?

Eileen Maier: There’s one right here.

Roopal Shah: Mic?

Audience Member: Hi. I can hear you. Yeah. Hi. My name is Vera. I’m early in my professional life. I, too, work at Oracle, first job out of college. I’m currently working in online business sales and I’ve been passively or actively looking to move into another line of business. Sales just hasn’t been for me, but I’m struggling quite a bit. I get calls and emails from recruiters for sales positions but not product marketing or product management or customer success, or other things I’d consider. Do you have any advice from your experiences transferring lines of business?

Eileen Maier: Yes. I think that if you have sales experience and you’ve been out with customers and you have had difficult conversations because selling is not easy. Selling technology is actually very, very hard. You’re facing a very skeptical audience. I think that you have incredible understanding and empathy for what that sales process is like. I

Eileen Maier:  think that there’s a ton of jobs out there in what’s called sales enablement and actually going in and saying, “Okay, I’m going to go inside the company but I’m going to think about the processes of what it takes to actually scale messaging,” because you know what it is that the sales team needs in order to be effective. Big marketing decks that have value propositions that start talking about the company first.

Eileen Maier: That doesn’t actually help you sell. You can come in and you can help organizations understand how do I actually make my sales team more effective? I know that there’s a huge hunger within the industry to actually be able to fulfill that. People typically go into marketing jobs and stuff.

Eileen Maier: I think that, given your experience, that’s a really good transition. I think one of the people that you should talk to, after we get done, is this woman over here because she worked with me but it was really her hard work that instituted sales enablement here at Guidewire. I think that we could probably give you a little bit more insight of some other things you could be thinking about, but that would be something I would think that would be worth you exploring.

Roopal Shah: I think there was another question. Yeah?

Girl geek asking a question to the panel at Guidewire Girl Geek Dinner.

Audience Member: My question is similar. I’m actually in sales enablement and I have a pretty diverse background but I want to scale back into marketing and I’m finding that, because my background is so diverse, it’s hard for me to market myself to a particular skill set. How should I go about doing that?

Eileen Maier: Okay. I feel like Roopal should start to answer some of these questions, too. With sales enablement and you’re thinking about going into product marketing or more corporate marketing or …

Audience Member: Corporate.

Eileen Maier: More corporate marketing. Yeah, I actually think that … I’m going to now reverse it but say the same message, which is I think the most important thing is to have empathy for the audience that you’re trying to communicate to. I think when there’s the corporate marketing mandate and what we’re trying to do in terms of increasing brand awareness and do demand generation or just brand establishment, I think that you’re doing it on behalf of the sellers who are out there in the field.

Eileen Maier: I think using maybe that angle of understanding the audiences that you’re trying to deliver value to, I think is really important because I think often times corporate marketing can get a little bit hung up in the brand and the message and lose sight of the connectivity to the people they’re trying to communicate to.

Eileen Maier: Again, going back to what I said about presentations. There’s always two sides to communication. Think about what advantage you bring, what experience you bring in terms of understanding the dynamics of that conversation because I think that you do have something special, having been on this sales enablement side.

Audience Member: I think that interesting observation during the corporate…

Roopal Shah: It might be working.

Audience Member: It’s impressive staying with the company 10, 15, 16 years, which is rare in the Valley not just from a startup perspective but the fact that you stayed with the company. I stayed at Cisco for 14 years. That does not looked very nicely. There’s a negative connotation. “Oh, you stayed too long. You didn’t change,” but then when I look at my career, I started in engineering, went to business, then went into marketing. I’ve done sales enablement. I’ve been through that journey and I lived globally, which has enriched my experience even further given the opportunities I had.

Audience Member: I’m just curious. Two questions. One is it seems like Guidewire did the right thing, at least from a diversity perspective. Seems like they retain all of you for all these years and encouraged your growth. What was the culture like with the founders?

Audience Member: The second question is how do you address the perception issue, which is completely opposite of what we see right now? Any advice or generally think?

Priscilla Hung: Maybe I can take a stab at that.

Eileen Maier: Sure.

Priscilla Hung: There a couple of things. Your observation is spot-on. I would like to go back to your first question first before we address the culture. Of course, we have a very, very strong culture in the company that we really live by. It’s not just marketing slogan. We really believe in collegiality. We really believed in working amount equals. I think that one of the reason why we stay here so long is I think, I don’t want to speak for all of you, but we genuinely are working among friends. We generally believe in the mission of the company.

Priscilla Hung: Also the company is in the last many years, it’s been doing very well progressively. We’re all proud of being the founding building blocks of it. This company really allow a very basically open view for all of us who developed. We all come into different job and end up here in a complete different job and in high places.

Priscilla Hung: It is an environment, an openness to so focus on not what you look like or what gender it is but it’s purely on how hard you work, are you a good worker, and you got recognized. Also, in my particular situation is, I don’t have a lot of experience in doing a lot of things but it’s so progressively I added more and more to my plates because my boss believed in you a competent person, a lot of the problems are … And, in fact, I think I can generalize it. It’s unlike you’re like a rocket science or you are doing something very specific. I think that going to school, it’s a lot more difficult than working.

Priscilla Hung: In fact, in working, it’s every day is your general competence and general problem-solving can go a long ways, of course, with hard work and dedication. The other thing is really dealing with people. A lot has to do with people, people communication. You work with people. I think that Guidewire paved a very, very good foundation for all of us who thrive and not just us, as all our male colleagues as well.

Priscilla Hung: But go back to the negative connotation as, “Hey. If I apply for another job, you’ve been here for 14 years,” but I think it go both ways. If I’m interviewing for someone right now who want a job at Guidewire and that person has stayed 18 years in a stagnant company, has a poor reputation, bad culture, going nowhere and you look at the progression of the companies, within the company, that person goes nowhere. Then, that is a negative thing but if you are looking at a company that has a reputation like Guidewire or other places that is small and going very big, it’s a is a very attractive profile.

Priscilla Hung: I think you have to put things in context in terms of what you mean by you’ve been a dinosaur in a company for a long time. I think the brand of the company and what it represents makes a difference in terms of the perception.

Sandia Ren: Yeah, and I would add to that, that when we’re recruiting, if I see a lot of short stints, that actually can concern me, maybe just because we’re used to people sticking around so long but it’s certainly something that I will ask is because at Guidewire, we do, we invest a lot in people. We all like each other. We want people to stick around and find their career in growth opportunities within the company.

Audience Member: I realize I’m in the audience but I think … I was at Guidewire for five and a half years, left for three and a half, just recently came back. The reason that I came back was because of the people on this panel and Guidewire cultural integrity. It really is a family environment.

Roopal Shah: Here, here.

Audience Member: Okay. I have a question.

Priscilla Hung: Oh. You’re…

Roopal Shah: Lisa Walsh, our Vice President of Alliances. There you go.

Audience Member: And anyway. You guys all talked about the importance of relationships and people in your careers. What do you look for, someone that you choose to mentor? Sort of like start to pay back. What are the attributes of someone that you think could be a good mentor, that you would like to mentor?

Sandia Ren: I can take that. Actually, as I mentioned, I was just in K.L. We were looking for a manager for the office and we were interviewing, met a number of a number of candidates who had managing teams for 10, 15 years, built teams from scratch, et cetera but then we met this girl or, I should say, woman. This woman who was … She even called herself a new leader. She’s only been managing team for a two and a half years but, as I talked to her, I could just tell that she was really smart, really clued in.

Sandia Ren: She really quickly picked up on the things that we were talking about. She was really excited to learn. She says she wants to switch jobs because she wants to be exposed to more people, different people because she thinks that that’s how she’s going to learn. I could tell the ambition was there and just the openness to learning.

Sandia Ren: Now, I’m like, “How do I hire her,” even though she may not be the best fit for what we’re looking for but it’s that eagerness to learn, that passion, I think that goes such a long way.

Lerk-Ling Chang: Yeah.

Priscilla Hung: I mean, for me, it’s just to build on that a little bit. I think, for me, it’s the chemistry has to work because I really believe in mentorship because I owe my career to my mentor, but first of all is you can’t mentor 200 … I mean, it’s the time with issue. I think that when you click, you click. Also, the second thing is just like Sandia said is, you will get a sense that whether the person in front of you actually is open due to change. If there is closeness there, it’s a waste of time.

Eileen Maier: Thank you, because we’ve been talking about this, Lisa. I think one of the things that’s really important is people say, “Well, you know, I’d like to find a mentor,” and, “It’s so important to find a mentor,” but I think this is probably the most important thing you need to know is that you actually get chosen. You can ask somebody to be your mentor but you are going to get chosen. It’s really how do you represent yourself as somebody who’s open?

Eileen Maier: You’re whip smart. Show that you’re whip smart but recognize that that’s how the relationship is going to happen is that you’re going to get chosen by that person because I think that if we think about people that we’ve mentored, it’s because that connection happened.

Eileen Maier: Make yourself available to those people that you’d like to mentor you and see if you can establish that connection in somebody’s … I guess I’m also saying you have to earn it.

Roopal Shah: I think we have time for … Okay. We’ll take these two as the last two. Do you need a mic?

Audience Member: Thank you. I’m really happy to have heard stories from all of you because just today I was feeling a little bit … “I’ve been at my company for 13 years. Am I considered a dinosaur?” Now, I don’t feel so bad because, like you said, I’ve done four different roles at the company. I know I’ve grown with the company. Thank you.

Audience Member: As a new manager, we were doing more focused on execution and planning. As I grew into a director role, it’s more about strategy. What is it at the VP level and at the COO level?

Priscilla Hung: That’s a very loaded question. I would say that, as a COO … I was just joking. Who I was talking to? My memory’s going. I was talking with someone today that I’m actually not really doing much. I’m not saying I’m lazy, but on a day-to-day basis, so my job is I’m thinking all the time. My job is to think about are we heading in the right direction? If we’re not headed in the right direction, how do I direct or influence the team to go to that direction? How do I make sure that people actually work together?

Priscilla Hung: That’s my job but I’m not giving you code. I’m not writing a paper. Sometimes I’m not doing my PowerPoint. Higher you go, it’s more about overseeing people. I would just say that when you’re individual contributor, you are measured on and you probably have something tangible that you’re delivering but as you get higher and higher, that becomes probably not majority of your job.

Priscilla Hung: Most of the time is you’re really thinking you’re working with people, you’re managing people. You’re making sure that you drive productivity in your team. I would just say that you think more and you work less. You produce less from the perspective when you get into more senior position.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

“The Customer Is Not Always Right”: Cindy Alvarez with Lean Customer Development (Video + Transcript)

Cindy Alvarez / Director, UX / Microsoft


Cindy Alvarez: I’m Cindy Alvarez, and right now I’m on a flight heading back from Johannesburg, so you get a pre-recorded talk from me. I’m going to talk today about why the customer is not always right. First of all, I want to start off by saying where did this thing even come from? Where did we start with the phrase, “The customer’s always right?” What does that mean? Why is that important to us?

It primarily comes down to the fact that we want our customers to come back and buy more things from us. We want them to like us, and that makes me realize that customers are not just external people who buy our products, they’re also people that we work with every day. There’s a lot of things you can do to make those relationships better, whether they’re talking to customers who are external to you, or the people who are next to you every day. Let’s dive right in.

First of all, let’s think about what’s our desired outcome. I think when we say things like this, we tend to think of things like units sold, or increasing usage, but fundamentally, we want to understand the underlying problems that customers have — whether that’s the one buying our product, or our coworkers because understanding is the only way we can hope to solve their problems.

We really, let’s be honest, want our customers to like us, we want our coworkers to like us, and we want to find the best possible solution for the problems that we’re facing.

That best possible solution isn’t always obvious. What do we do with that?

Most of us have been through the scenario where a customer came to us, they asked for something, we built it, we delivered it, and then it didn’t actually solve their problem. We’ve wasted time on something, and we have to support something that didn’t really meet their needs.

When people ask for solutions, they’re asking for their assumption of what the solution to their problem is. A lot of times, they haven’t really thought about the problem they’re really trying to solve, and honestly that’s not their job.

It’s our job as product people to think about how we’re going to come up with solutions.

For example, let’s say we had a customer who came to us and said, “I need to rent a car.” Now, on the face of it the best possible customer service would be to rush right out, and get that person into a car rental agency, and put them in a car as quickly as possible.

If we’re bound by things like SLA metrics and we have to respond to customers within a certain number of hours, or we have to respond to every single user voice, or every single email query, then we tend to game-ify ourselves into these non-optimal solutions.

Let’s think about why that person asked for a car. It could be that they’re about to go shopping, they don’t own a car, they’re going to buy a lot of groceries, and they don’t want to take them home on the bus. It could be that they’re having a vacation, and they want to drive along PCH and take in the scenery. It could be that they have a lot of relatives in town, and they won’t all fit into their tiny Prius.

For each of those scenarios, there are different solutions that may make more sense.

In one case, it may make more sense for someone to grab a Lyft, in some cases maybe you need to actually rent a van, and in some cases maybe you can borrow a car from a friend. If you’re actually out on a vacation and you want to take that drive along the coast, then sure, you do need to rent a car… but in that situation, you probably want to rent a fun to drive car. You don’t want to rent a beater or an SUV.

If we just rush someone straight to the car rental agency, we’d end up with that subpar solution.

As customer-oriented product people, we have to take a step back and ask “why?”

This is something that we also need to do internally, and I find this happens even less often internally because we assume that we understand the why’s.

As soon as we’re working in an office with someone, in a team with someone, we assume a shared context that doesn’t usually exist. There are people that I have literally sat across from for weeks on end, and yet we’ll still have underlying assumptions that are different from each other.

The “why” is always more interesting, it’s more useful, it’s more actionable, and it’s more trust building than the “what.”

That last one’s a little bit interesting because people think that asking “why?” sounds almost a little accusatory. If you have a small child, you know it can actually get pretty annoying when people keep asking why.

We need to add a little padding around it, but fundamentally when people ask why, what they’re actually saying is, “I want to know more about this. I am interested in you, I care about what you have to say.” As opposed to what, which has this whiff of, “how quickly can I get you out of my hair?” That’s not how we want to build these relationships.

Let’s think about it: someone comes to you, they have a request. It could be the customer who says, “I need to rent a car,” or, “You need to build this feature,” “You need to support this use case.” It could be your boss who has given you a task, “You need to do this thing. You need to manually sort through this data. You need to write this spec.”

The first question we really have is: “What’s the problem you’re trying to solve?” That can be pretty tricky to ask because if your boss says, “Do this,” and you ask, “Well, what’s the problem you’re trying to solve?” your boss might just say, you know, “Get on it. Let’s do it.”

Again, we need some padding, and so we want to say, “Okay, just to make sure I understand it seems like the outcome you’re looking for is X, is that correct?”

If your boss says, “Manually sort this data,” what they probably want is clean data. If someone says, “Get me a drill, “ they probably want quarter inch holes, but you need to validate that.

The best way to do it is give that, “Just to be sure I’m clear, this is the outcome you’re looking for, is that correct? Am I missing anything?”

This gives people the opportunity to step in and say, “Actually, this is the thing I needed,” or, “Actually, here’s some more information that I assumed that you knew, but you probably didn’t.” Or, “I’m sure I’ve told you this a billion times.”

Having been someone who’s managed teams, I have always had things that I’m pretty sure I said a billion times, and yet either I didn’t, or people didn’t hear me. It doesn’t really matter because the outcome was the same, and they weren’t privy to that information, and that meant they weren’t going to do as good a job as if they had the information they needed.

We want to ask why. Why do you need that done? What’s the outcome you’re hoping for?

A lot of us, when we’re interacting with customers, what we hear is basically a demand for features, and it’s hard to ask why because what they really want to get to is when. When are you going to build this thing?

It’s useful to take that step back. I like to announce it as such, and say, “You know what? Let’s talk about delivery deadlines sometime in the future, in a few minutes, right? But just a second. I want to be sure I understand something. It sounds like you’re asking for this feature. Just to be sure I understand, if we had already built it, what would it allow you to do? Essentially, how would it make your life better if you had this thing?”

The thing that I found surprising is that when you ask people some polite version of “how would it make your life better?” a lot of times you get a non-answer.

You’ll get an answer like, “Well, it would just be nice to have.”

We don’t have time for building things that are nice to have.

How would it make your life better? What would it allow you to do? When someone really needs something, they’ll have a story for you.

“Uh, you know, it would take me half the time to sort my data. Oh, I wouldn’t have to waste head count on this position. We could start coding tomorrow.” When people have a story, that’s something worth doing.

When a customer comes to you and you say, “What would it allow you to,” sometimes you get the non-answer of, “But your competitor has it.” That’s actually just pushing the can down the road a bit, and what you say to that is to say, “Okay, I understand. You’re right, our competitor does have that feature. I’m curious if you were using that feature with Google, Facebook, et cetera, what would it allow you to do?”

I can’t count the number of times I’ve had customers who say, “Well, it would just be nice to have.”

If the reason that you’re going to lose a feature sale is because of a checkbox feature that someone’s not even going to use if they go to your competitor, that’s not something we should be trying to win on.

You may lose a sale in the short-term, but you’re going to have someone who isn’t really having their needs met, and they’re ripe to be plucked back in a year or two.

We ask, “How would it make your life better?” Maybe we hear that it really wouldn’t, and then we proceed.

Sometimes you’ll hear a variation like, “It might be useful in the future,” and, again, kicking the can down the road a little bit, and you got to ask one more question which is, “Okay, I’m curious how do you see your organization changing in the future?”


“Well, you said it might be useful in the future. I’m curious how are things going to change such that this might be useful in the future?”

You want to be very polite and smiley, you’re very nice about this, but the point is if you don’t know how the future’s going to change, and I don’t mean the next five years because no one knows that. I mean the next six months.

If someone can’t give you an answer, then it’s not a real need.

It’s a wish, or maybe it’s leverage to try and get a deal. Or maybe it’s just someone who’s trying to look smart in a meeting, and I think we all know the people who are in meetings trying to look smart, being loud, man-splaining you, et cetera. “How will it make your life better? What do you anticipate changing in the future?”

The other thing is that when we jump in trying to understand problems, or provide solutions, a lot of times it’s useful to know what people are already doing, and how they feel about it. That sounds so incredibly simple as to be obvious and dumb, and yet I have been surprised by the number of times I’m in meetings where we really don’t know. Sometimes it’s, “Hey, could you take a step back? I’m just curious, could you walk me through what you’re doing today? I mean, I know high level, but I’d love to see the details.”

When someone walks through a process for something, you might see exactly where their pain point is. Maybe they asked for this feature that’s a widget, but you can see that actually they’re having a hard time with this other area, and the widget might be one solution, but as a product person you can see it’s actually a poor solution. Or it’s something that will be applicable to this customer and no others.

In a meeting, a lot of times, where there’s a debate between people who think that one side’s doing it right, and the other side’s doing it wrong, a lot of times that comes down to an assumption about what each side is actually trying to do.

“Could you walk me through what you’re trying to do,” is a good way to defuse that, and let people say, “Look, this is just what I want,” because that’s, at the end of the day, what we’re trying to do.

We’re trying to help people get what they want done.

If we do that, we will seem amazingly smart, and helpful, and kind, and everything else. It’s a really good hack.

“Could you walk me through what we’re doing today?”

This is also really great advice when you get put into a new role. Let’s say you got promoted to manager. Congratulations! You can’t just step in, of course, and say, “This is how we do things now.” You can try, but you’re going to get a mutiny.

If you join a new company, a new role, people have established practices. Some of them are good, some of them are bad, and if you don’t know the history of why people are doing one of them, you don’t have a lot of position to say, “Let’s do things differently.”

If you can go into a new organization, or a new customer, and say, “You know, walk me through what you’re doing today. Okay, that’s interesting. Okay, you know, how did that happen in the past? Do you have a sense for how that decision was made ? How’s that going? You know, if you could change anything about it what would it be?” This kind of conversational approach gets people to trust you, and it gets you a lot more insights than you would ordinarily have.

Now, I’ll note one thing here, is that taking that step back and asking supposedly “dumb” questions can be particularly tough if you’re a woman, and if you’re in a meeting where you think people are just a little too quick to think that you are asking dumb questions. This is where it’s useful to borrow a new person. This might genuinely be someone who’s new on your team, or you could literally just grab one of your coworkers and ask them to come into a meeting, this doesn’t work internally but it works well with customers, “Come into this meeting pretend to be the new guy, new gal.”

The new person has a lot more freedom to say, “Hey, I bet everyone in the room already knows this,” — hint: they don’t — “But could you walk me through what you’re doing today?”

You will get a ton of insight out of that, and the customer won’t actually mind getting to repeat history, and probably halfway through their diatribe they’ll be like, “You know what? We didn’t even tell you that our entire back end system changed in the last year, did we? (haha).” Yeah, that’s probably something that you should’ve known.

“What are you doing today? How is that going?”

Once they’ve finished talking about that, then you’re going to reiterate. This is active listening. This is the thing that makes you feel a little bit like a kindergarten teacher, but trust me, it works. I wouldn’t tell you this if it didn’t.

“It sounds like you need to do X, and Y, and Z. It sounds to me, like you’d be happier if magically these things were fixed. Is that accurate? Am I missing something?”

Give them that option to correct you, or to add things.

Now, there’s a magic thing that happens once you’ve reiterated back, which is that you have absorbed like 80% to 90% of people’s anger at this point, even if you don’t actually solve their problem. They’re amazingly happier that you took the time to understand it in the first place. There are studies to back this up. Stanford has been doing some research with doctors and malpractice, and they found with a control and experiment group, that surgeons who made a mistake, and apologized were much more likely to not have suits brought against them, or if there were, they settled for much less money. Essentially, if someone sewed a sponge in you by mistake… you really want to hear that person say sorry. If they don’t, you’re going to take them to court for all they’re worth, so we can do that.

That leads to my next point, which is — apologies are free.

Any woman who’s ever worked for me knows that I always tell them, don’t apologize. For your ideas, don’t apologize. Don’t say I’m sorry about this idea, or I’m sorry that I want to do something a new way.

But when it comes to a customer who is feeling wronged, who is feeling like, I’m already upset, go ahead and apologize because that de-fangs even the angriest customer.

If someone comes in and they’re furious, “I can’t believe that you’ve lost my data. I’m going to quit your account right now, this is ridiculous,” and you say, “I’m really sorry. We did that, we lost your data. That’s really awful, and I don’t want that to ever happen again. Let’s see what we can do about it.” Those are magic words.

It’s very hard for someone to hear something that is humble and accepting of responsibility, and keep yelling at you. They trail off, “Well, you better see what you can do.”

Give them out, apologies are free.

It doesn’t matter if it’s not your fault, if someone lost their data through something that was user error, it doesn’t matter. You’re not going to talk them into that. Telling people that’s the way the feature was designed has never made anyone happy ever, so apologize. You can do that.

Apologies are free. They’re also rare, especially good ones.

Follow-ups are also rare.

Even if you didn’t provide the answer someone wants, just the fact that you reach back out to a customer, or a coworker a couple weeks later to give them an update, “Hey, I looked into that solution. It turns out we’re not going to be able to address it. I’m really sorry, I just wanted to let you know,” people are amazingly happy about that because they never hear it.

The traditional vendor/buyer relationship is, “We’ll put it on the roadmap,” and then it’s on the roadmap, and it’s on the roadmap, and they never actually get the feature.

Internally, when people have suggestions, “Oh, we’ll consider it. Oh, put it in the suggestion box,” and there’s never any closure.

That’s the final thing I have to talk about: people like closure.

As humans, closure makes us feel satisfied. We like to know that something is going to happen even if it’s not the thing that we expect.

Customers and teammates don’t have to agree with your decisions, but they need to know why you made them. They don’t even need to understand why. It’s just that they need to know that you didn’t have malicious, or stupid reasons for making your choice.

“Based on these reasons, I am making the decision to do X.”

Someone may disagree, but they’ll grumble, “Well, I guess I understand why you’re doing it. I still don’t agree with you, though.”

That’s okay, customers in that scenario are going to be a lot happier.

In this case, someone’s come to us, they’ve demanded a feature, we basically talked them out of it, we’ve explained why, and yet they’re still not furious at the end. In fact, a lot of these customers end up being incredibly loyal because even though they’re not getting what they want, they know that you understand what they need. This works both internally and externally, and it’s been tremendously useful in my career.

The bar is really low for honest communication, and for digging in to find out what the underlying problem is.

We can do better, and it’s an amazing hack that most of the people around us don’t know about, so we should take advantage.

The customer is not always right, to be honest, no one is always right, but you can, and should control your own narrative. That means taking in what you’re hearing, and reflecting on it, and asking questions, and redirecting it to something that is more positive, and something that you can control.

I’m sorry am not able to take questions live, but I really do answer them.

I’m @cindyalvarez on Twitter or If you have questions, feel free to shoot them over to me. And that’s my last piece of advice: when people say it’s okay to ask questions, they mean it — take advantage. Enjoy the rest of your conference everyone!

“The Art of the Interview: How to Evaluate and Handle Candidates in Your Pipeline”: Aline Lerner with (Video + Transcript)

Aline Lerner / CEO & Founder /
Gretchen DeKnikker / COO / Girl Geek X


Gretchen DeKnikker: Hey, everybody, welcome to our next session. Couple housekeeping notes. Make sure that you ask your Q&A at the bottom and then vote them up so that we know which ones to ask when we get to the Q&A section. The videos will be available online just after the sessions wrap up. Without further ado, I’m so excited. I’ve just met Aline and I’ve become a super fan in one phone call. It was a little crazy and so now, I stalk her.

Aline Lerner is the co-founder and CEO of, which she’ll tell you a little bit about, but I think the product is pretty interesting especially for the audience here. She’s going to be talking about the art of the interview and really looking at it from the angle of not just you being the interviewer, but what are the other elements and what kind of feedback do you get? So, without further ado, welcome, Aline.

Aline Lerner: Hi, everybody. I’m also a huge fan of Gretchen’s and have been stalking her since I met her, so it’s definitely been mutual. Really excited to talk to all of you today about technical interviewing, probably through a lens that you haven’t seen before, because we have some cool data that normally you don’t get. Rather than talking about what makes someone a good interviewee, today I’m going to talk a little bit about what makes somebody a good interviewer.

I’m the CEO and co-founder of a company called We are a practice platform for technical interviewing, but we’re also a jobs platform, so if you do well in practice, you get to talk to top companies. The cool thing is everything is anonymous. We collected a ton of data and I’m going to share some of the things we’ve learned today. I’m just going to jump right in. All right. Great.

How it works — once you’re a user of our platform … this is a little bit of setup so you know where our data is coming from. Once you’re a user of our platform, you can see some time slots, grab one, and then, at go time you log in, and you get a mock interview with an engineer from a top company who is good at interviewing and good at giving feedback. After each interview, there is feedback, which you’ll see in a moment. Top performers actually get to interview with companies right on our site and those interviews are anonymous as well. After each interview, whether it’s real or whether it’s practice, there’s some metric feedback. This is the feedback form for interviewees. Hopefully, you can see some of those questions. We ask things like, how was the interviewee’s technical ability, communication ability, and problem-solving? Then, the interviewer will ask stuff like … we’ll ask the interviewer to rank the candidate on stuff like technical ability, problem-solving, and communication, and then, we’ll actually ask the candidate to rate their interviewer and this is what this talk is about.

Normally, as most of you know, an interview is one way. You don’t really get to rate your interviewer and if you do it’s a survey afterwards and it’s not right then. We ask everything from whether you want to work at this company and with this person to how excited you’d be. Then, we also ask how you think you did and that will come up at the end. Great.

We have a ton of data. We’ve done about 20,000 interviews on the platform and I’m just going to get down to brass tacks and show you feedback snippets. We try to distill a lot of signal from these and come up with a few broad categories for what the traits of good interviewers look like. This is going to be more data-driven and hopefully less about platitudes. I’m really excited to answer your questions at the end.

Before I get into the details, one thing, a lot of people think that if you work at a company with a top brand, it’s a really good crutch. If you work at a Google or a Facebook, you don’t have to be as good of an interviewer. That’s not strictly true from the data that we’ve seen. In fact, we saw no statistically significant relationship between brand strength and whether people wanted to work with employers on our site. Brand will get candidates in the door, but once they’re in the door, they’re essentially yours to lose, so keep that in mind.

The first kind of huge thing that we noticed among candidate feedback was that when you’re interviewing people, it’s important to be a human being. So, for each of these broad categories, I’m going to show you exactly what the candidates said. For instance:

  • “I like the interview format, particularly how it’s primarily a discussion about cool tech as well as an honest description of the company. The discussion section is valuable and may better gauge fit. It’s nice to have a company that places value on that.”
  • “Extremely kind and generous at explaining everything they do.”

When I’ve listened to some of these interviews, and I haven’t listened to all 20,000, but I’ve listened to a lot, the best interviewers are people who take the time to get to know the candidate even though interviews are anonymous. So, what are you working on? What do you want? Then, they’ll create a narrative where their company is the next logical step in that candidate’s journey. So, everything you’ve ever worked for is going to culminate in you working here.

Here’s the bad:

  • “A little bit of friendly banter, even if it’s just, ‘How are you doing?’ at the beginning of the interview would probably help the candidate relax.”
  • “I thought the interview was really impersonal. I could not get a good read on the goal or the mission of the company.”

Choosing the question. This is a very erudite topic of discussion, and I know everybody has opinions on what makes for a good interview question. We just have the data, so I’ll just tell you what the data said. So, here’s feedback from people that thought the question was good:

  • “This is my favorite question I’ve encountered on this site. It was one of the only ones that seemed like it had actual real-life applicability and was drawn from real or potentially real business challenges.”
  • “I like the question. It had a relatively simple algorithm problem and then built on top of that.”

One of the recurring themes here is that candidates are used to these generic algorithmic problems and what really gets them engaged is taking it to the next level and tying it into something that your company actually does. This is especially true if people may not have heard of what you do, or you are in a space that by default doesn’t get people excited. Anything you can do to get in the candidate’s head and ask them something interesting and then have it stick after the interview is over is going to be good. Then, candidates also feel like you put in effort.

One of the things that we’ve noticed is that whenever there’s this notion of value asymmetry in an interview, so the candidate is expected to put in work, but the interviewer is not putting in work, that’s not good. You want it to look like you’ve put in work yourself.

Here, let me show you some of the examples of bad questions:

  • “This is not a good interview question. A good interview question should have more than one solution with simplified constraints.”
  • “Question wasn’t straightforward and required a lot of thinking, understanding of setup.”

You don’t have a lot of time with a candidate. You want to make sure that the time that you do use is used on being able to build a connection with them and then actually seeing if they can think rather than jumping them through hoops.

“Is there any way to sharpen the image? Text is blurry.” We’ll send out these slides afterwards. Sorry about that, guys.

That’s really one of the most important things too is making sure that you are getting signal from these people in a way that’s not arbitrary. Setting up the problem to gauge whether somebody’s a thinker and a problem solver rather than catching them on arbitrary a-ha moments.

Writing a really good interview question is hard. It takes time, especially if you’re going to tie it something you do at work, it’s even harder.

One of the best tricks that I’ve seen for doing this is coming up with a shared Google Doc for your entire team or really any collaborative software. Doesn’t matter. Any time you do something at work that made you think, and it doesn’t have to be cool. The bar for whether it’s cool can be really low so you don’t have to worry about it, but any time you do something that you think was non-trivial, just throw a quick line in that doc.

Then, you can come back later and look at all the cool things your team has done and use that as a jumping off point to craft questions that are unique to you. Then, candidates will be like “You did put in the work,” and you do stick in their heads a little bit.

Asking the question itself — One of the best interviewers I ever met was a chief architect at a large software company. He used this expression that I really liked. He said that the purpose of an interview is — “can we be smart together?” That just really stuck with me, and I think that the way you ask the question can really determine whether you can be smart with somebody else or not.

Here are traits of good interviewers when it comes to asking a question:

  • “He never corrected me. Instead, asked questions and for me to elaborate on areas where I was incorrect. I very much appreciate this.”
  • “The questions seem very overwhelming at first, but the interviewer was good at breaking it down. I like the fact that you laid out the structure.”
  • “I’m impressed by how quickly he identified the typos in my hash computation.”

Engagement is, of course, important when you’re asking the questions so you actually have the opportunity to see what it’s like to collaborate with somebody.

Another really important part of this is, and you can see this in the feedback, is layering complexity. This idea of taking a question that can start off very simply at first and then building on it. Building on it in a number of different ways and you can set up benchmarks and say, “A candidate that’s good enough is going to get through the first three portions of the question. Somebody who’s really good is going to get through four and someone who’s exceptional is going to get through five. Then, somebody who gets past that is probably going to challenge the interviewer. The sooner you can turn something into a discussion between equals and an opportunity to collaborate and problem-solve together, rather than a one-way exercise where you’re trying to see if somebody’s stupid or not, which is the worst way to interview, the better it’s going to be.

Here are some examples of poor interview feedback:

  • “It was a little nerve-racking hearing you yawn while I was coding.”
  • “What I found more difficult about this interview was the lack of back and forth.”

Anything you can do to engage with candidates and build on a question is going to be the best and if you can couple that with the previous point, come up with questions that are original to your company and layer complexity in ways that other people couldn’t, then you’re going to be in a very good position.

What happens after the interview? This is one of my favorite takeaways from our data, and it’s completely counter-intuitive. As you recall, we ask people how they think they did on the interview as candidates and then we also ask interviewers how the candidate actually did. We actually graphed this. The x-axis here is the actual score on a scale of one to four for somebody’s technical ability and then the y-axis is their perceived score.

As you can see, there’s quite a bit of imposter syndrome. In fact, we found that imposter syndrome plagues a disproportionate number of our users. So, what is imposter syndrome? It means that you think you did poorly when you did well. Now, here is the crazy part. If a candidate did well and they think they did poorly and you don’t give them immediate actionable feedback and let’s say you let them sit on it for days, they’re going to get into this whole self-flagellation gauntlet.

They’re going to leave that interview and then they’re going to start thinking one of two things: either they’re going to think, “Man, that company didn’t interview me well. I’m good at what I do, and I don’t think that company knew how to get it out of me, so they suck.” Even worse, what’s going to happen is you’re going to think, “Oh, I’m a piece of shit. Now, they know I’m a piece of shit, and I totally didn’t want to work there anyway.”

What ends up happening is unless you tell people they did well, immediately after they did well, you end up losing a lot of good candidates because, by the time you get back to them, they’ve completely talked themselves out of working for you.

So, don’t let this happen. Don’t let them gaze into the abyss, and give people actionable feedback as soon as possible.

Actually, I saw one of the comments. I want to leave a few moments for questions, but one of the comments on the side was “imposter syndrome is a women’s curse.” We ran some data on our platform to see if imposter syndrome is more prevalent in women or whether it’s distributed across both genders. As it turns out, both men and women are equally plagued by imposter syndrome.

The other interesting thing that we learned, and we haven’t written about this yet but we will, is that the better you are at interviewing, the more prone to imposter syndrome you are, and the worse you are, there’s the opposite called the Dunning–Kruger effect where you think you did well when you, in fact, did poorly.

Thank you, guys. I’m really excited to answer some questions. My email and Twitter are also on the slide, and I’m happy to answer them offline as well. Sorry, I saw some of you said some slides were blurry. We’ll send them out afterwards, not sure why that happened.

Gretchen DeKnikker: Great. Thank you so much. We have a few questions here and you guys still have time to submit and vote some up. So, the first question is, what do you think of take-home projects instead of whiteboard style coding interviews for those who grew to dislike them?

Aline Lerner: Yeah. I wish I had it with me. I drew this picture a while ago called the value asymmetry graph and I mentioned it in the talk as well. Value symmetry is this notion that we have two sides, both of them are putting in equal amounts of work. I think that if you’re a company with a top brand, you can get people to slog through a lot more shit than if no one’s ever heard of you. When you’re deciding as a company whether you want to use take-home challenges, that has to be one of the things you consider is, how badly do people want to work for you? If you’re Google or Facebook, at least before you get into the interview, at which point the playing field doubles a little more, people are probably going to be much more motivated to work for you than some company that just started and has no funding. You can’t always look to those companies and say, “If they use a challenge, we can too.”

If you do use a challenge, just like with interview questions, the best ones tend to be ones where it’s thoughtful and where it’s representative of the actual work because then the candidate is getting some value out of it… doing this work is going to be like the stuff I’m going to do every day, so here’s a preview.

On, what we’ve seen is the customers of ours that have people do coding challenges after their technical interviews — and if those challenges take longer than an hour, the best people tend to drop out because in this market, engineers are flooded with opportunity. If you make them do work, they’re probably not going to do it unless it’s really, really interesting work or some companies pay people. If you have something that’s going take five or six hours, consider giving them a consulting fee and see if that changes anything. But, you should probably just have a really good challenge that people want to do or not have a challenge at all. That’s different for data science and engineering also. Sometimes that makes more sense. For software, you probably don’t want to do it.

Gretchen DeKnikker: I have one I want to put in because we’ve heard this a few times. We’ve been recommending it and everyone’s like, “It’s in private beta. How do we get access?”

Aline Lerner: Yeah, we’re opening it up really, really soon. So, we have a really long waiting list and we’re so excited to get through it. For now, we’ve been favoring people that are still in the U.S. because it means that we can place them a little more easily. We don’t always look at it, but in tie situations, we’ll potentially look at someone’s years of experience because we have more job openings for senior folks than junior ones. Regardless, we are working on a way to open it up and I expect it will be opened up by next quarter. So, everyone that’s on the wait list, I’m really sorry and if you have job interviews coming up soon, send me an email and we’ll see what we can do.

Gretchen DeKnikker: Yeah. Awesome. Yeah. It’s a great problem to have though, right?

Aline Lerner: Well, we really want to move on from that and just open it up, but yes..

Gretchen DeKnikker: Next question is, I’m unfamiliar with it but very interested. How anonymous is anonymous, first names, voices? Does that anonymity help level the playing field for women and people of color?

Aline Lerner: Yep. When we say anonymous, we mean truly anonymous. Everybody gets a handle. So, my handle on the platform is nihilistic defenestration. If you ever run into … I think I’m the only one that has that. IF you run into that, it’s me. It’s why I quote Nietzsche and wear black, but, in some cases …

For practice interviews, you can hear people’s voices. From voice, you could potentially glean gender and we don’t mask accents. However, we did just two days ago, we just got a patent on real-time voice masking. In real-time, we can make women sound like men or men sound like women or make everyone sound androgynous. If a company wants to use that for their interviews, then they have to turn it on across the board. If you let candidates decide, then there’s this other bias notion, who turns it on and who doesn’t. This way it’s turned on for everyone and we leave that at our customer’s discretion.

We did try making everybody change genders in practice to see what effect that would have and we found that surprisingly, at least to me, it didn’t really change how people did. So, women didn’t do better when they sounded like men and men didn’t do better when they sounded like women. We did notice that women were doing a little worse across the entire platform and I was confused by that ’cause I don’t think women are worse at computers.

What ended up being the case was that women were disproportionately quitting the platform after one bad performance in practice. Once you corrected for people that were quitting after one bad performance, well, the gender-based disparity went away entirely. So, we’re actually rerunning that data now that we have a lot more interviews and we’ll report back. Back then, we had a lot fewer, so all of that … As if the case with science, or in our case, pseudo-science, the stuff can be overturned.

Gretchen DeKnikker: All right. I’m going to do one more. We have 1,000 questions. I think we could mostly stay on this topic all day. You do have a blog, right?

Aline Lerner: Yeah.

Gretchen DeKnikker: So, if you want more from Aline, check out the blog on because she’s-

Aline Lerner: I’ll put it in the chat.

Gretchen DeKnikker: Perfect. Yeah. There’s a ton of information. So, I’m going to do one more. You touched on this a little bit, did any data, anecdotal or otherwise, bubble up around bias? For example, knowledge of algorithms, which can indicate recency of learning, younger candidates or those who got computer science degrees versus coming from alternative backgrounds?

Aline Lerner: Oh, God, yes. There’s so much bias. The most compelling bias or, I guess, the strongest signal of bias that we’ve seen has been against people with non-traditional educational and work backgrounds. If you didn’t go to a top school and you didn’t work at a top company, it’s going to be really, really hard for you to get in the door. What we’ve seen repeatedly, and this is the thing that blows my mind, is with some of the bigger customers that we have where they get a lot of inbound applications, people have applied, they’ve gotten rejected at the resume screen, so before anybody ever interviewed them, and then they came in … Then, they used our platform, practiced and got good enough to … or, in many cases, they were already good enough, but they got access to our employer portal, interviewed with those companies, and actually got hired.

Of course, once they unmasked after their interview, the recruiting team can see, “Oh, shit. This person is in our ETS but we rejected them six months ago before anyone talked to them. Oh, shit. There’s something wrong here.” In fact, 40% of the hires we’ve made in the last two years have been people that would have been [inaudible 00:21:03]. Companies admitted, they’re like, “Well, I never would have … What the Hell?” That’s why we insist interviews be anonymous, or they actually had been turned away by that employer.

Gretchen DeKnikker: All right. Well, I would love to stay doing this. Thank you so much for coming in and giving this talk. I think it was hugely valuable. Everybody, we’re going to take a short break. We will be back in 15 minutes, so go grab a snack and some coffee and we’ll see you at 1:20. Bye.

Aline Lerner: Bye, everybody. Thank you for having me.

Gretchen DeKnikker: Thank you.