In this session, Shayla Gibson emphasizes the importance of leadership and team management skills, stating that people skills are a superpower that can set project managers apart. She also highlights the need for technical proficiency, explaining that project managers should have a deep understanding of technical tools, processes, and industry standards.

Transcript:

Shayla Gibson: Thank you Amanda and hi everyone, I hope you’re doing well this lovely Wednesday and happy hump day everyone. So, welcome to Good to Great Strategies for Achieving Excellence in Technical Project Management.

Little about myself. My name is Shayla Gibson. I have roughly eight years experience in project management and seven of those have been in the banking and finance sector. I have led company-wide agile transformations and created and revolutionized project management techniques for small businesses and small corporations. Some of the topics we’re going to go over today; leadership and team management, technical proficiency, tools, and project management and its aliases.

So, first things first. What is the difference between project management and technical project management? And apologies in advance that this seems kind of obvious, but it’s the technical aspect. Technical project management requires knowledge and expertise in technical software development or specific technical domains and a deep understanding of technical tools, processes, and even industry standards. Project management can be on a wide range of projects and a wide range of industries, including construction, marketing, and event planning. It doesn’t mean that technical project managers cannot also be in those fields, but the difference mainly is just the technical aspect.

Now, leadership and team management. If you Google or search project management, this term is probably within the top five of that list of skills that you need. But I like to bullet down even further and for me, it’s the most important thing to have; people skills. People skills are your superpower. Can it be taught? Sure. But if you’re good with people, if you’re good at persuading people, sharing your narrative, your story, inspiring them, talking to them like a person, everything else can be taught. I can teach you the intricacies of how money truly moves through our banking system. I can teach you how to read NACHA Fedwire files. I can teach you change management or how a system works, but if you know how to manage people, manage a team, manage stakeholders and executives and everything in between, then that is your superpower.

And I want to make it clear, you do not need to be defined by being an extrovert or an introvert. I know plenty of introverts that are amazing people, have amazing people skills, and I know the exact opposite with extroverts. The thing you should focus on is, can you use your people skills? Because if you can, that is the excellence that’ll make you stand out from others.

Now, back to the defined bullet points that I have. You need to be able to lead a team, lead by example. If you are working hard, your team will also work hard. You need to inspire and motivate. Sometimes we have to work on things we don’t always care about or don’t even have the complete vision. Technical project managers and project managers need to have the ability to explain that vision and to keep their team motivated and going. You also need to have the confidence in the decisions that you have to make. For me, it’s making sure I have all the information, all the sides of the stories, and all the risks laid out. So even if I make a decision and it doesn’t pan out for the best, I was confident that I made this right decision with the information I had.

I have a little story time and it is going back to that bullet point about inspire and motivate. I worked with various technical teams. Some comprise the developers, engineers, product owners, product managers, you name it. This particular team was mainly developers and engineers and they worked well together. So this is not a story of where they weren’t working together, they didn’t mesh, and I came in and saved the day. No, they worked just fine with each other. If you ever meet me in person, I am this bubbly ray of sunshine and I actually really love bad jokes and dad jokes. So, every Friday that team will actually get a bad joke from me, a dad joke, and I can hear the eye rolls and the groans, and that’s what the team gave me too, but I still did it every single Friday. Eventually my team was, I’ll say, bold enough to all start ganging up on me and tell me how terrible these jokes are, that I need better jokes or some of them just don’t make sense at all.

Eventually, they started posting their own jokes. I didn’t even have to post anymore, but I still did because I love doing and I love hearing the groans. Eventually, they started giving each other feedback on these jokes, whether they were good, they were bad, whatever. Eventually, my team started going from working well with each other to great to excellent. Why? Because they were able to communicate together. They were able to collaborate and give each other feedback. Now, I’m not saying this is the secret sauce. All I know is that I’ve done it multiple times with various different groups and I’ve gotten the same outcome. So that is me using some sort of people skill, my superpower, to inspire and motivate a team.

Technical proficiencies. So what do you really need to know to be considered a technical project manager? It’s not knowing how to code. It’s not me logging into GitHub and looking at failed processes in sandbox, debugging it, and figuring out the best way to resolve it. It’s not me knowing that in a NACHA file for ACH payments, one place I can find a routing number is in the file header record, which always begins with 101 followed by the routing number of the originating sending bank. It also includes some date timestamps and the originating bank and the company name. But what language they use to code, knowing that some of their errors are flagged in GitHub, or even that my engineers need help debugging it, maybe this is an error that we see regularly.

Now, let’s go back to that technical statement that I just said, and feel free to fact check me on those routing numbers. Understanding the technical aspect, understanding the technical language and be able to spit it back out so others can understand is key. Things are always changing so you need to stay on top of them. So I’ll repeat that technical statement, then I’ll repeat it back in a more digestible way. So the technical statement. For a NACHA file, one place I can file a routing number is in the file header record, which always begins with 101, followed by the routing number of the originating sending bank. It also includes date time stamp, as well as the name of the originating bank and company name.

Here’s a more digestible way. ACH, it’s a payment type. And for banks to digest it, there’s a file format that’s called NACHA and this format is regulated federally. So there’s certain rules that all banks need to follow. The routing number, and I’m sure everyone here has opened a checking account or a saved account, we all get paid. You get a routing number. In this massive network of banks, that routing number is how we know which bank is which. So Bank of America, Chase, you name it, they all have their own individual routing numbers. So if I’m looking at this NACHA file and if I can find a line that starts with 101 and then right after that should be the routing number. Now there’s additional data in there and sometimes it’s going to look like a date timestamp or company name, but as long as I find 101, right after that should be the routing number.

That’s much easier to understand, right? And that is what we need to do as a technical project manager. Understand the technical and have the ability to rehash it for others to understand. We don’t need to be the smartest in the room, but we need to know how to talk to them.

Common questions I always get asked is, where can I get training for this? And the safe answer is starting with a PMP or a course in project management, figuring out what types of technologies or software are in your industry of choice, and learning about them. But this is the digital age and I can’t tell you countless times someone has asked me about a project or to look into something that I have never heard of. But YouTube has, Google has, and I am never shy to ask 1,000,001 questions. Sometimes it just takes that initiative.

Lastly, cross-functional understanding. Going back to the example I gave, not everyone in your company is going to have the same level of knowledge as you. So you need to be able to rehash those technical aspects and understand the 360 view. On the screen, you’re going to see a list of tools and you don’t need to know all of them and you may have came across them, you may not have, and you may, in your career as a technical project manager or a project manager. But if I can zoom in on one, the project manager software box, there’s two listed in there, but there’s probably 1,000,001 out there. Project manager software tools are all the same with different colors and maybe a slightly different feel to them. But let’s say you know nothing about Tableau or Power BI, once again, it’s a digital age. So somewhere someone has figured out how to use it and has put it online. Let’s work smarter, not harder.

Professional development. So I already touched a little bit about this, but just to give you some more color, I am actually currently studying for my PMP. So in the world of project management and technical project management, there is the infamous question, PMP or not? All I can say is this; whether you get one or not, experience is going to be what you can follow on. And also a PMP might help you get your foot in the door. I don’t have a complete answer, but just like anywhere else, if you look up professional development, you need to keep networking, talk to other project managers, technical project managers, certifications out there, especially within the industry of choice, and conferences and events like the one you’re attending now.

All right. Last but not least, technical project management and its aliases. So here’s a short list of what a technical project manager can look like in the world. It can be an IT operations manager, the DevOps project manager, or a implementation manager. If it walks like a project manager, or a technical project manager, it probably is, but just make sure you ask your questions too. Here’s an example.

So at my current company, my title is a Technical Operations Manager for our Technical Services Management team. I work on special projects that pertain specifically for the Technical Service Management team or the Operations team, and I mainly focus on our internal clients. But when I started at this company, I was actually implementation manager, getting our clients’ implementations up and running. It was very customer facing. I was working hand in hand with our solution engineers, our engineers, our product managers, and I was in charge of creating a template project plan and then following that through through its ups and downs. And before that, I’ve had many other tiles, but they were all within project management. Just make sure you do your research and you ask your questions when you’re seeing titles out there.

And thank you everyone. So glad you came to listen. Please get in contact with me on LinkedIn. Amanda, I don’t know if we’re open for questions. Do we still have time?

Amanda Beaty: Yeah, you’ve still got a couple minutes.

Shayla Gibson: I see one in the Q&A, I think this is a good one. So, how do you think AI is going to change up the situation for program or project managers? I think AI is a tool that we need to get used to and that we can use to our advantage. Going back to that this is a digital age and that there’s so much data out there, there’s so many tools out there, AI is just another tool that we can actually learn to use for our benefit. So, I would actually encourage you to learn a little bit more about AI and see how you could use it in your day-to-day. But I think they are only up to date until April this year, so be careful what you ask AI or ChatGPT or whatever you use, and it is just a tool, so you will have to actually read it and make the decision if you can use it or not. I will take one more if I can and I said-

Amanda Beaty: Yeah, go ahead. Yep, you’ve got three more minutes. Go ahead.

Shayla Gibson: So, do recruiters look for PMP or mainly reply, I think rely, on the experience of project management and roles? I think that’s really dependent on the company. Like I said, some really, really would love to see a PMP and others are more lax on that. So, unfortunately there’s not a really good answer, it really depends on the company, but sometimes the PMP just gets your foot in the door, and then other times if you can prove that you have the experience.

How to mention transferring skills in another industry when transitioning to tech. Are all program and project managers fundamentally doing the same core work? That’s a great question. If you have the basis of project management, you should have some of the basic skills to go and take that from one industry to another. However, I will say this, in tech it’s all about your experience with tech and how much you know. So if you are going to be transferring from a different industry into tech, make sure you do your research, learn as much as you can on some of the tech fields and most popular fields out there. And that way you can still use that within your resumes and your interviews to talk about that experience. I think I’m at time now.

Amanda Beaty: Yep. Let’s go ahead and call it and we’ll pop over to the next session. And thank you so much, Shayla, everybody really enjoyed this and thanks everybody for joining us and we’ll see you in the next session.

Shayla Gibson: Thank you.

Anusha Dharmalingam emphasizes the need for a culture of innovation within companies and provides tips on how to foster such a culture. She explains that innovation is about putting creative ideas into practice and highlights the importance of desirability, feasibility, and viability in the innovation process.

Transcript:

Anusha Dharmalingam: Hope you can all see the screen and can hear me fine. So here I am to talk about what does being innovative mean in the digital transformation. So a few words about me… Thank you for the feedback. It’s really hard to know if everybody can hear me. So a few words about me. I’ve been in the industry for 23 years, primarily in the technology industry. So today I play a role of an executive director and a senior architect at Athena Health. So I have played different roles, as you can see over the pie chat, like a software engineer, a development manager, program manager, and architect. I have been in the consulting, banking, and high-tech, and healthcare industries. My expertise is in the cloud technologies, and I’m really very passionate about women leadership and especially in the technical leadership. I’ve led digital transformation projects over at various companies at different roles.

On a personal note, I am a mother of two boys and I love to spend time with them and to bike when I can. So with that, let’s move on to our topic for today.

So I would like to start this presentation with a small story, a story that would ground us all on the essence of innovation. At the same time, give us the significance on why the culture of innovation is important for the long-term success of a company. It’s a story about a Stanford graduate back in 1990. He was a computer science graduate who started a company named Pure Storage… Pure Software, sorry. It was a company that built diagnostic software for unique based applications. Back in those days in 1990, that was very rare. So the company gained quick in popularity and had revenue that doubled year over year. So finally they sold the company in 1996 to National Software and they were very successful at it.

So this founder of the company had a moment in his life which would change the movie watching experience for all of us down the line. So the incident goes like this. He basically rented a videocassette from Blockbuster for a movie named Apollo 13. Despite his wife’s continuous reminder to return the cassette, he actually misplaced it and returned six weeks late. This incurred him about $40 of late fee. He had a very embarrassing experience on this that he decided not to even share it with his wife, and was constantly thinking about this on why did he have to pay a late fee for just misplacing his video rental. So he misplaced it and he was not very happy about it. So later when he went to the gym, he realized that the gym’s model of working was far better than what he had experienced while renting his movie.

So in the gym, all he had to do was pay $30 per month and there was no limits on number of workouts that he could do on a monthly basis, and there was no late fee concept. So he wondered, what if he applied the same concept, a concept of a monthly rental for the movie rental business? And that’s exactly what he did. After selling the Pure Software to National Software, he started a company which we all cherish today as Netflix. So in 1997, Reed Hastings, along with Marc Randolph, started Netflix as a movie DVD rental that would be delivered to your doorstep. So as customers, all you had to do was log into the website, choose your movies that you would like to watch, and the movies would be delivered to you at your doorstep for no late fees, but for a monthly subscription fee. So this whole business model took a while to gain popularity, but around 2000 they started making profit.

So in 2000s they went to Blockbuster, at that time a $4 billion company with 6,000 brick and mortar stores, went to them and said that, “Hey, could we partner?” Could you buy us by taking 49.5% of our share so that we could become a digital arm for you? But Blockbuster rejected that offer, and so Netflix went back to the DVD rental business. But Netflix did not stop there. They observed the digital era that was picking up in 2000s, so Hastings went to his board and said that we are in a pivotal point for our company. We either choose to stick to what we have been doing, or we embrace the digital transformation that is going on in the industry and start moving on to the steaming services business. Very reluctant, the board slowly accepted Hastings proposal and they invested on that proposal. And thereby in 2007, Netflix started their streaming services.

And from then on we all know what happened. Netflix thrived and thrived. And they did not just stop there. That is not the only reason they thrived. They actually build the culture of innovation within their company and they continue to innovate on a day-to-day basis. Some of their innovations that we are all familiar of are their recommendation algorithm, which suggests movies for us when we watch Netflix, or the ability of being able to stream out of Netflix on either your movies or on your phones or on your DVD players when those things existed, on your iPad, whatever. In all possible devices. So they worked with hardware vendors to make sure that that is possible. And apart from that, Netflix also started producing their original content. So we have their series, their movies, and whatnot today. So with all these things, they have now made their name as a common household name, not just across the United States but across the world.

While along the same lines, Blockbuster on the other side stuck to their original model. They did not adapt to any transformation that was happening in the world and they had filed bankruptcy in 2010, and they do not exist anymore. So these two companies gives us a stark contrast of the power of innovation, the transformation that it could bring in any long-lasting business. So this clearly sets us on why the topic of discussion today is super important. So with that, let’s start talking about… There are two things I wanted to cover in this whole session. One is to understand deeply what does innovation mean. And number two, to give you all some tips based on my experience on how to build a culture of innovation at your workplace.

So when you talk about innovation, it has a slightly different meaning than creativity. Creativity is about an idea, right? You have an idea on how to do something. Innovation is about putting that to practice. It is about making change to something that’s already working, something that’s already established. You’re making a change to that. It does have some few characteristics that are being shown here, the desirability. So for you to do an innovation, that should be a need, that should be a customer demand. And this demand can be implicit or explicit. Sometimes the demands are implicit. That might not be an explicit need for that, but it is somewhere there. There’s an indirect need for it. So that is the desirability. And the next important thing is feasibility. You can have whatever desires you want and you can come up with ideas on how to implement the desire, but that implementation should be feasible. It should be on top of your current operational capabilities.

You can’t be a company doing a movie dental business and want to suddenly provide cargo for airplane, or something of that sort. It should be aligned to your business. And the third important thing is the viability. The cost for buying that product, or the cost for building it should make some business sense. It should be possible, it should mean something, or it should be delivered in a medium that is possible to be consumed. So those are the viability things. So when we innovate, it is important that it has characteristics of all these three things and it intersects to meet at a sweet spot, which will give us a successful innovation. So with that, let’s slowly talk about why do we innovate. What better way to explain that than the story we just talked about, that clearly showed the contrast of a company that innovated and the company that stuck back and what happened.

But all said, the main thing on why we innovate is to meet our customer needs. So we need to deeply understand our customers, empathize on what they actually would need, and build products or solutions for that. It also helps us to have a competitive advantage in the market. Of course, it’s for growth, for us to make money and reduce the cost of how we are doing things. And overall, it provides adaptability. You’re constantly in the lookout of what is happening in the industry in the world, and you’re able to adapt to what it is. And no one said that innovation is easy and it can be easily done. It really involves some thought process and some investment to kind of get this going and to keep it up and keep it running in your workplace and the company.

There are different types of innovation that could happen. Let’s start from the right corner over here. The radical innovation. This is the one which I was talking about earlier about implicit demand. So when smartphone came into the industry, none of us knew that we needed a phone which could do all in all everything, where we can watch movie and listen to music. We didn’t know that we needed it, but it did come. So somebody radicalized and they introduced it into market and we soon adapted to it. So that’s a radical innovation. A disruptive innovation is in an existence market. So radical innovation creates a new market, a disruptive innovation is on an existing market, a totally different way of doing business. Say for an example, an Airbnb. We already had a hoteling industry and a lot of hotels, but Airbnb came up with a new model which would disrupt that and do something different.

Same with Squire. We all knew how to use credit cards, but to enable to swipe credit cards on a mom-and-pop shop using just a smartphone, that was a disruptive innovation. If you move towards the left, the architectural innovations are one where in an existing product, in existing market, whatever you’re doing, you’re doing a significant improvement. Doing something drastically different that would strengthen your space in the market, like the GE’s Ecomagination products. These are products that already existed, but to adapt to the climate change and being concerned on the environment, GE came up with these new set of products that made them leaders in those kind of products. Incremental innovations are one which I’m sure most of the companies are doing on a day-to-day basis. For example, the new versions of Apple iOS versions, which comes with newer features, or even the Netflix recommendation algorithms that keeps changing constantly. I’m sure all of us are continuously evolving our products that we develop, and those are all part of incremental innovations.

So at a different point of time, the companies would play a different role in each of these quadrants. And like already mentioned, innovation is not a one-time thing. It’s very similar to the agile methodology that is being recommended for a development process. It’s very similar to that, but it does have its own differences. So as you can see, the cycle starts here, right? You challenge the status code that you are in today. You say that you want to move away from whatever you have and you want to do something different. That’s where the creativity idea sparks up. You take the idea and you build, what we call as a prototype, or a minimum viable product, and that’s when the cycle starts.

You take the product and then you apply it to your user base and see if the product has its feasibility and is it viable to build it. And once it is done, you measure the metrics out of it. It is always a data-driven decision. So you measure saying that how much impact did it make in terms of the users, in terms of the revenue that it generates, in terms of the metrics that it provides you, performance. Whatever makes sense to that particular idea, you want to evaluate those metrics. And if those metrics are great, you would want to continue invest more on that and start to build that as a product and evolve it again and again. What’s very different about innovation cycle is sometimes it could so happen that these metrics clearly indicate that the idea that you came up with does not work. It’s not going to work. It’s either not viable or it’s not feasible, or it is not exactly meeting the demand that your customer wanted.

So in those cases, you happily pivot. You celebrate failure. What it means is you basically learn from what happened. You learn from what was done and how was it different than what was asked, or how was it different in terms of the cause that involved and whatnot. So these things are compiled and that is what is applied in your next set of learning. So this cycle continues and this is what is the innovation cycle. And this is very important that it continues on and on, and it does not stop. To have such an environment where these innovation cycles continue, you need to make it a part of your culture. It does not happen like one-offs, it has to be part of the company’s culture to do that.

For that, I would like to quote this from Grace Hopper, which totally resonates on this theme, “The most dangerous phase in the language is, We have always done it this way.” If you all stick to saying that we have always done it this way, then there is no way we are going to innovate. We have to challenge the status quo, that’s the first step. So you need mechanisms within your company or within your group, whatever level you can operate in, to promote those creativity ideas. And how do you do that? You do that by creating a conscious environment where those ideas prop up.

So you need to have forums where you can listen to your customers, where you bring in all people from different levels, from different groups all together, and democratize the idea generation process. You talk about the problem that your customer had presented, or you talk about the problem that the company is facing and democratize the ideas. So create an innovation lab. Innovation lab is where again, you are throwing a problem space and you are having people come up with ideas, and you pick few ideas that might work and you try it out. That’s pretty much it.

In all these environments, the hierarchy of your company structure is super important. It has to be flat, but it has to be strong. It has to be flat in the sense that the participants of the innovation group or the members or the employees should feel very safe, courageous, and should not worry about what would happen and things like that. So it should be such a safe environment for them. We need to enable the environment to be experimental, but it should be highly disciplined. When I say highly disciplined, it means that you should have proper focus on the scope of what you’re trying to achieve and the metrics that will be measured as part of that.

It’s quite successful in the companies that I’ve worked on when the reward structure is very much aligned to these innovation impacts that you make. So it naturally encourages and motivates the members when your reward structure is aligned to that. And also, it’s important that we provide the training and the tools required, especially in the technology area, so they can learn the new technologies. Like now gen AI is a thing and everybody would like to learn it, so provide the training for that. And always encourage collaboration. So it is not one kind of role, it has to be collaborative across multiple teams.

The last, it’s important to learn from these failures and treat them as opportunities, and also very important to have some fun when you do all these things. So with that, I would like to leave this whole session with the simple innovation framework that you all remember, especially on this winter month of December, FROST. So let’s remember this FROST. FROST is nothing but being focused, so the innovation group should be focused on what they’re trying to do. It should be regular. It should not be like, “Oh, I have a escalation today. I have an emergency today, so I cannot do it.” It should happen at a regular cadence. It can be once a week or once a release or once a month, whatever makes sense for your organization. It has to be on a regular cadence.

It has to be open, like it said. You do not have to specify this is how we should be done, it’s more open. Just take the problem, the ideas flow, and you will try and implement it. Safe. Everyone should feel safe, and people should be ready to accept their mistakes and learn from it. They should be ready to take risks. It’s a trusting environment. Trust each other kind of an environment.

Angie Chang: [Inaudible 00:18:11] sorry.

Anusha Dharmalingam: The most important thing, the output of this whole thing should be tangible. It should be tangible and it should be put to use for building your product.

Angie Chang: Thank you.

Anusha Dharmalingam: I want leave with this note, that remember FROST. So you can build an innovation culture within your organization if you adopt these few techniques within the group.

Angie Chang: Thank you so much for sharing this-

Anusha Dharmalingam: And to conclude the session-

Angie Chang: We are out of time.

Anusha Dharmalingam: Thank you so much for listening to me. And this is my LinkedIn, feel free to reach out to me and I will share the-

Angie Chang: Thank you. Thank you so much.

In this session, Tanvi Shah discusses the importance of metrics in product management and focuses on the concept of North Star metrics. North Star metrics are particularly important for prioritizing features, aligning with stakeholders, and measuring personal and company success. Shah outlines a four-step process for finding the North Star metric, which involves thinking about the business, identifying audience segments, brainstorming metrics, and narrowing down based on the stage of the business.

Transcript:

Tanvi Shah: Hi, everyone. I’m going to be talking about metrics and we’ll talk about interviews and we’ll dive into one specific topic, but let me quickly thank you, Amanda, for the introduction, but I just wanted to do a quick, better introduction here where I’ve basically worked as an engineer starting off at NetApp and then I transitioned into product management into the B2C world. Worked at a number of tech companies, both small and big, but being in B2C has helped me being a lot more metrics focused. I’ve learned a lot on the job as otherwise do. Personally, I’m also a mom. I have a six-year-old son. I also am a trained Indian classical dancer and I love reading books on the side. You might see me binging on two, three books at the same time. That’s a little bit about me.

Before we dive deep, I want you all to keep this in mind. Whatever I share today is like a toolbox. Use whatever you need, tweak it as you need it. It’s not gospel truth, you can change it. Don’t forget the big picture. When we talk about metrics, we get so deep into it, but we forget what the big picture is, so don’t forget that. Do a lot of mock practices when you’re thinking about interviews and prepping for interviews, and then run through more examples of interviews and compare it against the real world and quarterly statements of big public companies to understand what metrics they’re following. At the end of it, it’s really fun to understand metrics and to track them, so have fun while you’re doing this. It’s really important to keep the fun part of it here too.

All right, why do we need metrics? Let’s start there and then as we keep going through, I’ll ask a few more questions. Please interact in the comments. It helps me understand if some of this content makes sense or if not, we can diverge a little bit here. So why do we need metrics, quickly? We want to make decisions. We want to make projections. We want to have quarterly reports. We want to understand how an AB test works out. We want to understand what is the opportunity analysis for any feature that we’re trying to do, and that’s where metrics comes into play. There are three types of metrics, interviews generally. We talk about North Star metrics, there is a trade-off metric conversation or a diagnosis question. Diagnosis questions are not as much used these days, but I’ve still seen a few. North Star and trade-off, really big topics. North Star, there is another variation called dashboard. It’s kind of treated with the North Star here, so that’s why I’ve clubbed it together.

Today we’ll talk only about North Star because we just have a very limited amount of time. Why is a North Star metric important? Why do you think we should all care about it? Basically for three things. One, it helps us prioritize as PMs which feature is more important in the roadmap. It also helps align with stakeholders who might be talking about different metrics, and then you align metrics against company metrics and it ultimately helps succeed as a person because your performance review goals are definitely tied to this, and as a company it definitely helps them succeed.

Now, how do we find the North Star metric? It’s a four-step process. The first step is going to be thinking about the business. We’ll run through a mock question and we’ll go through the answers, but the first half, first piece here is thinking about the business. The second half of it is thinking about audience segments. These two then go into the third part, which is thinking about broadly brainstorming the metrics for the business and the audience, and then the last one, the fourth point, is to narrow it down based on where the business is at, in which stage is it, and we’ll go through all of them now.

Let’s run through this. What is the North Star metric for Airbnb? I think an example explains it better than simply giving ideas and how to do it in framework. For Airbnb, let’s start with what kind of business is Airbnb? In this case, the first thing that we want to start thinking about is the different types of services Airbnb offers or different business lines that it has, and the second one is it a B2B company or a B2C service? Again, they have business lines and what are the services here? With that, if you can take a stab at it, can you answer in the comments, what kind of services does Airbnb provide and what type of business models are these? This will help us dive into the next part. Add in the comments if you can answer the different types of services, types of business models.

Perfect. I see B2C. Yes, travel. Great. I’m seeing amazing things. One more thing to remember here is Airbnb has two major business lines. They do have the whole rental side of it. They also have experiences and that’s something that came up new. Now for each of these business lines, Airbnb travel is a B2C as some of you mentioned here, and Airbnb experiences both B2B and B2C because they do work with small businesses that are providing services. Now with that in mind, let’s talk about audience segments. How do we think about audience segments? Let’s run through a quick example.

For an audience segment example, let’s think about Amazon. For Amazon, there are three major types of audiences. We have the end consumers, there is the shopkeeper of B2B of business, and then there’s also the delivery person that’s involved in the Amazon side of things. [inaudible 00:06:22] is just an example to showcase the difference audience segment. What we are trying to do here is to then bring strong metrics. Who are the audience segments at this time for Airbnb? Can you answer in the comments based on the different business lines that we talked about? I’ll give a minute here. Who could be the audience segments for Airbnb Travel experience? Yes. Travelers host services. Yes, that’s correct. For experiences, who are the audience segments? Yes, consumers. Yes, yes, all of that is correct. I’m seeing good, but when we think about these different types of experiences, it’s necessary divided out.

For renters, there’s renters, there is the host, and for experiences, there are the experience seekers, the end users, there are hosts, and also there could be guides for physical tools or maybe there’s an in-between party that’s helped manage these services. This gives you the broad picture and now we go broad. Let’s try to find metrics for each business line and for each audience type. What we do is basically we use something like the Heart or the AARM metrics framework that’s out there to actually think about metrics for each of these audience segments. I’m going to pause here for a minute and ask again for the interaction in the comments. What are the metrics for each service type that we talked about? We have the Airbnb travel experience and the Airbnb experiences. What could the metrics be? If we go back to thinking about acquisition, engagement, monetization? What kind of metrics can we start thinking about for travel and for experiences?

Can some of you add it in the comments and then I can show what I came up with when I was doing it? Yes, number of bookings in a month, number of renters, hosts. Yes, very good. We also have, don’t forget, the Airbnb experiences. We want to ensure we are thinking of metrics for both service lines for the different audience segments. We have hosts on the platform, number of nights booked, and also visitors who are coming back. We have talked about metrics on all angles, also there’s revenue. Then on the experiences side, we are thinking about the number of experiences booked, revenue from these experiences, number of hosts, number of visitors. Again, all of the things that we did on the audience segment and the business side comes in here when we start brainstorming metrics.

Now, if we have a list of metrics, the next thing is to narrow it down to get to the North Star metric. What do you use to narrow down metrics now? We basically use stage of the product to define the company goals and to help us narrow down metrics. This is a rough framework where early stage companies are more about acquisition, product market fit goes into engagement. Growth is about user segment acquisition, engaging existing users, monetizing, and then expansion and maturity. Now, let’s think about the two business lines we talked about, Airbnb travel and Airbnb experiences. What kind of stage are they at? Can you answer in the comments again if you think Airbnb travel is growth or expansion or product market fit at this time? Yes, Airbnb travel is definitely mature. What about Airbnb experiences? Is it in the growth stage? Is it in the expansion stage? Is it in the product market stage? Yes. Airbnb experience is actually in the growth stage at this point. The comments are right on.

Mature phase, you think about monetization, retention as metrics. For growth phase for Airbnb experience, you’re thinking about engagement, monetization. Now this helps us narrow down from that bigger list of metrics to get to the North Star metric for Airbnb. Without giving away too much here, I wanted to basically take one beat here to really understand if we get the North Star. For Airbnb travel, who do you think is the North Star? We talked about a number of metrics here. Out of this, which is the North Star metric for Airbnb rentals or Airbnb experiences? Can anyone add it in the comments? Not sure would be one or two metrics at this point of time. Basically I know Priya asked what is the question? The question is trying to understand which is this one metric that really defines what should the company really aspire for?

Airbnb rentals, yes, nights booked is actually a good one. This is what it comes up with ultimately. Nights booked and revenue are actually the two metrics that they look at and they actually report this in all of the quarterly reports and Airbnb experiences is experiences booked revenue from experiences. Yes, the growth and maturity stage metrics look really similar, but I’m sure back in Airbnb they’re looking at a few deeper level metrics, but when we’re asked to report metrics, which is at a very high level, what is the North Star metric that I have to worry about as an Airbnb PM for renters or for Airbnb experiences or what is the CO looking at? These are the metrics they look at and they report on that. Basically this is a Q3 2022 readout of their quarterly report. Airbnb rentals and experiences, they reported 100 million nights booked and experiences booked 29% revenue year over year.

Now how are you going to use North Star metrics? You are going to use North Star metric as a PM most of the times to really prioritize your roadmap. But this exercise comes up a lot in the interviews and this is one of the frameworks of basically using it to get to your North Star metric in going broad and then narrowing down. I want to end it with saying that if you are interested in more metrics, if you’re interested in understanding more about trade-off and the others, well, definitely there is this … I think there’s a survey after this. Please express interest. But apart from that, these are the resources that you should look at. Lean analytics has been a Bible for all of my metrics things. I go back to it every time I interview.

I’ve heard about Gopractice.io. I have used a little bit of it. This is great for practicing and also don’t forget to mock interview with others, especially a lot of these execution interviews happen that way. That’s pretty much it. It was a quick rundown of metrics, generally takes longer, but if you have any other questions, please let me know. Amanda, I think we are right on time.

Amanda Beaty: Yes, you did great. Thank you. It’s a lot of interaction there. It looks like the audience enjoyed your talk. Thank you so much. Thanks to everybody for joining us and we’ll see you in the next session. Thank you so much.

Stephanie J. Neill discusses high-performing product teams and scaling effective product leadership. She emphasizes the importance of creating a high-performance culture and outlines the key elements of a high-performing product team, including clarity of purpose, psychological safety, and effective processes. Neill also highlights the importance of team composition, incentives, and managing underperformers.

Transcript:

Stephanie J. Neill: Thanks for having me Angie. And hi everybody. Wish I could see you. So I’m Stephanie and I’m here to talk to you today about high-performing product teams as well as how to scale effective product leadership. I do want to say before I get started, I lost my voice inconveniently today, so if my voice is cutting out, it’s not your computer, it’s just me. But yeah, hopefully it stays strong. Alrighty. So a quick intro. I’ve been doing product basically my entire career and close to I guess two decades now, I’ve been leading high-performing product teams. I’ve worked across a number of big tech conglomerates generally on e-commerce or content marketplace type sites as well as platforms and services. So the internal guts, all the fun stuff across federal government as well as private sector. I tend to enjoy leading smaller teams, so I’d say my sweet spot’s probably around 20. Smaller teams with outsized impact working on a mission-critical endeavor that helps vulnerable populations.

I guess the last part is probably the most important to me. I really want to feel like I’m having impact on people who need it. And then personally, I’ve lived all over the world as the child of a diplomat does, moving every couple of years. So I think it’s probably pretty small on the slide, but you can see a lot of the little blue dots just littered around. I’m also a Enneagram three for those in the know, which is basically very success-oriented and driven, but I guess always wanting to feel that I’m bringing value in everything that I do, which I think is probably a universal trait, but they ascribe it to the number threes. I’m also a Pisces, so sensitive and confused, I guess. Two fish swimming in different directions. And then I don’t have this on here, but I’m a big I little D on the DISC assessment, which means I’m basically a megaphone.

I love to amplify people and ideas and concepts that align with what I believe. And then I’m also a ENTJ, a commander personality, so very focused on getting shit done. And yeah, in general, I love these little personality tests and I actually see them as great tools for teams, to be honest, to compare and talk about themselves because one, it creates self-awareness, but it also creates shared awareness across the teams of why certain people might behave certain ways or might think certain ways. Of course the disclaimer is these are pseudoscience, so it’s really just a fun thing, but I think it really helps with culture and getting to know each other and all that. So that’s part of why I wanted to share with you today.

So a lot of you have probably seen this adage, I’m sure it’s been across the internet forever, but people plus process equals performance. I very much subscribe to this philosophy, and as I’ve mentioned, I’ve been leading teams for far longer than I’ve actually been shipping products, like hands-on product. And as many of you probably know from experience, when you move into management, you get further and further from a lot of the aspects of PM that initially drew you to it and gave you life. So there’s no more exhilaration of a launch day when you know you’re accountable for success or failure. There’s no more knowing every detail of how things work or what your customers need. You have to rely on others for that and you should, because no one person can keep all of this in their brain. And there’s no more deep camaraderie that comes with the pain and challenge of shipping a product with your building partners.

So this can be a huge mindset shift for people who move up into higher leadership. How I flipped that in order to keep my personal product power source strong was really to continue to apply product thinking at a more detailed level by thinking of my team as a product portfolio, like not the actual products they were managing. Of course, practically speaking, that is a portfolio, but thinking about the actual people as products in my portfolio. And so really looking at them and understanding what’s their vision for themselves and how does this company, how does this role, how do I fit into that and what does success in their life look like? What are they really trying to accomplish and achieve? Where are they’re trying to go? And then working with them to really put together a plan. So to me, each team member is a product and their success is ultimately my success, which is ultimately the company’s success. So this is sort of like the cynical take on servant leadership, but you’d be surprised at how easily you can manage your career and yourself like a product as well.

So, but today we’re going to talk about high-performing product teams. So one disclaimer I do want to give is that when I say product teams, I don’t actually just mean PM at all by any stretch and you can apply these principles to that, but I’m actually more thinking of the people who are accountable for building the right product in the right way at the right time for the right users. So not just the PMs, but really the triad of tech leads and UX designers as well as all the service-oriented groups who help make product launches a major success. So I’ll start with a definition. What does good look like for a high-performing product team? So I’ll state the obvious. A high-performing product team is one that accomplishes the outcome that they set out to achieve, assuming it was the right outcome, but I don’t actually see that as sufficient.

You can death march a team to success, they can achieve success or people can have a really great time together, but be chasing, I guess, the wrong outcome or not even moving any needles toward it. So I really see it more as a high performing team is one that accomplishes the outcome they set out to achieve, but they have fun doing it and they want to keep doing it together. And that last part is important. I’m going to double click on that in a sec. So that’s effectively the outcome or the output. You could measure it as an output. That you want them driving measurable impact to success metrics that matter to the business, and you want them aligned on values while also having high trust. So we talked about this people plus process equals performance construct. That is great and it does work well for a team, but it’s not necessarily scalable.

The way to make it scalable is to actually create that culture. So you’re not really creating a high performance team. It’s not what you’re really seeking to create. It’s really you’re seeking to create a high performance culture because culture will become self-sustaining, and that is how you can scale it. It’ll monitor people. It won’t be just you sort of having to look and check every box and make sure that every corner of your earth is tidy and perfect. There will be people within the culture who will do that. My favorite definition of culture that I’ve ever heard, I think about it all the time. Culture is the worst behavior that a leader is willing to tolerate. And I believe that so wholeheartedly. If you let infighting happen, if you let bad incentives be built, if you let people be rude to each other, it will detract from your goal of having high performing teams that are sustainable.

And I would be remiss if I didn’t point out the fact that it’s obvious to all I think, product is a journey. It’s not about shipping one big success and then like woo-hoo we’re done. So it’s easy for I think a team to get together and stay focused on a specific goal and make something happen, but it’s not sustainable over time and product needs to be sustainable. So it’s about consistently delivering value to your customers, having fun while you’re doing it, and you can measure the success in terms of team retention I think is a clear one. Team learning velocity, which to me is really, it’s really about how fast are we validating insights by shipping, so shipping does really matter. And then ultimately, what’s the impact to key metrics over time?

So I tried to sort these into the people, process sort of performance framework there, but the conditions that you as a leader at the highest level need to set, it’s really clarity. Clarity on outcomes. What does it mean to perform? What are the results that actually matter to the business and to our customers and how does that work in concert? So as the leader, you have to set clear expectations of what good looks like and how we know if we’ve achieved it or if we’re achieving it, but also the guardrails of how do we know we’re not achieving it? The team needs that structure, that top-down structure to be able to work backward from. And then you do need, I shorthanded this as Maslow’s hierarchy of needs, but you do need to start there. It is incredibly, like psychological safety for me is probably the most important factor for a team in order for them to really bring their all to bear and for everyone to trust that their special expertise and we’ll complement each other, we will find a way to complement each other.

And actually there’s a book, Culture Code. I forget who wrote it, but I really liked the way he described sort of a low trust culture versus high trust. Low trust is like everyone is like alone, scared guard dog barking at social threats, which creates interpersonal conflict and just all sorts of noise versus a high trust culture, which is basically a pack of wolves hunting down a shared goal together and winning together. And then lastly, but honestly probably most important because these are your feedback mechanisms of whether what you’re doing is even working. You need to have the right guardrails and mechanisms to take the guesswork as well as the busy work out of the repeated activities that lead to success.

So I’m going to talk a little bit more about each of these. Sorry for so much text on a slide. Hopefully it’s useful if we can share these slides. But talking about the leadership expectations, the clarity of expectations, there’s really three dimensions to shared purpose, right? So you need a vision, you need success metrics and you need guardrail metrics. That’s still important, but taken together, that’s purpose. Right. And in order to really institute shared purpose, you need the clarity, you need shared clarity of that purpose. People all need to understand. I often see these pithy sort of like vision statements or strategy statements, and those are good, but they can be interpreted many ways. So you need to ensure that there’s a shared understanding, a shared clarity against that purpose.

You also need to make sure that there’s actual alignment to that purpose because some people might not understand it very well, but just frankly disagree. And in many cases people can be kind of passive, I guess, aggressive against that. And it can keep people from rowing in the same direction with all their might. Some people might be just coasting on the oars or even digging into the water. And then that leads to empowerment and that empowerment of the team assigned, the accountable team to go after that shared purpose as hard as they can. So these are the three sort of management dimensions or leadership dimensions that I think are really important for you as the leader. And then of course, yeah, from a people perspective. So attending to Maslow’s hierarchy of needs. Again, psychological safety is super, super important.

I really don’t believe that you can accomplish great things for very long when that is lacking. I also think as a piece of that, we need to create space for more voices. So I often observe on teams, there are certain personalities that are very comfortable speaking up and sharing their opinion or sharing their thoughts or sharing their disagreements. But then there’s often many more that are not comfortable doing that. And so I take a lot of time with the teams and I instill this in my leaders as well to pause. I will oftentimes ask a question in meetings, and then I will sit there for an uncomfortably long period of time just looking at the team and smiling until someone is uncomfortable enough to speak up. Or for folks that I know are more introverted or less comfortable speaking up, I’ll ask them a very specific question.

I’ll be like, Hey, Fred, blah, blah, blah, because I was thinking, and then I’ll talk for about 30 seconds intentionally to buy them time to process or give them something to key off of. And then I repeat the question anyway, Fred, I’d really love to hear your opinion on X. So just those little tricks, like it creates a warmth I think, for people and a welcomingness that lets them bring their best to the table. And then being really explicit about your values, even writing them down honestly. What are your values? And then working with the team to develop team values, which I believe is great as a shared exercise because it makes everybody really think deeply about what they care about in terms of delivering value to this audience and solving the types of problems that they’re here to solve.

And then I also want to talk a bit about, I should highlight a bit about team composition. For people, it really matters. So I strongly believe in the triad, so PM, UX, and tech lead as the core components of the builder team. And I feel when you are missing any one of those, you’re not going to get to the right outcomes. You also want to guard against ratios. So if the ratios are off, for instance, if the PM to engineering ratio is more than one to 10, 10 is like a max, right? They call it a two pizza team at Amazon. If you go beyond that, which I also see too often, the PMs get so stretched thin that it becomes this feed the beast mentality where they’re just, they’re not thinking about the right work, they’re just thinking about getting engineers work. And that can, again, take away from the team being able to target and hit their outcomes that they want to and achieve their outcomes.

Of course, bad incentives, that’s like the quickest way down the wrong path. So you have to be careful. I know in some companies I’ve worked for, paths to promotion can sometimes create engineer or create an environment where engineers will seek out really specific types of work rather than doing maybe the less sexy, important work. So just making sure, like that’s just an example, but bad incentives are everywhere. So making sure that you’re really thinking about how are you incentivizing the team and how are you reinforcing it with your feedback mechanisms? What do you praise? What do you recognize? The people that you promote, what are the traits that they exhibit? And then I’d be remiss if I didn’t talk about making sure that you’re actively managing underperformers and thinking about the mix of people you have. If you have too many type A personalities, if they’re going to be like beta fish in one pot, you need them in their separate. So you need to think about who are the people that I’m bringing together and am I setting them up for success?

Woo. That was more than I expected to say on that one, but I just, I really care about people, I guess. And then as far as process goes, you need accountability checkpoints. So opportunity assessment, like make sure the team is coming to you and speaking with you. Oh shoot, I’m already up. Is speaking with you. Make sure co-escalation paths are clear and not fraught with terror. Make sure there’s an emphasis on learning and make sure that product teams are really doing proper stakeholder management and they’re communicating internally as well as getting information internally. Oh, and you have to make it insanely easy for them to access customers. That’s the last one I’ll say. And I look forward to seeing you all again.

Angie Chang: Thank you, Stephanie. Everyone connect with her LinkedIn and we’ll hop to our next session now. Thank you so much.

Stephanie J. Neill: Thanks everyone.

In this session, Savita Kini discusses the emerging area of enterprise consumerization and the impact of AI interventions in both enterprise and consumer settings. Kini highlights the three layers of transformation happening in AI product management (PM) roles in the enterprise, and discusses the opportunities and challenges in leveraging AI in the enterprise, including the need to balance personalization with privacy concerns.

Transcript:

Savita Kini: Hello. Thank you everybody for joining and good morning, good afternoon, good evening, wherever you are. I’m going to talk a little bit about a new emerging area around enterprise consumerization, and there is also AI interventions that are happening both in enterprise and consumer. So there are three layers of transformation that’s happening to the AI PM roles. And in the enterprise, how that’s changing, along with the consumerization of the enterprise. So there’s couple of themes and I’m trying to go through it quickly.

Okay, so what is really enterprise consumerization or the enterprise consumer? I think one of the things that happened over the last decade with the consumer apps is all of us who work in enterprise have expected that same kind of personalization of experience; like how we use app, how we do our performance reviews, how we file our expenses in the enterprise. How do we collaborate with our colleagues? That trend was already happening, even before the pandemic started. And then the pandemic happened and all of us worked from home. We were extremely reliant on the network, on the collaboration, talking to our colleagues via chat. And over the last two, three years, really that whole trend that began before the pandemic only got accelerated.

And then what happened? We had the whole LLM and generative AI explosion, and we are now getting into a whole new generation of enterprise SaaS where we as enterprise users and you, me, all of us, right? We want that same simplicity, the delightfulness, the creativity, the intelligence, the personalization. All of this experience that we see in the consumer domain, we want that in our enterprise experience, but not at the cost of losing the privacy: privacy of our customer data, privacy of our employee data. So there is a very unique transformation that’s happening, and I’m going to speak to it from the perspective of an AI PM in the enterprise.

I have a couple of little nuggets of transformation data that some of the research firms have been talking about. So like I said, hybrid work is here to stay. The Future of Work Research from IDC has put forth some very interesting data points around how our offices are transforming in the post hybrid, post pandemic era, because workplaces are becoming interesting watering holes. We are not going into work for … And this is true much more in the IT sector and since this forum is of women in tech, I will speak to it from the IT sector. It does not maybe apply to education or healthcare or retail. I think I’ll touch upon it a little bit later in my presentation.

But specifically we are going into work more to collaborate with our colleagues. We expect our workplaces to be, again, something of a draw, but not for our regular work, not for our regular mundane jobs. We are going in to collaborate. We are going in because we want to ideate, we want to create. And how do we augment that experience? A lot of companies are spending, according to IDC, over trillion dollars just in 2023 to redesign those workplaces. Now that’s the physical, but how do you create that same 10x better experience when you’re working remotely? And I think those two different trends are kind of colliding.

Now, let me just go specifically into what’s happening in the enterprise. Now this transition of AI in the enterprise actually started before the current generative AI efforts, and so there was speech recognition. I mean we all know about Alexa, Siri, and so on. But there were voice assistants already in the video conferencing space. There were computer vision models in the video conferencing space as well. That’s some of the experience that I come from, so I can speak to it. But what’s happening with the natural language-based model explosions is that that whole transformation is only becoming even more pronounced.

And there’s a huge opportunity. I think a lot of the AI talk with ChatGPT and so on, you talk about all the new opportunity to create your own video, write your own storyline script. That’s still consumer, but how does that change how we work on a day-to-day basis? What productivity gains are likely to happen? And there’s a lot of prediction. You can see everything. Like I looked at it, I was looking at some of the numbers. They’re changing anything from 130, 155 to 200 billion dollars by 2030, and that’s like a huge explosion of investment.

So where are these investments really going? They are going in different categories around AI infrastructure, AI chip sets, and neural accelerators. How they fit into the enterprise infrastructure; software, which is again, enterprise software. And I talked a little bit about the video conferencing space, and the collaboration space is another one with the large language models that we are seeing.

So it’s a gigantic opportunity. And how are we all prepared for capturing that transition and making impact? I think those are the key themes here, as for AI product managers in the enterprise.

Just a quick note that this transition, again, did not start today. It was already happening. There were machine learning models being used to optimize IT for robotic process automation and manufacturing in healthcare, in pharma, lots of different places where there were smaller models and innovations happening. What deep learning is transforming is in sort of the cybersecurity space, further optimization of enterprise infrastructure, sales, and marketing. So that’s where we are seeing some of the newer more game changing innovations.

Again, just to touch upon some of the industries where generative AI is accelerating that trend, you’ll see a lot of innovations in legal services, consulting, consumer and retail. How we personalize the experience for end customers, for example, in retail. Personalized healthcare. You’re going to see a lot of this kind of innovation in the next decade, which is just kind of starting. We are in the infancy zone as some of these viability of some of these products and business models gets fleshed out. So we are still on the hype curve. We have to get to this mainstream, what you say, viable business models, viable use cases, viable experiences. Because remember back to the original premise, enterprise is different from consumer because of just the data privacy concerns. And I’m going to go a little deeper into that in the next couple of slides.

So where are some of the innovations like I talked about? So manufacturing, supply-chain, you’ll start see some of the automation that was already started, but how to predict and make that even more informed and more intelligent.

Where the enterprises have the biggest advantage, which is lacking in consumer, is really the data. If you think about consumer, like let’s say take Google example, or Alexa; they rely a lot on our data, what we have produced. Even ChatGPT for example, there are huge concerns about copyright violations. That ChatGPT is trained on content of the writers and it has not credited them for their contributions, right? It’s just using that data, crawling the network and internet, and just using it to train the models. And that’s not okay.

In the enterprise, however, we are sitting on treasure trove of data from users coming to our website, who’s coming, what are they buying? There’s so much information across the customer journey that sometimes today sales is not able to make informed decisions. What should I upsell? What should I do better? HR, recruiting, there are so many of these interventions that are possible. One of the data points, for example, that I was reading about Copilot is that it has increased 30% productivity for developers. Our hiring practices, how we [inaudible 00:10:26] candidates, how we interview, how we train our interviewers, how can we do that better to make the hiring process simpler, more ethical, and unbiased.

AI can actually help us. There is a lot of talk about how AI has influenced bad hiring practices because of the data, but the other flip side can also be true. It can help us in detecting our own prejudices and biases. I think that’s where some of the interesting ways in which AI can help us do better, is what I think are some of the interesting interventions.

Anyway. So the big advantage for enterprise is that they have treasure trove of business data, which can be capitalized hugely to create very customized experiences for both internal employees as well as their end customers.

I want to show a quick video here about just an example of how we are doing it in Webex. Hopefully this will play through.

Narrator: In today’s fast-paced world, collaboration is key. Bring teams together effortlessly with real-time communication, no matter where they are. From home to office, or across the world. AI powered interactions break down barriers and make virtual collaboration immersive. Integrated meetings, messaging, calling, and events give you the tools needed to reach a global audience. Easily manage from a single place for uninterrupted productivity. Experience the power of seamless collaboration with the Webex suite.

Savita Kini: Okay, so now let me talk about the gory details. I presented a nice view of what the opportunity is out there when it comes to collaboration, business workflows, sales and marketing, healthcare. But what’s unique and different about the enterprise use cases is enterprises serve two stakeholders, ideally speaking. It’s the customers and then employees. Employees help us build the best products to serve our end customers. Right? I mean, that’s the bottom line. If you have good employees, good culture, you create the best customer experiences. And so when you think of enterprise apps particularly, it would be for one or the other stakeholder primarily.

Now, the second thing that I want to highlight that I have learned over the last five years of dealing with AI in the enterprise, is the issue around data governance and privacy. Unlike in consumer where you can get away by doing things like ChatGPT, just crawl the internet and release something, in enterprise we can’t do that. Because we are governed by stricter laws, our customers expect. We sell to both public sector and private sector. Like for example, if you sell into the federal government, you have to go through specific certifications. If you’re selling into healthcare, you are going through a lot of healthcare related regulatory compliance and certifications.

And so there are very strict governance policies that enterprise software and hardware and infrastructure vendors have to adhere to. And so that flows into how the apps have to be developed when we create these experiences for the enterprise use cases.

The other question is training data. So if you are building an app and you are building it for an enterprise, how do you acquire the data? If I’m sitting, I don’t have the data of a large bank. I might not have exposure to the conversations that they have internally in their meetings. How do I create a summarization using an LLM? Those are very interesting challenges that are unique in the enterprise space. So you’ll see a lot more of large enterprises actually building their own AI tools and experiences. So the opportunity for AI PM in the enterprise is both from an external vendor, but also internally in large enterprises. You’ll see AI PMs coming in to actually help with their own internal business workflow and optimizations.

There are restrictions to using third party and open source tools as well. Like at Cisco, we have very strict guidelines and tools and processes as we build our products on what third party or open source tools we can use. Then finally, the complexity of AI models, how they are deployed, transparency and aligning to public and private sector concerns.

Finally, in closing, let me just say, the big transformation here for AI product managers is not only do they have to do the enterprise PM role, but also there is this whole challenge of balancing personalization versus privacy when it comes to AI models. Because unlike in the consumer, in the enterprise, we have to disclose what we are doing with our models and what models are built into our features.

I know I’m running out of time, but-

Amanda Beaty: Yeah, I’m sorry. We do have a hard stop.

Savita Kini: Anyway, so those are the key takeaways. Final words, there’s enormous opportunity, but high ambiguity and chance of failures. Because finally, AI models are statistical models. The role and the concern and the focus for AI PMs will be how we bring the productivity gains through AI and deliver a more personalized and creative experience for the enterprise consumer. And it’s an interesting and challenging, but very hugely impactful opportunity in the enterprise.

So I’m happy to take questions offline. Please feel free to connect with me offline. And thank you for listening.

Amanda Beaty: Thank you so much. And thanks everybody for joining us. We’ll see you in the next session.

In this session, Anna Fuller discusses the important factors to consider before leaving a company, such as personal finances, family obligations, health, and legal status. Throughout the talk, Anna provides practical advice and recommends resources for further learning.

Transcript:

Anna Fuller: Awesome. Thank you so much, Angie. I’m excited to be here with you all. And while I get my slides up, I’m just going to publish a quick question so I can see what type of folks we have in the audience. So, I’m curious. I’m going to be talking today about leaving big tech to found your own company. And I just want to know of those of you in the audience who’s thinking about it, who is actively acting on it, and who is just here because they’re curious. So, if everybody could go ahead and vote, I’ll give you guys a few seconds. Okay. All right, so, we have a lot of folks who are in the thinking about it and curious stages, which is great.

All right, so I’m going to go ahead and hide this poll. You can keep answering, I think, throughout this session. But let me bring up my slides. All right, great. So today I’m going to be talking about starting a company, and specifically, how to start a company when you are leaving another company. So, who am I? I’m Anna. I’m a two-time founder. I’m currently working on a company called Halo, which is weekly flash deals for new moms. If you yourself are a mom or if you have mom friends, please head on over and sign up. I used to be a product manager at Google. And previously, I have a lot of startup experience, mostly on the product side and mostly in e-commerce and consumer.

Okay, so today we’re going to talk a little bit about that all-important question, when is the right time to leave your company? We’re going to go through the basics or what I call the important stuff, the meat of creating that new startup, so idea, traction, team, and funding. If we have time, we’ll go through some of the tactical stuff, but I’m going to leave them behind for you on slides so you can always come back. It’s basically the logistics of setting up a business and all the different software tools that I recommend. And likewise, if we have time, we’ll take some questions at the end.

Okay, why am I here today doing this? So it’s been shown that female-founded startups actually return 2-1/2 times the amount of revenue of others, but still, unicorns only have 14% female founders, only 6.2% of the CEOs in the S&P 500 are women, and under 3% of VC funding goes to women. These numbers are terrible. We need to do something about it. The thing that I would like to do to try and help enable you guys to come out and make these numbers better is share some of the hard-won early lessons from starting a company so that you can hopefully shortcut that process and get there even faster.

All right, let’s dive right in. So, first up. When is the right time to leave the security of your company, your current company, and start your new company? Okay, let’s think through a few things. What do you need to have in place before you leave? There is one thing on this list that everybody needs to have in place. It’s fully required. The others are going to depend on you. So, the stable personal foundation or your personal runway, this everybody needs to have locked into place. What do I mean when I say personal runway? Let’s look at a couple of different things. Think about your finances. What is your monthly spend right now? How long can you make it, how many months can you make it with the savings that you’ve accumulated, and how comfortable are you with burning down some of your personal savings?

Think about your family. Do you have any obligations right now? Are you caring for parents? Do you have young children? Are you thinking about starting a family? You can do a startup with all of those things. I have a toddler, and I’m doing a startup. But for you, you may want to just understand what that looks like and when you’ll be most comfortable to branch out on your own. So, health, make sure you’re in a good spot. Startups are really hard. They take a big toll both physically and mentally. So, just make sure you’re in a good spot with health. And then, lastly, consider your current legal status. So, if there are some folks who are here that are being sponsored or on visas, just think through those implications and make sure that you’re set up for success when you do decide to leave.

Okay, so we’ve got that one locked down. The reason I call it your personal runway is it dictates how much time you have after you leave your company before you need to have your startup providing that stability. And this depends on the length of your runway and also your own risk tolerance. So if you have more runway and a greater risk tolerance, you’re out the door. Go ahead and leave now, and you have time to figure it out. But many of us have a shorter runway. We want to check off a few more of the below before we actually leave our company. And I have the asterisk here. Don’t forget to check about your moonlighting policy. A lot of big tech companies will let you work on your own side projects, but you just need to disclose it. So, just read the fine print there and make sure you’re on the right side.

So think about these next four things, idea, traction, team, and funding, really, as things that we’re going to progressively de-risk. And you may choose to leave your company before any of them are de-risked, or you may choose to wait and leave until they’re largely de-risked, but it’s a gradual process. Okay, let’s talk about this. This is the important stuff. This is the meat of building your business. The idea, this is something that occupies an out sized mindshare. I think before you actually start a company, you have this idea of the founder, and they have this brilliant idea that comes to fruition, and they bring it to market, and everybody loves it. But that is a total myth.

In fact, I think too much emphasis on the idea is actually a bit dangerous because the reality is that as you get in and start working on your startup, the idea is going to change, and you don’t want to hold onto that too tightly. What you really want to focus on is the problem that you’re solving for your customers and how you can best solve that. And that’s likely going to change quite a bit.

So when you look at this handy little chart of this wood carving, at the end, you can see a fully formed statue. We do not want to start there. Don’t assume that your idea is what you’re going to ultimately build. Instead, start on the left side where you see this progressive, this outline of a wooden block that’s going to progress through the stages. So, how do we progress it? It takes a lot of time and effort. And to be honest, this is work that you’ll be doing over the course of your startup.

So, start with a problem area. How do you choose a problem area? Ask yourself a couple of questions. What are you the world expert in? It doesn’t have to be mind-blowing or groundbreaking. It’s just what is your expertise. If you were to start a consulting business tomorrow, what would people actually pay you money to do? Because odds are, if you were to solve that problem for them, there are some problems that you could solve for other people in that domain. So, think about that. Also, think about what are problems and frictions that you encounter in your day-to-day life. And then, lastly, make sure you pick a domain that you’re comfortable spending the next 10 to 15 years in because startups are long, and you don’t want to be bored. You want to be passionate about what you’re working on.

So in the next step, you’re going to navigate what we call the idea maze. So, really, what this is, it’s just understanding what the array of solutions that are already out there in existence are, and how are they solving the problem? How are they not solving the problem? You want to go out and talk to people in your space. You can talk to experts. You can talk to potential customers. You can talk to competitors. Just understand the lay of the land. And then start on your customer discovery. And this is where we start this lifelong journey of a startup because you’re always doing customer discovery. You want to get out and talk to your customer.

So refine your solution, move on, test prototypes, refine again, build your MVP, refine again, go to market, refine again. It’s just a constant loop of refining until you’re solving more and more of your customers’ problems, and what you end up with could look very similar to what you thought it might look like or it could look totally different. So, in startups, we have a phrase called pivoting. Sometimes, you just need to read the market and do a hard pivot, and you’ll find some other problem to solve. A book that’s super helpful as you think about customer discovery and idea validation is called The Mom Test. Go out and get it. It takes about four hours to read, and it’s going to really demystify that whole process for you.

All right, so traction. There’s really this continuum from your idea through validation and traction. But traction similar to the idea has this sort of mythology about it. It’s like, “Well, do you have any traction yet?” So, let’s break it down a little more. Traction itself is too vague. Let’s think about a couple of things. Why do you want to see traction? Is it because you want to de-risk the idea for yourself further, or do you want some validation that this is the right solution to work on? Is it because you want to get investors? Is it because you want to start getting revenue to fund your operations? All of these are valid things. But it’s important to understand why you want to see traction because we’re going to use that to understand what to measure and how much traction you need.

So, in terms of what’s important to measure, this depends totally on the startup that you’re building, but there’s a couple of key metrics that we always tend to go back to. One is users or customers. I differentiate the two because users are just coming back and using your product, maybe not paying directly. Customers are people who are paying you directly. We also want to look at maybe average contract value or gross merchandise volume if you’re a marketplace, anything that indicates that you have money that is either coming in or about to come in the door.

And then how much do you need? This is a really difficult question to answer. You need to set reasonable milestones based on your goals. And one thing to keep in mind is that this hockey stick growth is just not going to happen right away. So, try to prevent getting into the trap of thinking, “Oh, I’m just going to wait to bring on my team,” or “I’m going to wait to fundraise until I launch because then it’s going to be through the roof.” That’s probably not going to happen. So, set measurable, realistic milestones for yourself. I get asked a lot, and I think about this a lot as a founder, “How much traction is enough for fundraising?” And here I’m specifically talking about VC-backed companies.

This is not a straightforward answer, unfortunately, but there are some tactics you can take to figure out what the right answer is for you. So, first off, ask other founders. If you are in a B2C space versus if you’re in a B2B space, the metrics for raising that first round of capital are going to be much different. So find other founders in your space who have maybe just raised their first round and go and ask them. Ask them what the process was like. Ask them what their strategy was and what their metrics were like. Read funding announcements. TechCrunch is going to publish articles about companies in your domain who have just gotten funding. Read those to understand what their current status is. I would say there are two things to keep in mind here. One, founders like to paint a rosy picture, so take what you read with a grain of salt. And two, funding announcements can sometimes come months after the actual round happens, so they may have progressed a little bit further in that time.

And then, lastly, talk to investors. Reach out to investors in your domain. Let them know that you’re not fundraising right now but that you want to get their feedback and you want to get their input on your idea. Then, talk through with them what do they like to see before they write their first check for a company. Other ways to validate your idea. Like I mentioned, it’s sort of a continuum. You want to go through discovery, validation, and, eventually, real traction. So, discovery, there’s a number of ways you can actually figure out whether your solution is going to resonate without even writing a line of code. You can do surveys. You can do customer interviews for that qualitative data. Or you can set up what we call a fake door test where you put up a landing page with a call to action.

Let’s say, “Buy this product.” And you just measure how many people click “Buy the product.” And, of course, you don’t have the product yet. So they land on a page that just explains, “Hey, we are in the process of making this. We want to make something excellent for you. Enter your email address, and you’ll be the first to know.” So, we’ve gauged their demand because they were ready to buy even before we have the product itself. When you move a little bit further into the validation stage, you may want to run a beta with a small, trusted group of customers who will give you good feedback. You may stand up a wait list, people who enter their email address. That’s sort of giving up something of value to them in exchange for getting your product in return. That’s a good way to gauge demand.

Or if you’re in the B2B space, maybe it’s letters of intent. These are just letters that a company will write, or you can send it to them. They’ll sign, saying, “Hey, when you build this thing, I want to try it out,” and it’s not binding. So, it’s low risk for them that you can gather these to understand who your initial design partners will be and show some demand. Then lastly, real traction, customers and growth. There’s nothing that there’s no other secret sauce there. You just need to bring in people who are getting value from what you built. A really helpful book here is also called Traction. I highly recommend this.

Okay, moving on to team. So the traditional sort of triad that you might hear of in a co-founder relationship would be this hacker, hustler, hipster combo. And really, what that means is you need somebody to build the product, you need somebody to sell it, and you need somebody to design the user experience. Now, you don’t need a separate person for each of these things. Maybe you as the founder have two of the skills, and maybe your co-founder has two of the skills. So between each other, you have this overlapping skill set. But you do roughly need to know who’s going to build it, who is going to sell it, and who’s going to be responsible for that user journey.

So, to think about here, what are your own strengths? What do you bring to the table, and where do you need help? I would recommend getting very creative here because think scrappy. You’re going to leave your big company where you have a person to work on every small thing, and they’re a subject matter expert in that, and you need to be comfortable with wearing multiple hats. You might need to be comfortable with understanding what is good enough for now. Maybe you are a product manager, but you can do some wireframes, and so that is good enough to translate to a developer to have them help you build the initial product. Think about how you can get to your end goal and validate your ideas as quickly as possible.

Then, also, contractors and interns are a lifesaver. So, if you have no idea how to market something, you can look to hire an intern marketer who will get a lot of value from helping you in addition to you getting value from them helping. Do you need a co-founder? This is a big question, and I think this presentation is too short to answer it here. But I will just point out that co-founders are great ways to bring on a supplementary skill set for equity in the early days. They have skin in the game. And you don’t have a lot of resources, so you need people to contribute who have real skin in the game.

Also, founding a startup is an emotional roller coaster. So, having somebody there who can go through those ups and downs with you is essential. But make sure, on the other hand, that you’re not just bringing on a co-founder that you don’t know very well because it really is akin to a marriage. It’s going to be a relationship that you need to foster over the next 10 to 15 years of your life, so think carefully there.

Funding. Okay, a very important question. Why do we care about funding first off? So it’s not to say, “Oh, I raised this round of capital.” It’s because you need to fund your own company operations. You need to fund your own personal runway. Eventually, you’re going to run out of your savings. You need something to supplement that. So, let’s think through a couple of different types of company funding. So, we have broken down into three. Venture-funded, this is your typical equity VC-backed company. The exit event here is usually an IPO or a sale. And what to think about for this category is you’re going to go through multiple rounds of funding, so the founders will get diluted. You have to have an exit before the founder realizes all that value. So, you’re not going to have these yearly cash flows. There’s going to be a lot of pressure for growth, and billion-dollar outcomes are expected.

It’s really this big swing all-or-nothing thing. You’re also probably going to need to be full-time before you get that first check. Boot-strapped, usually this is the situation where a founder will put in initial capital, and then it really could go either way after that. It could be venture-funded. It could be revenue-funded. A lifestyle company, think here, these are if you’re going to start a physical business like a restaurant or if you’re starting a passion project like a blog. It’s any type of business that is not the sort of typical venture-backed Delaware C-Corp style business. This is usually revenue or potentially debt-funded. The business pays for itself.

A couple of key characteristics. It will have regular cash flows. So you can get this to a point where it’s actually paying you to operate the business, and you can start taking cash out of it. It’s usually more aligned with a more sustainable growth pattern, and you can start this as a side hustle.

Angie Chang: Thank you, Anna.

Anna Fuller: Okay, how are you-

Angie Chang: [Inaudible 00:17:42] time. We’re out of time.

Anna Fuller: Okay. Great. Okay, well-

Angie Chang: Thank you so much for your-

Anna Fuller: Yep, no problem. If you guys want to download the slides, feel free to grab this deck. And I’m always available for more questions.

Angie Chang: Great. And you can always hit replay in this Airmate software and they can just re-watch this session and get this QR code.

Anna Fuller: Awesome.

Angie Chang: Thank you so much.

Anna Fuller: [inaudible 00:18:08].

Nas Hajia emphasizes the importance of thinking like an architect and developing customized security solutions that fit the specific needs of each company. Hajia explains the elements of a risk statement and the importance of including them in a risk-based decision model. She also outlines the steps of the solution development lifecycle, including understanding the business model, defining the problem statement, and developing, testing, and deploying the solution.

Transcript:

Nas Hajia: Thanks for the introduction and thanks everyone for joining this session. My name is Nas Hajia, and we’re going to talk about Next Gen Solutions: Leveraging AI for Smarter Security Risk Decisions today. Amanda gave a great introduction to me, I’ll just go over some of my background highlights as well here. I’m a security architect. What I do is both on the product architecture side of things and enterprise architecture topics, both of them. I started my career when I was in Canada. I started from research and then I made my way to very different industries from telecommunications to financial. I moved to data industries where the business model was basically just acquiring and selling data again. About two years ago, I moved to the Bay Area. That’s where I’m living right now, and I work for an awesome company called Autodesk. I have my contact information there as well.

I’ll promise you that the QR code is safe. If you’re a trusting person, you can use that to find my LinkedIn page. I’m not sure if we have enough time for a Q&A today, but if we don’t, feel free to contact me on LinkedIn for any questions you may have on this topic or anything else. I want to start by this sentence because this is what will basically form the whole presentation today. I mentioned that I’ve worked in a couple of different industries and the one thing that’s always been true there is that you may perceive that a certain problem is the same across different companies, but very rarely can you actually implement the exact same solution in different companies in different situations and scenarios and try to get the same level of success or satisfactory measurements there.

What I want to do today is, instead of telling you what solution you should implement to get the best results, I want to walk you through how to think like an architect and basically be able to come up with your own solution that fits your own needs in your company and based on the problems that you are facing today very specifically. Let’s start from a quick poll. You should be able to access the poll if you go to that fourth or fifth option in your screen right now. We’re talking about risk-based decision making, so we, of course, need to know what a risk statement is first of all.

Take a look at these options. I can read them out loud while you’re going through them and we’ll see what the results are going to be. These are all statements that you will hear in many different security discussions in a company, by the way. None of these are fictional. I’ve come across these all the time. The first one says a company’s EDR solution are not scalable. Only 60% of managed devices can be set to have appropriately configured EDR agent. Second one is, we anticipate increased incidents in the next six months and we should proactively prepare by amplifying your security controls.

Option number three says, the absence of robust email-based data loss prevention controls increases the likelihood of disclosure of sensitive information due to human error, thereby compromising data confidentiality and exposing the organization to potential financial penalties. Option four is leveraging sophisticated email-based email security tools will minimize the risk of successful phishing attacks. The majority of you answered correct. The right one is option number three, which is obviously the longest answer as well. The reason for that is, each risk statement, there are three elements that it definitely needs to have. First element is the event. That’s basically the conditions that must be present for the risk to occur. The second one is outcome. What will happen when the conditions are present? The third one is the impact. What harm will it do and why should we even be concerned about that?

Optionally, you can add likelihood, risk factor, security controls to add to some of the other ones as well. This one is very important and we’re going to come back to this, and the reason for that is these are all the things that we need to include in our risk-based decision model. I’ve come across quite a different use cases for this topic specifically and also other ones as well. Unfortunately, not all the use cases that are proposed, they end up being successful. Some of them are rejected by either architecture boards or leadership. A lot of them could have actually had a better journey if they had gone through a solution development lifecycle phases here as well, because it’s one thing when you put together a POC or you want to test something in a sandbox environment and it’s a completely different thing when you want to implement it in an enterprise environment where, let’s be honest, the conditions may not be ideal.

The dependencies are a lot and you may have to consider a lot of integration prerequisites as well. The steps that we should start from before you start anything is you have to understand your business model, the existing architecture and processes very well. Then you want to move on to defining your problem statement. Next step is defining your solution goal and requirements. Step number three could be optional, but I usually like to start from this. This isn’t designing the solution itself. It’s a very high-level ideation of what the solution could look like, and then, based off of that, the next step you want to define your dependencies and prerequisites. Then finally, I’ve combined a couple of different steps here as well. You want to develop, train, validate, measure, improve, and deploy.

If you look online and if you try to do your own research or see probably different variations of the solution development lifecycle here, but what holds true is that they’re all going to be some sort of variation. This is basically, the difference will be based on how you want to differentiate between the steps. This is, I think, a good model to go forward with our talk today.

Let’s start from the defining our problem segment. We’re talking about risk-based decision making, so we have to start with the five Ws. That’s basically saying, the who part of the problem statement is any personnel who’s responsible and accountable for making and implementing security impacting decisions. That could either be leaders, that could be developers or operations team, people who are implementing patches or people who are making strategic decisions. The what of it is the fact that there are too many risk factors and they’re not standardized and they apply to a wide range of assets. It makes it difficult for manual process to pick up all of this. The win of it is actually, I would say that’s all the time because any decision we’re making could have security impacts as well.

The where of it could be is actually all of the assets on a company. Every single asset in a company could either have risk impacts or could be a risk source itself. The why of it is that, well, we want to make decisions that are aligned to business goals and at the same time we want to make sure that we’re efficiently using our resources. For some use cases, it’s not feasible to spend, for example, a whole year to make a decision and in some other cases, even spending one hour to make a decision is too long. We need to be able to find the right balance there. If we want to look at, let’s start going through the very high-level, what the solution would looks like. This is simple for a reason. You may want to think that’s what we’re trying to do here.

We have some sort of black box, some sort of operation and data analysis happens in it, you feed it some input, it spits out a certain output. The problem solves, you’re making your decision. Except that there’s a couple of problems here. One is that, we have no idea what the brain of the operation, the data analysis part of it, the security data analysis part of it looks like. The input gathering itself is actually quite difficult. It’s not just one. There are way too many and we’re taking a look at that in the next slide as well.

There’s a problem with the output of it as well. A problem statement, as you saw, it’s a bit too generic. It’s a very big problem and there are many, many different elements to it. Unless we break it down, we won’t be able to say, what are the outfits that we expect here? Another thing to keep in mind here, that the output can never be the decision itself. The output is only a decision aid for the human users and the human decision makers to utilize and facilitate the process.

Let’s take a look at some potential inputs that you want to look at. I’ll give a disclaimer here that this list is by no means exhaustive. It’s just intended as a way to show you what are some of the difficulties that we’re dealing with. The inputs come from all of the company’s assets, whether that’s people, process, technology, devices, data, all the events, millions and millions of events that are happening daily in a company, and it could include a combination of business and security data that is measurable. Something like what’s the customer usage rates and the number of users in a company, what’s the business reputation? What does the market look like? And then the security side, it could be anything from office type of security data like, let’s say, sensors and IOT devices to access events and security events.

We have application firewalls and code access and new vulnerabilities on anything that comes up. All of these have different resources, all of these have different formats, they’re coming from very different places. The idea is to combine all of this and give it to one centric model. In practice, that’s just incredibly difficult. If we want to consider think again to our solution development lifecycle phase, the next step was defining solution characteristics and requirements. There are some solution requirements that are applicable to most solutions anyway, regardless of whether it’s for risk-based decision-making or not.

For example, it needs to be functional, it needs to be learnable, maintainable. For this topic specifically, you have to be careful that your solution needs to be repeatable as in the exact same inputs should always give you the exact same output. It has to be interoperable, reusable, and there are a couple of other use cases that you want to consider. Those would be very specific to your use case. For example, you want to consider what is the amount of human interaction and decision-making power that you want to give to your human users. Or, what is the balance of accuracy versus performance of a solution that you’re comfortable with? Depending on who your main audience is or if you want to have different audiences, you may want to think about separating your security and business criticality criteria as well.

We looked at the problem, we looked at what our solution should have. Now, let’s talk about why we think AI could help with some of this. The big problem is that I mentioned with the input, there’s just too many security data. For difficult-to-find data, human-created processes and manual processes, even if they’re automated actually or human users, they will have blind spots. But at a well-implemented and well-designed AI model will not have that. That’s definitely one of the good things that we have to keep in mind here.

Second one is that your AI model would understand your entity state and posture. It could definitely, that’s a big could, but it could definitely speed up your process as well, and it could help you with training the people for any growing sophisticated attacks and the dynamic environment that the security risk sources are. You can, of course, also use AI bots for real-time adaptive security, and if you wanted to do that by using your human personnel, that would’ve been much, much more difficult and time-consuming.

We’ve talked about solution criteria and we talked about designing our solution for a very specific goal. We need to be able to test it and measure its success as well. First thing that’s important is to make sure that your solution is passing your repeatability test, and then you want to make sure that you’re testing it for very different threat scenarios as well. Then, some of the other thing you want to measure and test against, they could be based on the functionality of your model and you have very specific AI and ML measurements like accuracy scores and precision scores and learning rates that you might want to consider. But also, you want to think about what the goal of your solution was. For example, if your goal was to increase the average time that it would take your company and your personnel or the risk owners to remediate critical risk, that would be a good measure to consider here as well.

Now, let’s go through very quickly some of the lessons learned I’ve come across in my experience. First one, this could definitely, when it comes to your solution, make it or break it as a knowledge base and what your existing data architecture and governance looks like. If you don’t have that, then a hypothetical use case that works in sandbox would not actually be able to satisfy your requirements in an enterprise environment. You want to be very clear about what your impact criteria and definition is. Again, how much business criteria do you want to have in your model and how businessy do you want the output to be?

Next one is that you’re creating a solution for a very specific use case, and even if the measurements are great in terms of functionality, if it’s not actually being used in the environment and if it’s not something that there’s interest in your organization to adopt, that’s going to be a big problem. There’s going to be a question of why did we start with this anyway? Have those discussions early on as well.

Another thing is, if you’re starting everything from the scratch and you’re building it in-house, the time investment that goes into it could be so much that would skew your return on investment numbers. The time that you’re putting in and the time is one of the very important resources for any organization, that has to match what you’re getting out of your solution as well. Next one is talent and skillset. You can have a great use case, but if you don’t have the right talent and skillset in your organization to both design and implement it and also maintain it afterwards, then it will stay in ideation phase. Last but not least, like any other initiative, stakeholder management and expectation management becomes very important.

When you’re working with your stakeholders, you have to be able to convince them that the solution you’re proposing is actually satisfying a need of theirs. Then, when you’re talking with them and you’re negotiating with them, you could have stakeholders at both end of the spectrum, so you could have ones that think AI will solve all of your problems, and then you can have stakeholders that are extremely fearful of AI and think that it’s the greatest evil, for lack of a better word, and you need to be able to find the balance and to convince them and negotiate them and persuade them of the benefits of your solution as well.

Couple of considerations for any AI model, for your AI-based solution model is that AI hallucinations I think they will happen, so be aware of them and put appropriate guard lays in place. If they do happen, patching them is still an open problem and we don’t really know how to address them. Next one is with using AI, you may have new threat vectors. You may want to consider adding detective and preventive controls and including some very specific language and line items around that in your center response playbook, privacy and data protection, and then using how your data is being used and accessed and what the data lineage looks like is, of course, very important whether you’re using a third party model or if you’re building your own.

Now, the next one, very important to keep in mind is that there is no solution that is a hundred percent accurate. There is no solution that you can say never fails. Instead of designing for a fail-safe solution, you may want to think about designing solutions that are safe to fail and make sure that you have contingencies in place. Just to continue very quickly, since no solution is fail-safe, if you put your solutions on a critical path, there’s always a chance that they will lead and result to incidents, so be aware of that and have those discussions early on and see whether you’re comfortable with that or not. The other consideration is that there are some very good third-party solutions out there and it’s not always as easy as just going in and adopting them. Sometimes the cost that comes with them and sometimes they’re not very available, it makes it difficult for them to be immediately usable for your organization.

We talked about what problem we want to solve and how AI goes into that, but if we want to do it the other way and we want to say we want to use AI, we want to see where to use that in the security environment, we have a couple of good options that I don’t think we’ll actually have time to go through, so I’ll quickly go to the next slides. The most important thing in this presentation that I want you all to go away with are these points that I’ve put here. When you’re designing a solution, always be very careful with all the steps. Don’t skip anything. Always play the devil’s advocate and make sure you are considering and actively thinking about what parts of your solution could fail and document them. Make sure that you’re aware of the fact that there isn’t one solution that can solve or your problems. Plan for all your dependencies. Okay.

Amanda Beaty: We have a hard stop. I’m sorry.

Nas Hajia: Okay.

Amanda Beaty: All right. Thanks everybody for joining us. Thank you so much, Nas, and we’ll see you all in the next session.

Nas Hajia: Thank you.

Olivia Ouyang discusses how to leverage design thinking in the job search process. She shares her personal experience of being laid off and finding a new job within two months using this strategy. Design thinking is a five-step process that involves understanding the goal, identifying the problem, being creative in finding solutions, testing those solutions, and iterating as necessary.

Transcript:

Olivia Ouyang: Sorry about the long description about myself. Actually, this is highly relevant to what I’m going to share today, about how, not just for any designer, actually for anyone who is job searching, right now, how we can actually leverage some of the strategy I will share next, to help you to land on your next job that you really want.

Hi, again. I’m Olivia, and as you probably can catch some of the keywords that inform intro, in the past four or five years, I have a course of different startup experience, various sizes and industry. And a lot of people ask me, “Oh. How can you just jump from, for instance, a consumer banking app, selling to an enterprise global trade, whatever, logistic kind of platform? How do you make that connection, and make the dots?” And this is what exactly I’m going to talk to you about.

Unfortunately this also, I went through that personally this year in January. I got sudden layoff, and the situation I face because of some of the visa situation that I have to really quickly figure out. This has ended up, I landed out my current job within two months, also leveraging the strategy and process I will introduce next.

With this talk, I’m going to introduce the process. It is called, Design Thinking. I am sure a lot of people, especially in tech, are pretty familiar with this. But how do you actually apply this in terms of a job searching progress, is something we’ll be interested. And after all of that, I will want to share some personal perspective in terms of, if you unfortunately have to go through the layoff process, and have a stressful timeline, how do you manage the burnout, and some tips that really can help you to go through this process.

Next, I’m going to introduce this amazing problem-solving process called Design Thinking, again. It’s a five-step process. Basically, I would say, not only I use it every day in my work, and also I use it on my daily life as well. For anything, small and big, you really can just start it from understanding, “Okay. What is the goal here? What do I really want to achieve?”

And next up, rather than jumping to a solution right away, you want to understand, “Okay. What is the problem here? I can’t get it.” And trying to understand much of the context and constraints and everything. And then, you can start being really creative, saying like, “Oh. With all of the resources I have, with all the tools I can access, what are some of the way I can potentially try to get my goal?” And then, with a bunch of the lists, and then, you can just try out and test. If you are an engineer, I guess a more familiar kind of way to think it is, whenever you are writing a code snippet, you’re definitely going to test it, and then get the compile results, so that you will know what to try next. It’s basically like that.

In terms of for a real job searching process, first off, you want to understand, “Okay. What’s your job searching goal?” The key message I want to emphasize here, the job searching is not a one-way problem. It’s actually a two-way communication, which is a match and fit process. When you’re being evaluating as a candidate, you are also need to evaluating the other party as a company. It’s important to understand, what do you really want, as a job seeker, and then, think about what you can offer to actually really stand out from other candidates.

With that in mind, you can set a goal. Remember, we need to understand the goal first. You can make a list of the next job you want to land on. For instance, I wanted to go smaller or bigger company, specific industry you’re interested in, team culture, the management style, so on. Everyone has a different priority list in this goal list. And you can always adjust as you go through this looping process, and evaluate as you go in the job searching process.

Next step is actually to think about, “Okay. This is what you want, and what I can offer.” This is not just simply list every single thing you have ever done in the resume, but really, really think through what that experience mean in terms of the skillset. You can quickly match that up to any job description you are seeing on the job posting. That will make the hiring manager easily to understand why you are a good fit.

Talk about my own experience, because I have an engineering background. Before, when I was searching for the new grad first design role ever, I need to tell a really good story in terms of how my engineering background will actually help me design. I highlighted, strong logical thinking skills, and my capabilities of communicate really effectively with other engineers because of my background.

Similarly, when I’m looking for the other job in enterprise space, and no experience in designing for platforms and so on, a really complex domain, I actually highlighted my startup experience, and how I deal with all of the ambiguity in such a small team, to show the potential I can deal with unknown spaces, even though this is a really complex and unknown space I have never worked in the past.

Next up is to define. We talk about your needs, and also what you can offer. Next step is the reality check, really, to understand, what is the job market really like? This is an example that I use personally to evaluate myself as a designer. And this is actually coming from one of the company I really, really like. They really being transparent and show this metric of skill metric on their website, and show, “Oh. This is what we use to evaluate everyone in the team, and we are looking for someone who can compensate all of the skillset to this team.” I do this to myself to evaluate how good match I am in terms of to the team, if I really, really like to join it.

And next up is, you can do another exercise is to make another list that, “There’s some of my strong suit of the skills, and also some of the skills I want to develop.” And also, what is your career interest, and things you want to try out next. And the more you can find a match in this inner cycle between you and the target job post, the better fit for both of you, of course. This is a good exercise you can be evaluating and iterating.

From Step Three to Step Five is really a loop. You need to constantly adjust, and then, take that feedback, and then, tweak a little bit, and try the other thing throughout the entire process. And it works through the entire job searching funnel, as well. I can give you some example from really beginning.

First job, you just starting to say, “Hey, I wanted to land an internship or a new grad job, or my next role in a certain timeline.” And you need to plan it backward, to say, “Okay. I initially think maybe I need a couple of two weeks for my portfolio, my resume, and then some times for technical, and then, some times prepare behavior, so you have that rough timeline.

You can idea it in terms of, what are some way I can be creative to plan that and evaluate that. And then, later on, when you put into practice, to build all of the things, you will know the gap between what you thought you could do versus the reality. In recording your process, you can come back, and then tweak it, and then to make a more realistic timeline for yourself.

And same thing apply when you get stuck in a particular interview round, you can do that, too, for saying, “Oh. I have my draft, and then I start sending out a lot of application, but why I’m not getting any screening calls?” You might want you to think of, “Okay. There are some other things I probably can try other than direct applications. I might also want you to be really active on the social media, LinkedIn,” and then share about your experience, your skills to catch more potential hiring manager’s eyes.

And when you’re doing a referral, is there some unique message you can help to try and send it out? And also, linking outreach, what are some different message you can play around that will help you to get more feedback, or some private talent pool that you can join, and reach out to those VC funds, talent pool share, across the portfolio of the companies. Those are some of the creative way that could get you more exposure. I’m just making example here.

Of course, you can use the result of, are you getting more calls, and what is the feedback for the outreach, to evaluate how effective different things you have tried. And the other example, similar to the last one is, for example, you are getting stuck with the technical interview. Other than just waiting for the next opportunity, of course, you can try, “I can probably just record my own session if no one’s practicing with me.” You’ll be amazed by how many thing you can cut that you get stuck, just by recording yourself. And also, you can help others to prepare interviews and then learn from how other peoples respond. Anything they can improve and then reflect upon on yourself, as well.

Of course, depends on the actual interview, and even mock interview feedback, you can quickly iterate on, what I can improve next. And along this way, actually because AI tools are so blooming right now, I remember when I got laid off personally, almost a year ago, I’m basically still using the old methods of writing everything on pen and paper, and putting all together things in the Figma tool, which is the public tool designer all use, and of course a lot of documentation, and all of that. But today, really with ChatGPT, and a lot of other AI tool that can help you mock interview, I will not say they replace your own role as a job seeker to make your own material, but they can really help you quickly to put up the first draft, so you don’t get stuck in the blank canvas struggle, I would say. But at the bottom line here, job searching problem is still finding the right match. This tool doesn’t change the fact that this is the problem you’re going to solve.

And lastly, I want to share some of my personal notes. And this is more callback to early on, when I’m saying I have some visa sponsorship needs, and also with a tight timeline. I would say, transparency really goes a long way. I personally learned the hard lesson. If you ever invest in particular situation where you have a clear bottom line about, “I need specific support. I really can’t go onsite every day. And I have a compensation bar,” or whatever thing, where you, in a really later station interview, you need a deadline, and then close it very fast, and so on. Really, really be transparent on the first call with HR. Because if you are being transparent, then they can help you the best as they could, and also it save both of you time. It’s much, much way better than you’re at a later offer stage, and then figure out there is something you can’t agree on in the very beginning, but you already both spend that much time to get through everything. That will be a really sad and unfortunate situation.

Next up is, of course. Talking about burnout. I also personally have that as well. And you probably often hear about people saying, “How can you give constructive feedback?” And thinking the other way around, how you can take feedback constructively. What that mean is, for saying you failed the interview today, of course you will feel sad, everyone will, no one will like the feeling like, “Oh. I just failed.” But you can take a while and accept the fact that, “I’m really sad.” Acknowledge that, “I didn’t do well today. But okay, what have I learned? Is that because I’m not prepared enough, so that I learned from today’s lesson that I needed to spend more time prepping certain problems before going to the next interview. Or I shouldn’t probably rush to schedule that interview that early, if I’m not prepared. Because it’s wasting both of our time as well.

But if really the feedback that the other party give you is, “Oh. We are really looking for the candidate that have specific skills or experience,” that you wouldn’t be able possibly get in such a short time, that is not a good match. Or if they just tell you, “Sorry. We just filled it with another candidate, upfront.” That again, is not your fault. You really need to understand what is actually the reason behind it, and then turn it to a really actionable item for you to move on, and really use every failure as a stepping stone for your next interview, and eventually get you to success.

To close that note here, I talk about how you can use design thinking, which mean you can think like a designer even you’re not, to trying to problem solve every single little thing during your job searching process, to really customize that to your own goals and needs. And also for tips here, really trying to be smart, leveraging a lot of tools to help you get started faster, so you don’t have reason or excuse anymore to procrastinate. Also, I talk about how you should be really transparent with the hiring managers, so that both parties can move on really smoothly. And lastly, how to take the failure and feedback constructively, so you can always take away and learn from every single interviews that you have, so you can perform better, and know yourself better next time.

And this is all my session, today. I know we don’t have time for Q&A session, but if you do have any follow-up question for me, you want to chat with me more, feel free to contact me here with my contact info. But with that, thank you everyone for attending my session. Really happy to be here. Thank you.

Amanda Beaty: Thank you so much, Olivia.

Thanks to everybody for attending. And we will see you in the next session.

In this session, Michelle Maraj discusses the importance of data visualization and how it can be applied to any job. She uses the example of her travel blog to demonstrate how data points can be interpreted differently depending on the context provided. Michelle also discusses various job roles in data visualization, such as dashboard developer, data analyst, and journalist, and highlights five key skills for data visualization designers: data skills, statistics, knowledge of tools, design skills, and storytelling skills.

Transcript:

Michelle Maraj: Thank you so much. And thank you everybody for choosing to join my session today. Really appreciate taking your time to join me.

Well, I’ll get into a little bit more detail about my history in the field, but before we get started, I did want to give an introduction as far as why data visualization is important and why it can really be applied to any job. And so even though there are some roles out there, I personally am in a role that does center around data visualization, I do believe there is a very cross-disciplinary skill that you can build that can be really contributional to nearly any type of career.

So thinking about why data visualization is important, an example I want to use today is my travel blog. I personally love to travel. It’s one of my favorite, I guess, hobbies, and I did have a travel blog for a few years. So if I was looking at the statistics on my blog, I noticed that in March 2023 I had 16,000 views. So when I give you this data point, this is one piece of data, one row of information, you don’t necessarily know what to do with it. Is 16,000 a lot? Is that a little? What exactly are you trying to tell me? And so even though data can be valuable, it’s not going to be actionable unless you provide additional context around that data point.

So what I’m going to do is I’m going to put it into a data visualization and show you that, compared to last month, I am seeing a slight increase in the views on my blog. So you can see that in March 2023, I had 16,000 views, compared to February, I had maybe 15,000. So this would be a great visualization to show the steady growth in my blog over time. And maybe I want to make the argument that we need to invest in more writers or we need to start making more posts because travel blogging is improving, people are getting more interest in the blog.

However, if I look a little bit further back and expand my dataset, and compare it to, say, March year over year, what I might find is that the 16,000 is really great compared to, say, 2022. But compared to March 2019, my views have actually dropped quite a bit. You know, we had the COVID pandemic and so a lot of people were not traveling as much and not really looking at travel blogs as frequently.

So there are a few different arguments you can make here. You might say that travel blogging is slowly coming back, and so, again, we can still make that investment. Or maybe the story we want to tell is we want to pivot niches. Instead of focusing on travel, because we’re not reaching our full potential here, maybe we want to go into lifestyle or cooking or things like that. And so it’s really tricky because, again, it’s the same data points, the data is correct and accurate, but the data visualization that I show tells a completely dramatically different story.

And the question is, which is the chart that I should show? So that’s why data visualization skills are so important, because it comes down to that designer, that analyst, to make these different types of decisions. Depending on what type of story that you want to tell, you can, I want to say manipulate the chart to show that. And so, again, the data is accurate, but choosing what type of data to show and when and how just completely changes your story. Again, it’s really tricky to tell what type of chart, because both would be correct in this scenario. It just depends on what you’re trying to say.

So data visualization is so important because, no matter what role or industry you’re in, you’re going to be using data and you’re going to have to communicate in some way. And so your data vis skills is really that art of figuring out what’s the right format, what’s the right data, and getting it to the right person. And so making sure that you have that right context so that your story is understood.

So today in this talk, I do want to go through my experience in the field of data vis, what jobs exist, what skills you need if you’re interested in pursuing a career in data vis, and then how to build a portfolio. So for me personally, again, my name is Michelle Maraj and I am a full-time dashboard developer. And so what that means is that my users are people within our company who are looking at data, and so I’ll put together a type of view where people can go in and really look up data that’s relevant to them, whether that’s filtering down to certain market or region or maybe pivoting the data in a way that makes sense for their role.

So in my role at Gigpro, I do Tableau dashboard development where I support teams across a variety of different departments. And so that’s marketing, sales, operations, finance. It is a startup, and so we are helping build dashboards for pretty much anybody who needs data. Before that, I was working at Lyft, again doing Tableau dashboard development, but specifically for finance. And then prior to Lyft, I was in consulting where we were building dashboards for pretty much anybody who needed it.

I did not study data visualization in school. My background is in information system. So I did have that data background, but a of the stuff that I learned was actually on the job through consulting. As I was practicing those skills, it’s something that I started developing. So I love talking about data visualization. I think it’s just such a cool field to be in.

What jobs are there in data visualization? Now, I mentioned that data vis skills are going to be helpful across a variety of roles and industries, but if you are super, super passionate about it, like I am, there are a couple of jobs where it is going to be a much larger portion. So I mentioned being a dashboard developer. So a dashboard is, again, a view where people can interact with data, and so it is sort of a user experience development type role. I personally use the tool Tableau, but there are also similar tools such as Power BI, Looker, Data Studio. It really just depends on what the company is using and what tool you’ll need, but that would be dynamic report building.

Another different type of role that is really going to use data vis skills is going to be either a data analyst or a business analyst. And so you’ll find these types of roles, again, across nearly any industry or department. So if you have an interest in, say, fashion or home goods, you can probably find an analyst role in those different niches and really drill down into data related to those. And so as an analyst, typically what you’re going to do is you’re going to be responsible for pulling data out of a system, cleaning it, manipulating it, and then presenting any findings. And so typically, with an analyst role, you are creating static visualizations that are going to go into maybe a report or a presentation.

Another area where we see a lot of data vis skills being used is with journalism. Though you might have seen, maybe, infographics online, or even a lot of articles that are starting to incorporate charts to help better communicate stories. So I think that the journalism field is one of those where it’s growing quite a bit as far as thinking about how we can incorporate data into communicating a little bit better. And so those are three main different types of roles that you can look for if you are interested in a data vis role. But again, there are lots of other jobs where you’ll probably use data vis skills in them.

So if you think that the field of data visualization is interesting, let’s say, for me personally, I grew up loving technology, loving my computer loving video games. I considered being a graphic designer. I also really like math. If you’re interested, I think that data vis is one of those fields that really just combines all of those different types of interests. And so if you are interested in pursuing a career where you’re really using data vis skills, what kind of background would you need, or what could you work on to improve?

So I think that there are really five key skills that will help you be a really strong data visualization designer. So the first skill that’s really helpful to have is data skills. So knowing SQL, understanding data vis, figuring out how to get data out of different systems, how to manipulate it for analysis is going to be really, really valuable. Because no matter what type of tools you end up needing to use to create your data visualizations, you’re probably going to have to format your data into some form so that way you can get it into the tool in a way that you can create your visuals. So having that data background is going to be really helpful.

Then you’re going to want to think about statistics. So I didn’t necessarily… I’m not a data scientist, you don’t necessarily need to know how to do regressions by hand or anything like that. But having that background, depending on the type of role that you have, can be very beneficial to create either really complicated data visualizations or really just help you summarize your data better. Because with data vis, you are summarizing data points. And so thinking back to school when we had to figure out the differences between mean, median, and mode, and trying to figure out which of those is going to be the most effective, those are the types of decisions that you will make as a data visualization designer. How you want to aggregate your information and what types of summarizations are going to be the most accurate and the most valuable.

Then you’re going to want to think about what tools you’re going to use. And so this is going to really vary across different companies. I mentioned that I’m a Tableau developer, and so Tableau is my area of expertise, and I’ve really built a lot of Tableau skills. But, if you’re interested in, say, maybe a specific company, not every company is going to use Tableau. And so what I do recommend is looking at job postings for the companies they’re interested in and seeing what types of tools they’re currently using. Because even if I go to a company that doesn’t use Tableau, and I can’t always convince them to purchase it because that’s a pretty big investment, and so making sure that you either are building skills in a tool that companies are interested and use, or making sure that you can be flexible in the skills that you’re building.

So, for example, being flexible between Excel or G Sheets is going to be really valuable because a lot of companies will use one of those. But then also thinking about, even if you are a Tableau expert, at least being familiar with maybe how Power BI works could be helpful in your job search.

Then coming back down to it, we’re looking at design skills, and so thinking about how to make your charts visually appealing. Because if you think about, say, Excel, you can create a chart in Excel and it’s going to give you some type of basic chart, but what are the chances of you leaving the chart the way it is? You’re probably going to want to make some edits to it, whether you’re cleaning it up or moving chart junk, changing the colors, things like that. And so understanding different design elements will make your charts more visually appealing.

And then finally, storytelling skills. And so it comes back down to understanding your audience and understanding how you’re going to need to communicate your data is going to be helpful so that way you can communicate it the most effective way. Some of your users might like a dashboard where they can interact and drill down, but others are going to need a presentation or a printout of the data. And so knowing how people are going to consume your information is going to be really valuable so that you can put together the best visual for them.

So how do you practice your skills if you’re interested in this field? I do think that you can read books, take courses, and then, honestly, it just comes down to practicing, as I said. So, if you’re interested in different vis books, these are some of my favorites that I would highly recommend. So across the top, Alberto Cairo is one of my favorite authors that really gets into the psychology of how we are, how we read charts, how we interact with data and art. So love, love, love all of his books.

If you are going to be creating static charts, and so whether that’s charts that you’d put in a report or presentation, Storytelling with Data by Cole Knaflic is a wonderful resource with a lot of really, really practical tips. Nathan Yau’s Data Points is another book. He’s a data journalist, and so that’s a great example of how you can use data vis to communicate stories. And then Steven Few’s book Information Dashboard Design is one of my favorites if you are building dashboards where people are expected to interact with your data and drill down. So love all of these different books.

As far as courses go, again, there’s lots of free resources online. I know that with Tableau specifically, Tableau does have their videos online and free so you can follow along. So I recommend that.

But then when it comes down to practicing, this can get tricky because you think, where do I find data? How do I practice? Coming up with the case studies can sometimes be the hardest part, so I do recommend different community driven programs such as Makeover Monday. So you can Google this, but what they do is it’s a group of volunteers that find a chart that could use some help, or, essentially, a makeover, and they’ll share the chart, share the dataset, and then it gives the community an opportunity to try and recreate the chart. And so this is really great because not only do you have data to work with, but you can also see what other people are creating and get some ideas and inspiration from that.

So creating a portfolio. So let’s say you have been practicing, you’ve been building those skills, and now you want to share your work with the world. And so, again, a portfolio is going to be really, really valuable if you are interested in pursuing a career in data visualization. And so, for me, as a Tableau dashboard developer, I like to show examples of my Tableau dashboards because that shows the employer that I’m telling the truth on my resume and I do know how to create these visualizations and I can really show off my skills. So if you are looking at a specific tool like Tableau, you can use Tableau Public where you can post your dashboards and essentially build up your portfolio that way.

You can also use social media. And so you can use LinkedIn, or there’s actually a really big data vis community on Twitter, and you can essentially build up your portfolio by posting screenshots of your work or links to your work across those different types of platforms. Also, I personally have a personal website where I’ll put my visualizations and it’s called TheChelleCurve.com. I believe there’s a link in my profile here in the chat. But, essentially, what I do is I have a blog where I will post, well conferences that I’ve been to, but also some of the work that I’ve done as far as data vis creations, and I’ll put a little bit more detail as far as either how I got the data set or why I made certain design decisions. And so I built my website on WordPress, but again, there’s a lot of great resources out there.

And if building a website’s a little bit too intimidating, that’s totally fine. You can also put together just a Word DOC or a PDF of your work, and then you can upload that to either a job application, typically there’s room to upload additional materials, or you can upload it to, say, your LinkedIn, and that way people can see your work that way.

So if you are interested in this field of data visualization, a couple of things to remember from today is that, again, data visualization is such a valuable skill to have, especially in this tech world. However, with all of the data we collect, data is only going to be helpful if you can get it into the right format in the right hands. And so if you are presenting, say, a chart to a CEO, if he’s only going to read your emails, he’s not necessarily going to visit a dashboard and spend the time to drill down, then you need to make sure that you are summarizing the data and putting it in a format that they can read.

And then, again, data vis can be valuable to any rule. Data literacy skills are going to be valuable no matter what type of industry you’re in. And so, again, highly recommend at least freshening up on those skills. But if you are interested in pursuing a career in data vis, you just continue to learn, read those resources, practice, build a portfolio, and that way you can show future employers the work that you’ve done and what you’ve been working on.

So, again, thank you so much for attending my presentation today. If you want to connect, my name is Michelle Maraj. This is my website. Thank you, Beth, for sharing the chat the link. And then I can also… Happy to connect on LinkedIn or answer any questions there. Thank you again and hope y’all have a great afternoon.

Angie Chang: Thank you, Michelle. That was a really great talk and we’re going to end the session and go to the next one. So thanks everyone.

In this session, Duaa Gettani discusses the importance of standardizing UX research and the different stages of UX research teams. Gettani explains that research can be engaged at any point in the product development cycle, from foundational to generative to evaluative research, and suggests standardizing research processes and normalizing research’s impact to incorporate research at all stages.

Transcript:

Duaa Gettani: Hi. Thank you for that lovely introduction. Thank you all for joining, and thank you all for joining a talk on standardizing UX research. I know that doesn’t sound that exciting, but I will do my best to make it exciting. So we’ll get right into things. I know we only have 20 minutes.

Just a quick intro of what I’ll be discussing. I’ll briefly introduce myself, talk about what the importance of talking to users in different stages of UX teams and how that manifests itself in different structures of UX research teams.

So brief introduction about me, I do come from an academic background. So I know a lot of folks in the UX field come from very eclectic backgrounds. There are more traditional human-computer interaction backgrounds, but I come from an academic research background. This is some of the research that I did at the Institute of Transportation Studies where I got my MS in transportation and some of the publications that I worked on and the research that I worked on there.

I talk about this as a way to reflect on how I got into UX. So I come from an academic UX background and wanted to get into a research space in tech and found UX research as a great way to compliment my experience. So just a quick overview of my career trajectory. I started off as a research coordinator at Google and then a research associate at Waymo and then a UX researcher as well as an ops manager. So I did help with the ops management at Lyft, the operations that are associated with UX research. Then my current role here as a Senior UX Researcher.

I think a lot of folks probably joining this talk may have a question as to how I made that transition from academia into UX. My answer to that is that what I did was that I started off by joining in as a contractor at these specific companies. So the companies here in the box are the ones that I joined as a contractor. I think that’s a great way to introduce yourself into the UX space, to understand if it is a value to you and something that you’re interested in continuing, and a great way to network and get to know other folks in fields that you would potentially want to grow within. So just wanted to give that quick overview of how I transitioned into UX in case it’s beneficial to anybody.

I will get right into what I wanted to speak about today. So just talking about research as a researcher. We understand that basic research is about talking to users. So within a UX space, within a product space, the crux is that talking to users and conducting user research is essential for creating products to meet user needs that are easy to use and have a greater chance of being adopted and successful. So setting up processes allows everyone to benefit from talking to a user. So to prevent that ad hoc talking to users and have a structured standardized process in connecting to users.

So the first layer of the onion is talking to the user. If we simplify it, that first layer is just talking to a user. That surface level way to learn about products is to just simply talk to a user. Why they use a product, how they use a product, who they are? Within Square, where I’m currently at, our users are sellers, so people building products for people. So this next layer is understanding a user’s needs, goals and behaviors. Digging a little deeper, within Square we learn where seller’s pain points are, what are their goals as a business and as small business owners?Then where you’re trying to get to, that crux is to understand a user’s pain points and address them, building a seller focused culture for us at Square across all functions in Square banking. Then that center, that sweet spot here is where we want to get to continue to build empathy.

So when we’re talking about research, and we’ve all probably seen this similar structure and we’re understanding at what stage should research get involved in. The short answer is research can be engaged at any point in the product development cycle. From foundational, so learning about people and their problem space to generative research where it’s prioritizing challenges to solve and focusing more on the solution space and then that evaluative research where you’re more tactical and iterative. So research is not meant to be a one-off surface, but rather part of an overall process. Ideally, we incorporate research at all of these stages, both when we know what to build and when we can have something we’re iterating on. So how can we do that? By standardizing research and having and normalizing research’s impact.

So where you want to be and where it’s okay to be. So we know that research structures in different companies can be very different, and that’s okay. So research can have impact at any stage. So I have this oversimplified structure of where you’ll find yourself as a UX researcher jumping into a company. So sometimes you’re the sole researcher, sometimes you’re in a mid-sized team, two to five researchers, and sometimes you’re a fully embedded team, and this is ideally where every researcher wants to be. There are challenges at each stage. So I’m going to cover a few of them, but mostly focus in on the mid space, where a lot of researchers end up being.

Sole researchers, there can be probably the biggest challenges with one researcher is that educating on how research works within your company and having impact to drive growth and drive the growth of the research team as well as prioritizing projects that will allow for that to happen. Then where we will focus today is that mid-size team, and that’s where I am Square banking. The challenges here is that you may or may not have an ops person to help with structuring the projects and there’s still challenges with prioritizing and balancing projects and balancing prioritizing the right projects and consulting on other projects.

So where do we start? There are two ways to begin to, at least the way I see it is, two ways to take in research. You either have a road mapping process, which you can combine both of these processes, and a research intake form, because a lot of the times priorities within the company change, projects change and you may not stick as strictly to the road map as you would hope.

So we’ll start with the research intake process and including this road mapping process. What we do, at least with Square, we align research road map with the product road map. This is yearly, quarterly. It’s not a one size fits all. It’ll depend on what stage of the product you’re in.

So with the stakeholder alignment process we do is that we meet with stakeholders. We then ask about various aspects associated with these wants, and we then align on timelines, priorities in a general way, and we input that into a road map that can be referenced for all stakeholders.

Then the other way to intake these research requests is an intake form. This is a lot of the times a way to allow ad hoc projects that may come up outside of the road mapping process. So road maps change, projects pop up, and that research isn’t aware of, and we want to make sure that research is always aware of these projects. So who fills out these forms? It could be a product manager, it could be marketing, it could be designers. So we have them fill out the background objectives, the impact that they hope to see out of this project to allow us to see if this is a project that we would find value in partaking in.

So inputs to consider when scoping a project, the purpose. So consider whether the work is tactical, low level, reactive or strategic, and where in the product cycle it is, whether the work is, again, low level reactive, or whether it is strategic. So we ask these questions, why are we doing this? Why do we want to know? What do we want to know? Where are we in the product development cycle? Then we think about company priority. Who is involved? Is this a high company priority? Is this a low company priority? Then the timeline, is this something that research can actually get involved with? Does research have the bandwidth for this? Will we have time to complete this project in time for a potential decision and to consider the amount of time and resources that this project will take away from other projects? The level of involvement. How much research support is required? Is this something that research can just support in a way of reviewing research plans or help in other ways, or is this something that requires research’s full level of involvement?

Then finally we look at the priority matrix and how does this fit into that structure of a priority matrix? When I say the priority matrix, I mean defining the involvement can be impacted by this priority matrix, which we folks have probably seen before. If the risk is high and the problem clarity is very low, that’s where we would want research to focus in on as much as possible and have research heavy. Because of such a high risk and low clarity, research has a great place to allow itself to be a part of the foundational structure of a work stream.

Then finally, when we were discussing defining the level of involvement, one way to structure this is to think about a full service model versus a guidance/partner, and finally a consulting/self-serve structure. So in a consulting capacity, in a full service way structure, we’ll start with that is the researcher takes on the project from start to beginning and takes on every aspect of the project, including the research design. Whether if there’s a vendor that needs to be involved in this creation of discussion guides, recruitment, the field work, everything. Then when you’re thinking about it from a guidance/partner perspective, you may have a partner heavily involved with the research with you, whether that’s a PM or a designer who is involved with the structure of the research plan involved with taking notes and synthesizing the presentation. It’s more of a partner capacity. Then there’s the consult slash self-serve model, and that’s where a researcher may not have bandwidth at all to take on a project and they’ll advise on the research approach and review any questions, discussion guides, test plans, and help with every aspect of the study, but not fully complete it on their own.

Then how does this work in practice? So I’m going to have a quick example here from Square. So a case study that I have here with Square, and this is a project that particularly came up in a road mapping in our road mapping process. The larger crux of the problem here was that multiple teams wanted to better understand the needs and perceptions and jobs to be done of Upmarket sellers as it pertains to Square banking. So we started off with meeting with stakeholders and differing stakeholders because this was such a topic that covers different product teams.

Then we scoped out these different aspects that we need to consider as the inputs for scoping. So this spans multiple product teams and impacts overall strategic, generic, and strategy early in the product development cycle. Then it can influence strategy. It was also a high risk and low clarity project. So it allowed me to think about this as a covered all of the higher levels of these inputs and made me want to really push for research to be involved with this project and invest a lot of research’s time and bandwidth into this project, because of it hit all of the inputs.

Then finally because of that, we decided that this particular project would be a full service research project, and this project would require the full support and complexity and all the logistics and it was worth the researcher’s time.

So the impact and approach of this research, what we ended up doing was we did desk research/a literature review, which you see here on the left. Then we did a question prioritization workshop with all of the differing stakeholders. Then finally a third step in this project was that we interviewed specific sellers who fell into that Upmarket category. Within these three different stages, we presented all of this data in a research deck and did a company share out throughout these different stages and also did a final share out as we went through with the different stages of this research.

Then just to quickly go over other practices to build out. When we’re thinking about research, we all see the value of those foundational projects, but the iterative launch evaluative research stage is also important. We just went through a foundational strategic project, but evaluative research can be just as impactful and important, can be a major contributor to justifying the increasing of headcount and moving your team towards more researchers. So small, but meaningful impact. You don’t always have to take on those strategic foundational projects for full service involvement projects.

So this is just a really quick overview of other ways to expand your research toolkit to document new processes, create those research templates and onboard research tools and vendors that you can easily access when needed.

Finally, advocating for research throughout research findings, whether that’s research decks, newsletters, and a research repository, and finally research readouts. That’s it. Gets you to that stage of making yourself that much closer to that goal of being a fully embedded researcher.

Amanda Beaty: All right. Well, thank you so much. You finished just in time, so everybody got to hear your whole talk. Thank you so much for taking the time to join us and thank you to everyone watching, and we will see you in the next session.