“Next-Gen Solutions: Leveraging AI for Smarter Security Risk Decisions”: Nas Hajia, Security Architect at Autodesk (Video + Transcript)

Nas Hajia emphasizes the importance of thinking like an architect and developing customized security solutions that fit the specific needs of each company. Hajia explains the elements of a risk statement and the importance of including them in a risk-based decision model. She also outlines the steps of the solution development lifecycle, including understanding the business model, defining the problem statement, and developing, testing, and deploying the solution.

Transcript:

Nas Hajia: Thanks for the introduction and thanks everyone for joining this session. My name is Nas Hajia, and we’re going to talk about Next Gen Solutions: Leveraging AI for Smarter Security Risk Decisions today. Amanda gave a great introduction to me, I’ll just go over some of my background highlights as well here. I’m a security architect. What I do is both on the product architecture side of things and enterprise architecture topics, both of them. I started my career when I was in Canada. I started from research and then I made my way to very different industries from telecommunications to financial. I moved to data industries where the business model was basically just acquiring and selling data again. About two years ago, I moved to the Bay Area. That’s where I’m living right now, and I work for an awesome company called Autodesk. I have my contact information there as well.

I’ll promise you that the QR code is safe. If you’re a trusting person, you can use that to find my LinkedIn page. I’m not sure if we have enough time for a Q&A today, but if we don’t, feel free to contact me on LinkedIn for any questions you may have on this topic or anything else. I want to start by this sentence because this is what will basically form the whole presentation today. I mentioned that I’ve worked in a couple of different industries and the one thing that’s always been true there is that you may perceive that a certain problem is the same across different companies, but very rarely can you actually implement the exact same solution in different companies in different situations and scenarios and try to get the same level of success or satisfactory measurements there.

What I want to do today is, instead of telling you what solution you should implement to get the best results, I want to walk you through how to think like an architect and basically be able to come up with your own solution that fits your own needs in your company and based on the problems that you are facing today very specifically. Let’s start from a quick poll. You should be able to access the poll if you go to that fourth or fifth option in your screen right now. We’re talking about risk-based decision making, so we, of course, need to know what a risk statement is first of all.

Take a look at these options. I can read them out loud while you’re going through them and we’ll see what the results are going to be. These are all statements that you will hear in many different security discussions in a company, by the way. None of these are fictional. I’ve come across these all the time. The first one says a company’s EDR solution are not scalable. Only 60% of managed devices can be set to have appropriately configured EDR agent. Second one is, we anticipate increased incidents in the next six months and we should proactively prepare by amplifying your security controls.

Option number three says, the absence of robust email-based data loss prevention controls increases the likelihood of disclosure of sensitive information due to human error, thereby compromising data confidentiality and exposing the organization to potential financial penalties. Option four is leveraging sophisticated email-based email security tools will minimize the risk of successful phishing attacks. The majority of you answered correct. The right one is option number three, which is obviously the longest answer as well. The reason for that is, each risk statement, there are three elements that it definitely needs to have. First element is the event. That’s basically the conditions that must be present for the risk to occur. The second one is outcome. What will happen when the conditions are present? The third one is the impact. What harm will it do and why should we even be concerned about that?

Optionally, you can add likelihood, risk factor, security controls to add to some of the other ones as well. This one is very important and we’re going to come back to this, and the reason for that is these are all the things that we need to include in our risk-based decision model. I’ve come across quite a different use cases for this topic specifically and also other ones as well. Unfortunately, not all the use cases that are proposed, they end up being successful. Some of them are rejected by either architecture boards or leadership. A lot of them could have actually had a better journey if they had gone through a solution development lifecycle phases here as well, because it’s one thing when you put together a POC or you want to test something in a sandbox environment and it’s a completely different thing when you want to implement it in an enterprise environment where, let’s be honest, the conditions may not be ideal.

The dependencies are a lot and you may have to consider a lot of integration prerequisites as well. The steps that we should start from before you start anything is you have to understand your business model, the existing architecture and processes very well. Then you want to move on to defining your problem statement. Next step is defining your solution goal and requirements. Step number three could be optional, but I usually like to start from this. This isn’t designing the solution itself. It’s a very high-level ideation of what the solution could look like, and then, based off of that, the next step you want to define your dependencies and prerequisites. Then finally, I’ve combined a couple of different steps here as well. You want to develop, train, validate, measure, improve, and deploy.

If you look online and if you try to do your own research or see probably different variations of the solution development lifecycle here, but what holds true is that they’re all going to be some sort of variation. This is basically, the difference will be based on how you want to differentiate between the steps. This is, I think, a good model to go forward with our talk today.

Let’s start from the defining our problem segment. We’re talking about risk-based decision making, so we have to start with the five Ws. That’s basically saying, the who part of the problem statement is any personnel who’s responsible and accountable for making and implementing security impacting decisions. That could either be leaders, that could be developers or operations team, people who are implementing patches or people who are making strategic decisions. The what of it is the fact that there are too many risk factors and they’re not standardized and they apply to a wide range of assets. It makes it difficult for manual process to pick up all of this. The win of it is actually, I would say that’s all the time because any decision we’re making could have security impacts as well.

The where of it could be is actually all of the assets on a company. Every single asset in a company could either have risk impacts or could be a risk source itself. The why of it is that, well, we want to make decisions that are aligned to business goals and at the same time we want to make sure that we’re efficiently using our resources. For some use cases, it’s not feasible to spend, for example, a whole year to make a decision and in some other cases, even spending one hour to make a decision is too long. We need to be able to find the right balance there. If we want to look at, let’s start going through the very high-level, what the solution would looks like. This is simple for a reason. You may want to think that’s what we’re trying to do here.

We have some sort of black box, some sort of operation and data analysis happens in it, you feed it some input, it spits out a certain output. The problem solves, you’re making your decision. Except that there’s a couple of problems here. One is that, we have no idea what the brain of the operation, the data analysis part of it, the security data analysis part of it looks like. The input gathering itself is actually quite difficult. It’s not just one. There are way too many and we’re taking a look at that in the next slide as well.

There’s a problem with the output of it as well. A problem statement, as you saw, it’s a bit too generic. It’s a very big problem and there are many, many different elements to it. Unless we break it down, we won’t be able to say, what are the outfits that we expect here? Another thing to keep in mind here, that the output can never be the decision itself. The output is only a decision aid for the human users and the human decision makers to utilize and facilitate the process.

Let’s take a look at some potential inputs that you want to look at. I’ll give a disclaimer here that this list is by no means exhaustive. It’s just intended as a way to show you what are some of the difficulties that we’re dealing with. The inputs come from all of the company’s assets, whether that’s people, process, technology, devices, data, all the events, millions and millions of events that are happening daily in a company, and it could include a combination of business and security data that is measurable. Something like what’s the customer usage rates and the number of users in a company, what’s the business reputation? What does the market look like? And then the security side, it could be anything from office type of security data like, let’s say, sensors and IOT devices to access events and security events.

We have application firewalls and code access and new vulnerabilities on anything that comes up. All of these have different resources, all of these have different formats, they’re coming from very different places. The idea is to combine all of this and give it to one centric model. In practice, that’s just incredibly difficult. If we want to consider think again to our solution development lifecycle phase, the next step was defining solution characteristics and requirements. There are some solution requirements that are applicable to most solutions anyway, regardless of whether it’s for risk-based decision-making or not.

For example, it needs to be functional, it needs to be learnable, maintainable. For this topic specifically, you have to be careful that your solution needs to be repeatable as in the exact same inputs should always give you the exact same output. It has to be interoperable, reusable, and there are a couple of other use cases that you want to consider. Those would be very specific to your use case. For example, you want to consider what is the amount of human interaction and decision-making power that you want to give to your human users. Or, what is the balance of accuracy versus performance of a solution that you’re comfortable with? Depending on who your main audience is or if you want to have different audiences, you may want to think about separating your security and business criticality criteria as well.

We looked at the problem, we looked at what our solution should have. Now, let’s talk about why we think AI could help with some of this. The big problem is that I mentioned with the input, there’s just too many security data. For difficult-to-find data, human-created processes and manual processes, even if they’re automated actually or human users, they will have blind spots. But at a well-implemented and well-designed AI model will not have that. That’s definitely one of the good things that we have to keep in mind here.

Second one is that your AI model would understand your entity state and posture. It could definitely, that’s a big could, but it could definitely speed up your process as well, and it could help you with training the people for any growing sophisticated attacks and the dynamic environment that the security risk sources are. You can, of course, also use AI bots for real-time adaptive security, and if you wanted to do that by using your human personnel, that would’ve been much, much more difficult and time-consuming.

We’ve talked about solution criteria and we talked about designing our solution for a very specific goal. We need to be able to test it and measure its success as well. First thing that’s important is to make sure that your solution is passing your repeatability test, and then you want to make sure that you’re testing it for very different threat scenarios as well. Then, some of the other thing you want to measure and test against, they could be based on the functionality of your model and you have very specific AI and ML measurements like accuracy scores and precision scores and learning rates that you might want to consider. But also, you want to think about what the goal of your solution was. For example, if your goal was to increase the average time that it would take your company and your personnel or the risk owners to remediate critical risk, that would be a good measure to consider here as well.

Now, let’s go through very quickly some of the lessons learned I’ve come across in my experience. First one, this could definitely, when it comes to your solution, make it or break it as a knowledge base and what your existing data architecture and governance looks like. If you don’t have that, then a hypothetical use case that works in sandbox would not actually be able to satisfy your requirements in an enterprise environment. You want to be very clear about what your impact criteria and definition is. Again, how much business criteria do you want to have in your model and how businessy do you want the output to be?

Next one is that you’re creating a solution for a very specific use case, and even if the measurements are great in terms of functionality, if it’s not actually being used in the environment and if it’s not something that there’s interest in your organization to adopt, that’s going to be a big problem. There’s going to be a question of why did we start with this anyway? Have those discussions early on as well.

Another thing is, if you’re starting everything from the scratch and you’re building it in-house, the time investment that goes into it could be so much that would skew your return on investment numbers. The time that you’re putting in and the time is one of the very important resources for any organization, that has to match what you’re getting out of your solution as well. Next one is talent and skillset. You can have a great use case, but if you don’t have the right talent and skillset in your organization to both design and implement it and also maintain it afterwards, then it will stay in ideation phase. Last but not least, like any other initiative, stakeholder management and expectation management becomes very important.

When you’re working with your stakeholders, you have to be able to convince them that the solution you’re proposing is actually satisfying a need of theirs. Then, when you’re talking with them and you’re negotiating with them, you could have stakeholders at both end of the spectrum, so you could have ones that think AI will solve all of your problems, and then you can have stakeholders that are extremely fearful of AI and think that it’s the greatest evil, for lack of a better word, and you need to be able to find the balance and to convince them and negotiate them and persuade them of the benefits of your solution as well.

Couple of considerations for any AI model, for your AI-based solution model is that AI hallucinations I think they will happen, so be aware of them and put appropriate guard lays in place. If they do happen, patching them is still an open problem and we don’t really know how to address them. Next one is with using AI, you may have new threat vectors. You may want to consider adding detective and preventive controls and including some very specific language and line items around that in your center response playbook, privacy and data protection, and then using how your data is being used and accessed and what the data lineage looks like is, of course, very important whether you’re using a third party model or if you’re building your own.

Now, the next one, very important to keep in mind is that there is no solution that is a hundred percent accurate. There is no solution that you can say never fails. Instead of designing for a fail-safe solution, you may want to think about designing solutions that are safe to fail and make sure that you have contingencies in place. Just to continue very quickly, since no solution is fail-safe, if you put your solutions on a critical path, there’s always a chance that they will lead and result to incidents, so be aware of that and have those discussions early on and see whether you’re comfortable with that or not. The other consideration is that there are some very good third-party solutions out there and it’s not always as easy as just going in and adopting them. Sometimes the cost that comes with them and sometimes they’re not very available, it makes it difficult for them to be immediately usable for your organization.

We talked about what problem we want to solve and how AI goes into that, but if we want to do it the other way and we want to say we want to use AI, we want to see where to use that in the security environment, we have a couple of good options that I don’t think we’ll actually have time to go through, so I’ll quickly go to the next slides. The most important thing in this presentation that I want you all to go away with are these points that I’ve put here. When you’re designing a solution, always be very careful with all the steps. Don’t skip anything. Always play the devil’s advocate and make sure you are considering and actively thinking about what parts of your solution could fail and document them. Make sure that you’re aware of the fact that there isn’t one solution that can solve or your problems. Plan for all your dependencies. Okay.

Amanda Beaty: We have a hard stop. I’m sorry.

Nas Hajia: Okay.

Amanda Beaty: All right. Thanks everybody for joining us. Thank you so much, Nas, and we’ll see you all in the next session.

Nas Hajia: Thank you.