About this podcast episode

Agilists are ideal candidates to run complex 🤖 AI efforts

We are thrilled to introduce you to a machine learning and artificial intelligence leader, Zorina Alliata, Sr Global Machine Learning Strategist, Amazon, Chair AI Committee at AnitaB.org, Georgetown University Adjunct Faculty.

Sharing real-world projects and stories, Zorina and Bill Raymond cover a wide array of topics, from defining AI and machine learning to building AI solutions and measuring their success.

In this podcast, you will learn the following:

✅ What machine learning and AI can do to improve your business

✅ How AI works without any technical jargon

✅ Metrics to ensure your AI efforts will be successful

🎉 The roles required to deliver AI solutions with agility successfully

Transcript

(transcripts are auto-generated, so please excuse the brevity)

[00:00:00] Intro clip

Bill Raymond: Maybe about 10 years ago or so. I remember one use case from an airline They just had all these algorithms, machine learning algorithms in a way that we call unsupervised learning, which means, I don’t know what I’m looking for, just find interesting things in the data Go and look.

Zorina Alliata: one of the findings was that people who were born on a Tuesday tended to order vegetarian dinners when they were flying the airline. So there was an insight that no human person would’ve thought of asking that question.

But that was a pattern that the machine found, the algorithms found just by looking at the data.

[00:00:36] Intro

Zorina Alliata: Welcome to the Agile in Action Podcast with Bill Raymond. Bill will explore how business disruptors are adopting agile techniques to gain a competitive advantage in this fast-paced technology driven market.

Bill Raymond: Hi and welcome to the podcast. Today, I’m joined by Zorina Alliata, senior global machine learning strategist at Amazon, a chair on the AI committee at Anitab.org and a Georgetown University Adjunct Faculty.

Hi Zorina. How are you today?

Zorina Alliata: Hi, Bill. Thank you for having me. I’m doing great.

[00:01:12] AI Transformation and Agile Leadership

Bill Raymond: Yeah, I’m really excited about our conversation today. I think this is going to be a sweeping conversation that we’re going to have a short period of time to talk about, but I’m so excited to talk about this. It’s a AI transformation and agile leadership. Before we get started, could you share a little bit about yourself?

Zorina Alliata: Sure. So, as you mentioned, I am a machine learning strategist, and I work for Amazon Web Services in the Professional Services Division. I’ve worked in the US for the last 25 years. I’ve lived here in Washington DC area. And recently have moved to Germany, to Munich, Germany, also while keeping myworking for AWS.

So, that’s my main job. I love robots. I love machines. Been passionate about AI for like, ever. So, very, very exciting to see the times now when AI is so popular again and happy to tell people all that I know about it.

[00:02:05] AI years in the making

Bill Raymond: Yeah, I mean, I think we got a glimpse of AI over the years, right? Everyone got a chance to see some of the things that AI could do. Maybe they saw it improves search, maybe they could search for "cat" on their phone and all the photos of their cats will show up. But now we’re starting to see some, if you will, more real life scenarios and it’s amazing what’s happening just so quickly.

I know that it’s been years in the making, right? So ChatGPT just didn’t come out of the woodwork. I mean, we knew it was something was baking back there and we knew that with our Amazon devices we could easily talk to them and play our music or what have you. But now I feel like it’s just kind of coming to the forefront.

But you know, I think a lot of the way we get to see it is through conversational AI, you know, I want to talk to something, whether it’s with my voice or chatting on a keyboard.

[00:02:57] AI scenarios in the consumer world

Bill Raymond: Can you share some AI scenarios, some use cases where maybe this isn’t something that we’re seeing, if you will, in our consumer world?

Zorina Alliata: Sure. Absolutely. And first I want to say, you know, the chat bots have gotten really, really good. It’s no surprise they’re so popular. I was looking for a new car a few months back and I textedan agency and we texted back and forth for like 20 minutes before I realized it’s a robot.

Bill Raymond: Oh.

Zorina Alliata: And I know how they work.

So, they got very, very good at it.

Bill Raymond: They fooled you.

Zorina Alliata: Totally fooled me. But outside of that, it is a great thing, great help for a lot of companies. I’ve been very fortunate to work on really interesting projects that are real life, absolute real life applications. For example, if I go to the supermarket and I want to buy some strawberries, that is one of the companies that we’ve helped and we told them when to plant the strawberries to obtain the optimum crops. Especially during the pandemic, forecasting and predicting inventory levels and products and what to do became extremely important because everything changed so much. So machine learning stepped in and really helped fill those gaps.

Other interesting things, we’ve worked in healthcare quite a bit. When patients are in the hospital, what is the best course of action customized to that patient, based on everything we know about them, right? So my treatment would be totally different than your treatment based on our, you know, age and whatever other data they know about us.

What is the best way to optimize that outcome? And machine learning is supremely great at optimizing outcomes. So that was very helpful, understanding the hospital just data in general from all the sensors that they have there. Now they have all these intelligent beds and so forth. All of that data, somebody’s got to make sense of it, right?

So use machine learning to understand that, to make sure the nurses and the doctors are, I think, most optimum place they could be to really help. So really interesting applications in healthcare, in drug manufacturing, finding defects on production lines and making sure that the chemical reactions areoptimized as well.

So a lot of applications for machine learning there. And then for more, you know, if I pass by the street, I see a construction company just down the street here from me, that is also a company that we have helped with just finding defects in their buildings. As they build, they send out drones to take videos and the machinecomputer vision programs analyze that and if they see any type of defect, they immediately report it, saving quite a bit of money and making everybody’s life safer who works there.

So a lot of applications like that, that we use in all over the… you know, agriculture, commerce, supply chain and everywhere else in things that we do day by day.

[00:05:49] The need for custom software development

Bill Raymond: That’s really amazing. How many of these scenarios that you’re talking about need custom software development versus there’s just already some sort of AI out there, some products that they can use?

Zorina Alliata: That is a very good question. So there are already, for example, I’m going to speak from Amazon point of view, which I’m the most familiar with. We haveso-called AI services, and these are pre-trained models that for computer vision that I was talking about, called for example Amazon Rekognition, Comprehend, Textract, all of these already exist, so you just call them like you would call an API and it already knows and it translates, it transcribes, it understands sentiment, it does all those things automatically.

So they did, we did manage to automate quite a bit of, you know, some of the work that is going in machine learning. But also there is a lot of custom work every time, and that is because the data that we all have is so different, right? So the source of data, it’s always, you have to understand what is there.

And then everybody’s business problem is different. You know, what I’m optimizing for, it’s not what another person wants to optimize for. I want to optimize for speed, maybe they want to optimize for cost, right? So the variables are different in the algorithms and the data is different. So then a lot of these become custom engagement as well.

Bill Raymond: Oh, that’s really interesting. Thank you for that. Yeah, I think it’s so neat that you, you send a drone up to take some photos and then there’s this giant infrastructure back behind all of that, that just does all those computations and provides the results.

Zorina Alliata: In seconds.

Bill Raymond: Yeah.

Zorina Alliata: In seconds. Yes. amazing.

[00:07:27] Machine learning for dummies

Bill Raymond: That’s so amazing. So, all right. I have the big question for you and I know that we could probably spend at least an hour just talking about that, but could you kind of summarize for our listeners who are not AI experts, I mean, maybe we do have some listeners that are AI experts, but I think the general audience, what they’re interested in is learning how to maybe form their organization around AI.

But I think it would be really good if we had some sort of a understanding, a base understanding of how machine learning works. Could you give a kind of broad brush overview of how that works?

Zorina Alliata: Sure. And I promise I will try and keep it non-technical so everybody understands. I can start with an example, many, many years ago, and the machine learning was just starting maybe about 10 years ago or so. I remember one use case from an airline who did this thing. They just took all of their data that they ever had and they just had all these algorithms, machine learning algorithms in a way that we call unsupervised learning, which means, I don’t know what I’m looking for, just find interesting things in the data algorithms. Go and look. So they did that looking at their data, and then one of the findings was that people who were born on a Tuesday tended to order vegetarian dinners when they were flying the airline. So there was an insight that no human person would’ve thought of asking that question.

But that was a pattern that the machine found, the algorithms found just by looking at the data, right? Is it something that the airline can actually use to plan their inventory and to add more vegetarian meals? Maybe, right. Maybe it’s not useful information, but maybe it is. Maybe it, you know, it saves them quite a bit of money along the line.

So this is what’s famous about machine, what’s amazing about machine learning. Truly the machine learns by itself, it is not programmed. We do not program it to find who’s born on a Tuesday and what they eat. It finds it by itself by finding patterns in the data. It learns like that, right? Once it does that, it trains a model, so the software that the machine learning produces is called a model. It trains the model on the past data and then it applies that to future data.

[00:09:40] Automated Predictive Things

Zorina Alliata: So if you, you know, okay, let me take me as an example. I go to Netflix, they know that I love the Matrix movies, all of them, they know, right? All of the movies I’ve ever watched, they know about 90% of them are probably sci-fi.

So they would recommend me immediately, they take that pattern, that understanding and apply it to the new movies coming in. Is it a sci-fi movie coming in? They’re pretty sure that Zorina will like it, right? They apply what they’ve learned from my past into what’s coming in new into the future.

And that’s what they call deploying that model, right, to make this type of automated predictive things. It’s very high level, but that’s really what machine learning does and how it works. I hope this helps a bit.

Bill Raymond: Yeah, so it’s, it sounds a little magical that you just feed it data and then it sort of figures out what it finds, doesn’t it?

Zorina Alliata: Yes, exactly. But think about it. It needs a lot of data. It needs, it’s like a big monster with a big mouth and you need to feed it lots of data for it to understand that and that is a major, I think, gap. Right? Going on right now. That machine learning is not that AI that you see in the movies where the robots know everything and understand everything.

We’re not there just yet, right? We’re just looking very clearly at, you know, this is all I know from this data. This is the pattern that I’m going to do. What exactly does that mean, yeah, it doesn’t really do that just yet.

Bill Raymond: Right, right. Yeah. We still have to interpret it.

Zorina Alliata: That is correct. Yeah. And it needs a lot of data, right? Like a person, a baby, if they see a cat, they will then they see another cat, they know that’s a cat, right? So they only need to see one to figure out what a cat is. A machine at this time will need probably a million images of cat to understand and to predict that another image might be a cat. So the amount of data that you have to feed into the mouth of the monster is quite humongous. And it’s one of the things that takes a long time to develop machine learning.

[00:11:43] Agile and machine learning

Bill Raymond: Yeah. And you talked about developing machine learning and there’s a lot of care and feeding that goes into this, so that sounds like big and complex and there’s a lot of people involved, but so now I’m curious, what does agile mean to you?

Zorina Alliata: Yes, I’ve been an agilist for quite a while, I was lucky enough in a very long time, about 2011, I believe, or 2012 I worked on a government contract at the US Patent Office. And they were just starting to get into scaled agile, which is a SAFe methodology and they were actually working with folks from SAFe from the company, which at the time was tiny. It were only like five people. So four of them were there. And we stood up an entire scaled agile program and it was the first time that I was exposed to something that actually worked amazingly well. A project that was delayed, you know, two years and I don’t know how many millions of dollars were late.

We started implementing SAFe and following it with great care. And you know, in a year we had everything launched. So I saw it work and I was sold. It’s like, this works, this actually works. And to be honest, I’m saying that as a trained project manager as well, I have my PMP certification, I’ve done project management you know, traditional way, waterfall, you can’t really do that for machine learning.

As soon as I switched to machine learning programs and started managing those, you need to be agile. Because there’s just so much unknown, you cannot plan waterfalls. So to me, it became like a vital tool in doing this type of work.

[00:13:20] The two pizza teams

Bill Raymond: And I think famously at Amazon, you have this idea of the two pizza teams. Could you explain what that means?

Zorina Alliata: Sure. Basically if you need more than two pizzas to feed your team, then your team is too big.

Bill Raymond: Hmm.

Zorina Alliata: And it’s just a way of saying we have to be lean and agile and move fast, right? So, we try to keep our teams in between five and seven people assigned for every project.

Bill Raymond: Oh, that’s interesting. So it’s just about keeping up. So all these products and services you have, how does that break down? Like, I don’t want to get into a deep conversation about how Amazon is structured, but just from your personal perspective and working with these, how does that break down?

Because you have so many services that I assume are interconnected in some sort of a way that it’s just, it’s kind of mind boggling to think of a company with so many employees that have the sort of two pizza team focus.

Zorina Alliata: Yeah. And yet we do, yeah. All our products team are scheduled like that. And I’m part of the professional services division, which is a consulting arm. And we do all of our work in an agile manner as well. It’s the only way that works, right? I mean, we all know being agilists, we all know that it is the way to work to fail fast and to move fast.

Amazon culture, and I’m sure you know, because it’s public out there, their culture is very, to move really fast. They call it a day one. It’s always day one. It’s never day two and you relax and things just work, right? We’re always like, we’re just starting up. And a lot of the products and departments and development are moving around, like startups.

We like to say, everybody’s a startup, everybody’s doing their thing, everybody’s entrepreneur. So it is a very interesting dynamic. A lot of innovation happens in that environment. You’re not told what to do. It just happens organically. It’s really interesting, and the scale at which that happens, it’s also amazing, right, because Amazon hires more than a million people right now, so, yeah.

Bill Raymond: Yeah. No, it’s just amazing. So that’s interesting, that dynamic of really large company, but still keeping that small team focus, that probably provides a lot of opportunity for growth and for trying out new ideas without failing too big.

Zorina Alliata: Exactly.

Bill Raymond: So could you maybe talk a little bit about what’s happening right now?

You know, we’re hearing the term "AI transformation" and I do think that this sort of ChatGPT, Microsoft with Bing, Amazon devices, you know, these personal devices that we have, we can talk to them and get responses back. I think people are starting to see that this is important.

And as you mentioned, it’s not just chat, like you’re talking about helping organizations figure out like when the proper time is to plant strawberries and make sure that the buildings are safe. The use cases are, it sounds like they’re endless. So when we talk about AI transformation, the sort of the buzzword right now, can you talk a little bit about what that means to you?

Zorina Alliata: Sure. This probably could be a very long conversation as well, but I’ll try and touch on at least a few points. So I tell people, especially agilist program managers, project managers, there is a machine learning project in your future. And I put three exclamation signs because it’s coming! It will come one way or another.

It’s not just the bigger companies who are using AI now, it is very small companies. It is startups, it is, you know, medium-sized companies that are starting to use machine learning as the technology becomes more available and more common. So for sure, you will have to deal with some of this.

And your company will probably at some point go through an AI transformation. And I like to, if you’re an agilist, you probably know what a transformation is because it’s very similar with an agile transformation, right? Same idea. It comes all the way from leadership. There’s supporters, there’s change agents and then you go ahead and implement this.

So it’s exactly the way you would do an agile transformation, but this happens to introduce machine learning and AI everywhere in your product lines, product development, in your delivery processes for your company.

So these are huge initiatives, but because your competition is doing it, chances are your company will also do it. I think this quite a bit, right? Another insurance company is doing something with machine learning, then everybody else is like, oh yeah, we have to do that. Finance companies are extremely good at this, who can optimize a little bit better by two seconds by using machine learning. So it kind of spreads and rolls.

I think that this happens and I think that it’s important that it’s done right. Just like agile transformations, there could be all kinds of pitfalls. I’ve seen places where they hire a lot of very expensive data scientists to create this machine learning team, but then the delivery piece is not yet there, so nothing really makes it to production and to, you know, to be seen.

And that’s where our roles as agilists can bring great value, right? We can support this AI transformations. We know they’re coming. If you have an understanding of what this thing is and what it means, then you will see that as an agilist you can step in and support this adoption of AI and you can from your experience going to Agile transformations, you can really provide some value and help out the company to do that.

[00:18:39] AI transformation, a team to implement it.

Bill Raymond: Interesting. So you mentioned a few things there. You talked about data scientists and also needing to have all of the tools in place. And I guess I’m curious, you know, If we just step back and say, we have an interesting concept for how we might want to use AI in our organization, and so we say, let’s form a team to do that.

Could you talk a little bit about what that team makeup might look like at a, of course, generic sense, I suppose. And then what kind of tools, processes, frameworks that are out there that might help someone work through this AI effort?

Because most of us, whether we aren’t a programmer, but have been on a project at a company, we’ve had to deal with software, right?

Software almost is always there. And I think most of us know like, all right, this is how we work with developers, and developers know this is how we work with, you know, other different types of folks on the team that aren’t software developers. And so we’ve kind of got that, but I feel like AI is a slightly different animal because people are still learning it.

There’s different practices and practitioners that are involved. So I’m just curious to hear what your thoughts are on what that team looks like and any tools that they have at their disposal process-wise in order to make this a successful delivery.

Zorina Alliata: Sure. And also this is a very large conversation, but happy to follow up after this with folks who might be interested in, you know, getting more details.

[00:20:03] New roles in the team

Zorina Alliata: But for now, I can say that there are new roles in the team. So the development team does not look like regular software developers anymore.

The people who develop these machine learning models that we’re talking about and put together those algorithms to run it on that big data, they are called data scientists. And it’s a pretty new profession that maybe started 10, 15 years ago, so it’s quite new. In many places it requires a PhD, but at least a Masters. So these are highly educated folks, very expensive as well. If you’re a project manager, you’re responsible for the budget, you should know, hiring this type of skill it’s going to be quite expensive. But they are the ones that have this analytical programming skill to look at collected data correctly, analyze it correctly and then create this model.

So they’re the ones writing the code in Python. But then we have several other new roles, the most important one being a machine learning engineer. This is the person who takes that code and actually puts it into production. So it’s the infrastructure person doing those tasks and things like that.

Very important role, it’s also new, in many places, the data scientist also does the job of the ML engineer, like creates the model and also deploys it out. But we also see it a lot as a separate role, which I absolutely recommend because it’s a full-time job just to do the engineering around the models.

And that’s a pretty new role that has appeared. And then we many times see a data engineer or a data analyst who is supporting the data scientist into cleaning up and making sense of all the data before feeding it to the machine learning algorithm. And I can say one role that it’s quite common in this teams is some type of cloud engineer or cloud person.

And that’s because the cloud is awesome. I know you guys know, being in technology most of the time, but really, I used to be a developer and before there was the cloud, we used to do all these things by hand and now the cloud just does it for you. So it’s awesome. Any cloud, any cloud technology is just great, right? But you know, you need someone to understand how it works. So you do that deployment and all of those tasks correctly or else it will cost you a lot and will take forever, right? So a cloud engineer or some types of skills around that arena also are important in the team, and they do really well.

Bill Raymond: That’s a really good point. I was just working with a client last week and they were talking about how they wanted to translate the text that they had into voice. And they also had some audio that they wanted to translate into text. And so they’re going down this whole path of writing some application.

And I just said, you know, those services are available. You just need to purchase some time on that API over there.

Zorina Alliata: That’s exactly right. Yeah. Use what you have, use what you have. The technology is there. You don’t have to suffer anymore and write manual things. That is true. I can tell youa story when I first did AI, when I was in college, which was a very long time ago back in Romania where I’m originally from. My master’s year, we studied a neural network and it was a tiny neural network that had like five nodes and we were trying to make it recognize the letter A.

And we had one computer, one PC with some terrible Windows NT or something terrible a long time ago, an operating system. And we spent the entire year just running this little tiny model on this tiny machine that was very, very slow. And we were churning churning, churning for a year to create a couple of recognition models for a couple of letter A. We couldn’t do it because the technology was not there. The machine was trying, the logic, you know, was there. AI was the same idea, same paradigm. But the technology was not there. But now it is, right? And that’s why machine learning exploded, because the cloud technology and all that comes with it, that scales, right?

That now it can scale and it’s reliable and it’s secure. All of that has enabled machine learning as well.

Bill Raymond: So I have to imagine that these data scientists that you suggested, you know, be on the team and they’re expensive, but they’re probably going to have to spend months. I mean, I guess it all depends on the size of the project or the size of the effort that you’re trying to do, right? But I would assume that they’re spending months doing some of this work and then the data analysts cleansing the data and making sure that it matches what we need to do.

Would that be an accurate statement that… months?

Zorina Alliata: Unfortunately, more often than not, yes.

Yeah. This is part of just the lots of unknown that I mentioned that are part of this project. So when you are in charge as an agilist of managing the delivery and actually delivering the end result, it is very frustrating, right? Because you just don’t know, that airline had no idea they’re going to find, you know, people born on a Tuesday who are vegetarian. Like it never, they didn’t know the scope, they didn’t have a scope, they just analyzed data for months and data came back with this pattern. So how do you plan this if you are the agilist on, you know, when you have to deliver on that project, how do you deliver?

You don’t know they’re going to find something about people born on a Tuesday, right? So it is very hard to plan. For the people who are managing the delivery, it’s very hard to plan. Not impossible. If you have an understanding of how this whole thing works then you can work your way around it, right?

And you can, you can try and showcase the hard work that the team is doing, because believe me, it’s very hard work to clean all of this data and make sense of it to create the right business outcome. That’s a lot of work, right? So, it could take months, it’s true. But if you as the project manager, as the agilist in charge, you can kind of work with the team to understand exactly what they’re doing and communicate that back to business.

Then provide that visibility, right? That Agile is so famous for. I think that will help quite a bit in everybody understanding, oh, we’re moving forward, we know what we’re doing. We have some type of estimate for, you know, a roadmap of sorts, some type of milestones that we’re working towards. So it just gives a little bit more confidence in what’s going on, because otherwise, this whole unknown can be very, very stressful for the guys who pay, right? For the higher management.

[00:26:27] Metrics to measure the AI results

Bill Raymond: Yeah. And I think that’s the challenge right there is yes, we can continue to communicate and share what we’re doing and give those explanations. But I guess, are there some ways that we can show some value or some metrics along the way? , I guess one of the things that is probably going to be the challenge with AI is that it feels simple to us.

ChatGPT shows up and I can just type in a question and I get a response. I tell my device that’s sitting here behind me to play some music, and it plays some music, and it feels this like very natural. It gets me, you know? And now I’m saying I’d like to analyze my data to determine X, Y or Z.

So why is it taking months? All right. We know that now because data cleansing and data analysis, we might think about data in one way, but the machine needs it in another way. So I think we can get our arms around that. But now we’re that we’re kind of doing this and we have these expensive team members that are doing good work, how do we show value along the way? What are some metrics that we can use to measure that effort? How should we think about it from a leadership perspective? And as agilists, how do we think about that from a leadership perspective so that they know something is happening, even though there’s maybe it’s, there’s a little bit of silence on the other end?

Zorina Alliata: Yeah. And I think we have this challenge in some measure with regular software development, right?

Bill Raymond: Yeah.

Zorina Alliata: For example, infrastructure teams, they go and they " patch" a million servers or whatever, some very deep techy thing that nobody else understands what’s going on, right? And you hear from the management like, what is going on?

What are you guys doing? Ah, they’re patching something for three years now. So it’s hard to understand what is going on, especially with this type of really complex type of work, right?

[00:28:15] Three metrics

Zorina Alliata: So what I do is I, I’ve landed on using three types of metrics that I recommend in this type of project.

So first one is very low level, and it’s just literally the technical metrics that measure, is this a good model? Is this thing working? If I want to forecast when to, you know, plant the strawberries, right? How do I know my model is good? Well, let me go into the past, pretend it’s 2020, take that data, run my model as if it was, you know, March of 2020 and see, am I right?

If I was going to predict to plant in June, am I right? Is it going to really yield right? Look at the data that’s out there. Try and do something to test. There are all kinds of technical ways to find these metrics and figure out if your model actually works. Is it good? And those are very technical.

Unfortunately, they are extremely, that could be extremely technical. I think the worst I’ve seen was from a team that told me their model is awesome, it’s great and they obtain a wonderful result. And they were so proud, right? So I’m like, excellent, let’s brag about it. Let’s take it to, you know, present it to leadership and say, look, what a great job you guys did. Can I have the technical metrics so I can show them and they show me it was a dot, one dot. A graph with one dot. And they were so proud of it, like if it’s 80% and then 20% here and then this dot, I’m like, I can’t show a dot to anyone cause it’s meaningless to regular business people.

So we have to take that and transfer into something that makes sense. So they might be very murky when you ask your data scientist, Hey, how’s your model working? The answer you got back might be a dot, so be prepared for that. But if you ask a few questions, chances are you can translate that into something meaningful that business understands. Like, oh, this will improve my crops by 10% or whatever my yield, right? Or something. You can translate that into something that they care about.

The second metric that I use is just the regular agile metrics. How’s the work getting done? Because it has to be done. We are responsible of delivering results, right? We have to get things out of the door, so we have to measure that. If the data science team is stuck in analyzing data for three months and there’s nothing to show that it’s moving, that is progress and something has happened. So I try to say there is work going on and try to make that work visible. And the way you do that is by having the right type of user stories, right? There’s smart, we all know the creating good user stories and just clarifying the work that is happening so people can get credit for it.

And we can see you know, that burndown chart working the way it should. So agile metrics, good agile metrics are super important as well, just to show that the work is going on and that people are putting in the right effort and things are coming out of it. Right? And then the last one is the strategic metrics.

And these are usually very high level like, have I made any money with these models? Have I increased my membership or whatever I wanted to achieve? Have I done that right? I established it in the beginning of the project, when I started machine learning project, I asked the leadership, why are we doing this?

What are we trying to achieve? Are we increasing membership? Are we reducing risk? How are we going to measure that that happened and it worked? How do you measure your sales today? Can we do this after three months of running the model to see if it really worked and if really we achieved that? So get those in the beginning, right?

And then at the end I report on those as well, making clear that these are the things we started with and we really achieved that objective.

So between all these three, I find that most of the time it’s quite clear and everybody has a transparency that we agilists are famous for.

Bill Raymond: Right, and I really appreciate that. That’s good. Thank you.

[00:32:09] What should we learn?

Bill Raymond: And now as an agilist as most of the people listening to this podcast are agilists, or at least heading down that journey, and I guess, you know, what should you go learn? As you said, everyone that’s listening to this podcast is probably going to have some sort of an AI effort that they work on, whether they know it or not, it’s probably coming, right?

Bill Raymond: How do you get in front of this? How do you learn how to successfully run an AI project. What are some of the skills that you’ll want to pick up?

Zorina Alliata: Sure. So again, just being agile, you’re already 70% there because a lot of the concepts are the same. Agile was designed for situations like this where we don’t understand the full picture, we don’t have all the data, but we can iterate and go forward and learn and get feedback and keep building. So it was, it’s perfect to apply for machine learning.

So do not be discouraged, you already have a great advantage because you are an agilist in working in this type of project. That being said, there’s a lot of stuff out there available. When I went back into machine learning a few years ago, I went on Coursera, Udemy, edX, all of these websites with available courses, and I just took all that I could on machine learning. So, I remember, I had the bad idea of taking an R course, which is the coding language over the Christmas holidays. So yeah, spending Christmas day writing R code, what was I thinking? But it was very helpful because you understand, and you don’t have to take the coding one.

There are many out there just to understand how machine learning works in general. So, you know, you don’t have to code. You can just look at a couple of them on Coursera, specifically for managers and project managers. And I have a course on Udemy as well about managing machine learning projects and strategy.

So, yeah, let me know if you’re interested or look me up.

Bill Raymond: Yeah. Wonderful. I think that’s a great way to wrap up this podcast. I appreciate all of the time that you spent to prepare for this podcast and then to deliver this great podcast. We covered so many topics and I’m really happy that we managed to cover what we had hoped to.

Zorina Alliata how might people reach you?

Zorina Alliata: Sure. So I like Linkedin, I’m there every day. And that’s how I communicate with a lot of folks. That’s where I do a lot of my mentoring work. So please reach out on Linkedin, I’m happy to connect and chat over coffee or doing another podcast.

Bill Raymond: Yeah, that sounds great. And we will make sure that that Linkedin link is in there. Of course, I’ll get that Udemy course from you that you just mentioned. And you know, we found you through some of your writing at Georgetown and specifically there was an article titled, Artificial Intelligence, Machine Learning and Agile Practices.

And it breaks down a lot of the topics that we kind of maybe covered at a high level here in this podcast with a lot more detail. So I’ll make sure I share that link as well.

So, Zorina Alliata, thank you so much.

Thank you.

[00:35:04] Outro

Bill Raymond: Thank you for listening to the Agile and Action Podcast with Bill Raymond. Subscribe now to stay current on the latest trends in team, organization, and agile techniques. Please take a moment to rate and comment to help us grow our community. This podcast is produced in affiliation with Cambermast LLC, and our executive producer is Reama Dagasan.

Speaker: If there is a topic you would like Bill to cover, contact him directly at bill.Raymond@agileinaction.com.