Today, I’m talking with Natalia Andreyeva from Infor about AI / ML product management and its application to supply chain software. Natalia is a Senior Director of Product Management for the Nexus AI / ML Solution Portfolio, and she walks us through what is new, and what is not, about designing AI capabilities in B2B software. We also got into why user experience is so critical in data-driven products, and the role of design in ensuring AI produces value. During our chat, Natalia hit on the importance of really nailing down customer needs through solid discovery and the role of product leaders in this non-technical work.

We also tackled some of the trickier aspects of designing for GenAI, digital assistants, the need to keep efforts strongly grounded in value creation for customers, and how even the best ML-based predictive analytics need to consider UX and the amount of evidence that customers need to believe the recommendations. During this episode, Natalia emphasizes a huge key to her work’s success: keeping customers and users in the loop throughout the product development lifecycle.

Highlights/ Skip to

• What Natalia does as a Senior Director of Product Management for Infor Nexus (1:13)
• Who are the people using Infor Nexus Products and what do they accomplish when using them (2:51)
• Breaking down who makes up Natalia's team (4:05)
• What role does AI play in Natalia's work? (5:32)
• How do designers work with Natalia's team? (7:17)
• The problem that had Natalia rethink the discovery process when working with AI and machine learning applications (10:28)
• Why Natalia isn’t worried about competitors catching up to her team's design work (14:24)
• How Natalia works with Infor Nexus customers to help them understand the solutions her team is building (23:07)
• The biggest challenges Natalia faces with building GenAI and machine learning products (27:25)
• Natalia’s four steps to success in building AI products and capabilities (34:53)
• Where you can find more from Natalia (36:49)

Quotes from Today’s Episode

• “I always launch discovery with customers, in the presence of the UX specialist [our designer]. We do the interviews together, and [regardless of who is facilitating] the goal is to understand the pain points of our customers by listening to how they do their jobs today. We do a series of these interviews and we distill them into the customer needs; the problems we need to really address for the customers. And then we start thinking about how to [address these needs]. Data products are a particular challenge because it’s not always that you can easily create a UX that would allow users to realize the value they’re searching for from the solution. And even if we can deliver it, consuming that is typically a challenge, too. So, this is where [design becomes really important]. [...] What I found through the years of experience is that it’s very difficult to explain to people around you what it is that you’re building when you’re dealing with a data-driven product. Is it a dashboard? Is it a workboard? They understand the word data, but that’s not what we are creating. We are creating the actual experience for the outcome that data will deliver to them indirectly, right? So, that’s typically how we work.” - Natalia Andreyeva (7:47)
• “[When doing discovery for products without AI], we already have ideas for what we want to get out. We know that there is a space in the market for those solutions to come to life. We just have to understand where. For AI-driven products, it’s not only about [the user’s] understanding of the problem or the design, it is also about understanding if the data exists and if it’s feasible to build the solution to address [the user’s] problem. [Data] feasibility is an extremely important piece because it will drive the UX as well.” - Natalia Andreyeva (10:50)
• “When [the team] discussed the problem, it sounded like a simple calculation that needed to be created [for users]. In reality, it was an entire process of thinking of multiple people in the chain [of command] to understand whether or not a medical product was safe to be consumed. That’s the outcome we needed to produce, and when we finally did, we actually celebrated with our customers and with our designers. It was one of the most difficult things that we had to design. So why did this problem actually get solved, and why we were the ones who solved it? It’s because we took the time to understand the current user experience through [our customer] interviews. We connected the dots and translated it all into a visual solution. We would never be able to do that without the proper UX and design in that place for the data.” - Natalia Andreyeva (13:16)
• “Everybody is pressured to come up with a strategy [for AI] or explain how AI is being incorporated into their solutions and platform, but it is still essential for all of my peers in product management to focus on the value [we’re] creating for customers. You cannot bypass discovery. Discovery is the essential portion where you have to spend time with your customers, champions, advisors, and their leads, but especially users who are doing this [supply chain] job every single day—so we understand where the pain point really is for them, we solve that pain, and we solve it with our design team as a partner, so that solution can surface value. ” - Natalia Andreyeva (22:08)
• “GenAI is a new field and new technology. It’s evolving quickly, and nobody really knows how to properly adapt or drive the adoption of AI solutions. The speed of innovation [in the AI field] is a challenge for everybody. People who work on the frontlines (i.e. product, engineering teams), have to stay way ahead of the market. Meanwhile, customers who are going to be using these [AI] solutions are not going to trust the [initial] outcomes. It’s going to take some time for people to become comfortable with them. But it doesn’t mean that your solution is bad or didn’t find the market fit. It’s just not time for your [solution] yet. Educating our users on the value of the solution is also part of that challenge, and [designers] have to be very careful that solutions are accessible. Users do not adopt intimidating solutions.” - Natalia Andreyeva (27:41)
• “First, discovery—where we search for the problems. From my experience, [discovery] works better if you’re very structured. I always provide [a customer] with an outline of what needs to happen so it’s not a secret. Then, do the prototyping phase and keep the customer engaged so they can see the quick outcomes of those prototypes. This is where you also have to really include the feasibility of the data if you’re building an AI solution, right? [Prototyping] can be short or long, but you need to keep the customer engaged throughout that phase so they see quick outcomes. Keep on validating this conceptually, you know, on the napkin, in Figma, it doesn’t really matter; you have to keep on keeping them engaged. Then, once you validate it works and the customer likes it, then build. Don’t really go into the deep development work until you know [all of this!] When you do build, create a beta solution. It only has to work so much to prove the value. Then, run the pilot, and if it’s successful, build the MVP, then launch. It’s simple, but it is a lot of work, and you have to keep your customers really engaged through all of those phases. If something doesn’t work [along the way], try to pivot early enough so you still have a viable product at the end.” - Natalia Andreyeva (34:53)

Links

• Natalia's LinkedIn

Transcript

Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today I have Natalia Andreyeva on the line from Infor. Is it Infor Nexus or—Nexus is the product made by Infor, is that correct?

Natalia: Yeah.

Brian: Oh, it’s the unit. Okay, got it. So, you’re senior director of product management on the solution portfolio. So, does this mean you’re overseeing multiple AI applications or products that sit under Nexus?

Natalia: So, it’s like, it’s really Infor Nexus, but I’m working for Nexus, and Nexus is an independent business unit of the Infor, pretty remote from the actual parent. So like, we are not doing the ERP, and [unintelligible 00:01:45] that they do.

Brian: Got it. So, if I understood this right, one of your customers is kind of like the hub, and then they might bring on their shipper and their suppliers, and it’s kind of like a tool that they have to onboard into in order to be part of the ecosystem. Is that basically how that works? There’s one buyer of the Infor Nexus products, and then they bring all their suppliers inbound. Is that how that works? Because you said it was an ecosystem with multiple players.

Natalia: Exactly. So, our customers, we call them shippers, the ones that are shipping the goods end to end, and when we connect them to our solutions, we onboard their suppliers, or if they’re already in the network, we just connect with them as well, as well as we already have this logistic service providers and carriers connected in the platform, and if there are brand new ones, so we take care about bringing them on board as well, as part of the initial setup. And then it’s a collaboration in the platform that is enabled through multiple modules. And they can manage their orders, they can ship their orders, they can track their orders, they can look at the documentation. It goes from the manufacturing of the goods all the way to delivery.

Brian: So, can you—just, like, high level, we don’t have to spend a ton of time on this—but if you’re at a retailer, let’s say Levi’s jeans, or, I don’t know, pick a company, what are the job titles of the people inside Levi’s, and what are just some of the high level use cases that are done a lot. Like, what would someone be sitting down to accomplish in these products?

Natalia: So, there’s some of the titles, and some of the personas we work with are typically logistics managers or inbound analysts for the transportation pieces. We are also working with inventory planners. We are also working with supply chain operations. We’re working with account payables when we’ve also facilitating transactions between the vendors and our customers, financial transactions and supply chain. So, for that, we are working with account payables.

The list is really, really long, but a lot of that is our key personas. We’re working with quality managers that are tracking the quality of the goods with our suppliers. So, suppliers are working with our customers by producing the goods.

Brian: And can you give a little oversight of, like—overview of your team makeup? Like, are you primarily managing a group of product managers or product owners or designers? Do you own engineering? Like, what’s your data science or something—maybe give us a little, kind of, picture of who reports up into you, what kinds of roles?

Natalia: So, I am leading the group of product managers. My team is small and growing, but also working cross-functionally with our engineering who is responsible for the development of application and with our data science team, who are responsible for the model creations and AI product creation at core, and of course cross-functionally we work with sales and institutional consultants and customer support and professional services within the organization.

Brian: And do you have a design team as well, and do they functionally go up through product management, or is that a separate tier?

Natalia: It’s a good call out. Of course, we do have a design team, which is also growing. It is part of our product management family under the department of product management. And we have the design lead and a few designers that we are using cross-functionally across multiple products, and they also are doing a lot of user research and discoveries for us as well when we work on particular problems and solutions. They are very essential to everything that we build, especially in my area.

Brian: Got it. Yeah, I’d love to hear about that. Like, one of my questions because you said that you focus on the AI and machine learning portfolio, the solution portfolios, so does this mean you’re only focused on, like, features and capabilities that leverage AI across multiple products, or your whole product is effectively classified as an AI product, and so you run it end to end? You know, you’re the owner and product manager; you’re responsible for the outcomes of the entire application, or is it more just the functionality within features?

Natalia: In my current role, I’m responsible for the AI strategy. That is a mix of two things that you just said. So, it could be a very typical or topical applications where we are embedding AI solution into the platform, like, for example, predictive ETA, right, so it can be consumed by multiple modules in the platform. And what I’m currently working on is our new generative AI product, which we just, after doing the discovery, started to form as an actual product, so that I will be responsible for the outcomes of that product across multiple different verticals, which also will be co-owned with other product leaders because that’s the nature of the product, so they’re going to have to spread through the platform in different modules. But in my—

Brian: I see. Got it.

Natalia: —[crosstalk 00:07:04] lives, so I was just doing—I was responsible for the product and the product outcomes that was powered already by the AI and the ML solutions at its core. So, the entire platform was built on AI and ML, though there is a variety of different angles my profession can look at those things, so right now, it’s a mix for me.

Brian: So, you said design was integral to this, and I don’t think that’s a given in a lot of organizations. So, maybe you can unpack to me, how are your design partners—your design partners, meaning your business colleagues, your customers that act as design partners, as well as your internal design partners, your design team and those leads—how do they work with your team, and do they work with the data scientists and engineering as well? Like, how does that model look? What’s it look like for someone that was observing from outside?

Natalia: So, the way we are working together with our designers is we have launched the—for the new initiative specifically; so I always launch the discovery—and the discovery with customers, they always do with the presence of the UX specialist, right, so our designer. Though she comes with me to the interviews, and it can be mixed. I can be performing an interview and she listens or vice versa, but the goal is to understand the pain points of our customers by listening how they do their job today. We do the series of those interviews and we distill them into the customer needs, essentially distilling them into the problems we need to really address for the customers.

And then we start thinking about how, right? So, for the data product, it is, I feel like, a particular challenge because it’s not always that you can easily create a user experience that would allow them to realize the value they’re searching for from that solution. And even if we can deliver it, consuming that is typically a challenge, too. So, this is where we talk about what is the problem, what they do today, and my designer colleague is really creatively coming up with their solution of how the user experience can be for our customers. And we’re creating that concept, typically in Figma.

So, it’s not necessarily a wireframe at this point, it’s a conceptual design of the solution because what I found through the years of experience, it’s very difficult to explain to people around you what it is that you’re building when you’re dealing with a data-driven product. They just don’t get it. So, is it a dashboard? Is it the work board? Is it—so they understand the word data, but that’s not what we are creating. We are creating the actual experience for the outcome that data delivers to them indirectly, right?

So, that’s typically how we work. And latest challenge has been really interesting because we’ve been creating the user experience for our generative AI product that will be a conversational assistant. And with the adoption of, you know, [unintelligible 00:10:18] and other tools, it does force you to rethink how the users interact with your platform as your solution for the particular outcome. And I’d have to say, it’s kind of new territory for all of us, but I love how it’s coming together because it has to be driven by the outcome for the customer.

Brian: Is there something in this process so far that you’ve gone through that you’ve changed, like, when you’re doing discovery work for AI and machine learning applications, that’s different than you might for a traditional digital application, or is it effectively the same process? Like what, if anything, is different?

Natalia: Yeah, it’s a very good question. When I do the discovery, so I typically frame it around what kind of—we already have ideas for what we wanted to get out, right? So, there’s some preliminary conversation that happened, and we know that there is a space in market for the solution to come to life, we just have to understand where. And for AI-driven product, the insights that we have created, it’s not only about their understanding of the problem or the design, it is also understanding if the data exists and it’s feasible to build the solution to address that problem. And that feasibility is extremely important piece because it will drive the user experience as well. That’s what we have found.

So, I’ll give you an example. So, I had worked on a very interesting problem in my previous organization, and we created a brand new solution for the pharmaceutical industry. And we actually solved the problem that industry couldn’t solve for many, many years, for at least seven years to this point—the product just went live—and when we ran a very deep discovery, we had probably about five or six different prototypes on how to solve this problem that we created with our design team. So, that was a concept. And after we have created the first prototype that we actually had to discover tons of different details about the data that needed to be connected. So, we created a data journey of all of the multiple system that we needed to bring together to find the outcome for that solution.

And when we started to think about how to visualize it through the web application, oh my God, so we were ripping our hair out, all of us. Because anytime we would create a prototype, we would show it to the users, and they would not see the actual outcome we build it for. So, they would literally stare at the message that it says, “Your product is good to be used,” in text message, and they wouldn’t see it [laugh]. So, we would see everything else except for the actual outcome. And we had to redesign that particular solution, probably a number of times, maybe ten total. That’s how difficult it was.

Because everything, when we discussed the problem, it sounded like a simple calculation that needed to be created, and in reality, was an entire process of thinking of multiple people in the chain to understand whether or not the medical product was safe to be consumed, and that’s the outcome we needed to produce. And it was like one-page application. When we finally did this, we actually celebrated with our customers and with our designers, but it was one of the most difficult thing that we had to design. And then I was thinking, so why is this problem actually got solved, and we were the one who solved that, and nobody else had that product in the market? It’s because we have taken the time to understand current user experience through the interviews, and we went so deep and connected to all of those dots and translated it into the visual solution. We would never be able to do that without the proper user experience and design in that place for the data [crosstalk 00:14:32].

Brian: Got it. Did you feel like where you landed at it, is this one of those situations where a second competitor coming in could look at what you had done and leap and basically arrive at the same thing, but until you’ve gone through that proc—and nobody could do it the first time, it was really hard to do it the first time, but it’s easy to do it the second time, or do you feel like you la—all the technical work you had to do actually gave you a leverage above competition, where it’s like you could copy us, but you’re so far behind because you’re going to have to wire all these systems together, and there’s all this back-end work. Do you have any reflections on that?

Natalia: Yeah. You knew it’s been funny because the UX, you know, UI of the product, the front page of the product, was so simple.

Brian: Right.

Natalia: People were not getting why it is so simple and it took so long to build. And, you know, there were comments, it’s like, “Let’s not show it to anybody too early. They’re going to copy us, and they’re going to be able to replicate the solution.” And my answer was, “They will not be able to because it is so complicated and so difficult.” What we have done on the back-end, it’s not replicable so easily, unless you actually understand that process of the deep discoveries that we have done with multiple users.

It was interesting that when we were expanding to other customers, we explained them what was happening in their organizations, in where the pieces of data that they didn’t see, and they went back and they confirmed that it was indeed true, and we have connected the journey for them as well. So.

Brian: Yeah, was this generating a prediction? It sounded like it was generating a prediction: ‘safe to use,’ ‘not safe to use.’ Is that what it was?

Natalia: Yeah, it was a generating—it was factual, actually. So, it was factual. Generating the outcome so ‘safe to use.’ Or ‘not safe to use.’

Brian: Got it. And did you have to provide evidence of how the software came up with this conclusion?

Natalia: Yes.

Brian: So, was the hard work in figuring out how much of the evidence to surface and how to surface it in a way that people would believe the conclusion that you came up with? Was that what was difficult?

Natalia: Absolutely. So, it was really difficult to make it meaningful for them, and not overwhelming, then actually be able to trust it. But it was really funny that they couldn’t just see the conclusion. And the second we would point it out, and they were like, “Of course it’s there.” Like, “We can see it now.”

Brian: Yeah [laugh]. Well, just the fact that you’re checking that because—and this just goes to show our bias. When we look at the thing all day, it seems totally obvious, right, so the fact that you’re going out and validate—like, you go in with just that honest, totally neutral hat, and don’t assume anything about how good or bad it is, and just come in neutral, you’re open to learning, right, and you’re not carrying your bias forward about how great your baby looks, you know? [laugh].

Natalia: Exactly. You know, I can give you a different example of what just happened recently. And—

Brian: Mm-hm.

Natalia:

the validation of our beta solution for the recommendation engine already here in Nexus. And it was just the early version of it, it was not really great, you know, user experience designed around it, and we were looking to optimize inventory shortage. We needed to mitigate inventory shortage situations for them. So like, we needed to reorganize their inventories that are available so that we can resolve the shortage on time before it was a problem. And it’s an optimization problem from that perspective.

And when we ran the beta test with our pilot users, 50% of recommendations they made as invalid recommendations on viable recommendations. And when we talked to them, they couldn’t just understand what we were saying to them through the user experience we provided. So, 50% of all of the recommendations that were perfectly valid were marked as unviable because the user couldn’t figure out what we were recommending to them.

Brian: Can you give a specific example of that?

Natalia: Yeah, the specific example of that would be so, for example, you have a shortage in location A, right, and to resolve the shortage, you would need to move the inventory from location B and C, and probably D, to get to the quantities you need for the orders that are coming out from the warehouse.

Brian: Got it.

Natalia: So, our engine would find that, right, and then tell the users that you have to transfer inventory from point A to point B, then you’re going to have existing order in location A, and you need to take part of it and then move it to that location B. And to finish it off, you’re also going to have to tap into the inventory from the location D, and move, let’s say, 10,000 pieces of that to the location B as well. So, if you do that within, you know, next three days, you’re going to be fine—and then you’re going to have to replenish these locations, but it’s a different story—but you’re going to be fine with your critical orders that were supposed to be going out to your customers in a week from now, right? And we would give them the list of cards for every action that they needed to do, and with details around what to move, where to move, and by when, and by which mode of transportation.

And because when the customers looked at those they couldn’t, really make sense out of the sequence, out of the locations. They were like, “I just don’t understand why I should be doing this, and what is an actual outcome that I will be accomplishing by doing those sections?” So, the basic thing was unclear to them, what is an impact that I am going to get as a return for executing these actions? It wasn’t reading. It was all there, information was there, but they couldn’t make sense out of it. So

What we have done is we looked at all of these scenarios together with our design team. So, my designer colleague was really fantastic, and we designed for the most difficult case we could find where it would include ten different actions—transfers—and we gave it a completely different user experience. So, now the customers will see the summarized written summary of all of the actions that they need to do, as in snapshot, right? So, it’s like, “Those are three things you will need to do.” They will see the immediate mitigation impact, so you’re going to resolve it by 90%, right? So, like, maybe not 100, but you’re 90% going to be there. You’re going to be fine.

And then in we decided to do, like, a table view, where we separated every particular purchase order, every particular shipment that needed to be modified for the particular type of inventory, with a clear changes to from now to the new thing. So, for example, it’s going to have to be shipped not March 28th, it’s going to have to be shipped April 1, for example. So, and they could see it very clearly that they needed to move this inventory from this purchase order to that location by that date. And now it’s working, and now they can understand it. So, we didn’t really change how the engine worked; we only changed how the user experience presented to the customer, and it is now working for them.

Brian: It sounds like you spend a fair amount of time doing customer exposure time, whether it’s discovery work or validation of design, usability testing, concept testing. Is that correct?

Natalia: Yeah, it is correct. So, I think, for the AI and the ML products, and it’s a lot of buzz right now in the industry, and everybody is pressured to come up with a strategy for AI or explain how AI is being incorporated into their solutions and platform, but it is essential for all of my peers in product management to still focus on the value, right? So, what is that value that we are looking to deliver and drive [unintelligible 00:22:45] outcome, and to understand that you cannot bypass discovery. So, the discovery is the essential portion where you have to spend time with your customers, with your users, with, you know, champions, advisors, and their leads, but the users who are doing this work, this job every single day, to understand where the pain point really is for them and solve that pain point, and solve it with design team as a partner, so that solution can surface that value. I have, like, tons of different stories around it.

Brian: One of the things I’m getting out of this or that I want to surface that I think is a challenge for some listeners here is, did it take a while at Infor—I don’t know if this is just something you’ve done all the time through your career, but getting the time to spend on discovery and getting out of the deliver, deliver, deliver, build, build, build, build, build, build. When’s it going to be done? When’s it going to be done? It going to be done? When’s it going to be done?

You seem like you have a really strong focus on the value: understanding the problem up front makes the delivery part go a lot faster and easier because you know what you need to build. Did it take a while to get to that point where you were able to spend the requisite time on discovery? Because I think a lot of teams still are build-biased, and I see this with data teams particularly, they want to just be in the order fulfillment, give them what they want, and they think they understand the problem, and they’re not doing nearly this much discovery work. And then they have those problems like you showed, which is, like, what? The answer is right there?

Like, move these orders, move this shipment, and the customer is like, “Why should I do that? Why?” Like, it’s just a bunch of alerts, and it’s not clear why. And it’s like, nothing to do with the machine learning or the algorithms. Like, that’s all great, but it’s dying in the last mile [laugh]. So, how did you get to that point? Or was it always like this? Is this just, like, always the way it’s been, or have you built a model here to make sure the business understands the value of doing this discovery work?

Natalia: Well, so the framework that I typically apply, I came up with and built with in my previous companies that I was working. And we did have a really huge challenge there. So, we couldn’t easily access customers because the traditional approach has been always, build, build, build, build, build, right? So, talk to somebody else who can tell you what the customer problem is. And all of those stakeholders, like, our, you know, professional services, marketing, team solution consultants, they all provide really good input, but it’s just not enough for the good solution to be created.

So, for product managers, especially if we are not scaling and building a new solution, we have to talk to customers to understand them. It’s not always have to be super deep and long, right? So, it can be quick few conversations [unintelligible 00:25:31]. But that was a real battle, but after we start showing the results, it started to pay off. It opened the doors for all of the other teams to start doing the same thing, and we built trust with business.

In Nexus, when I joined, and I’m alwa—I am with Infor Nexus for almost a year—I didn’t have a problem like that because I could speak to customers anytime I needed to speak to them. But it is a little bit of—can be the you know, cart before the horse situation where they a lot of projects would start with development-first approach, and sometimes they work and sometimes they don’t. For example, with optimization engine, the example that I gave you, that’s a very strong technical solution, but because no—like, was limited research upfront done, we missed that important user experience problem. So, when I joined, we kind of went back a little bit and re-looked at that experience and fixed that problem for them. So, I think the advice I would give here is we kind of have to advocate for that discovery upfront, but the technical team can still do the feasibility, and POC, and prove that technology is working.

So, you just have to trust them to run with the POC stage of the discovery, and then you still going to have to merge. So, the willingness to pivot should be there for that relationship to work, and if it’s only the engineering-driven or technology-driven type of innovation, most of the time, it will end up on the shelf, right? So, it will not [unintelligible 00:27:10] customers. And unfortunately, I have those examples as well, really great ideas, blockchains, and even patented ideas, but they were never adopted by the market because it was never a problem big enough or valuable enough for the customers to adopt and solve for.

Brian: Yeah, yeah. You’re preaching to the choir here. I think that’s great. And yeah, I… amen [laugh]. Right now, with, like, with all the stuff happening with this, like, you’re working on a GenAI product, and you’re overall focused on AI and machine learning products, what’s the biggest challenge you have, like, in your work, from a product and design standpoint, when working on these solutions? Is there something that, like, stands out?

Natalia: It’s a very, very new field, and it’s a very, very new technology, and it’s evolving really, really quickly, and nobody really knows how to properly adapt or drive the adoption of the solutions. And I think general education is the challenge there, right? So, you know, like, right now, everybody talks about agents. A very small number of population actually knows what it is or how to adapt it into your solution so it brings value. But it is cool and hot, right, and it’s evolved so quickly, over the last six months, I would say it feels like it was just, like, yesterday we didn’t even talk about them, and overnight they showed up, right?

So, in that speed of innovation, in generative AI, in the AI field right now is what is a challenge, I think, for everybody. And where we have to be very, very careful, is that, you know, people who work on the frontlines so that, you know, product team, engineering team, we always way ahead of the market. And people who are going to be using those solutions, they are not trusting the outcomes that are coming out of them, and it’s going to take some time for people to become comfortable with that solution. But it doesn’t mean that your solution is bad or didn’t find the market fit, yet. It just, it’s not yet the time for it.

So again, educating our users on the value of the solution is also part of that challenge, and we have to be very careful about it. When we launch the products and when we talk about that, it has to be accessible, right? So, it has to be simple because otherwise it’s going to be intimidating, and people also not going to adopt the solution.

Brian: Yeah, and is this something where—when we’re talking about GenAI specifically here—is this something that was like a mandate that came down from above for you from Nexus senior leadership and saying, “What’s our strategy? What are we doing?” Or did you have customers asking for this? Which I find is very rare; I don’t hear that [laugh] too often. Where did it come from that that’s now even in your radar?

Natalia: Oh, that’s interesting. So, their idea of having, like, a digital assistant was already present when I joined.

Brian: Oh, okay.

Natalia: So, my role was created because of the AI emergence that was happening in the market. We have a giant platform, and it was just, like, we’re going to have to figure out our, you know, act together. We’re going to have to figure out our strategy with AI because we can do a lot. And that’s why my role was brought in: to understand exactly where the value is, where we have to invest into it. And idea of the digital assistant already has been present by the time I showed up, it wasn’t clear where exactly we’re going to add the value in it.

And we actually sold it in fortunately a really good way because we created a nice POC around our version of Copilot, right, and we based it on the story. When I joined, it was a POC, right? I was like, all right, cool, but what are we doing? So, I talked with a couple of customers, we created a story that was addressing the pain point for them, and we proved that it would work with the data. We already had enough proof that it would work.

And we showcased it to our customers at one of our customer events, and expanded that reach so that for me to get the partners who saw the value in adopting this type of solution for them because they saw, okay, I can actually shred tons of time on cooperating with our digital assistant versus searching for information in multiple sources, and they became my partners in crime. So, we’re now building solution with customers, we are driving it in right direction, we know it’s going to be bringing value to them because of the discovery we just concluded. So, that’s how we solve that with that because otherwise it could have been another [theory 00:31:57], right? So, about where the value is, and then we could have hit it or maybe not, right? So, and that’s the challenge with just technology-driven innovation. Sometimes it’s a hit and sometimes it’s a miss. So, we have an evidence it’s going to be a hit, so I’m very [unintelligible 00:32:14].

Brian: So, this isn’t out yet? This is in development right now?

Natalia: Yes. So, it’s going to be coming out.

Brian: Okay. Do you have a plan or a regular framework for how you measure user experience quality? Like, I’m assuming this is not a separate thing they pay for. Is this, like, a feature that’s in the product already, so effectively, it’s not a new source of revenue. That’s an assumption. Is that true? Or TBD?

Natalia: It is going to be TBD because it’s going to have different flavors. So,

their revenue-generating activities, but it’s also going to be part that they just built in into the platform which is part of it. What we do is we do have tools that track our user experience within the platform, and we track where they spend a lot of time, and what are the usage of different modules and applications. There is a telling story because not all modules are being used equally. And then our design team also follows up with the customers to understand what they like, what they dislike about the applications.

We do have a challenge with redesign of some portions of the platform right now, and giving customers new user experiences because the platform is pretty big, and solution exists for 25 years, so there are some parts of their solution is just a little dated. They’re going to be [unintelligible 00:33:36] and redesigned, and our design team has been leading that effort with customers, understanding what’s not working for them today. But overall, I would say it’s not always specific to the user experience. It’s specific to the customer satisfaction with our products, and those checks are being done regularly with all of our customers and led by our customer success groups, technically.

Brian: This has been a really great conversation, so thank you for being so candid about everything. Do you have any other, like, high-level closing thoughts that you’d give to someone that is an AI product manager or a data product manager, about focusing on value, user experience as it relates to business value as well? Anything like that that you’d like to share?

Natalia: Yes. I will say something probably very common: never fall in love with a solution. Like, never do that. And you have to repeat it so many times in your head. Do not fall in love with the solution. Fall in love with the problem you’re solving for your customers. The solution will always change. You will have to sunset products, you will have to redesign them, and when we get emotionally attached to our product babies, it’s really difficult.

but we actually have to put our energy and understanding of the problem.

Another piece of advice: when you engage in with customers into the discoveries, what I got from my experience of trying multiple different things, it works better if you’re very structured with that approach, and I always provide an outline of what needs to happen. And here is my framework. It’s not a secret. So, you do discovery phase, where you talk to [unintelligible 00:35:26] about particular problem, or you searching for the problems. You do the prototyping phase, where you also have to really include feasibility of the data, if you are working with AI and solution, right?

So, in that prototyping phase can be short, can be long, but you need to keep the customer engagements through that phase as well, so they should see the quick outcomes of those prototypes. And keep on validating this conceptually, you know, on the napkin, in Figma, it doesn’t really matter. You have to keep on keeping them engaged. Then once you validated that it can be working and the customer likes it, then build. Don’t really go into the deep development work until you know that. And when you build, create your beta solution. It only have to work so much to prove the value.

Run the pilot, and if it’s successful, build the MVP, then launch. Then launch. So, it’s the four steps: discovery, prototyping, development of a beta, and launch of the MVP. It’s simple, but it is a lot of work, and you have to keep your customers really engaged through all of those phases. Then I can guarantee you the solutions will work, or if something doesn’t work, you can just sunset it very early in the process, but most of the time, you just pivot. You just discover something very interesting, and you understand how to solve the problem better. And you pivot early enough and you still have a viable product at the end.

Brian: Excellent. Natalia, thank you so much for coming on, Experiencing Data and sharing. It’s been great to chat with you. Is there a good place to follow your work? If someone wants to reach out to you, where would they go?

Natalia: Yeah, they can connect with me on LinkedIn. That’s probably going to be the easiest way to connect.

Brian: Okay.

Natalia: I will be happy to have those conversations. It’s been a pleasure, so thank you so much for having me and for letting me share my experience with you.

Brian: Excellent. Cheers.