In this episode, I’m exploring the mindset shift data professionals need to make when moving into analytics and AI data product management. From how to ask the right questions to designing for meaningful adoption, I share four key ways to think more like a product manager, and less like a deliverables machine, so your data products earn applause instead of a shoulder shrug.

Highlights / Skip to

• Why shift to analytics and AI data product management (00:34)
• From accuracy to impact and redefining success with AI and analytical data products  (01:59)
• Key Idea 1: Moving from question asker (analyst) to problem seeker (product) (04:31)
• Key Idea 2: Designing change management into solutions; planning for adoption starts in the design phase (12:52)
• Key Idea 3: Creating tools so useful people can’t imagine working without them. (26:23)
• Key Idea 4: Solving for unarticulated needs vs. active needs (34:24)

Quotes from Today’s Episode

“Too many analytics teams are rewarded for accuracy instead of impact. Analysts give answers, and product people ask questions.The shift from analytics to product thinking isn’t about tools or frameworks, it’s about curiosity.It’s moving from ‘here’s what the data says’ to ‘what problem are we actually trying to solve, and for whom?’That’s where the real leverage is, in asking better questions, not just delivering faster answers.”

“We often mistake usage for success.Adoption only matters if it’s meaningful adoption. A dashboard getting opened a hundred times doesn’t mean it’s valuable... it might just mean people can’t find what they need.Real success is when your users say, ‘I can’t imagine doing my job without this.’That’s the level of usefulness we should be designing for.”

“The most valuable insights aren’t always the ones people ask for.
Solving active problems is good, it’s necessary. But the big unlock happens when you start surfacing and solving latent problems, the ones people don’t think to ask for.Those are the moments when users say, ‘Oh wow, that changes everything.’That’s how data teams evolve from service providers to strategic partners.”

“Here’s a simple but powerful shift for data teams: know who your real customer is.
Most data teams think their customer is the stakeholder who requested the work…
But the real customer is the end user whose life or decision should get better because of it.
When you start designing for that person, not just the requester, everything changes: your priorities, your design, even what you choose to measure.”

Links

• Need 1:1 help to navigate these questions and align your data product work to your career? Explore my new Cross-Company Group Coaching .
• For peer support: the Data Product Leadership Community where peers are experimenting with these approaches.  

Transcript

Brain: You are now experiencing data with Brian O'Neill. Experiencing data explores how product managers, analytics leaders, data scientists, and executives are looking at design and user experience as a way to make their custom enterprise data products and analytics applications more useful, usable, and valuable.

And now here's your host, the founder and principle of designing for analytics. Brian O'Neill.

Welcome back to Experiencing Data. This is Brian T. O'Neill. Today, the podcast title From Data Professional to Analytics and AI Product Management Mindset Shifts to Make. Why This, why now? Well, I wanted to talk about this. Transformation or this ways of thinking at a high level that I want to see data professionals make when they're going to shift from traditional analytics and data service type work into.

Product management where you're managing data, products that use analytics and ai, particularly for, uh, decision support, uh, types of activities. But even more broadly than that, what are the mindset shifts that you need to make to be successful in that role? Uh, and to actually embrace the spirit, literally the title of product management as well as the spirit of this role, the way I see it.

So that's what today's. Episode is about, so if you've been thinking about stepping into this role or you're just trying to kind of feeling maybe a little bit of imposter syndrome about what exactly is my role? I'm kind of winging it as I'm going here in this new data and AI product management role that you're in, or maybe you're thinking about how do I understand if my team is living up to the expectations of what this type of role is about?

This is kind of the who this. Episode is four. So a couple things here. First, let's talk about kind of framing the challenge, uh, today, analytics work in particular, uh, which is a lot of the work that data people do. It's about analyzing data, looking for answers, producing answers, making answers easy to give or to explain to someone else.

Uh, so we're often doing this. Preparation work, investigation work, this type of analysis, it's literally in the word analytics analysis, right? Product management requires us to define and frame what the right questions are to ask in the first place, and not necessarily to focus so much on doing the delivery of analytics work, solution creation type of work.

I think that. Ultimately product management, whether you're doing internal or commercial, typically there's going to be some type of fiscal outcome, a return on investment profit, reduced cost, whatever it is, there's, there's gonna be financial targets associated with this. Whether you actually have p and l responsibility or you're in an internal position, there's an assumption that you are gonna help create outcomes and not just deliver more technology outputs on schedule.

Um, this is not a project management role, it's a product management role. Design is often a gate to getting to these outcomes because design has a lot to do with making sure that the things we make are actually satisfying the needs of the people who they are intended for. And when we don't have use, then we don't get business or organization value because the solutions just sit unused.

So these mindset shifts are really important and a lot of this gets into that, how we're asking questions and things like that. So we're gonna jump into that. Are you managing deliverable creation or are you responsible for the team producing outcomes that users, stakeholders, and paying customers actually value?

And there's different tiers there based on the kind of work you're doing, whether or not who, who is your target? Customer, audience, user, stakeholder, even defining those words and what they mean, particularly if you're in an internal role. It's really important to understand how to define those words. So I've got four key ideas I wanna share with you today.

Let's jump into these. So the first one, this idea, we've sort of touched on it, but analysts give answers and product people ask questions. Again, I gotta, I gotta credit this framing to Andy Sutton from the Endeavor Group down in, down under, in Australia, one of our original, um, DPLC data, product leadership, community members.

Uh, he runs a large, uh, data and, uh, software organization and side. This hospitality company, it's the largest hospitality company down in Australia. And he really said, I forget when he said this, but I really latched on. It's such a smart observation. Analysts give answers and product people ask questions.

So part of your job is to ask really good questions in order to inform whether or not the delivery teams are engineers and our designers and our SMEs that we're actually building the right stuff because the game is no longer about build the stuff, deliver the stuff on a schedule. Then someone else is supposed to do something with it, and then maybe some kind of value gets created down the line.

No. Now it's about making sure that we're building the right stuff. Okay? So delivering the best possible answer quickly. This is kind of more of an analyst mindset, and I want you to be focused on. Uncovering the real problem that needs to be solved before deciding what to answer and or build. As is much the case when we're talking about, uh, data and AI products, it's easy to get lost in precision and accuracy and all your knowledge about the ins and outs about how well, what's a customer in the database.

And we have four different definitions here. And it's like, there's a, all, all these minutia details that, that really get into accuracy specifics. And it's not to say that that doesn't matter. It is possible to give a really highly accurate answer to a question that no one's asking. So most of the time this is gonna mean leveling up and making sure that the big picture is being addressed.

So let's look at, uh, uh, an example here. A marketing VP might ask for something like a, uh, I need a segmentation dashboard. And instead of just building it a product mindset, a product designer or a product manager might ask, well, what decisions will, will this solution, this dashboard help you make? If we were to have that, like right now, what kinds of actions are you struggling with today and what did you use your, your, whatever you do currently to answer questions that you like.

If you, if you want a new, is it a new segmentation dashboard or do you have some solution you're using right now? If so, what? Can you show that to me? And what was, when was the last time you used it and, and tell me what happened. Tell me a story about the last time you used this thing. What we're looking for here is the signal behind the ask and, and this questioning and this hypothetical might reveal the need for, say, a campaign prioritization tool, not a segmentation dashboard.

So this kind of leads me to this idea that there are latent problems and active problems. This isn't just true for data teams. I think this is true for even a lot of product teams, design teams as well, but a lot of teams, particularly when you get into larger organizations, they're doing answer fetching and, and project work and data drive through type of work and assignment work and the next stage of that.

That's kind of the first stage. The second stage, when you get past the, well, we follow the orders, we did what we are assigned to do, and it's not our problem if somebody doesn't use it. 'cause this is what we were told that we need to make. The second tier there is, is actually starting to question these problems on the surface, and generally we can get to a little bit, you, you kind of like pull back the first layer of the onion.

You know how the onion has this dry skin on the outside and if you just go down like a, a layer or two, you can kind of start to surface what's actually behind this, this need here. And hopefully in this case what you, you end up with is you have more. Satisfaction from users. You, you gave them a better version of the thing that they asked for, and the thing that they asked for was perhaps more or less correct because I'm, I'm not saying that customers and users never know what they need and we should completely not listen to that at all.

It's more this idea that requirements are friendly lies sometimes, and we need to know how to interpret that information. Ultimately, tier three thinking. Is actually solving the real latent problem, and, and this is the problem that users aren't actively expressing to you right now, and it's the problem.

They may not know that they have, or they've just never stated it because it might seem impossible or just so obvious that, well, of course we would love that, but there's no way we could ever do that, so we're not gonna ask for it because they've made a lot of assumptions about things. These kinds of insights are usually only things that emerge from deep habitual.

User research such that you can begin to see patterns that nobody else can see, or you're, you're seeing observations and you're drawing connections between things and it, it's kind of like if you, if you're inside the jar, it's really hard to read the label and people aren't obs you know, when we go about our daily life, we're not.

There's no like person looking down at us and how we're living our life and all the decisions we're making on all this thing, but in this situation, we literally are trying to watch that and we can't necessarily ask the person that's the character in this little movie that we're watching to explain everything about what they're doing and why they're doing it and all this.

Some of this we have to either probe for verbally or we need to simply observe it through behavior. And this isn't a one-time kind of thing. But really when we can get to what the underlying latent problems are, this is where we might be able to create really outsized value because we start to really address the underlying, uh, need that has not been stated, uh, as of yet.

So what can you do with this kind of key idea? Number one, I think before you start on product work, or even if you're kind of in the middle of stuff, is writing down, you know, three to five problem questions. That are not, that are assumptions or hypotheses that you really need to, to validate and go spend some time to actually validate this problem space that you're currently up.

Your solution is addressing some problem space, and I want you to go out and spend some more time in this space talking to users, observing users, to make sure that you've actually uncovered this space and see if you can get. Either to that kinda level two, where that first layer of onion is peeled back or three, if you can actually hit on, and you'll probably know it when you hear it, um, because the users probably as they're talking to you, they're gonna run into it and say something they've never really said out loud before.

Or it might seem totally obvious to them. You're probably gonna know when you start to run into that space, and it doesn't mean that's a guaranteed win. Go out and build that thing and everyone's gonna be happy. Obviously we'd be looking for patterns here across multiple different users to, to kind of revalidate that this latent problem actually exists more widely.

But the first thing to, in order to do any of this, you need to, to start out by having a sense of the questions that you're gonna ask. So that's kind of what I would recommend doing here, is to, at least if you're listening to this show, hopefully you're already kind of operating at, at, at tier two. And if you're not, that's okay.

But we really wanna get out of this. All requirements are correct, and if I just build what was asked for and I use Scrum or I, I go through some of the rituals of product management that I'm there for, creating the value that I'm supposed to, especially for people in leadership positions here, more senior PMs and things like that.

This game is really about creating value and it's not enough to simply go and do that delivery work to spec. Okay, so that's the first one. Um, number two. Change management is designed into the solution and not bolted on afterwards. So the old mindset is that delivery is the finish line. The product is in production, the model is in production.

Change management is someone else's job. The mindset shift from that position is one where we think, Hey, if. If the right people don't adopt this product and we see no meaningful change in our primary success criteria and benchmarks, then this solution has failed. Even if delivery was on time, on budget, all that good stuff and met Spec, this is a outcomes and results based approach to doing product work instead of a deliverables based approach to doing product work.

So let's, let's talk about like an example here. Of course it's going to use AI 'cause that's all you're hearing about all the time. Doesn't really matter. But let's say, um, the company wants to use, uh, ai, the company you're operating at wants to use, uh, an AI chat bot to manage all tier one customer support requests and hidden in that is probably a goal to reduce their operating costs around live human customer service people.

So. Traditionally, a digital and or data team might buy something or they might race to go out and start building a bot that could go and answer these things, and we'd train the data on all the documentation we have and all the recipes to resolve issues and figure out what all the top tier issues are in the first place, et cetera, et cetera, et cetera.

And initially, the customer service reps themselves, the actual humans doing this work. What they're probably seeing right now is the beginning of the end of their career or their job, and not necessarily a solution that will allow them to do higher value work that could prop them up, that could support them, that could allow them to really shine and do the things that they're great at.

And keep in mind, customer service people are often the ones talking to paying customers just about more than anybody else. Especially if you have a product that's kind of a set and forget it kind of thing. And there's really not a a lot of need to come to CSRs, but when they do, people want help. So this product and user experience kind of approach, we would be wondering how can we make CSRs lives better by offloading highly repetitive and simple tier one types of customer supports.

Uh, tickets. I will say like specifically to customers, you know, like customer support types of issues. Sometimes well fix your product and service and you may re reduce the need to have inbound customer service requests in the first place. But I'm gonna assume for whatever reason we're not doing that.

And this is where designers would really carry this kind of thing forward and kind of say, well, what's the actual, you know. First principles thinking here would step all the way back to like, well, why are so many tickets coming in about resetting passwords? Well, maybe there's something wrong with password reset user experience that people can't, that feel that they need to contact a human being to get help with that.

So you can see here where we're, again, we're starting to look at this from the customer's, uh, perspective, and we're really trying to get to the first principles, but I'm gonna ignore that for a second. Let's just assume that. You can't touch that, or whatever it may be. We're, we're living in this space of we have these inbound tier one support requests coming in and they need to be serviced by somebody.

So how could we make the CSRs lives better if we were to take that work away from them, offload that work from them? Are they excited to let go of that work? Are they afraid of that work going away? 'cause there may not be needed as much. What would be true in the future? Say X stays out from release. For these users.

In this case, I'm talking about like A CSR that's using an application that probably involves this bot doing some of their work. What would need to be true and better in the future for the org? What would need to be true and better in the future for the paying customer? The one who's actually on the other end of the chat, you could carry this forward to.

What about the managers of the customer service departments? The point is we're looking at all these humans in the loop and we understand that change is difficult and we need to think about change while we're making the thing that we're making. We can simplify change management by involving these parties, especially end users of the actual application.

In this case, it would be probably both paying customers but also. The CSRs, if they're the ones that have to manage the bots that do some of this work, we would involve them in the creation of the thing that we're making. They become a first class, uh, person, a first class, uh, stakeholder, a a design partner, sometimes language that we use for the org.

Baseline values that we can observe and test against could be, you know, things like volume of ticket requests, uh, going down or answering these questions faster, answering these questions 24 hours a day, seven days, a uh, a week, things like this. And we can obviously create estimates of fiscal value here, right?

By looking at cost reduction and, and all of that for our end users of the application internally, these CSRs. We might explore, say the idea of customer service reps actually managing their own little workforce of little AI agents that are doing this work with the ability to train this workforce, to work the way that they work because they've, they've known how to do this work for a long time to override conversations when necessary to establish incentives for them to keep low priority tasks handled by the ai.

For higher important ones to be human controlled, what would those incentives be? Are the incentives in place now? Correct. Given the change we wanna make. Where is their CSR labor work that these people hate doing right now? And could this new data product be refined to actually remove some of the stuff that they actively dislike about their job right now?

Even if some of that might be an ancillary to say. Building the model of the actual bot and the number of questions that it can answer and all this kind of stuff. We would also be, want to be dialed into what? What do CSRs find frustrating about doing their job efficiently with, especially as it relates to technology.

Might we explore the possibility of gamifying something like this, making it fun? Whose team of AI bots can outperform? The other team by using your own. Maybe there's a way for the agents to do some, uh, specialized training themselves to give shortcuts that might not be in the official documentation, but there's a way for, uh, you know, these managers to, to help influence the way the agents answer questions.

And so there might be a little bit of uniqueness between the different bot teams and this be, could become a friendly game. This is me just imagining solutions. I'm not saying you should go out and do this. What I am looking for is what are the in what, what? How could we delight this CSR such that we don't have to try to force them to use a new solution?

I wanna figure out, well, what's in it for them? If we are to take away some of this work? The only way to do this is to get involved directly with these people. We need to understand what it's like to do the work of A CSR. We need to shadow them. We need to talk to them. We mostly need to listen and watch what they're doing, but our goal is to design affordances into our user interface and to our experience so that this group is also successful, not just the business by saying, Hey, we offloaded all this human labor and look at the dollar savings.

That's part of it, but what are the downsides if that's the only thing that we pay attention to? Other stakeholders. What about the paying customer? Well, they, they likely have zero control in the first place about this, except that they might be able to give us feedback on the solution, depending, I, I'm making judgements here that the decision to do this has already been made.

And talking to paying customers, regardless of what they say. If there's a cost cutting goal in line, unless that organization is product and design mature, they're probably gonna do that and wait for the downside come to come later. 'cause they wanna see a short term financial, uh, benefit. But could we use sentiment analysis and shadowing some of these, uh, conversations to understand if these support tickets are being resolved without the company brand being soiled.

Could we continuously sample some of these conversations to see whether or not a ticket was actually closed in a positive way within a, an acceptable timeframe? And qualitatively actually look at some of these so that we can see whether or not, yes, it, it was closed and under two minutes, which is great, and it was closed at all, which is also great.

But what did the conversations look like when we sampled them? Were people angry? Can we see frustration in the dialogues that they're having with the bots? Might we give our paying customers the option to use human or the AI bot perhaps explaining that the bot might be faster and it might be worth a try, and the human might be slower 'cause you might have to wait.

But we'd give them a choice, especially early on in the deployment of such a thing. And could we learn from marketing how we might talk about what the benefits are to using. AI assistant to answer your customer service issues versus using a human. So now we have marketing potentially involved. All going back to this change management theme about putting the change management into the design of the solution.

As a product person, we'd also probably be thinking about placing bets, keeping our bets small, deploying this solution to a, a limited set of teams or people. Trying to incorporate fast, rapid learning cycles, get feedback early, look at where all the unexpected things are, and people are gonna kick the tires.

They're gonna ask silly questions that have nothing to do with customer service, and we wanna start seeing what the fringes look like and a probabilistic solution like this. And we want to contain that risk. We wanna move quick and we wanna learn and we wanna roll that feedback back in. 'cause we definitely don't want to do damage either.

While we can see, oh, cut human customer service equals direct cost savings. But what that's ignoring is, well, what are the byproducts and, and the side effects that can happen from that, which may not seem like cost right now, but they, they could become costs. So what can you do about this right now, this big?

Number two, I think spending time observing where users and stakeholders dislike their current solutions, processes, applications. This can help us start to understand. Where delight might be able to be brought in. And if we're thinking about delight as being something we're designing for here, that's gonna reduce our need to, to go through a change management process against people's will, because we will be building in desirability into the solution from the beginning.

You could also spend time uncovering where organizational culture, politics, or compliance things may create friction or roadblocks. This is something we do collaboratively with relevant stakeholders. We would be brave and we would surface some of these things that we found out and that we've heard, and we'd say, look, I, I know we wanna do this.

This is the overall, when we've talked to 15 people in this CSR job, this is what we've learned so far. This is how the company is perceived. This is a hypothesis. Maybe we need to go out and validate it, but we wanna bring this up because we don't want to build this AI agent. Try to roll it out. It's against everyone's will and then it falls on its face and we do some damage that we haven't really calculated.

No one's really asking questions about that. So let's talk about where friction is gonna happen such that we can make all of this part of the deployment of the product includes addressing some of these types of issues. And I'd wanna understand if I was in product, like where have things, where have rollouts like this not gone well in the past?

I'd probably start with the CSRs and ask them, when was the last time you had to change your technology solutions in a large way or the, your tooling or your processes? What was rough about that? What about your question scripts and the way you talk to customers? What could the company have done better?

What could technology have done better? What kinds of fears do they have? These are all more product oriented ways of thinking. Again, I'm gonna restate key idea number two. Change management is designed into the solution, not bolted on afterwards as a task that you or some other team will then go do.

Once they have this finished thing, this, this application or tool, someone else goes and tries to plug it in and get it used. I want you to kind of let go of that thinking and, and see that as a high risk way of doing things.

Number three, key idea. Number three, adoption of data products only matters if it is meaningful adoption. So the old mindset here is counting usage. ME metrics, and I see this quite frequently. How many unique users locked in? How many minutes did they spend on average? How many interactions do they have?

We look at. Google analytics tool, equivalence, whatever you're for, whatever your internal metrics are, and chances are if you're deploying something through a tool, you have some way to count things. So what's the shift? The shift here is that meaningful adoption isn't about sessions and durations. It's about reducing frustration, increasing delight, and then seeing downstream fiscal value emerge as a result of facilitating meaningful adoption with the solution.

This also assumes that if we're increasing satisfaction and making better things, that we have a baseline from which to measure this change from status quo to satisfied or to delight, depending on how far up you want to take things. If we cannot measure, or we do not have a baseline from which to measure where we're starting, then we really cannot measure the change that has happened.

And we're now relying on hearsay and squeaky voices to tell us if we did a good job. I do care about qualitative feedback from important stakeholders. I think that's, that's an okay thing here, but adoption actually needs to be defined. What does meaningful adoption mean? And if so, then what is unmeaningful adoption?

What does that look like? It's the change that we need to design for. We need to have a definition of meaningful because that's not the same thing as adoption. Well, why? Because high usage metrics, as you've probably heard me say on the show before, can actually mean there's a problem with the solution.

It's clunky. It takes too many, too much time to use the solution to get the thing you want out of it. Wow. Customers on average, used our customer support bot for 42 minutes a week. Whenever they had a ticket at average, 42 minutes. That must be great. Except we know in customer service world, more time is not good.

Generally less time spent is actually what we're going for here. The problem is, if you don't know how much time an activity should actually take, there's really no way to know if any of the signal you're getting is meaningful. Because adoption, a meaningful adoption rather, is a qualitative state of adoption, meaning.

There's meaningful and there's not meaningful. There's a range here, and if nobody can count what meaningful means or describe that, then it's all kind of hearsay about whether or not we're actually doing work that matters here. So like an example here, a sales team dashboard that requires, you know, 12 clicks per update.

You're getting lots of heavy usage. You can see they're clicking all over these different screen, these screens, et cetera. But when you actually go and do a one-on-one with one of these people, you find out that they're quite frustrated by this solution and how many clicks it takes. They have to export stuff, they gotta put it in their other tool, bring it back into the CRM, update some fields, recalculate some model score, and on and on and on and counting.

Analytics is really telling us how much is the product being used and when. But if we're actually measuring user experience and meaningful adoption, we need to be dialed into feelings. And I know it's scary to talk about feelings on a data podcast, but feelings have a lot to do with people's, whether adoption is gonna be meaningful, it's gonna tell us about how successful this person feels.

Success may be just simply, I just, I don't wanna spend any time here if it's like, it's like going to a public bathroom. The best experience may be simply the shortest, most invisible experience possible where you feel like you're never even in that bathroom. You're just in and out, and, and maybe there's a way to make that even better.

But there will be oftentimes, particularly in business software, where we're simply making, uh, usability and utility really high. That might be what we're looking for, but someone has to define what useful and usable actually means. What is not useful and not usable, mean? 'cause the change is what we're after.

The change is the score to the game that we're playing now. So what can you do with this one, this key idea? Number three, adoption of data products only matters. If it is meaningful adoption. Well establish a baseline of what it means to have meaningful. Adoption and non meaningful adoption. There's no right or wrong answers to these things.

Sometimes you have to put, uh, especially when you're doing zero to one product work, you might have to put a hypothesis out there, get it out, evaluate your test, your, your hypothesis against reality, and then. Recheck your assumptions. Maybe a only a 10th of the people have even touched this thing. Okay?

But now we have some signal back here. Maybe in round two when we work, when we go back and revisit what have we created, have we created value? Then we can adjust our baseline, uh, accordingly. But for those of you working in large companies, a lot of times there's already existing processes and product technology solutions and all this kind of stuff.

It shouldn't be that hard to establish a baseline. Of what good adoption means because chances are there's already bad adoption or bad user experience or usability problems, utility problems, trust problems, speed problems. It should be, there should be a way to define this. There must be a way, 'cause if you literally someone is paying for this product, it's probably 'cause there were enough negative attributes.

The issue is when we don't really get into defining these success criteria and metrics upfront. And instead we wait until the end only to then find out what is the score keeping method that the people in charge are actually using. And I will say this, you cannot ask users necessarily what a good user experience for them would necessarily be.

They're gonna start just thinking aloud creatively, and they might give you a bunch of information that could be false signal here. I would rather you really understand that. First of all, understanding past behavior is, is a better in indicator for you to set a, a baseline 'cause that's factually what's actually happening, regardless of what people are actually saying is happening.

And those aren't the same thing. But you should also know that stakeholders and or users, if you're involving them in this process, they may not know the answers to these questions. They won't be able to tell you what meaningful adoption means. They're probably, especially in the business and the senior, uh, our senior business partners.

If they're gonna be glad that you're actually thinking about the quality here, that there is a definition of quality here, and that there's a definition of low quality. And so it's not that they're hiding the information, it's that no one's asked the question. And so this early discovery work we do when we talk to stakeholders and users, this, this is an exercise at figuring out, well, what, what it does it mean to have meaningful adoption?

And the right answer is the one. That you and, and customers and product owners and stakeholders all come up with together. And that's, to me, part of the work of product management is to put definitions around these things such that we can play a game that can actually be measured so that we know the score of what we're doing.

And we're not scoring project management metrics such as did it, did we deliver the tech to spec on time? This is project management. It's not product management. Okay, number four, last one For today. Thanks for hanging in there. Solving active problems is good, but the big unlock is when you can surface and solve for latent problems.

Lemme say it again. Solving active problems is good, but the big unlock is when you can surface and solve for latent problems. So old mindset. My job is to respond to stakeholder requests and build and deliver this data-driven solution or AI solution as requested. The shift here is that the most valuable work that we might create for our stakeholders and organization is gonna be when we can solve for these unspoken, hidden needs that stakeholders and or users never thought to ask for in the first place.

I kind of mentioned this earlier, but I think this, this one deserves to be a key idea of the four in this particular episode. This is really important. So let's look at an example. You know, analysts may field weekly, you know, ad hoc requests for, let's say I need a list of top customers by revenue, uh, from the marketing department.

And so a PM might mindset might see a latent problem that looks like this. Well, it sounds like these marketing stakeholders lack a way to explore customer lifetime value. And they believe that looking at top customers by revenue, this is the technical delivery, ask the output or solution ask. They believe that this is the way to answer to, to in order to explore this customer lifetime, uh, value.

And, and this is a proxy for doing that. Furthermore, as a product person, if we or, and, and, and pro and researcher mindset, when we dig into why they care about any of this, we find out that they're actually really trying to do narrow band targeting in their campaigns, and they believe that customer lifetime value or revenue, which is a proxy for them, for customer lifetime value.

This is the way to do narrow band targeting in their campaigns. And why do they care about that? Well, they care about that because they want to be able to show, Hey, I spent all this marketing dollars and look at my, my CPL, my cost per lead, or CPA, my cost per acquisition, or whatever game that they're playing in their little world.

That's ultimately what they care about. So they've given you a solution that they think will help them go solve their problem and play their game better. Sometimes when we do this laddering exercise, these asking these why questions and asking 'em to open up, we might realize that, okay, really what you're doing is trying to do better targeting.

And you think looking at our top customers by their revenue spent, like the amount of revenue they generated for us is the way to find other clients like them that we could, uh, double down in our marketing effort or whatever it may be. And, and this is where the data person might say, you know. We could actually create a model here that could look at more than just customer lifetime value.

And in fact, we might know something about what actually drives people to buy and how would you, dear marketing person, how would you feel if we, if the data suggested that there were other feature, other metrics and machine learning language, we're thinking about features now that there's other attributes of this customer that actually might be more related to a buying signal.

So now we're into a different kind of conversation, but this VP of marketing is now hearing that you are dialed into their need, which is I need to show effective, ethical, and reasonable spend of our marketing dollars. I need to show an ROI ON when I go spend our money on marketing. And marketing is really hard and they don't want to go and spend a lot of money on campaigns that don't work because they have to answer for that.

Now that they know that you're on their team and you understand that underlying need here is really not to get a dashboard that has top customers by revenue, it's really to do narrow band targeting in their, in their, say, advertising campaigns. Now we're dialed in at a different level and we understand what's in it for this vp and how will we make that VP successful in their job.

So, again, let me restate the key idea. Solving active problems is good, but the big unlock is when we surface latent problems and we, and we solve for those latent problems that have not been necessarily expressed yet. So what can you do with this one? Again, the answer is usually going and, and, and talking to observing and mostly using your ears to learn to understand what it's like to be this vp.

To engage with them in discovery conversations, to understand what's hard about doing campaign targeting now. Like how do you know when you got it right and when you got it wrong? Are there, are there numbers that we can, uh, attach to effective and non-effective? Where did they get the idea to use customer lifetime value in the first place?

Is that, uh, best practice in the industry? Is it the boss's orders? So now we know there's some politics involved and, and belief about what matters and what doesn't, which may or may not be factually true, but it's an important thing for us to understand. If that's actually how the business operates, then that may be all that matters.

Right now at this stage, are there incentives aligned with. This VP of marketing following orders, quote, quote, to use the correct metrics and to deliver, or, or is it really about they are empowered to deliver the best results on their campaigns regardless of how they do it, we need to know what the, the, the mind, the incentives of this VP are because you might be able to come up with, uh, a machine learning model that could do a better job of segmenting.

The users, et cetera. But if this person can't back it up and can't say, I, I did use the CLV measurement customer lifetime value. I did use that as my main way of, of running this campaign. And here are the results. And maybe it didn't do super well, but I did follow instructions, I did do what we were asked to do.

You need to be dialed into that because you can be technically right, but you can be effectively wrong as, as you know, if you've listened to this show for a long time. Giving this VP of marketing a technically right, but effectively wrong solution that they're gonna ignore or not use, that is not a win.

That might be an academic win for your paper to go write at a machine learning conference about the propensity model that you built, but you're actually not creating any value for the org. And since you're in product, your job is to create value. So that's it for today. I'm just gonna restate these.

Thanks for hanging in there. There were four key ideas. For data professionals moving from being a data professional to analytics and AI product management. There's mindset shifts and there's four big ones that I was thinking about today that I shared with you. That first one, that analysts give answers and product people ask good questions.

The second one, change management is something we design into solutions. It's not something that's bolted on afterwards and, and, and delivered by somebody else. I'm being super black and white with that. I am not saying there is never a need to have any type of change management process. That's not what I'm saying.

What I'm saying is let's not rely on change management to be the determinant of whether or not this data product is successful or not. There is a whole lot that we can do to make better things that don't require as much. Forced change, encouraged change, training, threats, whatever it may be. That's not generally the best way to get people to change.

We wanna make it in their interest to change. Number three, adoption of data products only matters if it's meaningful adoption, which means someone has to define what meaningful and non meaningful adoption actually means. And big idea number four, solving active problems is good, but the big unlock is when we surface and address latent problems.

Problems that are not being currently expressed verbally or in writing. Okay, that's it for today. Uh, if you need help doing this type of work on your data, product, uh, initiatives or with your team, I am. If you got all the way to the end. I have been hinting at, and I'm happy to say that my new cross company group coaching service is in soft launch.

I am building the the first cohort out right now. You can hop onto that over@designingforanalytics.com slash group coaching. Uh, whether you're focused on your career advancement as a data and AI product management professional, or whether or not you're looking for some, uh, support, maybe you've been doing this kind of work and you, you need a little bit more, uh, support dealing with, uh, day-to-day challenges that come up in your unique environment.

That is really the, the promise of what my small group cross-company coaching is all about. You're gonna get support from me personally, uh, as well as, uh, from your peers. Uh, we'll be doing hot seats. There will only be, uh, 12 people in the, uh, first group. That'll be the cap. So once that's capped out, then um, people will have to wait for a seat to open and we will launch as soon as there are five people.

So if you move early, uh, not only will you get access to lifetime pricing, but you will also get the benefit of the smallest, uh, group size if you value, uh, a very small group size. So there's that, and if you feel like you have a good grip on what you're doing, but you kind of feel like you're working alone, we do have a forum or I, I do have a, a community that I started along with a, a large group of data product professionals.

It's called the Data Product Leadership Community. And this is really a place for you to both contribute your knowledge, your experience, and your learnings along this road. Uh, with others. And then of course, uh, to learn from them and to get feedback on your ideas because you are not alone. There are other people that are applying product management and user experience design methods of working producty, ways of working to data science and analytics work.

So, uh, I encourage you to, to visit that link. I'll give you those two links. Group coaching again is designing for analytics.com/group coaching. The community, the D plc, as we call it, the Data Product Leadership Community, is available over@designingforanalytics.com slash community. Till then, talk to you soon.

We hope you enjoyed this episode of Experiencing Data with Brian O'Neill. If you did enjoy it, please consider sharing it with the hashtag Experiencing Data to Get Future Podcast updates, or to subscribe to Brian's mailing list where he shares his insights on designing valuable enterprise data products and applications.

Visit designing for analytics.com/podcast.