Transcript:

Brian: Welcome back to Experiencing Data, this is Brian T. O’Neill. Today, why discovering valuable analytics use cases is so hard, even with AI. So, today I’ve been thinking about some principles from behavioral science and UX research that might change how you think about doing analytics product discovery. In other words, when I say this, I’m talking about the research to uncover what customers need or want from your data and insights product.

I’ll also chat about where I see the role of AI in all this, both in your product’s experience, as well as using AI in the process of doing discovery work, and how that might help or hurt you. My short answer to this is, as with anything, I think it’s yet another tool. It’s an amazing tool, I’ve been really impressed with what’s possible these days with it, but I’m always challenging myself about whether or not the information I’m getting is actually good and useful to me or not. I also want to help you, ultimately, with this episode and with all my episodes, to turn your products analytical complexity into commercial clarity, so that you don’t fall into this trap that I call building a technically right, effectively wrong product that’s full of features and capabilities, but ultimately it’s lacking in benefits. And benefits is what people pay for and benefits create products that people can’t live without.

So, let’s jump in here. The core discovery mistake that I sometimes see happening—so this episode was partly inspired by at least one, if not a couple recent discovery calls I’ve had with prospective clients asking about the process of discovering how to build an insights product, how to figure out what the needs are and all of that, and feeling that the typical design a prototype and get it in front of users and then get feedback on it didn’t seem to be working particularly well, and there was something unique about data products, and specifically analytical insights types of products, that is missing, that that process wasn’t working. So, I thought this was a good time to address that question, as well as, just broadly speaking, some more things about how I tend to think about research from a product management and design perspective, and not from an analyst’s perspective, someone that’s used to living with data and doing deep dive analyzes on different types of problems. So, let’s jump in.

The first one here, core discovery mistakes. So, one of the big patterns is teams start discovery process by asking, what data do we have? Like, what have we stored? What have we acquired or processed? And so, with this data, what models, reports, dashboards, integrations, what could we build with the data that we have? Because obviously we can’t build something with the data that we don’t have, right?

And sometimes I think there’s an assumption that because you have data set X, that your ICPs will want and care about solving problem Y with this data. And so, we have use cases that might get turned into product prototypes or even product features emerging from the stored data or from data exploration. And I think this is a very analytical and analyst-driven way to think about it, it’s a very tech-driven way to think about it, by starting with, what are the raw materials that we have, and I think discovery and product birth work when we’re birthing a new product, this is a creative act, and I think there’s a place for obviously looking at the resources that you have, including the data that you have, but you also need to wear this other hat, which is not turning a blind eye to what you don’t have, and being open to understanding that you may have an incomplete solution here, but it might be possible to bring it together with other data sources or other capabilities and then deliver some benefits there. Or maybe you need to go acquire some data. But the point is that your boundary is not limited by the data that you’re storing in the database.

So, why does this matter? I think it can lead to weak products because insights fall into the ‘that’s interesting’ bucket, but they don’t actually drive any action, so users don’t change any behavior, and you sort of end up with this thing, it’s like, “Yeah, I know I should be doing something about that,” and you know, you have this tool that’s just lingering in the background, and maybe they check in on it once in a while, but if you were just to cut it off, the phone wouldn’t ring, no one would be calling you to come and get their subscription renewed, there’s just kind of indifference there and apathy.

So, a little insight here, I think I mentioned earlier—I can’t remember if I did or not—but I wanted to tie some of this back to some behavioral science insights here, like what we can learn from that world and decisions often don’t begin with analyzes, right? So, people form a hypotheses first, and then they go seek out evidence to support or unconfirm—what’s unconfirmed? That’s not a word—to support or not support the hypothesis. I know there’s a better word in there. I was not an English major. I was a music major, so keep that in mind.

So the tactical takeaway here is not to start your discovery process with data sets, models, dashboards. And this is a very black-and-white recommendation. I’m going to try to be very black and white. There’s always gray areas and these things are spectrums, but I’m going to basically suggest today you don’t start with that, and instead, you work backwards from the end, from the customer experience with the product and instead, look at what decisions are they trying to make? And one way to inform that is what have they recently tried to—what kinds of questions have they been grappling with, whether they were able to answer them or not with insights?

Where do they have uncertainty and in particular, uncomfortable uncertainty? We all have uncertainty in our lives. We don’t really know what the weather is going to be, and we really don’t know, like, “Oh, I’m going to go on vacation to this part of the world, and is there going to be a war there or not?” Like we all live with, like, certain levels of uncertainty, but if there’s uncomfortable uncertainty, that can be a place where your product has an opportunity to shine eventually.

And then also starting with workflow friction. So, I’ve recorded a lot of podcasts about what I call FOWA, F-O-W-A or Flow of Work Alignment, but understanding where there may be existing analytical use cases that perhaps are already being solved, but the work to get to the answer to make a decision is very laborious, so where is there friction that could be removed there?

The second big item today is that insights do not equal decisions in the real world. So, a lot of real world decisions rely on pattern recognition, people’s intuition, contextual cues, things like this. So, I mean, just take right now, a whole business strategy may largely be based on the gut of a small team, or even just the CEO saying, “I think we need to go over there. That is the direction we need to be pointed.” They cannot give you all this data to prove all this.

Decisions are made all the time without data, so we need to accept that that’s a reality, and that ground truth analytics and insights aren’t necessarily going to be what drives behavior, so data can often be used to just confirm or challenge judgment. So, you should not assume, if you have an analytics background and you’re building product now, or a technical background, I think a lot of times, teams assume that if we just show them the correct data, that better decisions will happen. They also sometimes assume that because they know that data X is correlated with Y downstream outcome or change in the real world, that both customers know this and, perhaps even more importantly, that they care about this as much as data people think that they should. So, we have to operate in the reality of our customers’ head, and not what we think is good for them or what they should care about. That is not going to convince somebody just because you think that they need to care about it. That doesn’t create customers, right?

And lack of customer exposure time and getting deep into the user’s worlds means that incentive structures are often unseen by the team that’s doing the discovery work, and incentives may be what is ultimately going to drive behavior with your product. So, a basic example of this is, like, you may think that you have this amazing analysis that will help, you know, your B2B customers’ business make a really important strategic directional turn, but the end-user has very little clout in the organization. They’re making day-to-day operational decisions, and what they really care about is whether or not some number has dropped before 41 because if it goes before 41 then that’s a yellow flag, and that’s going to kick off some of their process, or it’s going to alert my boss that our numbers are below where they’re supposed to be, et cetera, et cetera, you can see where this is going. The business value that you promised was something else, but the end-user’s incentives are to make sure that the machine runs at 41 or higher and never drops below that, and your tool is there to assist them in making those kinds of decisions.

So, when you don’t have that customer exposure time, you can’t see these incentives. And this is stuff that LLMs are not going to tell you about, especially if these things are sensitive or they’re not things that are often talked about in public. They sit in the minds of users, they’re attached to people’s sense of belonging or what their value is to the organization. This is the kind of stuff that the models are never going to tell you about, that you’re really only going to get through customer exposure time. So, data without decision context creates interpretation work that falls on your customers’ shoulders.

So tactically, don’t design [laugh] try not to design your analytics features like charts and dashboards and model outputs and not think about them as these nouns. I like to think of them as designing around decision moments, around recommended actions, around moving uncertainty from a lot to a little, or even removing it entirely. The act of design is the act of making those changes and those improvements to the lives of the users, which ultimately will be reflected hopefully in the business value that the organization sees. And again, I’m generally assuming that you are working in a B2B context, you have human users of some kind, or beneficiaries, at your customer within their organization. But of course, the organization also cares about some kind of fiscal value that they’re getting from the product as well. So, you have at least two of those stakeholders that you’re trying to satisfy.

So, during discovery and design, you want to be asking what decision becomes easier if we solve this? And when you believe there is useful data to assist with decisions that customers may not think to use, this is where product design, user experience and UI design really becomes critical, otherwise, users aren’t going to connect the dots between what you’re showing them and the goal that they have or the decision that they’re trying to make. So, the more there’s a lack of current knowledge on the part of your customer to connect data set A with benefit or real world change B that they care about, the bigger that gap is, the more you need to invest in product design in order to close it if you feel like there’s a tremendous amount of business value you can deliver there. Because it doesn’t matter if, in theory, you have all the building blocks because theory isn’t what is going to sell. What’s going to sell is when the recipient feels the change, they feel the confidence, they have a good experience, and downstream from that, the business can attribute benefits to that decision clarity or confidence or you know, ROI changes, or you know, sometimes we can point to analytical changes as a result of the insights that we’re providing, not all the time. So, that’s, kind of, the tactical things you can take away from that one.

So, big idea number three, prototypes. Let’s talk about prototypes as a means for discovery. So, I think that prototypes have a place, but they can also bias analytics product discovery somewhat significantly. So, I think what happens, often, if we have, I will say, untrained researchers demoing prototypes to prospective customers, my general feel is, I’d rather you take imperfect action and do something rather than nothing, but this is where we get into what kinds of feedback are you getting based on the questions you’re asking? Are you asking good questions and is the information you’re getting back believable or not?

So, this becomes a question of, how well are you executing the prototype research that you’re doing in order to inform your product roadmap? So, users might comment on charts and filters, layouts and colors and UI tweaks and button placements and they’re going to react to what is visible. But the real question is often not seen, which is, are these insights, is this UI, is it changing any decisions or is it informing any decisions or not?

So, in the process of showing a prototype to figure out, hey, is this product something you would pay for, you’re actually getting feedback on your interface design, but not really on the product’s value. And I would even challenge you to say you may not be getting useful feedback even on the interface design based on what you’re hearing, especially if you believe that the customer is always right. Because customers aren’t designers any more than customers often aren’t analysts, necessarily either, but they are often going to try to help you because they want to be useful and helpful. They’re going to tell you all kinds of stuff. So, the question is, which of that stuff should you be believing as being really, truly informative to the direction you want to take the product in?

So, I would avoid conditional questions, meaning questions that often use the word ‘would’ in them, W-O-U-L-D, I mean [laugh]. So, “Would you use this dashboard,” for example? “Would you use this chat bot to do X?” Instead, I would look at your prototype, your design, or your dashboard, whatever the artifact is that you’re showing, I would think of this more as a prop. It is a prop to get discussion going. Especially if you’re not always sure what questions to ask, a prop can be a really good way to get conversation going.

So, you could ask them, “What if any questions at all does this data seem to be answering for you personally?” And, “How does this design answer those questions either better or worse than the current methods that you’re using?” In fact, “How do you currently answer this question at all?” Like, “Do you even try to answer the questions that you think this data is providing to you?” “What’s your level of uncertainty about what you’re seeing here? Is it high, medium or low? And if it’s low, what are you going to do about it? What do you currently do about it?”

And I’m always interested in these kinds of questions about current behavior. This is a better indicator for you of what to believe in the future is understanding what they’ve been doing so far, and again, watching out for those conditional questions about what people would do, which tends to be an unreliable question. So, asking what kinds of information has changed their mind in the past, and if they start giving you ideas, that’s fine. You thank them for those. I would love to see if those ideas that they’re giving you exist anywhere else.

Like, they got those ideas from somewhere. Is this because you currently do that in your Excel workbook that you built, this custom tool you built or whatever it may be? I would want to try to validate those by seeing that they’re actually using those ideas right now and they’re not just coming up with creative ideas and throwing you all kinds of potentially noise. There could be some signal in there, but you need to just be careful that you’re likely going to be biased by hearing this wish list spewing from them and thinking, if we build the wish list, they’re going to buy. This is the trap, and it’s hard to unhear these things unless you’ve really been trained in doing research and you know what kind of signal you really should be paying attention to or not.

Some other questions you could ask, “Do you have an alternative solution?” “Walk me through the last time that you used it, step by step.” And really, what we’re trying to do is validate here is, do they really have an alternative solution? Can they even remember what they did with it? This is going to tell us a lot about, is this solution really getting used or not, if they can demo it to us. And if they really can’t remember, you’re sort of calling them out that they’re not using it, and you have to be careful because the goal is not to shame them, but they may feel like, “Yeah, yeah, I’m supposed to be going look at this thing, but I just kind of use my gut, and it’s pretty—you know, no one ever says anything. It’s good enough.” Be mindful of that context.

You could ask, “Where is it taking too much time or effort to feel like you’re moving forward with this?” Because the goal, you know, when we think about analytics and insights, the goal is usually not to go and do analysis work. The goal is to find an insight that helps you reduce uncertainty and make a better decision, or even make a more confident decision. You could ask, “Is there a point where you’d need to revert to another tool?” Such as exporting data to Excel the classic one, or putting this in into an AI model, or connecting it with another service? Where are they doing that now? Again, go back to current behavior, current examples of this stuff, but what we’re trying to do here is illuminate whether or not there is a perceived gap there.

And again, they may give you solutions to stuff that does not exist here, but you can ask questions about, “Oh, so you would want to connect this to Zapier. Well, why would that help?” Because they’ve just given you a feature requirement—connect it to Zapier—but the customer’s goal is not to go and connect software to Zapier. That is not the goal. There’s something downstream from connecting the service to Zapier that they think would provide them a benefit. So again, you’re listening to what’s coming back. And this might have been inspired by your prototype, but really, what we’re doing is we’re problem finding. We’re not doing design validation work; we’re in a problem-finding mindset.

Number four, the big idea number four, data availability bias. So humans, I think, sometimes assume that available information must be valuable. And I think analytics teams, or teams that have analysts and technical people on them building products, assume that oftentimes, if a data set exists, it must contain useful, informative product roadmap information in it. Or, rather, sorry, I should say it contains information that would be useful to have in the product. We can see where bias kicks in all the time.

I mean, look at if you’re—I’m sure almost everyone here by now has been using ChatGPT or other Claude or other large language model interfaces and we can all see how often they, without doing some tuning, they are very sycophantic, and they like to tell us what a great job we’re doing, and all of this. That there is bias here. You can even just challenge the response half the time, and it will give you a different response just by challenging it. Even if you believe it’s totally correct, it may bow down to whatever you said, “Oh, you’re totally right. That’s not a great answer.” Right?

So, there’s bias in the models now. There’s also bias in the data sets, which I’m not going to spend a lot of time there. We know that there can be bias in the data sets for all kinds of reasons, but there can also be bias in assuming that because you have acquired all of this data, that you’re going to now productize, that there must be value, so there must be a reason to show that. And you might as well show everything, expose everything, just in case. At some point, someone might need to drill down into that telemetry and correlate it with something, some use case that no one can actually think about, but just in case, we should surface it.

And so, there’s a cost to that. All this data being made available that’s not actually providing any signal, this can cause problems in the product. So, customer needs really need to be driving all of this. And it doesn’t mean you can’t use available data to see if you can figure out what needs might be answered with it, but I don’t think the idea is to just provide all the data all the time because people may assume that the reason they put this here is because it’s valuable, not the product manager didn’t really know, so they just made everything available in the tool, just in case. Because we don’t really know what someone wants to do, and we don’t want to think about it, so we’re going to push the question-asking and analysis work and the design work onto the customer. Here are all the building blocks. It’s a giant pile of Legos with no picture. You have no idea what it can do. It’s just all the bricks. Now, go assemble it into something. Figure out what the problem is. Right?

So, don’t ask what we can build with the data that we have, and don’t assume that customers know that data set X is going to improve outcome Y for them or that they care about that, even if they do understand it, that they necessarily care about that. So instead, we want to be asking what decisions are currently being made with poor information. Who owns these decisions? What do they need to feel confident about making a better decision, particularly if the data might tell them something that they didn’t want to hear? They came in for confirmation and they’re not getting the confirmation that they seek. So, you want to be working backwards from this point of decision back into what your product’s current available insights and data are.

Number five, let’s talk about AI and particularly large language models here for discovery and product user experience. I think there’s an emerging belief in traditional, like, BI and analytics products, that dashboards may eventually become unnecessary, that even having a structured UI and UX may not be necessary, and that instead talk to your data. So, give them a way to talk to their data. Users can just ask the questions and get the answers. So, talk to your data, tell it what you want, and that assumes that the answer to the question is going to be a data readout.

So, why does this sound compelling? Well, having language models involved lowers the barrier to asking questions. So, we presume NoSQL is now required that can all be translated in the back-end by looking at the prompts that are coming in and translating those into SQL behind the scenes, even if we’re going to show customers what the eventual SQL was that did the number pulling and that kind of thing. It also promises no dashboard or product training would be necessary. And you have probably most of all this flexible exploration tool, infinity number of use cases that theoretically can be satisfied. We can’t even imagine all the use cases that are there. So, it feels like discovery can just kind of happen on demand. We don’t need to put these tight guardrails around the product.

I would say to this, that behavioral science, or even my gut, would just suggest there might be a problem here, which is that users may not know what questions to ask. So, if discovery is already failing when users can’t articulate use cases, or teams work backwards from data to arrive at a hypothesis, or users haven’t even used traditional BI reporting successfully such that they have what you might call data literacy, then I think conversational interfaces might even amplify some of these problems. Because now the user must also invent the questions to ask, which might be the hardest part, this blank box, this cold start problem here, now they have to decide what’s important to ask.

The analysis work, they have to know how to keep following up. And again, prompts, these tools are getting—the models are getting better at actually prompting us with good questions, and I do like that. Again, we get back into this issue of, like, is this line of questioning I’m being asked, is this driven by what’s right, or is the model just guessing here? So there’s, again, we get back to this idea of quality and ground truth. How do I know that these questions are relevant? But there is more analysis work being put on the customer instead of the tool.

And then there’s that judgment of, what am I being told, both in terms of the questions the AI is asking me as well as the responses that I’m getting back. The new failure mode here then is that you ship a product or product features, which it’s basically this infinite exploration tool, but it’s not a high value decision support tool that’s so indispensable people can’t live without it and they want to pay for it. And those things aren’t mutually exclusive necessary, but I think that’s the risk is that slapping a large language model interface, a chat interface, into your tool turns it into an infinite exploration tool, but the customer’s goal is not infinite exploration. That is not the goal.

I also have this hypothesis. This is not grounded in any data or anything like that, but I have this feeling that when there is so little effort required to get a useful answer or insight by which I might make a decision with data, there may be lower user confidence in making a large, significant decision based on that information. Time will tell about this, but I think there’s something here about the speed at which we can get answers to analytical questions, data questions, that it may actually slow people down in terms of making decisions. Now, that actually might be a good thing, that we’re using that kind of System 2 thinking, where we’re being analytical about what we’re getting back from the model, so I think that’s not necessarily a bad thing, but it’s something to be aware of here, in the speed of delivery of responses versus the quality and confidence the end-user is going to have. I think this is a really interesting UX space to be considering if you work on analytics products.

So, yes, what else? So, the users ask questions, like, you know, “Show me revenue trends,” or, “Show me which customers churned,” but the system still hasn’t solved the real problems, which is, which decision should the user make differently today as a result of these kinds of questions? Like, what decision would they eventually make when they’re asking these kinds of questions? Users may also ask data questions based on what they think is stored, so what they think the product’s scope is. So, we’re back to the traditional search, when you think about search UX, the search problem of understanding intent, not just literal queries that are being typed into the prompt box.

So, I think successful implementations would need to spend time to understand queries and challenge users on what they’re really trying to solve because I think a lot of the time it’s not really—show me the count of how many times something happened. Show me revenue trends. Okay, count past revenue and show the revenue. That’s actually not probably what the customer is necessarily asking. It’s as compared to what? What would you do differently based on what you see in the revenue?

So, there’s that exploration versus the decisioning thing there. And I don’t think most people are paying a lot of money to just get better exploration tools, necessarily, unless you’re an analyst and your job is to do data exploration. But if you’re selling into a business customer, they are not probably sitting down with a goal to do analysis work. So, when you add this chat interface, just be mindful that you may be facilitating exploration work, but that’s not the product’s value or what’s going to make them buy.

So, big tactical takeaway, I don’t think GenAI is eliminating the need for intentional product design in your solution. I think it increases the need for decision framing, for having workflow context, recommended actions, having opinionated insights that come from the tool. So, I think really good AI-driven—or I should say, AI-assisted—analytics products will combine user experience around making structured decisions here. They’ll have some flexible conversational exploration involved, but they’ll know something about decision context, and I think they will be nudging or driving users to certain directions that have more to do with making the decisions. You would know the kinds of decisions users are trying to make, and your conversational interface would also be trained on this information so that it is constantly aware of how a data question, “Give me the report,” “Give me the numbers about something that happened in the past,” is tied to this intention that somebody has.

And then I think they will also be looking—and we’re seeing this more often now, if you’re just using these AI tools in general—but dynamic user interfaces. So, leveraging certain types of design patterns or design outputs. You know, any time someone asks a question that touches on revenue, we’re always going to show—or they mentioned campaign and revenue together—that the system knows, oh, this is probably a question about, “How is the campaign spend affecting the revenue downstream?” And whenever we have a question about campaign spending, we’re always going to put that information relative to historic spend or periodic spen—let’s say this is always—it’s, you know, it’s Christmas and holiday period in December, so we’re going to assume that kind of context. We’re going to provide trends from previous years. By default, we have a design pattern that would be going along with key questions and prompts that people are asking so the system knows what kind of response to give that may not just be prose answers to analytics questions.

Because sometimes you need to provide charts and data visualizations and things that are not suitable for words because our visual system processes information very differently than reading, and sometimes we may need to just look at a historical trend, or look at a pattern, some periodicity in the data that would be really informative to helping answer the user’s question. And that is not stuff that you necessarily want to write out in prose format. And if you’ve done any work right now around—I’m actually working on this right now with a client where we’re looking at using language models to provide a prose summary of what the dashboard is actually showing. And sometimes what you end up with is a very verbose text-based explanation of what’s on the screen and it’s not any faster or more useful than just using your visual system to eye the data with a well-designed interface that understands what questions are relevant to the data that’s being shown. The UI itself and the data visualization is the answer. This visual information is the proper way to deliver that insight, not prose. So, just because we have language models does not make them the best way to answer all kinds of decision support questions. So, just use the right tool. Like with any technology, the right tool for the right job.

The invisible intelligence gap is the next big idea. I just did a whole—the prior episode to this one, you can go listen to that entire thing, but the invisible intelligence gap is really this idea that analytics products generate insights and predictions and signals and charts and data, and often they fail to connect them to decision contexts, to workflow simplifications, to action in the real world that will generate some kind of value for the customer downstream from that. So, the benefits and the outcomes that the customer seeks or needs or wants are not connected to what the product’s features and capabilities are. As such, your intelligence is invisible. Your product’s intelligence, its technical capabilities, its analytical prowess, is essentially not visible here.

So, humans need context and narrative and confidence justification, and oftentimes, you know, analytical tools, they deliver, you know, hard numbers without meaning, and so you need to support decision framing, recommended actions or conclusions, so you can go see my CED framework, Conclusions, Evidence, Data, at designingforanalytics.com/ced which might help you address some of those kinds of challenges. Analytics products should also support explanation to help users justify their action. I love it when tools that are doing AI for BI are showing queries or they’re helping users understand why this data, how is it presented there, what did the system think about that I might not have thought about such that it’s showing me something different than I might have asked for?

Analytics products should also support that while exploration is sometimes needed, for most business users, doing analysis is not the goal. Getting to the decision in action, getting the confidence to move forward, that is where the value happens. It’s not in the exploration there. So, your product is successful when it creates these outcomes, these improvements to the lives of the users, using the interfaces and experiences that your product affords, and then the downstream business value that emerges as a result of that usage.

So, that’s about it for today. Just kind of big takeaway here, the big discovery mistakes, starting with data models, your database or your data stores, and then building dashboards and then asking the wrong questions or reacting to what you hear and assuming that is ground truth, that you can take away all the wish lists or changes that customers gave you, or the critique of what they visually saw in the product, and assuming if you just fix that, you’re going to win because UI/UX validation is not equal to product value validation. You can keep iterating on a design for a product and keep making the design better, but not create something that somebody is willing to pay for because the value isn’t there. You can get the interface correct for a given situation and the situation that the user might understand, but do they care about it and understand it? That’s the key. They have to care about it and want it. There’s a desirability thing there as well as a usability and utility thing.

So, the better path here, start with the humans in the loop. Start all the way at the end, the place where the people are going to touch the technology and the product. What are the decisions they’re trying to make? Where is that uncomfortable uncertainty—not just uncertainty, but uncomfortable uncertainty—where and how will this fit into existing workflows? What are the risks and worries that they have now, or that your product could end up shining a light on?

This is a big thing, I think, when we start thinking about agentic AI, and we start having to expose where the model has to be able to make decisions, and when things go wrong, who owns those decisions, especially if they were being made with data, which, of course, that’s how all these systems work, is it’s based on what the data is there. Your solution might be shining lights on uncomfortable truths. Is that something someone’s going to pay for or not? I think that’s a place where you need to spend a good amount of time to validate that even though you’re moving forward in one direction, you may be creating new forms of risks that didn’t used to exist.

So, we want to frame this stuff as how could we make our customers lives better if we solve these things in a really unique way, and of course, in order to solve them in a unique way and make their lives better, it means that you really need to understand the current baseline of how good or bad their experience is, using the tools, not just experience but the value they’re getting out of current status quo. What is their status quo? What are the tools that they’re using right now? How would we grade how good or bad that experience and value they’re getting out of their current tool set is? We need a baseline there before we can know if we’re actually making it better.

Because you thinking that, “Wow, I’ve replaced all this whole tool chain with this one system now and it automates the whole process from A to Z,” that is an assumption that you are making, that you have automated the right things, that you have not illuminated new fears or risks that didn’t used to exist as a result of removing all of this work, for example. We don’t know any of this until we go in and actually talk to customers. So, they have to own that value. We can’t assume that what we think is good for them is stuff that they’re going to value as well. They’re the only ones that get to own the decision about whether your product is valuable or not.

So, closing insight: product delivery work is getting easier. I’m sure almost everyone here is probably using—or your teams—are using coding agents to help write code and deliver product faster. It’s never been easier to build product fast and to do delivery work faster, but the real question is, are you making the right product or not? Do you understand the problem context well enough? Is there a sense of urgency and need that someone would be willing to change their behavior to get their hands on your product? It’s something they can’t live without that’s going to make their lives better.

So, if you can dial into that, improving the lives of your customers, giving them wins, making sure that the benefits are obvious and that you’re talking about those benefits the right way and they can feel that in the product, then you’re going to be on the right track here. Good luck. This stuff is hard. It’s not easy. I think analytics products in general have special challenges, but I wish you all the best.

If you need help and you’ve tried doing this stuff yourself, you feel like you’re not making a lot of progress here and you need some outside perspective on this, I can help you out. Just head over to designingforanalytics.com/contact. You can find my email address there, and you can also set up a free discovery call where I can do a rapid assessment of your situation, and we can talk about whether or not there is a way that I can help your team and your product get ahead, so you can ultimately turn that technical complexity into commercial clarity. Good luck.