Reasons your next sprint, product, or project might fail

Good design happens at the intersection of discovering real user needs/wants and business goals that are ACTIONABLE (by design and engineering). Yes, there's a little magic/instinct that creeps into good design too, but you can get far without a lot of this magic. It's really more about nailing the problem set, and having really clear goals. But what does that mean?
When I am talking with a prospect about a new project, I ask them tell me their business goals for the project/app/product. More often than not, I get a reply like this: 

"an elegant user interface"...
"easy to use"
"supports our data infrastructure"
"like Apple/Iphone"
"easy for engineering to build"
"a new dashboard/visualization"

This usually requires a follow-up conversation, as none of these are really business objectives at all.

They are design objectives...the desired end state of the user interface. And guess what?

They're ones that almost EVERYONE wants for their application. It's almost like saying, "we want funding for our project so we can move it forward."

#metoo !

Most of you on this list are stakeholders; PMs, execs/founders, analytics leaders, engineering leaders, and data scientists. Whether it's your role or not, demand or help create clear, design-actionable goals for the project. Without them, you're simply at higher risk to fail because ultimately, by the end, your customers, users and superiors are definitely going to be passing judgement in retrospect. Why not establish the means for "passing judgement" at the beginning and share it with the team?

Here are some other warning flags to keep in mind when creating or evaluating your project's goals: 

  • Engineering implementation requirements masquerading as business goals ("The app will use API xyz v2.0.1")
  • UI implementation detail masquerading as business goals ("will have a button that allows XYZ filtering and mapping") 
  • Little to no discussion of the goals prior to execution work commencing (whether it be engineering, data science, design, etc.)
  • Lack of a UX person or knowledgeable facilitator who can facilitate the maturation of the goals from vague-->design actionable.  (No offense to resident UX folks, but as a general rule, I find that internal resources aren't usually going to push as hard to get the goals stated clearly. Why? They usually feel like they have less skin in the game.)
  • Because we use "agile," it's less important to establish design-actionable objectives

You can always adjust the objectives as you move forward and learn, but when you get the core ideas defined up front, EVERYBODY involved wins and EVERYBODY can usually come together on course corrections.

And one final note: you can still have goals, even with projects such as machine learning where you may not know ahead of time what the outcome is. A business goal can still be stated for this, e.g. "Perform a minimum-effort lab-style experiment to determine a set of future business problems/opportunities [along the lines of X, Y, and Z] that may be possible to satisfy using [data set x]." In this case, our outcome is not a solution, but a set of possible future problems. The point is, we tried to make it CLEAR and actionable to the stakeholders, and the team, from the start. 

The difference between design and Design

I am guessing if you're reading this, it's because there's room for your analytics service or data product to get better, and maybe you know that simply adding more data sources, algorithms, bug fixes, or [insert today's hype cycle tech], isn't necessarily all there is to making it better.

Like some of my clients, you may not know exactly what is wrong or could be improved in your service, but you know there's room to improve, and maybe you're concerned your service is getting too complicated or complex. Maybe you're seeing low engagement, growing attrition, sales that are harder to close, or workflows that are getting more complex. Maybe it's just "ugly" and doesn't seem elegant/valuable.

A few months ago via my social feeds, I saw a design thought leader mention something along the lines that most bad design out there isn't really "bad design," but rather, it is the result of "no intentional design." I think that tends to be true. In fact, it's even harder for most people to identify bad design now because so many plugins, repos, and libraries have made it easier to put reasonably good looking surface layers on top of any data product or analytics service. From charting packages, to CSS grids, internal design system/components, and third party tools, it's so much easier today to get a decent looking something out the door.

What I hope you'll remember today is that design isn't just about pretty UIs that look polished. While the paint and visuals do matter (despite what the usability police will sometimes tell you), they can also hide a multitude of UX problems that ultimately may lead to, or may already be causing business problems.

Good Design–what I sometimes call—"Capital D Design"—has the power to make your data sing, delight customers/users, bring new/better ROI to your organization, provide inspiration to teams, reduce complexity, reduce engineering cost, save time for users, and expose new value in your existing service. However, the big gains usually don't come from focusing on the surface level alone. Better data visualization cannot fix every data product and analytics problem.

Good Design starts with a deep understanding of your customers' and users' behaviors and needs (not just desires) and a clear definition of what a successful business outcome looks like from leadership that are then encapsulated in a clear product strategy by the product owner. This probably sounds all handwavy, but you'd be surprised how many times my clients and prospects cannot quickly and clearly state these things to me (or more importantly, state them to the team doing the execution). When they can't, it's usually a sign that there is more design than Design going on, and that there's a lot of room to improve the service.

What kind of [d]esign is your organization doing?

Design KPIs – what improvement did you celebrate in your last analytics software release?

I know to a lot of software teams, getting features/fixes/releases out the door feels like improvement. However, did you actually create or improve the value of your service?

To to that, you have to understand what your users actually value, so you can align your efforts accordingly. Most of the time, these nuggets of useful customer information do not come right out of a sales conversation, or via the Leave Feedback link, or through your customer service/support channels.

When was the last time your measured your software updates in these types of terms?

  • You helped routine task X take Y mins less to accomplish by making the workflow easier. This helped your customers get in/out faster since they really don't want to be in the analytics any more than absolutely necessary. (In your research, you also found out your service is most used on Friday afternoons, and ultimately, what you're really doing is getting a working parent out the door, on time, to get to that evening band concert, soccer game, or dance recital). That is value.
  • You made it possible to discover the relationship between X and Y, which can enable C cost savings. Perhaps the data was always there, but the design didn't facilitate making that discovery. Now it does. And you just helped the buyer justify the cost of your service.
  • You made an internal entry-level employee or worker feel valued through the use of your analytics servicePerhaps an insight they found made them stand out to their manager, and gave them the feeling that they, and their role, really do matter to the organization.
  • Your design change allowed improve a downstream health outcome, or encourage a provider/patient intervention to happen at the right time. Not only did some downstream stakeholder (e.g. a patient) get value from the software update, but you also made the user of your analytics feel wonderful about their impact on the patient.

The # of bug fixes you released is a good engineering metric, but it's not a directly a customer value KPI. To find those, you have to get out, talk to users, and observe the quality of the experience that your design is actually facilitating.

How to solicit *real* needs from users via UX research interviews

Readers of DFA know that I'm big on not immediately giving customers what they asked for, and instead asking the question "why" to learn what the real latent customer needs are. And for you internal analytics folks, remember your employees, vendors, etc. are your "customers" whether you think of them that way or not! Anyhow, some of you may be wondering why engagement is low, or you're not getting the results you hoped for. If you're not sure where to start, here is a super easy script:

  1. Recruit your customers for a 1-on-1, 30-60min screen-sharing meeting, or in-person meeting (even better). Tell them you're doing some customer research to learn about what is working and not working about the tools/solutions you manage and work on. You can also share that no advance preparation is needed and let them know their feedback would be extremely useful in making your service more useful, usable, and productive in their work. Note: scheduling can take some time, and you can even outsource this effort. One other thing we sometimes do here with research is to avoid sharing the specific thing we're going to discuss, to avoid users going and doing "homework" ahead of time to familiarize themselves. This may not be possible though, but if you can obfuscate it a little, that is sometimes a good thing. Your customer is likely to feel like they are being "tested" during all of this, and so your job is to help them learn that you're there to evaluate the service, and not them. Avoid using the word "test" and use the word "study."
  2. Open the session by asking them to tell you their background/bio. If possible, get permission to record audio/screen capture and mention it is only for internal review purposes. At the session, ask the person, if you don't already know, what their overall job responsibilities are and then ask how your service fits into that. At this point, the customer is self-reporting, so take this with a grain of salt. If they immediately start showing you interactions with your service, that is GREAT. Let them run wild, and keep asking questions that encourage them to demo the product back to you as they use it. Encourage them to "speak out loud" and give praise for feedback. I usually end up repeating the phrase, "thanks, this is awesome feedback" 20 times in a session. Note that you aren't praising their specific actions: you need to say this whether they do the right or wrong thing with the service because the feedback itself is what is being praised. Anyhow, chances are, after the short "bio" chat about their job responsibilities, they probably won't open up any tools as they will be expecting you to lead. As such, you now want them to open the tool and proceed further.
  3. Ask the customer to open up the service/tool you plan to discuss. Note: the study has already started at this very moment. Take special note to focus on what you see them DOING much more than what they are SAYING. Take note of things like
    1. Was the service bookmarked in their browser or easily accessible?
    2. Did it look like they were fumbling around just to launch it (e.g. haven't used it in a while, but don't want to admit it?)
    3. Was the login/password forgotten or not immediately accessible? These are all good signs customers aren't utilizing the service.
    4. If they need help after a bit, help them, and state, "If you haven't used this in a while, it's not a problem. I can help you get access to the tool." Note that this is called an "assist," and you want to do this only after you have concluded that it is rather obvious the customer can't even get past the login. Typically, in research, your job is also to avoid assisting.
      • Remember too, that this is NOT a training session but a discovery session to learn about what is happening in the wild when you aren't around.
      • Additionally, your goal isn't to scold them for not using your service, but to try to solicit useful information and honest facts from them. As such, during this simple act of opening/accessing the service, this a great example of where Actions speak louder than Words. Your customer might have told you they "use it all the time," but in reality, if you see them fumbling to try to simply open your service, you can see that what they're saying may not be quite as true as what they are doing. Keep this concept of "doing" over "saying" in mind as self-reported behavior is often very misleading. This is one of the core things that I see my clients/stakeholders getting wrong. You cannot necessarily believe customers/users' needs as verbally stated. They do not always know what they need, and their reporting of past behavior is often flawed. Which leads me to my final / next step: the recent history question.​
  4. "When did you last use the [service] if you can recall? Can you show me what you did specifically, speaking aloud as you go through it?" This question is specifically worded in such a way that you're not asking them in general how they use the tool, but instead, you are asking them to demonstrate a SPECIFIC use case they worked through to get some useful insights. This is much better as it forces them to use the service and show you their UX. You are likely to learn a ton here, and one of the best things you will learn is stuff you never even knew to ask about! You might see strange behavior patterns, ping-ponging between screens, opening up of external tools/spreadsheets, etc. This is all very good feedback.
    1. If the user fumbles quite a bit with your request and it's obvious they don't know how to use the service, it's ok to just tell them, "if you haven't been in here in awhile, that's ok. Can you tell me what you think this service might be useful for? What might you be able to use this for?" At this point, you're now observing their clicks, and encouraging them to "keep thinking out loud." Note that this is unscripted intentionally, so you want to let them take tangents and follow their instincts. Your job is simply to collect information and not judge their skill with the service.
  5. ​At the end of the session:
    1. Invite them to ask you any questions they may have.
      1. If your service DOES have a way to solve the question they have, don't tell them this and instead ask, "Do you think there is a way to do [do that task]?" Invite them to "try" themselves. If they get entirely lost, but your service does have this feature/need met, provide an assist to them, and then ask them to continue. Remember to encourage them to think aloud the entire time, and tell them, "we're here to evaluate the design, not you." Most customers feel like they are "dumb" when they fumble for too long (we all know that feeling when we can't open a simple bottle, or figure out how to open the door, or some other poorly designed system that seems like it should be easy!).
      2. If your service does NOT have a way to answer their need/question, encourage them to explain to you what end goal they have and what would make the service awesome. It can be pie in the sky; that's ok. What you want to avoid is encouraging them to start designing out the system in that moment, and instead, focus on what they personally, would think is valuable. Users also have a tendency to want to talk for others and think they are unique so watch out for, "I think most people would X, but I probably wouldn't." You want to learn about Y and not X in this case so keep coming back to them with things like, "that's great feedback thanks. Can you tell me what YOU would need/do/want that's different from what you think everyone else needs? I am really curious about your own particular needs and it sounds like you think they might be unique."
    2. Thank them and ask them if you can be in touch with them again in the future as you integrate their feedback. Your job is to develop a long term relationship and let them know that you need continuous user feedback to make the service better, and that their feedback contributes to a better user experience. Most customers love helping out.

Need help? Set up a free micro-consultation call with me on my contact page.

The Easiest Way to Simplify Your Product or Solution’s Design

Ok, you probably know this one, but let's dig in a little farther.

I recently started to explore using the TORBrowser when surfing on public wi-fi for more security (later finding out that using a VPN, and not TOR, is what will enable safer surfing). However, in the process of downloading and trying the TORBrowser out, it provided me with a golden example of what you should not to do in your product.

The very first screen I saw when I launched TOR was this:


So, what's the big deal here? First, I will share the answer to today's subject line with you:

Remove everything that is not necessary.  

​Yeah, yeah, you probably have heard that before. Famously, the pope asked Michelangelo how he knew what to carve while creating the statue of David, and his response was along the lines of, "I removed everything that wasn't David." Nice.

Are you removing the cruft and noise from your product?

If we take this thinking further, I would say that today's core takeaway for you is to "remove choices by making good assumptions in your product, wherever possible." You might be wrong sometimes, but you'll be a right a lot of the time.

Jumping back to the TORBrowser UI example above, there is more you can learn from their design choices:

  1. This UI says, "This [option] will work in most situations." Well then, why isn't this automatically selected as a default choice?
    Does this screen now seem necessary to you? Why does the product not just "try" that setting by default, and present the other option as a fallback, if the default fails? Nobody downloaded TORBrowser with a goal to "set it up" with the right networking settings. This entire step is not necessary. It literally says it's not "in most situations."
  2. Right the wrong time. 
    I haven't needed to use this pane yet as the default setting worked (surprise!), but it's an example of the developers trying to present helpful information. That's good. The design problem is that it's appearing at the wrong time in my experience. I don't need this right now, and I don't even want to think about how the networking is configured. It's completely irrelevant. Are you presenting choices at the right time in your product?
  3. Most users don't care how your software works; don't expose the plumbing. 
    ​There are sometimes exceptions to this for certain technical products, but even when there are, once most users have "learned" what they need to learn about the plumbing, it quickly becomes irrelevant. The value has to shine, or people stop paying for the service. That includes products built for technical audiences.
  4. This UI and UX is not fun at all...especially as a first impression.
    It's a needless distraction, it's not fun, and it's got me focused on, "how hard will it be to get this app working?"
  5. The visual design attention (or lack thereof) is undermining the mission of the product. 
    This is the hardest one to teach, but a combination of graphic design choices (probably unconscious ones) here contribute to this UI not feeling particularly safe, secure, and careful. The goal of TORBrowser is to "protect" the user. If you think of words like protection, precision, stability, and safety, then the visual design should reinforce these ideas. The topic of graphic design is hardly something to be captured in an email, but I can leave you with a few suggestions and considerations. Refer to the diagram for a detailed analysis:Image

    1. What could be removed from the TORBrowser UI sample?
    2. Are the invisible things (like padding/margin whitespace choices) consistent, meaningful, and deliberate?
    3. While a single graphic design choice sometimes has the power to impact usability or even the financial the bottom line, typically, it is the sum of numerous small design choices that account for the overall perception of your product's quality and aesthetic.
    4. It's possible to follow "all the rules" and still not have a great product aesthetic or utility. (That's why we have designers.)

What internal analytics practitioners can learn from analytics “products” (like SAAS)

When I work on products that primarily exist to display analytics information, I find most of them fall into roughly four different levels of design maturity:

  1. The best analytics-driven products give actionable recommendations or predictions written in prose telling a user what to do based on data.  They are careful about the quantity and design of the supporting data that drove the insights and recommendations being displayed, and they elegantly balance information density, usability, and UX.
  2. The next tier of products are separated from the top tier by the fact they're limited in their focus only on historical data and trends. They do not predict anything, however, they do try to provide logical affordances at the right time, and do not just focus on "data visualization."
  3. Farther down the continuum are products that have progress with visualizing their data, but haven't given UX as much attention.  It's possible for your product to have a *great* UI, and a terrible UX.  If customers cannot figure out "why do I need this?," "where do i go from here?," "is this good/bad?," or "what action should I take based on this information?," then the elegant data viz or UI you invested in may not be providing much value to your users.
  4. At the lowest end of the design maturity scale for analytics products are basic data-query tools that provide raw data exports, or minimally-designed table-style UIs. These tools require a lot of manual input and cognitive effort by the user to know how to properly request the right data and format (design!) it in some way that it becomes insightful and actionable. If you're an engineer or you work in a technical domain, the tendency with these UIs is to want to provide customers with "maximum flexibility in exploring the data." However, with that flexibility often comes a more confusing and laborious UI that few users will understand or tolerate. Removing choices is one of the easiest ways to simplify a design.One of my past clients used to call these products "metrics toilets," and I think that's a good name! Hopefully, you don't have a metrics toilet. *...Flush...*

What level is your product at right now?

Video Sample: O’Reilly Strata Conf

This is recording of my presentation at the O'Reilly Strata Data Conference in New York City in 2017.

Do you spend a lot of time explaining your data analytics product to your customers? Is your UI/UX or navigation overly complex? Are sales suffering due to complexity, or worse, are customers not using your product? Your design may be the problem.

My little secret? You don't have to be a trained designer to recognize design and UX problems in your data product or analytics service, and start correcting them today.

Want the Slides?

Download the free self-assessment guide that takes my slide deck principles and puts them into an actionable set of tips you can begin applying today.

Getting confidence in the value of your data

(As shown to customers in your UI)

I'm talking to a prospective SAAS client right now, and they're trying to expose some analytics on their customers' data so that the customers can derive ROI from the SAAS on their own. The intent is that the data can also be useful to the SAAS sales team, as a tool to help prospects understand what the possible ROI might be.

I had a question for Dave around whether the project would be successful if we talked to the users, designed a bunch of interfaces, solicited feedback on the design outputs, and found out that the data, while interesting, didn't really help the customers derive ROI. Would the design engagement still be productive and a success in the end? Ultimately, I didn't want to take on a project if we had hunches that the data we had, while being the best possible available data and elegantly presented, may not help the end user or buyer calculate ROI.

Here's what Dave told me:

Yes, the design engagement would still be a success. It provides us a punchlist of what else we need to do, which is in-and-of-itself is useful; and presumably defines what the analysis/reporting needs would be once we get that data. Less of a success, or more of a delayed-gratification one, but still useful.

I thought this was interesting to share, and I hoped Dave would say this because it shows that sometimes, you have to do some design to figure out what the final design needs to be. You can't always plan ahead what the right solution is and moving from designing on assumption to designing on fact is powerful information to inform your product.

Conversely, you can also spec out the entire project, including all the data/queries that customers said would be useful, write it into a spec or backlog, code it up, skip design, and then still have it not be successful because customers couldn't actually experience the ROI that your data was supposed to convey. A product backlog does not = a viable product. It's just a bunch of user stories or features. The glue holding them together, and what helps customers realize the ROI, is design.

Tips to help focus your analytics design/engineering efforts on results

If you are starting out on a new feature design, or analytics effort, can you clearly state what the value will be in quantifiable terms at the end of the sprint?

Are you building an "exploratory" UI, or one that is supposed to drive home conclusions for the customer?

When clients come to me about product design engagements, I spend a lot of time trying to understand, at the end of the project, how success will be measured. Frequently, my clients haven't thought this out very much. I think that's natural; when you're close to your data, you can probably see a lot of ways it could be "visualized" or that value could be pulled out. But, when it's time to get specific, are you and your team able to clearly share a vision of what the desired end state is?

Here are some examples of goals and success criteria I've seen with my design clients. Having these types of success metrics makes it much easier for everyone involved on the project to know if they're on track to deliver value:

  • SAAS Example: Make it easier for the sales team to sell our product by surfacing interesting analytics that help customers see the value of the product. Ideally, a 30-day closing period for a sale drops to a 1-week closing period.
  • IT Software Example: Remove unnecessary troubleshooting time from the customer's plate by running analytics to either surface a problem, or eliminate what isn't the problem. This is a reduction in customer tool-time effort. If we can drop troubleshooting time by 50%, that is worth $X per incident (the business impact time + the manpower/labor time saved).
  • Generic example: Help the customer understand what interesting outliers are in the data so they can take action. There are opportunities to exploit if the outliers are interesting. Our analytics should help surface these outliers, and qualify them as well. If we can save 10hrs a week of "exploration" time the customer has to do by surfacing this data early in the UX, that is a substantial labor savings ox $x as well as overall product quality/delight since the users can now focus on what's really important in their job (not the "hunt").

This is the start of making any design engagement successful.

Got some of your own goals/metrics to share? Hit reply; I would love to hear them.  If you're embarking on a design project and need help getting these defined,  you can schedule a free micro-consult with me below.

“Post-truth,” data analytics, and omissions–are these design considerations?

Post-truth. The 2016 word of the year.

Yikes for some of us.

This got me thinking about UX around data, analytics, and information, and what it means when we present conclusions or advice based on quantitative data.

Are those "facts"?

If your product generates actionable information for customers, then during your design phase, your team should be asking some important questions to get the UX right:

  • What risk is there to our customer if the data is wrong or could be interpreted incorrectly [easily]?
  • What information might we want to include to help customers judge the quality of the information the product generates?
  • If our technology analyzes raw data to provide actionable information, are there relevant analyses that the product did not run that the customer might need to contextualize the conclusions drawn?
  • Is our product (and company) being genuine, honest, and transparent when appropriate?
    (That's how I roll at least, and few scenarios suggest this ever is bad advice.)
  • Is the display of supporting data considered and as unbiased as possible?
    (Notably: did you design the presentation of the information before coding it, or did you just dump it into a charting tool?)

Part of getting a product's design and UX right is knowing what questions to ask.

Let's take a quick example many of us without pensions can relate to: retirement savings.

Let's say you work at a financial institution and you're supposed to design an online tool that can help customers understand how much income they need for retirement, and specifically, what monthly savings target they should have in mind to reach that goal. A wise design-thinking product owner will be considering issues beyond how the UI works, the sliders and input fields on the forms, and the way the output charts look.

If we're talking about "truth" in the context of design, I'd hope the product team considered:

  • How confident are the displayed estimates?
  • Since this is a predictive tool, did the app run one than one type of simulation before generating advice?
  • Did the app factor in unique characteristics of the user, such as their own behavior to date (if known)?
  • Does the design clearly mention relevant variables that the tool cannot control for, and also how much those variables might affect the predictions that are shown?

How much, and how loudly a tool answers these questions depends on the content, risk, customer problems, and needs.

Sometimes these things don't need to be "answered" literally in ink because not all customers will care, or they might just assume that your calculator already does all this magic.  And, there are times when all the ink might just be noise (e.g. weather forecasts).

All that said, I am not sure "post-truth" fits in anywhere with good product design.