What's my #1 engineering bias to overcome when designing good analytics products?

I've worked with a lot of talented engineers in my 20+ years of designing websites and software. One of the things about analytics that can trip up some engineers: imperfect data, conclusions, and evidence.

Analytics tools rarely provide exact answers, but this doesn't mean there isn't value to your customers.

If your data, analytics, or AI–in whatever form they are in–can substantially:
  • Reduce tool time required to obtain useful knowledge (tool time can mean your customer hiring a consultant, hiring a staff member, building in-house software or models, or studying their ad-hoc analytics)
  • Reduce the effort required, or add some quantitative weight to justify a new business decision (reduce risk, or appearance of risk)
  • Grow revenue
  • Reduce risk
  • Maintain business continuity
...then you're probably on the right track. The key here is "how good does it have to be"? 

The answer to that is that the cost/effort involved to set up, use, and understand what your product provides has to exceed the cost (financially, and the perceived amount of tool effort and time).   The larger the spread, the more value there is. 

If your customers can form conclusions that are at least "in the ballpark," that can still be a good win for your product. This is often where I've had friendly friction with engineers.  Whether it's building a data model, or troubleshooting software relying on analytics, they know all the places that could make the models better, and where all the "holes" and "lies" are.  

The more productive way to think about this in my opinion is "is our ballpark analysis/conclusion/help better than what they have now?" That's a product question. You can almost always make things better over time; you have to decide when you've arrived in the ballpark and whether your value is outweighing the aforementioned costs.  

On a final note: if you aren't conducting 1 on 1 usability evaluations (observing customers performing structured tasks), you likely have no real understanding of the full costs and tool time being imposed on your users. Don't fool yourself thinking you know; you don't know what you don't know. It's probably the most valuable tool you can use to make your design better as it helps you see what really needs attention.

If you're not sure how design ties into all this, or how to run a usability study, check out the links below.

Need to improve your product's design?