Transcript:

Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today, Bhaskar Sunkara, CEO of Bicycle AI. What does Bicycle have to do with analytics [laugh]? Tell us.

Bhaskar: [laugh]. Yeah, I think that’s a good start. So, I think just to define Bicycle first, and then I’ll basically come back to, kind of, what it does, Bicycle is basically, like, an AI analyst that’s assigned to a revenue-critical KPI. So broadly, obviously, whether you call it, you know, AI for analytics, AI for, sort of, consumption, BI, et cetera, but it’s an AI analyst assigned to a revenue-critical KPI. And so, that’s essentially an agent, and what we do is we give it a job, which is around a revenue-critical KPI, so it could be take the simplest example, like search conversion, checkout conversion, something like that in retail, right, or inventory availability, or something like that, what the agent does is it looks for meaningful changes.

You know, once it finds a change… so the meaningful change could be, hey, my checkout conversion or my search conversion is down for a particular category… like, could be toys, could be electronics, or whatever it is… and then it actually investigates the likely business and technical drivers. So, is it because of inventory? Is it because of suppliers? Is it because of, like, something technical that happened? Your app is, you know, down for a particular version of iOS, et cetera.

And then, you know, when it looks at the drivers, it recommends or basically takes a governed next step. Now, the reason we call it Bicycle is because the analysis, the investigation of why the data movement happened is across both business and technical factors, and those are the two wheels of the business, you know? The business is the business, and then the technology actually runs it, and we, kind of, just visualize that as a bicycle as the two wheels of the business, and then we have to look for factors across both. So, hopefully that makes sense.

Brian: Ah. It’s very, very clear when you put it together that way, the wheel analogy. So, partly what I wanted to chat with you today about, I want to rewind a little bit. So, you were the founding CTO of AppDynamics…

Bhaskar: Correct.

Brian: Which Cisco acquired, as I understand it?

Bhaskar: Yep, yep. Absolutely.

Brian: So, in my prep, when I was, like, doing some research on your background and work, I came across an article, I think, when you were a venture partner. Are you still there at Decibel?

Bhaskar: Yeah. I mean, not as, you know… once I got into the meat of Bicycle, then spend less time. But yeah, still pretty actively involved with them, yeah.

Brian: Yeah. And so, you were reflecting, I think, on your exit with AppDynamics, and you said something along the lines of, like, “I still remember how hard it was to get our first 20 customers, and I still see every entrepreneur trying to climb the same steep hill.” So…

Bhaskar: Yeah.

Brian: … the, like, AppDynamics, my understanding is you had to convince buyers they had a problem that maybe they didn’t know have, like, application performance issues in the data center, like, the IT environment. Corre… am I basically correct about what AppDynamics was doing? So, how did you make… like, if you rewind to that business, which was I think objectively a success? How did you make that invisible problem feel urgent enough that someone wants to pay money for this? And then did you take some playbook from that or learnings that’s like that… there’s some concrete stuff here that wasn’t just luck, or we were at the right place at the right time? Or maybe it was, and you applied that to your next… I’m assuming you’re applying some of that to Bicycle, but tell me back at AppDynamics, how did you get someone to care enough about analytics and insights on application performance and say this is worth paying for?

Bhaskar: Yeah. No, I think what was happening, so we started in 2008, and one of the things that was happening then is the contribution of software to running a business was just becoming more and more, right? Like, more of your business was being run by software, and as a result, everybody understood that, like, if I have issues in software, if my software is not available, my software has issues, then my business is suffering. It’s a direct hit, right? You’re actually losing revenue. So that’s, kind of, the starting point.

And that’s when people realize that, you know, monitoring is not just a question of, well, you know, it’s a nice to have, but it’s a must have. You have to absolutely have it. But the issue with, you know, the monitoring part of it was the construct with which you monitor was very, very, sort of, low level, so you’re monitoring CPU, you’re monitoring code, you’re monitoring things like that. So that’s, kind of, what we really changed, where we said, look, I mean, you should not be monitoring constructs that, you know, as an operating team, you might not understand because you never wrote the code. This is also when, you know, DevOps was truly… like, the operations team really was running operations because it was a significant job versus, yeah, I wrote the code. I’m going to take a look at it when I get a chance, right? Like, you had an operating team that was actually running it.

So, then you start looking at, okay, you know, can you track it? Can you monitor it if you don’t really understand the code, if you don’t really understand the queries, which was the case? So, we gave them a construct which we call the business transaction, which we said, okay, tie it to user experience, and it could be logins, it could be checkouts, it could be, you know, bookings, et cetera, and that’s what you’re tracking, and we will give you the layer of tracking all of that, and then help you, sort of like, get to the next level of what is really happening. So that’s, kind of, how, you know, we really pitched the problem. Because the problem was there, but people were really, really struggling with it.

The other thing people were struggling with was the way the space came about, you know? Now, it’s called observability, but back in the day when it was application monitoring, there were agents… I mean, not AI agents, obviously, but there were agents that are running inside the application that were part of your code, right? Like, you wove yourself in the code, and then you’re, kind of, like really running there. And the stability of them, the way they were designed, there were a bunch of issues as well, so we had to do some technical choices to make that, sort of, more robust and things like that. But the pain was already there, and that’s how we really went to market with it on… you know, as the operating team, focus on transactions and don’t focus on all the low-level constructs, and that really got us the momentum.

Brian: Yeah, when you talk about the low-level constructs, like, having done a fair amount of work in that space, what I remember is the environment provides you with thousands of different telemetry metrics, but getting to an actionable insight within the telemetry metrics, especially if you don’t know what, like, swapping at the disk level is and how that, like, creates, like, queuing, and then that’s slow performance, and then slow performance means the boss can’t check email, or like, you have to, like, ladder it all the way back to this, like, business concept there…

Bhaskar: Correct.

Brian: … how did you make sure that you… like, there’s a sweet spot there between of a business valuable conclusion and insight and the evidence that backs up that this is indeed what is wrong and what you should do about it. And here’s all the telem… and this is where the analytics people, I think, live is they stop at the evidence part. Here’s all this evidence [laugh].

Bhaskar: Yeah, yeah, yeah, absolutely.

Brian: So, how did you, like, think about that so that you’re not just shoveling what an old client called the ‘our metrics toilet.’ Here’s every metric we have. You can create whatever you want with it, and it’s like it turned into the metrics toilet. How did you do that?

Bhaskar: Yeah. I mean, I think the most important thing is: are you anchoring on the right metric? And also, by the way… and I’ll go into details of that, right… but also, is it, sort of like, fine-grained enough? Like, so what we anchored on was take a transaction, right? Like, you know, the response time of checkout. Like, let’s just say checkout is the core money transaction for what we’re talking about.

And so, what you’re saying is, I’m going to look at response time. I’m going to look at the error rate. I’m going to look at, sort of like, the load and everything else. That’s the core metric, but if you just look at response time of a metric, it’s sometimes not fine-grained enough because what can be happening is, there can be a certain set of customers based on the path that they’re hitting, they can be having issues, and the aggregates might be great. So, then the next level for us was let’s look at each individual checkout and grade it and have a scorecard, and then say these, you know, let’s say, a hundred checkouts happen, you know, eighty of them were normal, and then, you know, ten of them were very slow; ten of them actually stalled.

And now what you’re getting is a very, very solid way of how the user is experiencing it because all the other metrics, like you said in your metric toilet, can be perfect, but those 20 transactions were very meaningful because they might be your best customers, they might be your most loyal customers, or whatever. Because that’s when you’re starting to get real with what those transactions are. So, if you anchor on the right metric, you know, everything then becomes, well, I want to chase down these ten transactions that stalled, and if that stall had an issue on a disk swap, that’s when that metric is worth it. Until then, that metric is not worth it.

Brian: Yeah.

Bhaskar: Right? So, anchoring on the one that’s the most meaningful, and definitely we’re applying those lessons in Bicycle as well because the difference in AppD and Bicycle fundamentally is that AppD started with how do you go from a technical signal to action and Bicycle is about how do you start with a business signal to action? And the business signal could be, you know, my search conversion is down for toys or electronics or, you know, apparel or whatever it is, and that’s, sort of like, something even the CEO understands that okay, people are not buying enough toys, right, or electronics or whatever it is. But then, how do you then take it to… and it should be fine-grained enough, you can’t say I’m just tracking my revenue KPI. It just doesn’t tell you the story, right?

So, right metric, right granularity, I think is the starting point of, like, do you have the right detection layer? And then when you go into the explanation layer, then you can go as deep as you need to figure out, like, why that particular granular metric is off. Does that make sense?

Brian: Yeah, I understand, and we could probably do a whole episode just on this because, especially when, like, the granular metric has bearing on multiple business KPIs, like…

Bhaskar: Correct, correct.

Brian: … you know, like, application slowness or, like, you know, failed checkout, whatever it may be, it could mean different things. But the main thing I think at a high level here was just your experience of avoiding the common metrics toilet problem, where it’s like tons of telemetry, but it’s not connected to business value, which is what a fiscal buyer…

Bhaskar: Correct. Yeah, would care about.

Brian: … wants, even though an end-user may need telemetry-level evidence to back up a decision. Like, it’s the difference between buyer need, a fiscal person and an end-user who might be more operationally-minded.

Bhaskar: Yeah.

Brian: Do you have that now? If we, kind of, fast-forward back to Bicycle then, are you dealing with, like, the two-world problem: your fiscal buyer here, but also then the person that’s going to live in this and have to maybe defend decisions that the tool is… especially if it’s making decisions automatically. Like, “What? You shut down the, you know, West Coast e-commerce server? Like, what do you think”… It’s like, “Don’t worry, it’s better.” Like, [laugh] tell me about the two, the end-user, like, selling and designing for the end-user versus the buyer, especially during the POC. I’m sure there’s some kind of POC experience, all that.

Bhaskar: Yeah. No, at a very top level, I think the business definitely cares about, can I get insights faster, can I get investigations faster, can I get value, can I save time, can I save money, and all of that type of stuff, right? But you’re so right. Like, I mean, when you apply that to, you know, who’s actually using it, you know, whether it’s an analyst, whether it’s sort of like the data team, whether it’s, sort of, the business person who is actually really getting advantage of the insights, the way they look at it is completely different, right? Because the business KPI owner, like… let’s say you’re, like, a, you know, merchandising manager, somebody who owns a KPI, or you’re an account manager and you own the KPIs for the particular account that you’re working with… you’re just really focused on, like, did something move? Like, what’s the right KPI to really figure out, like, where the problem is with? You want to understand why that’s happening, and you want to know what you do about it.

But if you look at the team, the data team, the analytics team, they’re so focused on, is it trustworthy, is there enough evidence to, sort of like, back it up, is it picking it up from the right pipeline, you know, and all of that type of stuff, right? And if you go to the exec, the execs basically, like, well, did we save time? Did we save money? Did we lose revenue? You know, stuff like that.

So, the surface area has to attack both the exec as well as the data person who actually owns the data and is responsible for making sure the data is good. And then the business person who’s actually making the decision, you know, they have to, sort of like, have their layer of that. So, that’s one of the most challenging things to make sure you’re attaching or attract, you know, like, really attacking the surfaces based on who the user is and how we’re, kind of, really building that. And one of the things that we’ve done in the past is I think we over-rotated a lot on the business user and making that simpler, et cetera, but the trust and accountability layer, and then how you get these, sort of like, data team involved in it so that it’s all certified and people can actually… by the way, I was just reading your framework, right? Like the, sort of, the conclusion, you know, the evidence and the data, and really, sort of, and that’s a very, sort of like, easy way of summarizing it, but how do you dress up the conclusion, how do you have the evidence where the data team feels, you know, really solid about it, and then, what data are you actually using and can you trace it all back?

And the point is not just saying hey, you know, let’s make a decision, et cetera, but the decision part, the automation part, has to be completely based on how costly would being wrong be, right [laugh]?

Brian: Yeah.

Bhaskar: I mean, if you’re just sending a notification to someone and it’s automated, yeah, go for it, right? Like that’s fine. If you’re turning off, like, something, like you said, if you’re basically changing your routing, if you’re changing discounts on the side, et cetera, those better be, sort of, done with a ton of guardrails where whoever is responsible for it, they still need to be accountable for it, even if AI is doing it.

Brian: Yeah. Can you tell me at a real practical, like, product level, how do you think about designing the product to ensure it’s revealing its business values? So, say this is, like, the classic, what does the exec see? Like, why are we paying for this? Renewals coming up? Yeah, what do we get for that Bicycle thing versus the guy that’s, like, the revenue operations person or some more operational-minded end-user who’s actually dealing with telemetry, and then oh, the system decided to reroute. It’s changing coupon codes. Should it do that? Can I bless that decision? Or it already made the decision. Do I want to live with it, or do I want to override it?

That’s very different, so how do you think about pro… like, are you sprinting, like, this week we’re going to address, you know, metrics dashboard for, you know, KPI values saved or money earned and then next week we’re going to work on, you know, telemetry, like, proving out the investigation evidence make it better, whatever. How do you think about that? Is it one thing? Is it two discrete audiences that you’re… two mouths that you feed? Like, how do you think about it?

Bhaskar: No, fair enough. Like, I think there are really two layers to it, right? Like, if you keep the, sort of, data layer and the actual, sort of like, core data layer and the operational layer aside, one layer is basically the actual decision-making layer of, look, I mean, I found something off with search conversion. I found something off with payment approval rates, and here’s all the drivers I investigated, right? And the drivers could be like I was saying earlier. It could be inventory. It could be suppliers. It could be like basically, you know, technical drivers.

You know, are you looking for errors in log files? Are you looking for issues that an observability tool is reporting? So, looking at all that surface area. And so, for that, what you’re really doing is, you know, summarize, come up with a conclusion saying, you know, “We think this is, sort of like, the primary cause for it.” Wherever you can be deterministic, be deterministic.

So, I’ll give you an example, right? So, let’s say payments are failing. Let’s just take something that’s really, really simple. Let’s just say payments are failing, and you’re like, oh, you know, payments are failing in California, and it’s pretty specific, and that’s the KPI that you’re going after. Now, the first thing you need to do is, in a deterministic way, can you analyze all transactions that were failing in California, identify the dimensions where most of the failures were?

So, let’s say you identify saying, “Well, most of them were happening on iOS 14, right? And they were basically transactions that were going through for American Express, and maybe another dimension on top of it.” So, what you’ve done there is that you’ve actually, in a very deterministic way, identified what to focus on once you detected something was off, right? And that needs to be transparent. And the person who’s making the decision, if they can’t trace it, if they can’t actually look at the fact that they actually looked at all transactions in that time range where the KPI was off, you know, they actually really derived the focus set from it, the focus from it on these three dimensions, now what you can do is, you know that it’s iOS particular version, you know that it’s American Express, you know that it’s, like, a particular county in California, right? So, now the analysis that you’re doing after the drivers you’re investigating are very specific to that.

I mean, it’s not Visa, so don’t bother about that. And so, now the analysis that I’m going to show you, the drivers that I’m going to investigate and tell you are very specific to these three dimensions that I’ve isolated out. And then you go into the theory of probability, right, because you probably made… somebody made a change to that particular iOS version, and someone checked it in, right? And unless you’re able to read that code and see if it’s wrong or if it had an obvious error, you can’t basically just say that, look, you know, deterministically, I tell you that this change caused it. It might have. It’s a theory at that point.

So, be very transparent on what you’re deterministic about, like, where you’re, like, really, sort of, crunching data and coming up with, like, accurate results, where you’re using things like machine learning where you’re actually doing, like, the right level of, like, you know, analysis for it, but if you have a theory on, like, look, I looked at these three things, you know, oh, bookings were up because of July 4th, et cetera. Whatever, right, like, because that’s a possibility, and so that’s a theory. So, for that particular user, giving the explainability, giving the transparency of everything that you’re doing, that’s one.

The other mouth, so to say, right, is when we are coming up with, like, you know, the triaging of all of this, under a time period, you know, are we saving revenue? How much revenue are we saving, right? Because the decision was taken and something was reversed, and then suddenly you don’t have those payment failures anymore, but over a period of time because of that decision, how much revenue did you save, right? It has to have some baseline. So, that’s more like the exec end, you know, looking at that.

Brian: But how is that experienced by the executive in the product versus the analyst or the person doing what you call investigation or root cause analysis?

Bhaskar: Yeah, totally.

Brian: Those are two different experiences. Do you have, like, separate… I mean, I’ve seen this model. It’s like there’s a dashboard or screen, it’s really targeted this user who’s looking at it maybe once a month to, kind of, justify the existence of the product to begin with and the licensing fee or whatever, and then someone else looks at it once a year, but then operationally they live in this tab, blah, blah, blah or whatever. I mean that’s a very simplistic way, but how are you thinking about, like, those two different constituents as you prioritize product work? Like, how do you think about that?

Bhaskar: Can I go into, like, the surface area of the product a little bit?

Brian: Please. That might, yeah, help me visualize it, yeah.

Bhaskar: Yeah, absolutely. So, we have four features, you know, core surfaces in the product. And I can split them into two, right? Like, I call them push and pull, right? Push, you know… actually pull… start with pull. Pull are dashboards and chat, right? Pull is basically what you’re pulling. You know, like, you traditionally do in analytics a lot of times. You’re asking the question. You’re opening up a dashboard. You’re deciding which dashboard to look at the right time, right? Like, so that’s traditional analytics, and we have that, right?

Our core features, the way we really, really differentiate is the push features, and the push features, there are two of them. One is alerts, right? And the other… let me talk about alerts first. So, what alerts do is, let’s say you had a dashboard… and there’s a very easy way of understanding how alerts work, taking a dashboard… so let’s say you have a dashboard. You’re, like, search conversion, let me look at that, and I have a hundred items or a hundred categories I’m selling in, you know, a hundred different regions, right? I’m selling toys in California. I’m selling toys in, you know, Atlanta, you know, whatever.

So, now you have to, sort of like, look at all these metrics, and let’s say using a dashboard, you’re looking at, like, search conversion for X category, Y category, Z category across all these regions. Now, the only way you understand something is going off is for a particular region, for a particular category, you spotted something and then you take an action on it. What the alert does is, the AI watches all the hundred categories… let’s just say if they’re a hundred categories… you know, uses machine learning, uses baselining, uses, sort of like, you know, time of day, time of week, et cetera, all of that, you know, aspect, and comes up with, hey, search conversion is down for electronics. That’s the starting point, right? Like, so that is an alert. That is meant for the tactical owner of the business KPI, right?

Now, if you have an exec, the exec’s not reacting to, you know, an alert happening at a given point in time. What the exec is saying is, take the KPI search conversion. It can be more, you know, it can be hierarchical, it can be even things like click-through rate, et cetera. But let’s just say it’s search conversion, and I am looking at it over a week, over a quarter, right? Bunch of stuff happens… sometimes it was electronics, sometimes it was apparel, sometimes it was, like, books or something else… but can you digest all of that and do a deep analysis, group them together, cluster them together, add impact to it, and can you create me a narrative, and that’s basically what we call a data story, right?

And a data story is usually, like, your weekly, monthly, you know, like, a Monday morning briefing in retail. You know, like, it’s a quarterly business review, it’s a weekly business review, and so it almost reads like a deck, but it’s deep analysis on all the alerts that have happened, all the insights that we’ve found, along with the causal, you know, definitions along with the causality and all that stuff. So, when you really think about, like, how we do that in the feature, it’s really either/or. Like, you’re focusing on the alert, which is very, very tactical, and the data story is basically very strategic. So, that’s basically how we define it.

Brian: Got it. Got it. Yeah, we could have a whole thing just on alerting and telemetry. Like, just, you know, when do you raise attention for something and understanding normalcy and abnormalcy and all this stuff. But I don’t want to go too deep on that. That’s a whole episode.

Bhaskar: Yeah, yeah, yeah.

Brian: Tell me about this process of, like, getting your first 20 Bicycle customers. Who’s in the room? Are you trying to satisfy the, we’ll make your life as an e-commerce manager easier: less time investigating, less time arguing about which thing caused what, or are you selling to this, you know, executive who’s, like, you know, I’m losing revenue over stupid stuff. It takes us forever to figure out what the root cause was. It ends up being something fairly minor, but the investigation took two weeks, and we lost Christmas electronics or whatever. Is there a pattern emerging here? Like, it’s really about that analyst or the ecom manager. Like, no, no, no, no. It’s about this executive who knows that there’s opportunity here. Like, who’s in the room? How do you think about the go-to-market here? Like, tell me about that.

Bhaskar: Yeah. I mean, I think it’s really like two personas, right? Like, one, you can start with the executive, like you said, and it really depends on who your first touch point is, and we usually enter through them. But it’s either the executive who understands I’m losing revenue, especially because we focus on transactional businesses. If the exec is basically like, look, I mean, I’m actually losing money, I’m actually losing time, I’m actually running inefficiently because the time to insight, the time to action is really, really bad, and they really believe in it, and they are looking for different ways of what is, sort of like, the next generation. And so you have to have that visionary because this is still an evolving category. It’s still, kind of, playing out in terms of how people have really accepted on how to do this. So, that’s one persona.

The other persona is somebody who’s, like, head of data, head of analytics, and who’s, like, look, every quarter, as my business grows, I actually have to, you know, hire, like, four or five or six more analysts because when we detect motions in KPIs, when we detect movements in KPIs, and we then figure out, you know, what is actually really happening, that’s so much of tribal knowledge, that’s so much of, like, work for an analyst that as my business grows, as we do more type of business, as we grow through different territories, I have to start attaching analysts. And so, they’re like, I can’t keep up with it, right? And so, I need something which is some repeatable motion of I’m in control, but I’m able to, sort of, run these investigations, you know, and the metrics and everything else.

And that’s where, you know, our pitch to them is that, look, we’re, kind of, like a cursor for your analysts, right? Like, cursor multiplies what you do and it creates a system for you. It makes you, sort of, 10x, and as an analyst, you can, kind of, do the same thing. So, it’s one of those two personas depending on who our touch point is because how the business is looking at it and how data is looking at… because when data gets that, sort of like, leverage, the unfair advantage of using AI… because if you think about it, predominantly the most the dominant category is using coding, you know, using AI for coding, basically, but that advantage hasn’t yet, sort of like, really, you know, percolated into the analytics area where you feel like you can use it as powerfully.

And so that’s, kind of, really how we talk to… but some of that has been recent. I think we started more with, you know, really focused on, you know, selling to the business, and business basically loved it, right, but we never really had the right, kind of, value prop, the right way of talking to the data team, to the analytics team. And so, that obviously was where we started off super promising, but we couldn’t get to the finish line right, and so that’s been a little bit of an evolution on having that, sort of, core pitch for both the business as well as data because business gets the benefits, but they don’t really own the tool. They don’t really run it themselves, and without the help of, like, the data and the analytics team helping them, they really can’t, you know, actually figure it out.

Brian: Got it. Let me play this back to you if I understood it right; you tell me if I got it right. It sounds like from a positioning standpoint, it made more sense for you all to shine light on the, you know, hiring human analysts is expensive, it’s slower, they have to build tribal knowledge, in order for you to keep up, you need to attach bodies. We can actually make the existing bodies more efficient by giving them better tooling, which makes your analytics team look like you’re creating real… we can show you the tangible business value you’re creating to your CDO or CIO or whoever you’re reporting into, and so you’re helping the analytics team be champions inside of the enterprise.

Which is different than, like, help the chief revenue officer, like, say, you know, find revenue holes in the business there. And did I also hear maybe you were, sort of, if you position this, and this, I think, for founders and stuff, like, this is where the positioning is so important because it sounds like what you’re saying is, if you were to approach, like, you know, a chief revenue officer, a business stakeholder, and an executive level, you have a blue ocean kind of product. Like this is not replace… they don’t see this as, like, replacing some status quo; this is, like, magic stuff that all of a sudden can find revenue holes. There’s probably some risk there because it’s like we’ve never had a magic thing like this before…

Bhaskar: Absolutely.

Brian: But the analytics team’s, like, oh, this is, like, a bunch of humans, like, working all night, all day, helping me do the patterns that we know, and so it’s not magic. I mean, I’m sure the technology is really cool, but they can understand conceptually what it is. Same product, two different positionings and mindsets. Is that, kind of, where you landed? Was, like, the easier path, less friction path is going through?

Bhaskar: Absolutely.

Brian: Okay.

Bhaskar: Yeah, and there’s some parallels with, you know, AppDynamics also, right? Like, we really focused on selling to the head of ops, the operations team, right, because the value prop of AppD was very clear to the business on, well, I mean, you know, it’s mean time to repair, it helps me with that, like, it helps me with downtime, it helps me with availability, it helps me with performance, so it’s a no-brainer. Like, if I don’t have it, my business is suffering. But you know who used to run it? Like, you know the operations team was installing AppDynamics on every server they had, right?

And so, if you don’t get them in the boat of, like, this is better for you, that’s kind of really hard, and, you know, took us a little bit longer than, you know, we would have wanted at Bicycle to get the same thing with the data teams. Because you can’t go in and say, oh, you know, you don’t need the data team. You just like plonk it on top of your warehouse and start. But it just doesn’t work because, you know, governance, et cetera. Et cetera.

Like, even if the tool is so easy, and we actually have built that, sort of like, that onboarding layer, et cetera, that you can put it on top of the warehouse and start getting answers, the point is, you don’t know whether they’re accurate, there’s no layer of governance, there’s no layer of, like, the data team certifying that what you’re doing is right and what you’re working with is actually right. So, you need both of that, and so that’s, kind of, where we landed.

Brian: Yeah. I don’t know where you are, like, stage wise, in terms of numbers of customers, such that you have patterns that are emerging, but I’m curious, are there patterns in your early buyers here, and is there anything that turned out to be wrong that you went in the door thinking one thing, and you’re like, whoa, we got to recheck that in the product or the way we’re positioning it or anything? Like, so patterns you saw both in the buyers and also was there anything that you were wrong about in terms about that audience?

Bhaskar: Yeah, absolutely. So, I think if I were to pick one or two things, right? Like, so what we started off, you know, initially was, kind of, being, like, an automated data analyst, like, just doing all the data analysis, you know, finding patterns and, kind of, things like that. And one thing we underestimated, or one thing that was a problem in that layer, was that the lift of how do you get from zero to where Bicycle is working really well as an automated data analyst, that started being a problem because it took a long time for everybody to say, “Let me get all my KPIs to you. Let me get my mappings. Let me teach the system so that it can actually do what it does with the automated data analysis.”

And so, what we did with this is, especially post-LLMs, we basically built like an automated business analyst on top. And what the business analyst does is really it has one job, right? Like, it starts with a level of… it doesn’t start from scratch. It understands retail, right, it understands travel, it understands payments, but what it’s doing is it’s figuring out how to compound the company intelligence or the company context, you know, on top of it. So, that’s one layer I felt we, I think, you know, started off, kind of, you know, later.

And then the other part I, kind of, already talked about a little bit, which is we, you know, over-rotated on, you know, selling to the business, talking to the business too much, and we didn’t really figure out, like, what’s the holistic pitch of, like, how does this tool, like, really get bought and, like, how does the data team really, like, govern this tool and have a way of, like, really administering this out to the business and everybody else, like, basically, you know, doing that, right? So, and then maybe the other part that we initially maybe I could say they got wrong or, like, you know, learned is, when you build a platform, the initial inclination is to, sort of like, have, like, a good large horizontal area and then get people to, sort of like, do what they need to do with it, but as we became more vertical, we started then thinking about look, the shape of this has to be an agent that drives an outcome. So, what happens now is, if you have someone in retail, you know, let’s say they have some data in Snowflake, they put Bicycle on top of it.

We go through the data, we look at the metadata, we analyze everything, and we give them options on here’s six agents we can do on top of this data. One of them is going to basically help with, you know, your checkout conversion. One of them can help with loyalty and administering this and that. So, being more specific in what the outcome is and what we’re going to do, in some sense saying it is an AI analyst that’s assigned to do a job, and that job is, like, really protecting the KPI. You know, working through this KPI and guarding this K PI or analyzing this KPI. That’s when we’re like, okay, it has to be super-specific. It cannot be like this horizontal area where people come in and say, oh, I have this great platform. Let me now spend X amount of time teaching it this and teaching it that. I would say those were the big lessons.

Brian: Do you feel like there’s converging on some, kind of, moat? Like, do you feel like you have a moat? Because I’m like, if I could get, you know, a million bucks, couple million dollars, hire some great engineers, why can’t I just rebuild Bicycle myself as a startup competitor, or even internally? I’m a bank. I got 4000 engineers. I’ll just build this myself. What’s your moat? Do you feel like Bicycle has a motive? You figuring that out? Like…

Bhaskar: Yeah, yeah. No, I think I’m actually pretty clear about it, right? Like, so I feel like, you know, compounding operating context or compounding company intelligence is the moat, right? Like, that’s, sort of like, an easy way of describing that. Because at the end of the day, every analytics tool is just, you know, it starts with data, and for us, we start with understanding how your business works, right?

Like, and the first thing isn’t, you know, building a dashboard, but it’s more of, like, an operating model of your company. And so, let me just go into maybe a little bit of detail, right? What are the events that matter, right? Like, what are the dimensions? And when I say events, it’s things like searches, you know, checkouts, et cetera, et cetera. And those are specific to the vertical. Which dimensions describe your business, right? Like, how are your KPIs actually calculated? What patterns are worth paying attention to? Is it, sort of like, search conversion? Is it like payment approvals?

Again, that’s the knowledge that you’re not learning specifically from a company, but you, kind of, know about that vertical. What kind of things typically cause those patterns? Are those typically caused by supplier issues? Are they caused by technical issues, et cetera, et cetera? Again, that’s broad knowledge.

And what typical actions do you typically… do you take, right? Like, for example, if you introduced a bug in a release, you would typically want to roll that back. Again, that’s general knowledge. This is all vertical knowledge. If you’re a retailer, Bicycle understands products, promotions, categories, stores, et cetera. Et cetera. If you’re a payments company, it understands authorization, right? You know, bins, issuers, et cetera. So, that is day one, right? The product carries that knowledge, right?

Then what Bicycle does is learning about your company, like, your KPIs, your definitions. How you define a KPI, and how you really calculate a KPI is different from another company, so we want to make sure that if you already have that slotted out somewhere, we’ll pick that up. What is your business hierarchy? Because your business hierarchy is different from something else. What is your terminology, your products, your suppliers, your regions, right?

So, as the system keeps running, the operating context of the company intelligence keeps getting richer because which KPI movements actually matter? Which anomalies are just noise? And this is where there is, like, the human-in-the-loop that comes in, right? Like which causes are common? Which explanation of me drilling into a KPI movement actually proved to be correct, where you’ve actually validated that for me?

Which actions were, you know, less of a blast radius and were accepted, and were like, okay, that we can actually do that? Which actions actually improved the KPI? That’s where you’re like, every investigation, every decision is improving and teaching the system something, and you’re, sort of like, you know, compounding. Because look, I mean the basics are the same, right? Like, everybody will have agents, everybody will have, you know, if you look at the tooling, everybody will have agents, you can build semantic models, you can build… everybody is going to have that, right?

There’ll be LLMs powering it. Everybody will have chat, but what is durable for, in my mind, is that how does a business actually operate and think of that as building an operating manual, and that’s when you can be very, very specific on this class of revenue-critical decisions. Can we help with decision support and decision-making? Because the more specific you get, that’s when you can be super, super useful. Because at the end of the day, intelligence is such a commodity that if you can create a trusted layer, if you can create something that is very, very specific to the customer’s work and to their, sort of like, business, and eventually create an outcome that’s measurable, that’s the bottom line. And for that, you have to be very, very specific. And so, to me, just to summarize that, like for us, the compounding operational intelligence or the compounding company context is basically the moat.

Brian: Yeah, this compounding institutional knowledge, as I think of it as, kind of, one of the three possible moats for I think, like, for AI for BI companies and analytics. So, can you make, just in a minute or something, give me a concrete example of how the tool compounded a piece of knowledge and got better the second time, to illustrate your point about how the longer we sit in here, the better value you’re going to get? Is this because humans are, like, “I like that recommendation. You changed this thing. I approved it. Do this some more next time.” And so, the system learns that, like, oh, it likes it when I do that, you’re reinfor… literally reinforcement…

Bhaskar: Absolutely.

Brian: Or not. I don’t know. Tell me… give me a concrete example of the compounding institutional knowledge.

Bhaskar: A hundred percent. So, we work with, like, a large quick commerce company, which is basically, like, you know, heavy volume of orders. Before I go into that, like, the one simple way of understanding the technology is we describe it as DEAL, like, Detect, Explain, Act, and Learn. Detection is just, like, detecting the KPIs, you know, explanation is both factors: technical and business. You know, acting is, like, act with guardrails, and learns basically where there’s reinforcement, like, where whatever you’re, kind of, learning from the.

So, let’s say come back to search conversion, right? Like, let’s say there is basically, you know, a hundred neighborhoods in San Francisco, and, you know, you’re selling a hundred different items, and you’re basically looking at search conversion, the agent is looking for search conversion and the movement in that. So, I’m in Pac Heights, so let’s just say you’re not, you know, brand is not selling well enough in Pac Heights, you know, quick commerce, Instacart, think of it as that way. Now, why could that be happening? It could be happening because it could be an inventory problem because you’re out of stock. It could be happening because, you know, Uber Eats is selling at a lower price, right? It could be because, you know, your catalog has an issue. It could be a technical problem, et cetera. Et cetera.

Now, once you get into a particular area, let’s say inventory problem, out of stock, could be a supplier thing, could be something as basic as you haven’t ordered because… or maybe, you know, there’s volatility in ordering, and you’re like, oh, you’re suddenly out because you didn’t anticipate that, right? As you start going through that playbook, what we’ve done with this customer is starting to add more context, more intelligence on how do you deal with out-of-stock. Like maybe there’s a scenario that’s very specific to the company where they’re like, well, we actually have ordered but haven’t gotten the stock yet. But we ordered but basically we only have half of it. How do you deal with that?

So, as you go through the decision tree, keep adding, you know, things that are very, very specific to the company, and then that remembers it for the next time, that basically acts on it for the next time. So that’s, kind of, like a good, you know, maybe a retail type of example. But something similar exists in travel as well, where you start getting into revenue losses because there is basically issues with suppliers. So suppliers, you know, are processing, and, you know, especially when you have online travel agencies, suppliers actually give you the tickets, and sometimes there’s issues with how suppliers are processing, and sometimes they’re typical… there are more issues with one type of supplier than the others.

How do you teach that system so that the next time around, the degree of certainty on this is on the brink, but the supplier has a history of issues, and so you can actually lean more on let’s take the supplier out because that’s actually what we do with that customer. We actually take the supplier out, you know, of the loop so that you stop losing revenue. So, there’s some, sort of, heuristics and decision-making that you can bake in, and those are, sort of like, examples of how it learns, you know, over time. If I don’t go into just, like, metric detection and things like that, like, which, like you said, that’s a pretty broad area.

Brian: Yeah, yeah. Thank you for taking that little tangent and making that concrete for listeners. I want to give you the final word here and let people know how that they can get in touch with you, but I wanted just one little user experience question about how. You talked about how you have to do a bunch of definition collection at some point. In order for the software to do its thing, it needs to know something about the business. This kind of thing means, like, the tool’s value is dependent on some activity that users have to do that could be perceived as a tax, like, extensive setup or investment, and like, oh my gosh, I have to redefine again what a customer is.

Like, they’re not dead, they bought something in the last 12 months, they live in this country, and like, because there’s no standardized definition of what this is, how do you make that not feel like a tax? Is this, like, we have a bunch of logical defaults because we’ve done so many e-commerce companies. You start with, like, a generic e-comm, and then you can make it better if you want to get more specific, but that’s the playbook? Or like, how do you think about the tax on the user thing?

Bhaskar: Yeah, I think that’s exactly how it is. You know, if someone signs up, we actually launched self-service yesterday, and so someone can go on the website and basically start for free and then just bring their data, you know, CSVs, et cetera, and get started. But that’s exactly what we do. We start with the baseline of what is retail, and as we do, like, X amount of retail companies, we learn more, and the baseline keeps getting better. But as a company, if you have to now put in your specific operating context, that’s where a lot of the agent stuff, like the LLM stuff, really helps, right?

Because you can say, “Hey, here’s my KPI document that I’ve been working for the last couple of years, right? I’ll give you that.” Like, it’s unstructured, it’s raw, but we can figure that out. They’re like, “Well, I have defined all my KPIs in my semantic layer on DBT. Here’s my connection, right?” Like, OAuth to DBT, we’ll figure all of that stuff out.

Here’s my, sort of like, ticketing system where this particular class of problems I have, sort of, triaged through. Connect me to it, I can figure out what to grok from it, right? So, I think you start with the baseline is the most important thing because that’s where the hardest work is, and it’s also starting to get into more specific things about, it’s not just retail, but quick commerce is very, very different from large marketplaces because, you know, you’re delivering in 15 minutes versus your order will come to you in a couple days, [laugh] right? Same thing with, you know, payments, right? Or travel. Like, a car rental company is very different from an online travel agency, right? So, the specificity that you start with is, you know, point number one. And then you build on it, so it’s just compounding, compounding, compounding. And in the addition of context, I think be smart in how you absorb it, like, how the agent can actually process some of it, and that makes it easier.

Brian: Bhaskar, thanks for being so transparent about your journeys here with both companies. I really appreciate it. I’ve been talking to Bhaskar Sunkara, the CEO of Bicycle AI. I want to give you the last word. Any closing advice for other founder operators, heads of product at B2B analytics companies, getting time to value, closing those first 20 customers, just, kind of, open ended for you?

Bhaskar: Yeah, absolutely. No, so definitely, as, you know, as I mentioned a couple times, obviously think about not just who’s the user, but how is the tool going to be governed? Like, who’s going to actually own it? Like, the data and the analytics team is a very, very integral part of it. And then think about, sort of, how the business is going to use it, but like, getting intelligence to be, you know, trustworthy and to be very specific is very important.

So, that’s a very big part of it. You can’t just build, like, a, you know, horizontal platform area. You have to start building, you know, much more closely to how they’re, kind of, doing. The user experience is actually really, really important, and AI isn’t going to suddenly, like, remove user experience. If anything, it makes it more important because it’s the connection between agents, decisions, and humans, and accountability, right?

Like, and a lot of times, it’s that feedback and how you build that, you know, UX layer intuitively, where you’re teaching concepts, you’re reinforcing, or you’re saying, “Hey, now you’re safe to do this because I’ve put guardrails in it.” Those parts are very, very important and very atypical from what you’ve done before, so a lot of this learning as the new world comes up. Yeah, so that’s, sort of like, maybe on the advice side. And from our side, we actually just we’ve been working on all of these, you know, whatever we’ve talked about, and, you know, have incorporated a lot of this thinking. Check us out at bicycle.ai.

We just launched self-service yesterday, so if you’re in travel, retail, or payments, and if you have some data, you can literally just go start for free, kick off a trial, and the AI will actually onboard it. And then I’ve already talked about the surfaces of the product on how we do push and pull analytics. So yeah, I’m excited to get more people to, you know, try it and, you know, just, sort of like, experience, like, what we really created. And then just work with us because we’re looking for the forward-looking, you know, thinkers who are really thinking about, like, how to do this in the new world.

Brian: Yeah, yeah. Where can people get in touch with you? Is LinkedIn, or do you have a website? Tell us… LinkedIn?

Bhaskar: LinkedIn is great. Yeah, LinkedIn is definitely great. Yeah, absolutely.

Brian: All right, cool. Well, I’ll put the link in the [show notes 00:50:27]. Bhaskar, thank you so much for coming on Experiencing Data and sharing your experience.

Bhaskar: Absolutely, yeah. Thanks so much, Brian. Thanks for having me.