138 – VC Spotlight: The Impact of AI on SAAS and Data/Developer Products in 2024 w/ Ellen Chisa of BoldStart Ventures

Experiencing Data with Brian O'Neill (Designing for Analytics)
Experiencing Data with Brian T. O'Neill
138 - VC Spotlight: The Impact of AI on SAAS and Data/Developer Products in 2024 w/ Ellen Chisa of BoldStart Ventures

In this episode of Experiencing Data, I speak with Ellen Chisa, Partner at BoldStart Ventures, about what she’s seeing in the venture capital space around AI-driven products and companies—particularly with all the new GenAI capabilities that have emerged in the last year. Ellen and I first met when we were both engaged in travel tech startups in Boston over a decade ago, so it was great to get her current perspective being on the “other side” of products and companies working as a VC.  Ellen draws on her experience in product management and design to discuss how AI could democratize software creation and streamline backend coding, design integration, and analytics. We also delve into her work at Dark and the future prospects for developer tools and SaaS platforms. Given Ellen’s background in product management, human-centered design, and now VC, I thought she would have a lot to share—and she did!

Highlights/ Skip to:

  • I introduce the show and my guest, Ellen Chisa (00:00)
  • Ellen discusses her transition from product and design to venture capital with BoldStart Ventures. (01:15)
  • Ellen notes a shift from initial AI prototypes to more refined products, focusing on building and testing with minimal data. (03:22)
  • Ellen mentions BoldStart Ventures' focus on early-stage companies providing developer and data tooling for businesses.  (07:00)
  • Ellen discusses what she learned from her time at Dark and Lola about narrowing target user groups for technology products (11:54)
  • Ellen's Insights into the importance of user experience is in product design and the process venture capitalists endure to make sure it meets user needs (15:50)
  • Ellen gives us her take on the impact of AI on creating new opportunities for data tools and engineering solutions, (20:00)
  • Ellen and I explore the future of user interfaces, and how AI tools could enhance UI/UX for end users. (25:28)
  • Closing remarks and the best way to find Ellen on online (32:07)

Quotes from Today’s Episode

  • “It's a really interesting time in the venture market because on top of the Gen AI wave, we obviously had the macroeconomic shift. And so we've seen a lot of people are saying the companies that come out now are going to be great companies because they're a little bit more capital-constrained from the beginning, typically, and they'll grow more thoughtfully and really be thinking about how do they build an efficient business.”- Ellen Chisa (03: 22)
  • “We have this big technological shift around AI-enabled companies, and I think one of the things I’ve seen is, if you think back to a year ago, we saw a lot of early prototyping, and so there were like a couple of use cases that came up again and again.”-Ellen Chisa (3:42)
  • “I don't think I've heard many pitches from founders who consider themselves data scientists first. We definitely get some from ML engineers and people who think about data architecture, for sure..”- Ellen Chisa (05:06)
  • “I still prefer GUI interfaces to voice or text usually, but I think that might be an uncanny valley sort of thing where if you think of people who didn’t have technology growing up, they’re more comfortable with the more human interaction, and then you get, like, a chunk of people who are digital natives who prefer it.”- Ellen Chisa (24:51)
  • [Citing some excellent Boston-area restaurants!] “The Arc browser just shipped a bunch of new functionality, where instead of opening a bunch of tabs, you can say, “Open the recipe pages for Oleana and Sarma,” and it just opens both of them, and so it’s like multiple search queries at once.” - Ellen Chisa (27:22)
  • “The AI wave of  technology biases towards people who already have products [in the market] and have existing datasets, and so I think everyone [at tech companies] is getting this big, top-down mandate from their executive team, like, ‘Oh, hey, you have to do something with AI now.’”- Ellen Chisa (28:37)
  • “I think it’s hard to really grasp what an LLM is until you do a fair amount of experimentation on your own. The experience of asking ChatGPT a simple search question compared to the experience of trying to train it to do something specific for you are quite different experiences. Even beyond that, there’s a tool called superwhisper that I like that you can take audio content and end up with transcripts, but you can give it prompts to change your transcripts as you’re going. So, you can record something, and it will give you a different output if you say you’re recording an email compared to [if] you’re recording a journal entry compared to [if] you’re recording the transcript for a podcast.”- Ellen Chisa (30:11)



Brian: Welcome back to Experiencing Data. This is Brian T. O’Neill. Today I have my friend Ellen Chisa on the line. Ellen, how are you?

Ellen: Good. How are you?

Brian: Yeah, I’m doing great. It’s been a long time. I think we had a cocktail at some point, a decade ago, maybe, in Somerville. I think we were both in travel, and that’s probably what instigated that.

Ellen: That was it. I forgot—

Brian: You were at Lola.

Ellen: —you were in travel, too. Yeah.

Brian: Yeah, I had that startup, and you had moved over to Lola for a while. And I forget how we, like, reconnected, but I saw that you had moved into the VC space, and I thought we could talk a little bit about data tooling, and what’s happening in AI from the VC perspective, and all that, today. So, that’s kind of what I wanted to jump in with.

Ellen: Yeah, absolutely. Let’s do it.

Brian: Yeah, yeah. So, you’d been in product for a long time, and some design. You were kind of these hybrid product slash design professionals. And I’m curious, now that you’ve moved into the venture capital space at boldstart ventures, and you guys focus on, like, SaaS, dev tools, things like this, is that correct? Like, developer community?

Ellen: Yeah exactly. The big thing we’re focused on is we want to be working with people from before they even start their companies. We call it ‘at inception.’ We want to meet you before that, and then within that, we’ll do anyone enterprise, but that can be a lot of developer tooling, data tooling. So, I’m sure tools people who are listening use all the time. Yeah, and other SaaS.

Brian: Got it, got it. So, I’m just kind of curious, like, what’s it like being on the other side of this—[laugh] the other side of the wall. The fence? I don’t know what you would call it. You know, you’ve been on product teams, and now you’re on the VC side, but particularly like right now, with all this, like, explosion of stuff that’s been happening in the AI space, and like, any just general reflections on being on the other side now?

Ellen: Yeah, it’s really interesting. It’s exciting. So, I feel like when you’re on the operating side of this—and I remember this very much; Lola was obviously in a chat application, kind of in our last wave of AI and NLP and chatbots—you were thinking very much about the micro-interactions of how the product work, and how you might add it into the product you have and your feature and whatever is happening there. And so, you’re kind of looking at, I have this new technology. How do I use it? And from the other side of the fence, as a venture capitalist, you’re thinking much more, “Oh, we have this new wave of technology. What are the big shifts that it’s going to enable, and what sorts of foundational technologies or developer experiences or user experiences are going to enable the next wave of huge companies?”

Brian: Mm-hm. Is there anything that’s particularly changed for your company, or just how you’re seeing the landscape, given what’s happened in just, even, like, the last 12 months—particularly with the generative AI space and all of that—is that changing what you’re seeing in terms of the types of pitches that you’re getting, the types of founders you’re seeing, what they’re doing? I’m just kind of curious. Because there’s so much, like, “AI is our strategy,” and, like, “What are we doing GenAI?” All these large enterprise organizations are getting these, kind of, big, broad mandates with not a lot of clarity, and I’m kind of curious, from the tiniest companies coming up, like, what’s happening more down at that side?

Ellen: Yeah. It’s a really interesting time in the venture market because on top of the GenAI wave, we obviously had the macroeconomic shift. And so, we’ve seen a lot of people are saying, the companies that come out now are going to be great companies because they’re a little bit more capital-constrained from the beginning, typically, and they’ll grow more thoughtfully and really be thinking about how do they build an efficient business. And then on top of that, we have this big technological shift around AI-enabled companies, and I think one of the things I’ve seen is, if you think back to a year ago, we saw a lot of early prototyping, and so there were like a couple of use cases that came up again and again, one of them, for instance, focusing on developer products. I saw a lot of people who were saying, “Hey, we can use these new generative AI products to automatically write unit tests for functions.”

And so, that’s obviously something, most developers don’t love writing unit tests, it can be good for code quality to be able to have a robust testing practice, but that’s not something that makes a huge standalone company, necessarily. But it is something where it was very easy for one engineer to pick up the tooling and just try to build something on the weekend and kind of end up with a cool prototype, and start thinking, “Hey, this might be a company.” And so, I would say over the last year, we’ve sort of evolved from those use cases that are kind of, “Hey, what’s the first thing I can build and try this out,” all the way through to… more differentiated products.

Brian: Mm-hm, got it. I’m curious about, we think about founders as being product people sometimes. Founders are rarely designers sometimes, founders or engineers sometimes. Is there a class of founder that’s coming out of data science now or some kind of data thing where that’s their native place, and they’re moving into the business side [laugh], you know, to run a new business? And I’m curious if there’s stereotypes or trends or the way they see the world that’s different maybe than your other classes of founders?

Ellen: It’s a really interesting question. I’m trying to think. I don’t think I’ve heard very many pitches from founders who consider themselves data scientists first. There’s definitely some from, like, ML engineers or people who think about data architecture for sure, like, but I think people tend to pitch it more as a skill set rather than the role that they had. I’m also trying to think. Back in 2021, there was a big wave of what people tended to call modern data stack, and I don’t know how many of those products were truly founded by data scientists so much is people who were data engineers or product people who knew there were problems around data.

Brian: Yeah, I would assume that maybe the data engineering space might have been more reflected in the people coming to you, I don’t know, because there’s so much tooling and stuff in that space. I could see efficiencies, like, “Oh, God, we have to do this again. Like, we should have a product that could do this.” You know?

Ellen: It’s interesting because, like, early-stage startups and data are, like, not antithetical, but a lot of the times, you don’t have very much data at the beginning. You’re kind of saying, “Hey, I have this hypothesis about something I want to build. I’m going to build it and test it somehow.” Usually, the early testing is more qualitative where you’re just talking to a few users and seeing what their deep pain point is, and getting them to use it, which comes more from the design side, and then you have to get to some amount of scale before the data becomes interesting and significant.

Brian: Yeah. I assume at the level you’re working on, or when you’re meeting the people that you’re considering for funding, are these usually ones and twos amount of people? It’s a pair? It’s an individual—

Ellen: Yeah.

Brian: —it’s a trio, something like that. Quite small, I assume?

Ellen: Exactly. It’s usually—two is I think the common default for assuming how many founders there are. But yes, usually, it’s one to three people who have some idea and problem they want to solve. For us, it’s usually at least one of those people as technical, sometimes both.

Brian: And it looks like you were pretty involved with helping them get off the runway kind of thing, at some point, when you had started there. Is that correct? You were kind of in the weeds, like, boot—getting their company going, a little bit from a product strategy perspective, some of that? Is that correct? When I was looking at—

Ellen: At Lola?

Brian: —on your LinkedIn, kind of, when you’ve—uh, no, when you first came into boldstart, it looked like you had—

Ellen: Oh, yeah, so at boldstart, the fund has been around since 2010. My partners, Ed and Eliot, co-founded it, then our fourth partner, [who invest 00:07:19] Shomik joined a little bit before I did. And so, now it’s the four of us investing. But Ed and Eliot really refined the model around technical founders, enterprise product, high [conviction 00:07:29], small number of deals per fund. What I kind of came in to start experimenting with that we’ve continued on was, sort of, what is the best set of things we can have to help our founders, how do we provide them the most options and leverage to be able to go forwards.

And so, we did some stuff with events, we did some stuff around content, which of course, lots of VCs do, and then as I ended up moving more into the investing side, we had a new operating partner, Anna Debenham, who joined us from Snyk where she had been in one of those cool hybrid product engineering, design, data types of roles. And so, she’s really picked that work up. And so, we now have the developer founder’s toolkit that she wrote a bunch of, we have another set of internal resources that are available to our portfolio founders. And so, we really think of it as being very targeted. Like, we think of it is, we’re a product for technical founders, often building for other technical people, and so we try to make that product better by having the resources that helps that specific set of founders.

Brian: Got it, got it. Is there a core skill set or gap that you find that this class of founder comes in with where they need the mo—like, you know, 80% of them need help with X, you know? I don’t know product-market fit, or how to—I don’t know what it is. I don’t want to seed your head with stuff, but I’m curious if there’s patterns there with where they need the most help. I’m guessing it’s not the technical part, but I don’t know [laugh].

Ellen: It’s definitely not the technical part, I feel like you should not be asking your VC with help—

Brian: Right, right [laugh].

Ellen: —on the technical part most of the time. I definitely have had people where, when it’s only two people, you kind of need to get out of your head, and you want to talk to someone else, and I’ve spent some time talking about specific bugs with people, but usually, one of the things VCs are most helpful for is, we get the advantage of we get to talk to a lot of people all the time, so like I get to talk to you, and so we can often be helpful for making customer introductions, where we’ll just happen to meet someone at an event and go, “Oh, you’re telling me all about this problem you have. I know a founder who is solving that exact thing. You should connect and talk about it, and it’ll be mutually beneficial.” So, that’s definitely a big one. Another one is that, frankly, most founders don’t love spending all of their time fundraising; they like spending their time solving the problem. They want to solve and work on their product, and VCs have the benefit of spending time with other investors, and so we can be quite helpful in future fundraising processes as well.

Brian: What’s in the toolkit, though? Like, so what’s that about?

Ellen: Oh, that is all much more tactical, and so I think this stuff is a lot of fun. So, that can be stuff—like, one big question that used to come up for us all the time is I feel like—like travel businesses, you know, a good conversion is, travel is. You know what, like, a good hotel booking rate is. I don’t remember what it is off the top of my head, but people who work in travel generally have a pretty good sense of what standard metrics are.

There historically hasn’t been that for developer tooling companies. It’s been much more, like, “Oh, like, you know when it’s good when you see it, people start talking about it, maybe you have some GitHub stars.” And Anna went through, and she got a dataset that OpenView, a fund who’s local to us here in Boston, had produced, and she went through and pulled out only the data for developer-focused companies, and created a set of benchmarks that our founders could look at to see what’s going on, and, like, how successful their activation rates were, for instance.

Brian: Oh, got it. Got it. Do you find those are tracking pretty accurately across the spread of things that you do? Like, it’s a good enough benchmark for—because I mean, you cover a fairly wide set of stuff, even within I mean, tooling and SaaS and all of that there’s still, a broad—it’s cross-industry, right? I mean, depending on the industry, there might be, you know, a few customers, but each deal is a major bank, so it’s like a huge amount of money versus, like, I have to have a thousand customers or 10,000 customers that are all $2999 a month. You know—

Ellen: Yeah.

Brian: —I don’t—I’m just riffing here, but I’m curious.

Ellen: Yeah, yeah. So, that data set is specifically, she looked a lot at individual developer activations, which, of course, we also talk about the sandwich model a lot. So, a lot of the companies we work with—and I think data tooling is like this, too—you’ll have an individual developer or an individual data scientist who will adopt something and start using it within the organization, and then it might spread to a couple of their teammates when they add them to a project, and then to another team when someone switches teams, but then eventually way down the line, if you’re thinking about a six or seven-figure contract, there is someone out there who’s having the steak dinner and closing a big top-down sale with that organization. And that’s usually when you’re thinking about more access controls and more security features on top of the product. And so, that data around, like, what is good revenue retention, what is good net churn, there are pretty standard examples across industry that you can look at—Clouded Judgment by Jamin Ball is a really good newsletter that looks at some of those big metrics and how things are performing publicly—but there wasn’t nearly as much data about the bottoms-up side, which we tend to think of as being an early indicator of tools that are going to continue to be successful.

Brian: Got it. Got it. Is there anything that as a product person, you would have immediately changed, like, in your work at Lola or other places that you had been at, knowing what you know, now in the VC chair? Is there anything you would, like, actively just perspective-shift or anything like that?

Ellen: Yeah. I thought you’re going to ask me the opposite question. I feel like I know the opposite answer for what I’m—

Brian: [laugh] Well, I am going to ask you the other one too [laugh].

Ellen: Okay cool. I’ve got that one in the back of my head now. Yeah, I’m trying to think. So, after Lola, I was at Dark where we were building a programming language, and we very much went bottoms up to individual developers. I think one thing that I’ve definitely learned a lot more about since being on the VC side is just how important it is to think about your story from the top down and kind of like, what is the big market that’s changing, and why, and how is this going to work in the long run?

I think one thing we got to with Lola, was that we moved in the direction of being a platform for business travelers who are traveling regularly that was a much more constrained technical version of the problem, and I think with how early the NLP technology was, at the time, I think we should have constrained that down sooner than we did, so that’s certainly one of the shifts that I would have made. I think, in general, people usually go overly broad with their user groups, especially for tools for data people or developers—and travelers, I guess is another case that’s pretty broad—where people are like, “Oh, it enables you to do this thing in data, so any data scientist can use it.” Or, “It enables you to do this thing in engineering, so any developer can use it.” But developers and data scientists aren’t monolithic groups, and so I would encourage anyone who’s thinking about building tools to really narrow down on, like, okay, data scientists in this size company, who are or aren’t working with the data engineer, who are or aren’t working with a designer, and kind of figuring out, like, what the profile is, where it really shines. And then that’s not to say you can’t expand it to other groups after, but I think getting that first shining profile helps a lot.

Brian: Yeah, it’s—I mean, it’s the same thing in the services business. It’s just like, no one needs another generalist software engineering shop, like, that’s completely unspecialized that does everything for everybody. Same with design.

Ellen: Yes.

Brian: Like, you’re better off picking a hyper-niche, and kind of really understanding what it’s like to be a data scientist at a bank, you know, or with a giant team and a lot of money and resources versus—or a startup, you know, and knowing what that’s like and designing around that to get traction. I mean, it makes sense to me. Because you can always scale later, but—[laugh].

Ellen: And it’s like I think of that in venture, too. Like, at boldstart, we’re really good if you are a technical founder starting at inception, which is quite different than being a generalist fund, which some other people do and do very well, but those are different models. And I like being on the specific side of the model where you can really focus in.

Brian: Yeah. I’ve used this before. One of my former business coaches calls, like, it helps you have a Rolodex moment because it’s like, oh, the fun that starts out at incubation with SaaS and developer tools. Oh, that’s Ellen’s company—

Ellen: Exactly.

Brian: Versus, like, “do you know anyone at [these 00:14:47]?” Well, what are you doing? It’s like, “Okay.” You don’t really have any of those because it’s so broad, but it really helps you park to have that Rolodex moment. So, yeah, that’s cool. So, tell me—it’s not like you were going to anticipate—like, what does the product mindset bring into the venture capital share, like, coming out of product? Like, so answer that one [laugh].

Ellen: Yeah, no. This is the one that I think is funniest is that as a product person, and especially as a product person who works at high growth companies, you’re always thinking about, okay, how do I make this work more smoothly? How do I automate this? How do I have this keep going? How does this work when we suddenly have ten product people? How does this work when we have 10,000 users?

And venture doesn’t scale in the same way. Like, you’re a relatively small partnership, so your relationships are one-to-one; you’re not thinking about when our team is 10x larger next year. And similarly, like, the number of founders you’re working with, while you’re investing in new founders all the time, it’s, again, a constrained number of people, you definitely know. You’re not saying, “Well, we have a million customers, and I’m not going to be able to talk to each of them.” And I think it came in with the inclination to maybe over-make process for us, when it isn’t really necessary.

Brian: Oh, got it, got it. Back to the types of help that your founders need, the types of people that you’re working with, when we talk about designing experience, I think sometimes it’s difficult for people to get beyond interface design, and realize, like, well, a developer can have a user experience as well. There’s always an experience, it’s whether it’s designed or what I call ‘byproduct design,’ where it just kind of emerged from a bunch of other choices, and there’s your [laugh] experience. So, I’m curious, like, the companies and pitches that you get, and the product pitches and the people, is that ever a place where you have to work on where it’s like, well, that’s great. You know, you’re solving this really difficult data engineering issue, but man, is it clunky to get from A to B, or A to D, or whatever the path looks like. I’m kind of curious if you used that language, or is it even an issue? Do you use the language of design to talk about that with founders? Like.

Ellen: It is definitely an issue, and it comes up all the time. And I think one of the things that we’ve noticed is we’ll actually ask founders who are technical to pull up—I mean, this does go to interface—but, like, their Figma box, or, like, a walkthrough or a demo, and we don’t necessarily care about anything any—most designers would say this, too—at the stages people are working at, we’re not like, “Oh, does it look pretty? Oh, did you make a nice logo?” We’re thinking like, “Oh, can we see how you’ve logically thought through how someone is going to use your tool.”

Brian: Right.

Ellen: And there’s going to be refinement, and most of the people we’re working with aren’t necessarily design professionals, but you can start to see, are they thinking about it, where they’re putting the user first, like a designer would.

Brian: Mm-hm. Does that skill come easily? Do you feel like that… having done some advising at MIT with the Sandbox Fund, my general feeling is that younger people coming into the startup space are more intuitively aware of design matters, even if they’re technical. They have a little bit more mindset than I’d say engineers my age, you know, technical people that I worked when I was, you know, in a W-2 job and stuff, completely less aware, tend not to care as much. Like, just kind of, there’s an age difference. I don’t know. I’m just curious if you’ve seen that where they’re actually aware of this experience, the user experience thing, more than, I don’t know, more seasoned [laugh].

Ellen: I agree with you. I don’t know if it’s necessarily only age, but like, I’ve definitely worked with designers who maybe were at an organization that really valued UI, until, like, the value was how many screens can you make how quickly—

Brian: [laugh].

Ellen: And that was like something they were rewarded on, which I think is no longer a thing that organizations reward or think about in the same way because I think people are more likely to say, “Okay, we have one screen, and now we have a design system.” And I think when you have an experience like that, and like, the making of individual screens isn’t as valuable, you think about things a little bit differently, so you probably do think in a workflow-centric way.

I think another piece of it is—we’re probably relatively contemporaries—like, I always think of it as, like, in the old internet, you could just look at things and see how they were built, and everything was like not that great, and so it didn’t feel that daunting to make something new. And when I think about new generations of technologists, like, if you’ve grown up with an iPhone and every app you have is this, like, finished, polished thing that you can’t deconstruct, I think you probably have a very different sense of what software needs to be and how it works. But that might mean that when you start thinking about making a piece of software, you think about that holistic package of it being good from the beginning.

Brian: Yeah. Are most of the companies you’re working with right now, are these founders coming out of, like, “I had that pain. That was my job. Now, I want to start a business to solve that thing for people like me.” Is that the bulk of what you’re getting?

Ellen: Frequently. And I think in enterprise use cases in particular. A lot of enterprise problems are narrow enough that you experience them while you’re working in a large enterprise. It’s not the same thing—like, working in travel, I feel like everyone has had the idea of, wouldn’t it be great to have a personal travel itinerary app that just knows exactly where I want to go and what I want to do. Like that definitely is the thing that, like, lots of people just happen to think of, whereas I think the founders we work with tend to have some specific technical expertise that’s on the edge, and have a sense of what market is interesting for that technology.

Brian: Mm-hm, got it. Does the current velocity of all this stuff that’s happening right now with artificial intelligence, I’m curious, does that end up providing fertile ground for more of these opportunities around data tools, data engineering solutions, things like this? I’m curious if it spawns out all this other opportunity because like, “Oh, now we can do X, but in order to do X, we have all this infrastructure things that we need to build,” or it creates opportunities for that stuff. I’m kind of curious if it’s just spawned more opportunity there because of what’s happening in the AI space right now.

Ellen: I think, definitely. So, we obviously, like, we have a set of companies that are foundational model companies, and I think those need to be relatively well capitalized to train those models. Like, that’s just an expensive pursuit, you need a lot of technologists. But then even the layer beyond that—and I don’t think any of this is settled yet—if you have these models, and you have to prompt them, then you need to be able to test your prompts and understand which are better. And so, we’re starting to see tooling around A/B testing of prompts, which I would think of as being data tooling.

And then you need to have observability of, like, what is your model doing over time as you either change the prompts, or if you fine-tune or, like, depending on the context you get using [unintelligible 00:20:54] or whatever that is. And so, yeah, you start spawning off all of these different areas. And I don’t think we know exactly which of them will be buttoned together or not. So, is your prompt A/B testing and observability put together into one thing, or is that also with the hosting of however you’re doing your model, or do you end up having something that’s more like a serverless function where you call out to a model and then put it back into a classic application, and you’re monitoring the overall application output? I think there’s still a lot of open questions there, but yes, there certainly space for a lot more tooling.

Brian: And this is a little outside of my domain, so it might be kind of a green question, but I’m kind of curious if, like, the companies that own these large foundation models, it’s a lot of upfront expense, but then they’re going to have this considerable edge over even, you know, other companies that might want to build on top of that and create, you know, like, a subset service that relies on, like, say, a large language model for the, you know, the basic grammar and functions of communication and stuff, but then it’s been trained, you know, hyper-specifically on some space, does that feel like, from a VC, would you say, well, that’s a threat because, you know, the people that own the underlying models can eventually take over that. If they see there’s an opportunity there, it’ll be easy for them to jump in that space? Or do you see it more like, that’s just infrastructure. It’s kind of like Dell EMC, NetApp, they’re not going to want to get into the application space; they’re storage infrastructure, networking, compute, and then they stop, right?

And then you build your stuff on top of that, and they’re happy with that business to stay out of, you know, maybe getting into application layer, something like that. So, I’m just kind of curious with, like, the LLMs and some of that, if that’s how you see it, or just any reflections about, do they have this, like, edge, you know? Does OpenAI always going to have the edge over anyone building on top of OpenAI because they’re going to be like, “Oh, that’s a nice revenue stream. Let’s just build our own,” you know [laugh]?

Ellen: Yeah, I think it’s like kind of always about, like, what is the opportunity cost of something, and, like, what sparks the infrastructure? I think it’s interesting because, like, obviously, AWS, I feel like it’s the last infrastructure behemoth, which grew out of Amazon needing their own web services and having such complex infrastructure needs. And so, I guess I’ve always thought of OpenAI as being more parallel to, like, an AWS, GCP, Azure type of experience than Dell or EMC, although I like the hardware parallel as well.

Brian: Yeah.

Ellen: I think there is, like, a big technical debate around the idea of if we’re going to have these large models—which is where OpenAI and others thrive now—or we’re going to have many small models. And I feel like the many small models would be quite different because that would result in everyone kind of building their own, if they are capable of doing it. That would require it to become cheaper, that would probably require us to have more GPUs available. Like, there’s a lot of things that would need to happen for us to get there, but I think that would kind of remove that incentive. But either way, I mean, I don’t think it’s really, usually, logical for an infrastructure provider to start going after every single vertical application that could be built on top of their infrastructure.

Brian: Yeah, no.

Ellen: I think it’s just hard to have that many areas of expertise within one company.

Brian: Yeah. There’s discussion, I’ve heard about this, and I think I’m aligned on this initially, that a lot of user interface eventually may go away as we get better with conversational interfaces, whether text or voice, and I’m kind of curious, well, you can reflect on that with tools that have a GUI, you know, SaaS applications through the browser, but I’m curious if you think that will change the developer space at all, as well, that there will be less tooling—or I don’t know h—I don’t even know, but just whether or not even the experience of being the developer—because obviously we can already write code with these tools; I mean, I’ve been doing this the last couple of weeks with just stuff where it’s great. It’s just—and I’m learning a ton just watching it, like, fix it, and then I paste it in the error, and it translates the error into English I can understand, and then I watch as it fixes it. It’s like, jeez. Like, and then I think about, like, some of these companies, and I’m just curious what the disruption is going to be to, like, you know, data tooling person with this.

Ellen: There’s so many good themes in here. So like, one thing I find very interesting is I still prefer GUI interfaces to voice or text usually, but I think that might be an uncanny valley sort of thing where if you think of people who didn’t have technology growing up, they’re more comfortable with the more human interaction, and then you get, like, a chunk of people who are digital natives who prefer it. But then I think you get another set of people are kind of like, “Oh, well, we can just go beyond this,” and the new interface is voice, and they’re expecting it to be good instead of expecting it to be bad, like I do. So, I think that’s, like, one very interesting thread is how that plays out, especially as many of the people building the technology today are still what I would think of as being GUI first or more digital-native people. So, that’s one thing.

And then I think [sigh] I think it’s not that we won’t have any interfaces; I think we just might have quite different ones. There’s some really cool work that Amelia Wattenberger did, she was a designer at Adapt, and then has also been at GitHub Next that, like, talks about how using AI, you can have different levels of zoom for looking at something. And so, like, if you’re looking at Airbnbs, you maybe could zoom in and see one Airbnb, but then zoom out and see a map, like, not just a physical map of locations; you could also say, like, okay, the two axes I care about are these two axes, and then make a dynamic plot of all of the Airbnbs for, like, their location to your conference center compared to their price, so you can, like, see the trade-offs dynamically and zoom in and out of that. And so, you might be able to ask for that in a text-based way, but you’re still, like, seeing results in a graphical way. And so, I think we’ll kind of have these hybrid interactions that are enabled by this.

Brian: Yeah. I don’t think it’s going to zero anytime soon, and I think certain stuff, our visual acuity is so fast, like, when you plot stuff on a graph than, like, listening to it read off, here’s the one hundred things in order of—like, you would not want to sit and listen to that read to you. Your eyeball can say, “All right. I’m going to look at those three in the top right [laugh] corner.” Like, so there’s definitely still relevance there, but I think even just when you think about navigating context, switching all this kind of stuff that involves, like, not—what we would call Tool Time in the design space versus Goal Time, where it’s like, I’m actually getting some work done.

It’s like, no, I’m trying to figure out how to link up my bank account, so I can go do X. It’s like, I really just want to send some money, but [laugh] like, I got to go through all these steps, you know? And so, being able to say, like, “I want to send money to a new bank, and here’s the routing number,” and it’s, like, [burrrr], you know, and then it wires all that stuff up. So, I’m kind of thinking of it more that way, where—and I’m kind of curious if that, if you see that that’s going to disrupt on the developer tooling side as well, even if, you know, those tend to live in the terminal [laugh].

Ellen: I think it—my—the other example—and this isn’t a developer example, but you reminded me of it—is the Arc browser just shipped a bunch of new functionality, where instead of opening a bunch of tabs, you can say, “Open the recipe pages for Oleana and Sarma,” and it just opens both of them, and so it’s like multiple search queries at once. And that seems like the sort of thing that a developer might do. So say, you’re a developer, and there’s, like, a live incident, and you’re like, oh, I want to open this graph in Honeycomb to see the tracing, and I want to open, like, this internal webpage, and I want to, like, load up this bug in Zendesk that a user reported, and this email from somewhere else. And like, being able to do all of that very quickly and kind of end up just in a workstation, I think, is something that seems not very far off.

Brian: Mm-hm. Those were some local restaurant, Ana Sortun restaurants that you’ve mentioned, for people listening not out in the Boston area [laugh]. Two of my favorites, though.

Ellen: They’re good ones. I try to always—good easter eggs with podcasts [laugh].

Brian: Very good [laugh]. Excellent, excellent. There’s a new Sofra opening, too, in Brighton, by the way.

Ellen: I’ve never been. This is on my list.

Brian: Ah okay. Yeah, definitely worth a stop, and there will be a second one, so that’s cool. That this has been, like, super fun to chat with you about this. Any questions that I didn’t ask you that you think I should, for, kind of, my audience of data product leaders that are out there, just in your space right now, that you wanted to add?

Ellen: Yeah, I guess one thing I think is I really do feel like this wave of technology biases towards people who already have products and have existing datasets, and so I think everyone’s probably getting, like, this big, top-down mandate from their executive team, like, “Oh, hey, you have to do something with AI now.”

Brian: Right.

Ellen: And I don’t know, I feel like there are going to be, like, little quick hits where people just want to staple a little thing onto their UI to do it, but I feel like there’s big opportunity here, and I think there’s big opportunity for differentiation, and so it’s worth data and product leaders, like, sitting down and really thinking, like, “Okay, what can I do with this technology?”—

Brian: Yeah, yeah.

Ellen: —in a more ambitious way.

Brian: A lot of times, we don’t want to—we try to—at least in my design work when I’m doing training, it’s trying to teach not this, like, solution-first, solution in search of a problem kind of issue, but I do feel like there are times where it’s like, there’s this massive new tool that does all these amazing things with it, and then sometimes is a time where it’s like, we might need to start with the tool to get us to understand the problem space. And prototyping with this tool, like, it’s so disruptive and so new, and in this—particularly the LLMs, I think, in the enterprise space—you’d be foolish not to be playing around with it to figure out where maybe there is opportunity. And so, there’s times to break those rules a little bit [laugh] with maybe starting with a solution is okay, sometimes. It’s just how you—it’s kind of like what’s your goal here in the initial stages? And I would still be, like, looking at it as I’m trying to find a problem where this new thing would be really useful, but I’m playing with it in order to expose that space. I don’t know [laugh].

Ellen: Yeah, and, like, playing with it in order to understand what it can even do. Like, I think it’s hard to really grasp what an LLM is until you do a fair amount of experimentation on your own. And the experience of asking ChatGPT, like, a simple search question compared to the experience of trying to train it to do something specific for you are quite different experiences. And then even beyond that, there’s a tool called superwhisper that I like—although there’s other versions of this—that you can take audio content and end up with transcripts, but you can give it prompts to change your transcripts as you’re going. So like, you can record something, and it will give you a different output if you say you’re recording an email compared to you’re recording a journal entry compared to you’re recording the transcript for a podcast.

Brian: Oh, so like—because you want the email to be short, like—

Ellen: Yeah.

Brian: —take my blabber, but, like, condense it down to an email. Like—

Ellen: Yes. Take my blabber and turn this into, like, an email with the most germane three facts.

Brian: Oh, I see.

Ellen: Or take this, like, stream-of-consciousness thing about the meeting I just had and turn it into an outline of meeting notes. And you can give it a lot more specifications about what that is. But I think it’s hard to understand what is easy or hard for an LLM without spending a lot of time with them.

Brian: Right.

Ellen: And so, I think that playing is just valuable because you’re not going to know how to apply the solution until you actually understand what it does and what it doesn’t.

Brian: “Have you been told you talk too much? You should try superuser”—[laugh] or whatever the company was [laugh]. I mean, you think about the niching though, like, “Hey, I know I’m, like, a little bit verbose. Like, [laugh] maybe I should try this tool.” That’s pretty funny.

Ellen: I do think there’s, like, also just going to be cool stuff like that, like, a lot of the meeting transcription tools will tell you who’s talking for what percentage of time in the meeting. Like, meta-analysis of that information is something that should be pretty easy now because you’ll just be able to pull it out and say like, “Okay, I’ve copy-pasted the transcript in. What is, like, my average amount of talking in meetings compared to other people?” Like, that’s a pretty cool query that I… I mean, actually building up the dataset would be annoying. Using this, maybe not so bad.

Brian: [laugh] Yeah, yeah. There’s a lot of cool, a lot of cool stuff going on right now, and fun things, too. Where can people, like, get in touch with you? Where do you hang out? LinkedIn? Like, what’s the best place for people to find you?

Ellen: Yeah, I guess, LinkedIn, I’m Ellen Chisa pretty much everywhere on the internet. If you can find me, I exist there.

Brian: Cool.

Ellen: Yeah, or ellenchisa.com. Or I’m ellen@boldstart.vc if people want to email.

Brian: Cool, yeah. And that’s Chisa, that’s C-H-I-S-A. Ellen, it’s been really great to just catch up with you and riff a little bit, and I’m happy you’re in the new space here with the VC side. So, hope that’s going well.

Ellen: Yeah. It was great to see you again.

Brian: Cool. All right. Take care.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Subscribe for Podcast Updates

Join my DFA Insights mailing list to get weekly insights on creating human-centered data products, special offers on my training courses and seminars, and one-page briefs about each new episode of #ExperiencingData.