Discussing Langchain with Max Tee

In this episode, the hosts discuss the importance of building AI products and the popularity of Langchain as a leading library for developing AI applications. They highlight the power of Langchain as a building tool and the benefits of using JavaScript for AI application development. The hosts also explore monetization strategies for AI products and the challenges of technical defensibility in the AI market. They discuss strategies for success in a competitive market and the importance of timing and niche markets. The episode concludes with the hosts sharing their AI winner of the week, Multiply.ai, and discussing the potential of multimodal large language models.

Takeaways

• Langchain is a popular library for developing AI applications and is highly sought after in the market.
• JavaScript is an important language for AI application development, as it allows web developers to easily integrate AI functionality into their projects.
• Speed to market is essential in the AI space, but companies should also focus on building technical innovation and defensibility.
• Finding a niche market and specializing in specific industries can be a successful strategy for AI product development.
• Multimodal large language models have the potential to revolutionize AI applications by combining text and other media.

Episode Transcript

Rod Rivera: Welcome to another episode. In these conversations, we discuss how to build AI products from both technical and business perspectives. We explore software engineering, AI foundations, market dynamics, business models, and industry players. We approach these topics from a first principles perspective. Joining me again is Max, a veteran in the VC industry and innovation space. Max, how are you today?

Maxson Tee: I'm feeling great! It's good to be back for another episode. Today we're talking about LangChain, so let's dive in.

What is LangChain?

Rod Rivera: For those unfamiliar with LangChain, it's the leading library for developing AI applications. It boasts over 45,000 stars on GitHub, possibly even more by now. When you think of building AI applications, LangChain is the name that comes to mind. If you search for AI jobs on LinkedIn today, the requirements almost always include LangChain. This proves not only the library's popularity but also its high demand in the market. Max, what are your impressions of LangChain?

Maxson Tee: That's fascinating. I wasn't aware that LangChain was so frequently required for AI roles. The fact that LangChain is almost synonymous with AI application development is intriguing. It's reminiscent of how Google became synonymous with search.

Personally, I find LangChain appealing because it makes deploying applications much easier. It allows developers to focus on solving problems rather than getting bogged down in the underlying workings of Large Language Models (LLMs). As someone who's not very technical, I find that aspect particularly attractive. I can envision myself potentially building an application using LangChain someday.

From a builder's perspective, what are your thoughts, Rod? You've deployed it yourself, right?

LangChain's Impact on AI Development

Rod Rivera: I agree with you. Building AI applications is a new space, and we haven't yet fully defined best practices or established rules. There aren't many books or university courses on the subject. So, if someone wants to build an AI application, the fastest way to get started is by downloading LangChain and experimenting with it.

LangChain offers several advantages:

1. It has everything you need, including integrations with most major commercial and non-commercial systems - over 650 application integrations.
2. It provides all the necessary building blocks for creating applications.
3. There's an extremely active community sharing best practices, tips, and helping each other.
4. The community is very fast in adding new functionality. Even for obscure preprocessing techniques, you're likely to find something in LangChain that will make your life easier.

So, if you're thinking about creating an AI application, LangChain is the first step.

Maxson Tee: Perfect. So, if I were to start building an application today, I'd download LangChain and use its framework and guardrails to figure out how AI can fit into my application, whatever problem I'm trying to solve, as well as get all the right integrations in one place. I take it that's the power of LangChain from your perspective?

Rod Rivera: Yes, the power lies in its comprehensive set of building blocks, best practices, and utilities. It's important to note that LangChain isn't an LLM itself; it doesn't provide compute or "intelligence." Instead, it offers a set of guardrails and best practices that enable you to build AI applications efficiently.

You still need to make decisions about which LLM to use (like OpenAI's GPT or an open-source alternative), which database to employ, and so on. But you use LangChain as the glue to connect different systems and create your workflow. For example, if you're building a "chat with your data" application, LangChain helps you manage the entire process from uploading PDF files to getting the answers you need.

JavaScript and AI Development

Maxson Tee: That's interesting. So, developers still need to think about all the workflow aspects, but LangChain handles much of the heavy lifting and integration work. I particularly like how it serves as a guardrail for developers, lowering the barriers to entry for different players to enter this space and develop AI applications.

Rod, you recently had a great chat with Jacob about using JavaScript to deploy AI applications. I'd love to hear your thoughts on this. During my university years, most AI and machine learning work was done in Python. Why is writing in JavaScript so important, and how will that further lower the barriers to entry for others?

Rod Rivera: Yes, it was a fascinating conversation. Jacob is responsible for leading the efforts to bring LangChain to JavaScript. This is important because while Python is popular for AI, data science, and machine learning, not everyone knows Python. There's a whole world of JavaScript developers - estimated to be 40% more numerous than Python developers - who have primarily been building web applications, websites, and backends.

More and more, these JavaScript developers are saying, "I also want to do things with AI. What can I do?" Historically, JavaScript hasn't been used much by data scientists and machine learning engineers, so there hasn't been much in the market to enable web developers to build data-driven or AI-driven applications.

LangChain is trying to help bridge the gap from web development to AI application development. Its JavaScript implementation is now close to parity with the Python experience. This means a new wave of developers who may not be as technically versed in AI can now build AI applications relatively simply.

Maxson Tee: That's super interesting. I always find democratization and lowering barriers to entry fascinating, as it allows more people to access certain capabilities. From a market perspective, it's a smart play. As you mentioned, with 40% more developers in JavaScript, you're opening up the field to more players and tackling a bigger market by bringing AI application capabilities to those developers.

It reminds me of how web developers transitioned from Web 1.0 to Web 2.0, and then with crypto, there's Web 3.0. Now we have AI coming into the picture. It feels like we're approaching a utopia of combining different technologies, which is really expanding the potential for LangChain.

The Future of JavaScript Developers in AI

Rod Rivera: Exactly. When I talk to JavaScript developers, many of them ask if they should start learning Python because that's where the AI-related developments are happening. They wonder if they'll still have a career five or ten years down the line if they remain JavaScript developers.

At the same time, many are noticing how the low-hanging fruit in web development, like creating landing pages or small applications, is starting to become automated. With LLMs and AI technology, it's becoming easier for non-coders to create reasonably good websites and applications.

This means that the classic cases of building websites for restaurants or small studios are starting to disappear. As a result, these developers are saying, "This is affecting how many products are available and our market rates. We need to move up in the chain and try to do something more sophisticated." In this case, that means building AI applications.

Maxson Tee: That's a great point. In the past, everyone needed a web presence, so everyone was trying to build websites. But now websites have become so responsive, and people want to have applications on top of them. This makes it easier for non-technical folks to build AI applications, not only to help themselves but also to create more smaller applications because it's easier.

It's a bit like how building websites was really hard in the early days, and then platforms like Blogger came out, and suddenly everyone had some sort of blog online. I think we'll see something similar with AI applications. The big players will probably build very sophisticated ones, but for everyday people like you and me, we'll probably need one or two relatively simple AI applications that could help us a lot.

LangChain: Starter Kit or Long-term Solution?

Rod Rivera: Yes, exactly. One thing you often see on websites like Hacker News or Reddit is more advanced users or sophisticated developers asking, "Who needs LangChain? Is it really necessary?" or saying, "I've graduated from LangChain." This comes from the idea that LangChain is a starter kit to get reasonable results when building with AI.

But once you've understood more and know exactly what you want, LangChain can become a bit of a crutch. Because it has so many models and integrations, when you really know what you want, everything else can feel like unnecessary dead weight. At that point, developers might ask, "How can I just do this on my own in the leanest possible way, with only exactly what I need?"

So indeed, for most everyday users, LangChain is the way to go. But as they become more sophisticated, they might graduate from it and decide not to use it anymore. From an investor's perspective, how do you view a product that's popular but that users might eventually decide they can do better without?

Maxson Tee: That's a very interesting question. It seems like they might have a retention problem eventually because you have activation, you make it a lot easier for everyone to do very well, and eventually, they have to move on and do something more sophisticated. I think there are multiple ways that LangChain is trying to address that.

Personally, I think the integration aspect is interesting. If you already know specifically what kind of integration you want, that's a different story. But many people want the flexibility and optionality to change whichever integration they go for as they progress. You don't want to get locked into one thing, as we've seen with previous large enterprise software.

To answer your question about something that allows you to make money and remain popular, I think e-commerce has some ideas like this. In e-commerce, you often have one product that grabs everybody's attention, a second product that actually makes money, and a third product that's there to show you why the second product is the best to buy. LangChain could do something similar because the buying psychology is pretty much the same.

I think the focus should be on driving traffic in at the top of the funnel and then figuring out how to activate and monetize. That's the second thing you eventually have to work on. And for retention, it's all about coming up with something new, some new integration, making it harder for people to leave the LangChain ecosystem.

I'd love to see what those Reddit posts are most dissatisfied with about LangChain, and then it's just about plugging those gaps. There's a possibility to build small, bespoke pieces for people who want something a bit more tailored. Alternatively, LangChain could decide to just target the everyday builders, which are numerous, and make it even easier for people to deploy AI applications.

Monetization Strategies for Open Source Projects

Rod Rivera: On these parallels we're drawing with e-commerce, another analogy we can use is that the monetization of LangChain is not through the open-source product, the LangChain library itself. Instead, they're doing a closed-source SaaS offering called LangSmith that's complementary to LangChain but not necessary for its use. While LangChain is a toolkit to build applications, LangSmith is a toolkit for things like metric evaluation and error tracking - monitoring what's going on with your application. It's a very different product. How do you view this strategy of having a popular product that you're not making money from directly, and then building a second product to monetize?

Maxson Tee: I think they're trying to build adjacent services on top that allow them to add extra value for users who are heavy users of LangChain. I'm pretty sure within the LangChain community, there are many great people who don't really want to move away from it. So LangSmith sounds like a great way to monetize from those users because they're so invested and believe that this is the way to go.

I personally think it's actually a smart way of doing it. You can see this with multiple different open-source projects. I always bring up this company called Hazelcast. They do something similar where they have a distributed framework to help you build distributed systems. But then on top of it, they also sell you extra add-ons that make it a bit better. In this instance, LangChain allows you to monitor your application, while Hazelcast might allow you to monitor your distributed system. So there are parallels you can draw with other open-source projects. My take on this is that I think this is a good approach. I don't think they're doing anything wrong per se.

LangChain's Dominance and Strategies for Competitors

Rod Rivera: One other thing to mention is that while LangChain isn't the only library in the market for this purpose, none of the others come close to its popularity. This is really a situation of right place, right time. LangChain came out to the public around the same time that ChatGPT took off. At that point, there wasn't really anything available in the market to start building AI functionality, and it immediately became the name that everyone knew about.

As time has passed, others have tried to replicate its success. Their products might be more advanced or better implemented, but they don't come close to the popularity and name recognition of LangChain. In these cases, Max, when you see a follower theme where it's becoming evident that it might only be possible to replace the incumbent, or where even second place is a very distant one, what are the options for those products and teams? They might be building something great, but for reasons outside their control, they can't be as successful as the original entrant.

Maxson Tee: I think there are two ways to think about it. There's definitely the first-mover advantage, which is where LangChain is at the moment. And then there's also the "riches in the niches" approach, where the followers that come in want to find a niche place to start and really service those specific customers and go deep.

For example, within financial services, the integration might be more focused on things like Bloomberg, FactSet, and other specific data providers, which is less relevant for LangChain. If I were building a second library, I would probably not try to compete directly with LangChain but find spaces where LangChain users are dissatisfied. For instance, you mentioned that a lot of people say they've "graduated" from LangChain. There's a possibility to build an application to serve those more advanced users.

My general sense is that for a fast follower strategy, which many large corporations believe is their strategy when it comes to technology, is to work with the incumbent. Find special niches where LangChain has decided not to serve because it's not a big enough opportunity for them, but it could still be a meaningful market that can spawn new businesses.

Rod Rivera: That's correct, and it relates to our discussion about LangChain's JavaScript version. One could say that the Python version is already 100% entrenched in the market, so it's very unlikely that anyone will be able to push it aside. But the JavaScript market is perhaps more up for grabs, partly because fewer JavaScript developers are familiar with the AI ecosystem. They might not have even started thinking about doing things in AI - it's still the very early days.

Potentially, there might be an opportunity for someone to focus solely on serving JavaScript developers and offering a better experience to them. But here's what I'm thinking, Max: JavaScript developers are still at the beginning of their AI journey. They're interested in doing something with AI, but they're not as sophisticated as Python developers. So they need more education, more hand-holding, and perhaps more convincing.

This, of course, takes effort. It's much easier when people are coming to you organically, and you just need to offer them a service, versus having to convince them to start building with you in JavaScript. When teams come to you in a similar situation where perhaps this market is underserved, but maybe it's underserved because the market is not sophisticated and still needs to be educated, do you say it makes sense to go for it? Or do you think the education will be time-consuming and distracting, and might not make that much sense?

Strategies for Entering Underserved Markets

Maxson Tee: I think context matters in this instance. It really depends on a couple of things:

1. How big is the market really?
2. How uneducated are they? Eventually, they'll get there, but what timeframe are you aiming for?

Because eventually, everyone will learn something and catch up. If AI is going to be the next big thing, every one of us will have to learn AI one way or another. So when it comes to things like how big the market is, whether you have the right resources, and the right time to spend on that market - those are all things that one has to think about. It's a strategic choice because it's a deliberate decision to go after that market.

There's a possibility that it works, and there's a possibility it doesn't work, but that's what innovation is about. You're trying something new. You've found something in that market, you go for it, it may work, it may not work, but in the process, you have to weigh the pros and cons. If it works, the reward is really high; the risk is that it might take 10 years to get there.

It's a bit like when Elon Musk started Tesla. That was a long time ago - I think around 2003 or 2004. He didn't actually see success until much later, but he was going after a big market. So some sort of similar thinking can be applicable here as well.

Rod Rivera: Indeed, Tesla has been around for a very long time. I think it was founded around 2003, and it's been a long journey.

Certainly, I'll continue and finish the rewritten transcript:

Maxson Tee: Yeah, exactly. But he's just been chugging along. So I guess understanding what kind of person you are is super helpful too. It requires some introspection, knowing if you're okay with "eating dirt" for the next 10 years in order to hit a bigger market. So I think it's a strategic choice you have to make both professionally and personally.

Balancing Speed and Innovation in AI Startups

Rod Rivera: Indeed, there's a lot to think about. I also see this situation from the perspective of the companies using LangChain. If you go to their website, you'll see logos of international household names - everyone is building with LangChain. However, if we're a young team trying to do something new in AI and we decide to use LangChain just to be faster, isn't this potentially a risk? Instead of trying to build some sort of intellectual property advantage by implementing things on our own - maybe things that are different or unique - we're getting speed and can go faster into the market. But at the same time, our technology stack becomes more generic and therefore potentially less attractive in terms of how valuable the company is.

When teams come to you with this question of being fast and getting to market quickly versus trying to have technical innovation, how do you frame it? What do you tell them?

Maxson Tee: I think, again, context matters in the sense that you need to consider what problems they're trying to solve. Is AI a big part of it? If AI is a very big differentiator for them, then maybe building it yourself might make more sense. However, if AI is just an add-on to the problem you're solving, then I would say going with LangChain makes more sense because it helps you get to market quicker, and it's not a core feature for you.

For example, let's say you're building a lending company in financial services, which is what I'm familiar with. If you're building a lending company, ultimately your job is to lend. So you cannot outsource credit analysis, and if you're using AI to do credit analysis, then you really cannot outsource AI. However, if you're using AI to do, say, customer form filling, then I would say use LangChain quickly because it's not really the most differentiating thing you could do. Yes, maybe there are user experience benefits in that part, but it doesn't actually contribute to the core fact that credit analysis is still your differentiation.

So I think you have to consider speed and whether or not this AI is a core differentiation point for you and the team. Those are the two things that I would flag off the top of my head. How do you think about it? If you were to build it yourself, what would your approach be?

Rod Rivera: It's a very valid point that you're making. I was thinking about it from the perspective of what one calls, a bit disparagingly, "AI API wrappers" or "GPT API wrappers." This means just calling the API of OpenAI and then packaging the responses in a nicer user interface. But in the end, the heavy lifting is all happening externally in OpenAI.

In those cases, I do think that speed to market is paramount. Being fast and trying to set a foot in the market is essential. This is also because so many new applications are being launched every day in the AI space that the differentiation you might have comes down pretty much to being the first one. Therefore, if you use something like LangChain, you can have this speed. But nevertheless, yes, it is something that, from a technical standpoint, is less defensible.

This has happened to quite a few companies. Because they say, "We are building on top of OpenAI, we use LangChain for this, and then we focus on experience," what has happened is that later, OpenAI or other providers come along, and pretty much what was an independent product becomes just an additional feature of their existing offering.

For example, up until this summer, it was a great idea to have AI agents that have your same style and can chat with you based on your knowledge. So you could say, "I'd like to chat with Albert Einstein," and this digital Albert Einstein would answer in the same way that the real Albert Einstein would. Quite a few companies were built around this concept.

Maxson Tee: Yeah, it's like the Microsoft Copilot.

Rod Rivera: Exactly. So instead of a little digital assistant, it's someone famous, like Elon Musk. You want to talk to Elon Musk, and the AI uses all of his tweets as data. Then you can talk to this AI version of him. Quite a few companies built around this idea. But technically, it was just gathering the data (like Elon's tweets), giving this to OpenAI's GPT, processing these results, and presenting it nicely.

But late last year, OpenAI announced this functionality themselves. Yes, it might not be perfect, it might only be 70% or 60% of what these startups were trying to do, but it seems good enough. And then it becomes a question of why would someone now go and use these startups when they can just stay with what OpenAI offers?

So I see this trade-off of being fast, but once you have set foot in the market and maybe you're even successful, given that you have this very limited technical defensibility because you are heavily dependent on OpenAI, as soon as OpenAI or someone else similar comes with very similar functionality, then it can be game over. Maybe not, but it puts you under a lot of pressure.

In those cases, Max, what are the strategies? Let's say you have a successful AI-powered lending application, and it becomes a hit, but it doesn't have too much of a technical moat. Then OpenAI decides to do "lending AI" or something similar. What would be your strategy here?

Maxson Tee: That's super interesting. I think for financial services, it's a little bit different because there's regulatory capture to it. Not everyone can process payments or lend money. But from a technical perspective, especially for consumer applications, a lot has to do with branding and distribution.

OpenAI is so powerful because of the internet. They have something very powerful at the back, but at the same time, they're also talked about everywhere. So it's a little bit like making quantum computing available for everybody to run compute on. In that sense, I think you kind of have to know from your own application what you're using AI for.

I agree with you that from a technical perspective, when you have a solution that is not really defensible, which means it's easily replicable everywhere, it's a bit like money - it's a commodity. Everyone has money. If you have enough money, you can just give it to anyone if you want. Certain applications are like that, and it makes them more replicable by others.

My general sense is that because of this, there will be applications that are doing very well, built by one or two persons, that will last for a certain amount of time. But eventually, a new wave of applications will come and take them over. To a certain extent, you could see that with Web 2.0 applications. When Facebook first came out, they didn't have Instagram or their chat app. That's why you had other apps on the side that allowed you to chat, like MSN and ICQ. But now in the US, the majority of people just use Messenger or iMessage, and then WhatsApp. Suddenly there's a consolidation of those applications by the bigger players.

OpenAI is kind of doing some of that. However, there are still other places that some of these larger applications or larger solutions will not go to. So I guess really picking the problem that you're trying to solve will help you understand whether or not technical innovation or technical defensibility is needed. That's one thing that I would think about if I were to build a company myself.

Conclusion and AI Winners of the Week

Rod Rivera: Yes, I see that the takeaway here is that time to market is essential. And if something like LangChain helps you get there, well, so be it. Amazing. But then don't base the whole value proposition of your company on this AI functionality that you've built with LangChain, but rather on other elements that are harder to replicate.

Here you were mentioning with financial services how there is all this regulation, how it has high hurdles to entry. And then if it is, let's say, a finance-related application that is sprinkled with some AI functionality, then this way you have a little bit of the best of both worlds. On one side, you have the AI functionality that you brought fairly fast. But then your main value is not the AI itself, but rather these financial services that you're providing. And this is something that requires a lot of specialization and not so easily will OpenAI come and try to do it instead of you.

Maxson Tee: Yeah, I think OpenAI ultimately is, as Benedict Evans said, still a pattern recognition technology. So there are things that they recognize very well, but then you also know that your life is a little bit more complicated. Patterns cannot be 100% correct all the time, and there are still a lot of areas that need 100% accuracy in order to serve a customer's specific problem.

So I think understanding the problem that you're trying to solve is super important, and then understanding how your solution can be defensible for that specific problem is also important. And then the technology of how you apply those technologies is where this conversation came about, right? Understanding what's at the back calculating for you, what you're going to expose, and therefore what your clients are going to experience. All that needs to be taken into account, at least in my head.

Rod Rivera: Yes, absolutely, Max. So I think this can be the main message for the listeners on how to think about building with LangChain, what to do and what not to do. And just to finalize, who was your AI winner of the week? Is there any company, any product, anything you saw that you'd say, "Hey, this got my attention, and I really was attracted to it"?

Maxson Tee: I saw a company called Multiply.ai. It's based here in the UK. They basically provide financial advice using AI. I find that very interesting because, as you know, financial advice is regulated, and it's also very specific. Financial advice for you and me is different because we have different preferences and needs. So I find that super interesting. Hence, it ties into my earlier answer about finding a problem that's very specific and how you can solve for that.

Rod Rivera: In my case, what I see as the winner of this week is the topic of multimodal large language models. The idea here is that at the moment, pretty much all language models are text-oriented, but more and more they're being combined with other modalities. I had a chance to play around with one called Lava in the last few days. It works with images and text, and it's fantastic how it's possible to provide an image and then ask it to describe what's inside, including text recognition.

The use cases are enormous. There's a whole industry of optical character recognition (OCR) with specialized tools and software. Now, maybe not for all use cases, but for a lot of them, it's no longer necessary to use these very specialized libraries. Instead, you can just use these types of multimodal LLMs and get the results.

Historically, you'd need to find the right library, preprocess the image, maybe crop it a bit, and even then, the object recognition might be a bit flaky. Here, I'm really astonished by how we've already mastered text, and now it's about these combinations of other media with text that will enable so many things in the coming months.

Maxson Tee: I find that super interesting. I recently read a thread, I don't know if it was on X.com or some email newsletter, basically saying how inefficient books really are. They're just not a very good way of sharing knowledge, but it's the only way that we've known how. Hence, when it becomes multimodal, it becomes super interesting because it engages all our different senses to gain knowledge. I find that fascinating, especially considering that about 70% of our brain power is devoted to visual processing. So I'm particularly excited about this development.

Rod Rivera: Fantastic, Max. This has been a great discussion this week. So for everyone, check out LangChain, download it, especially the JavaScript version. I think it opens so many doors to a universe of developers who until now were pretty much underserved. Thank you so much for being here this week, Max, and to everyone else. Let's reconnect next week. This has been AI Products. See you next week.

Maxson Tee: See you then. I'm going to duck out.