Discussing Unify with Max Tee and Chris Wang

The conversation revolved around Unify, a company that evolved from Ivy, an open-source framework for machine learning, to a "model hub" aiming to provide a Unifyed API endpoint for various AI models. The participants discussed Unify's business model, market positioning, branding challenges, and potential future directions. They explored analogies from other industries, considered the company's strengths and weaknesses, and debated the viability of their current approach in the rapidly evolving AI landscape.

Takeaways

1. Pivot Challenge: Unify is struggling to transition from a successful open-source project (Ivy) to a profitable business model with their "model hub" concept.
2. Branding Issues: The company faces significant branding challenges, including difficulty in search engine visibility and potential confusion with other similarly named entities.
3. Market Uncertainty: There's uncertainty about whether the market will consolidate around a few major AI providers or fragment into many specialized offerings, which affects Unify's potential value proposition.
4. Enterprise Trends: Many large enterprises are moving towards in-house, on-premise AI solutions, which could limit the appeal of Unify's model hub.
5. Cost Optimization: While cost savings is a major selling point for Unify, companies have other ways to optimize AI costs without switching providers.
6. Niche Targeting: The company might benefit from targeting industries with thin profit margins where cost optimization is crucial.
7. Community Leverage: Unify has a strong developer community from its Ivy project, which could be leveraged for its new offerings if properly incentivized.
8. White-labeling Dilemma: There's a tension between providing a white-label service and the desire of AI companies to maintain brand visibility and customer relationships.
9. Potential for Acqui-hire: Given the challenges, one likely outcome for Unify could be an acqui-hire by a larger AI company.
10. Evolving AI Landscape: The rapid pace of AI development makes it difficult to predict long-term viability of current business models in the space.
11. Alternative Approaches: Concepts like AI agents that can autonomously choose between different models might compete with Unify's model hub approach.
12. Analogies from Other Industries: Parallels were drawn with Unifyed messaging platforms, financial services, and logistics to understand potential pitfalls and opportunities.

These takeaways highlight the complex challenges Unify faces in establishing itself in the competitive and rapidly changing AI market, as well as potential strategies and considerations for its future direction.

Episode Transcript

Introduction to Unify and Its Evolution

Rod Rivera: Welcome everyone. Today we'll be discussing Unify, a company with an interesting evolution in the AI space. Let me provide some context on Unify's journey:

Unify started as Ivy, which was essentially a framework for machine learning experts. The goal was to bridge the gap between technologies used by enterprise developers and those used in academia, research, and cutting-edge startups. Ivy acted as a meta-language, translating between different machine learning frameworks.

Ivy became incredibly successful, garnering thousands of GitHub stars and building a large community of active developers. In fact, Daniel, the founder, mentions managing around 500 developers at any given time.

However, to build a sustainable business, Unify is now shifting focus. They're promoting what's called the "model hub". This is intriguing because:

1. There seems to be limited synergy between Ivy and the model hub.
2. The 500 developers working on Ivy may not have strong incentives to engage with the paid model hub offering.

The model hub concept:

• It's an API endpoint for machine learning models from various providers like OpenAI, Amazon, Azure, etc.
• The idea is to have one API endpoint to rule them all, similar to Ivy's approach with frameworks.
• As a developer or machine learning expert, you can be assured that your request will be routed to the most cost-effective model in the market.

Max, you were part of the discussion with Daniel. What were your thoughts on Ivy and Unify?

Max Tee: From my understanding, Unify's website promotes access to a suite of models via the Model Hub, helping optimize your deployment stack. In simple terms, I see it as a "best execution" for models based on the problems you're trying to solve.

To elaborate, there are multiple models you can query to solve a problem. The Model Hub helps optimize and calibrate based on your priorities, whether it's speed, cost, or depth of answer. That's my interpretation, though I could be mistaken. Rod, do you have a different technical perspective on this?

Rod Rivera: Not entirely, but Chris made an interesting analogy offline. Chris, could you share your thoughts on Ivy and Unify?

Analogies for Unify's Business Model

Chris: Although I wasn't part of the conversation, based on what I've heard, two images came to mind:

1. Trading: It's similar to a limit order where whichever market hits the target price executes the order.
2. Unifyed Messenger: It reminds me of services like Trillian or Meebo. These allowed you to connect accounts from various messaging platforms (ICQ, AOL, Microsoft Messenger) to a single service, enabling you to message everyone across different platforms.

Rod Rivera: Those are excellent analogies, Chris. However, if these analogies are accurate, they might be a bad omen for Unify. We all used these Unifyed messenger programs in the past, but we can't recall their names now. Today, they're not the primary way to interact with messaging apps.

This could suggest that the market for API endpoints might coalesce around a few major players. Users might simply prefer to use Azure or OpenAI directly, deciding that potential savings don't justify using a third-party provider with its own support and recommendations.

Another challenge I see is branding. During our research, we struggled to find information about Unify. Even when searching for "Unify AI", their website was nearly at the bottom of the results. We also found similar companies in the AI space that weren't Unify AI. This presents a troublesome situation where the brand gets lost in the space.

Max, given these challenges - a potentially generic name and difficulty standing out in the space - what advice would you give them?

Branding Challenges and Strategies

Max Tee: This is a challenging question because we're all confined by language. It's similar to URLs - all the single-word domains are taken, so new companies often use two words. In the future, we might see three-word URLs.

I can't comfortably answer the branding question, but what I typically see is companies owning a segment. For example, they could be "the Unify for AI machine learning models". In Malaysia, there's an internet package called Unify, and when people there talk about Unify, they think of internet packages.

The key is differentiating yourself between different markets and the players you want to serve. Given that Google search is generic, tailoring your brand to your specific market or user base is crucial.

Many startups use creative names, like how you named this podcast "AI Products". It's wordplay, it's fun, but it gives a different shade to your users at the very least.

Rod Rivera: That's a good point about having two names depending on your sector. Chris, when you see teams coming up with names, and considering the idea that you want to be broad enough to accommodate potential market or product shifts, what do you advise as the right approach to finding a good name?

Chris: Honestly, I think the name "Unify" is good. It conveys the value proposition of what they're trying to achieve, and it's simple. The main focus should be on ensuring top-notch SEO so they can be easily found. They also need to consider what other words to use to provide context to their name, given that a Google search for "Unify" yields various unrelated results.

However, I'd like to shift our focus a bit. I think the name is fine, and it's more about marketing and growth strategies at this point. Rod, from a technical standpoint, how significant is the pain point that Unify is addressing? Can you comment on that?

Market Opportunity and Pain Points

Rod Rivera: That's a point where I'm a bit ambivalent. On one hand, one of the main complaints companies have about the AI wave is the cost explosion. It's one thing to do a small proof of concept with OpenAI's GPT API for a few users, but it's another when your entire customer support department of 700 agents needs to start using these assistants. Companies are often surprised by how quickly the bills can escalate.

So, the idea of saving money and being more efficient is certainly appealing. However, the way Unify proposes to do this with the Model Hub is not the only way companies can save money.

In the business model for these API endpoints or cloud models, there are primarily three levers they can adjust:

1. Which model to use for the request (cheaper models tend to be more basic but less expensive)
2. The input passed to the model (longer requests cost more)
3. The length and accuracy of the answer (more detailed responses cost more)

Companies have a set of levers they can adjust without necessarily switching providers. When I talk to companies, nobody has really mentioned that switching providers is something they're actively considering.

Bold statement: It seems that at this moment in the market, companies are more focused on optimizing existing models with their current provider rather than looking for the best offer across the market.

Max, you have a good analogy with this marketplace idea. Coming from a finance background, I imagine there was a time when something similar happened in financial markets. Can you draw parallels between the financial space and Unify?

Max Tee: Yes, let me think about this. There are two components to consider: why this problem exists, and why it's relevant now.

As Rod mentioned, people are currently just trialing and trying to do their best with whatever models they have. So at the moment, Unify's solution may not get massive adoption. But I can see that from a parallel perspective, if and when some of these models become super specialized for certain types of tasks, that's when you'll want to figure out how best to route your questions based on your criteria.

The reason for this is that all these large language models are trying to be as generic as possible, and any sort of specialized model they're developing in-house. There's a possibility that this solution can be developed in-house. For example, if you're a large bank, you might have a model for private wealth management that may or may not interact with your model for your markets trading business. But there could be a use case that requires both models to answer a certain question. Could you then do routing to that instead of just to a generic ChatGPT that you have on the side?

I'm trying to figure out different use cases that may or may not exist because I'm just thinking out loud. But I would be quite interested to see if those specialized models start manifesting themselves within large corporates.

On the flip side, large corporates also try to present themselves in a Unifyed front. If you read recent reports from big banks, everyone will have something like "One Goldman Sachs" or "One Barclays". The idea is to combine all their different products and offer them to clients in one interface.

From a technological standpoint, if you have one interface where people can just speak to, and then you have multiple models at the back running and giving you some answer, that's where the routing becomes important. But that's just how I'm thinking about it. I could be very wrong tomorrow if something massive happens.

Chris, do you see something like this from your past experience in innovation centers, large corporates, or as a builder yourself?

Challenges in Market Penetration and Monetization

Chris: First, I need to apologize. I actually called it "Unifyer" earlier, but it's "Unify". So maybe there is something about the marketing that needs to be addressed.

I like what Max said about timing. As I think about this image of messengers and unifying messenger platforms, we do live in a world where we still have many different messengers. I have Telegram, WeChat, WhatsApp, iMessage, Signal, and Slack, and I feel pretty overwhelmed. Yet, I don't have one unifying messenger platform that I'm using to deal with all my messages.

I've been thinking about why I'm not using such a platform, and I can come up with two answers:

1. We may be living in a world of unbundling rather than bundling at this point.
2. Because these apps live on my iPhone, I feel like there's no need to switch or condense them. I just go onto an app, and the barrier to entry or usage is actually quite low.

The reason I'm bringing up this image is because I wonder if this technology we're discussing, having to transfer from framework A to framework B, is enough of a pain point. Is it enough of a barrier for people to develop and really harness? Rod, I'm not sure if I heard that this was really a big problem.

I guess the question I'm really asking myself is:

1. How big is the market?
2. If you were to build a business around it, how would you actually want to build it?
3. What is your exit route?

For this company, I could see an acqui-hire scenario, basically being bought by an AI company that uses the technology to make something easier. It could just fold into a feature or something like that. What are your thoughts on that, Rod?

Rod Rivera: Yes, I want to go back to something Max was saying about enterprises. On one side, enterprises are deploying things in-house and they are doing their own models or using existing models that are deployed on-premise. Once you have this on-premise and on your own servers, the question of which provider you're using doesn't exist anymore because you're using your own machines and hardware.

Bold statement: This is really the way I see most relevant enterprises going. They'll say, "We want to have everything in our hands because this way I can guarantee the quality of the service. I am not at the mercy of an API failing or any data leakage. I can control everything."

The other thing is the idea of routing and using different providers. It works nicely in a world where there are many models with very different capabilities and requirements. But we're seeing two trends:

1. Companies are ending up using more or less the same models, let's say three to five.
2. There's a trend towards fine-tuning. Companies take a model they like (e.g., Llama from Meta or Mixtral from Mistral) and adjust it for their company data and specific use cases.

As a result, this idea of taking models from the open market and routing them may no longer play a role because companies will have their own fine-tuned models for specific departments, which aren't available in the open market.

Bold statement: So here's what I see: this is definitely an acqui-hire play, as you mentioned Chris. It's a very good way to give a successful outcome for the company.

At the same time, this is a growing market that's evolving rapidly. This market didn't really exist a year ago, so who knows where we'll be five years down the line. It's possible that there will be an explosion of providers, with more regional companies starting to offer models. In that case, it could become very confusing, and you might really need someone who acts as this broker.

Another topic we keep highlighting in our discussion is branding. If there's one trusted brand in this space, let's say Unify, then companies might go to them even if they're using very nice, powerful models in the back that don't have a strong brand themselves. Companies might come to Unify because they use the Unify brand as an umbrella brand.

So, if we believe that the market will explode in use cases and technologies, then yes, there's a big opportunity for Unify. If, on the other hand, we think it's coalescing around a few specific providers, and large companies are using this on-premise and in-house anyway, then I'm a bit more skeptical.

Another thing I keep thinking about is that Unify has built a very strong brand in the open-source space with Ivy, with all these developers and awareness. But it seems they're struggling to bring this into their paid offering.

Max, if a company has found success with one product but they can't bring this success to their next offering, what options do they have?

Strategies for Leveraging Existing Success

Max Tee: I don't think I can give advice, but I can share some observations, especially when a company has already built a big brand. I'd like to draw a parallel to the influencer space. Look at how Kylie Jenner became famous - she built a very strong brand for herself, and eventually, she started selling products, and everyone just followed.

I was thinking whether there could be something like that within this space. If they've already built a huge following of different AI engineers, there will be some things that all these 500 engineers are talking to each other about that may eventually flow into problem statements or even features that could extend beyond your model hub.

Instead of thinking about monetizing, I would focus on solving some of the emerging problems for your 500 engineers that you're so captivating. Because like you said, a year ago, this market didn't exist. And five years down the road, who knows what kind of problems we will have.

Bold statement: My general sense is to follow where these people would like to go. I know it sounds a bit non-capitalistic, but I guess the money will come if you have a lot of those people following you around, as we have seen in recent years, especially in the influencer space.

Another thought I had was around internal solutions and open source. You folks are probably very aware that a lot of financial software is built on open-source frameworks. So basically, you just take something open source, bring it in, and some engineers decide to add a couple of things that specialize for internal usage.

If I were to draw a utopian world where a lot of the work and knowledge can be replaced with models instead of just human routing requests, that might be something interesting. For example, let's say you're partnering with a large bank, you want to run an event and you want to be able to use certain marketing materials, get approval, make sure it's right, and have legal sign off. Today, it's a person that does that. Is there a world where all the different models are being deployed in the different departments where simple requests can be routed and you can get sign-off or input for those marketing materials a lot quicker?

If that's the case, then I can start seeing that the larger software players like Microsoft, who are already in all these large corporates, will be able to utilize something like that to help optimize those routes. The reason I'm thinking about this is I had a conversation with a startup that basically helps people route emails because a lot of the requests are still coming via emails. If models can help to take away those emails, then there is a need to do the optimization because today, nobody does the optimization of emails. All of us get CC'd into giant emails that have nothing to do with us.

That's just my thought. Chris, I saw you nodding a little bit. Maybe you resonate with the email stuff, but I'd love to hear your thoughts around this.

Chris: Just to piggyback on what you just said, what I'm hearing is what you'd love or what you would like to have. Maybe I can give another example: an AI that coordinates the creation of a business plan. You have an AI that focuses on the financial part, one that focuses on the marketing part, maybe a creative AI that comes up with all the fancy images, and then essentially having something that coordinates and creates that business plan on point. Maybe you have a quality assurance AI as well. You do it today with a team of different skills, and the future idea would be to have this all be done by AI.

I think there's a lot of value in what you're calling routing. But isn't it something that we already see today? For example, if you were to use ChatGPT-4 and you wanted to have an image created or there's some input that you give in multimedia or rich media, you'd get a certain format. So isn't it maybe something that we're already seeing that will actually just be much more powerful in the future?

I wonder if this use case we're discussing right now is a little bit beyond the Unify case we're seeing.

Max Tee: Yeah, possibly. What sparked this idea was, you know, when you talk about the iPhone where you have one device that technically pulls all the different messaging apps together. So the iPhone kind of makes it easy to pull everything together, and every single application is so easy to switch between that there's not really a big pain point.

I'm just trying to think if the big pain point to switch between things is that my experience is with big corporations where you have to go around and ask for information to pull things out. Hence why I'm thinking, you know, can the switching between one application to another be the problem? However, if the application itself makes it so easy for you to use a lot of things, which in this case, all the messaging stuff are so easy already, there's really no reason to switch between them or there's really no reason to optimize the routing because it's really so easy to switch to begin with. So that's just, I guess, my thought process.

I don't really have the answer. Like I said, I'm just conjecturing.

Rod Rivera: One idea that came to mind now that we're talking about this is the concept of agents. The idea behind agents is that you have, let's say, simpler AIs that are cheaper and specialized in specific tasks. One does text creation, one does images, another one is a summarizer, and so on. And there is a master AI, a more sophisticated one, that is orchestrating and sending the requests, then gathering them and doing something with it. This is a very active and thriving topic in AI research.

I was thinking, this could also be a way how, going forward, exactly this problem of limitations and then finding the best provider, the cheapest one, and so on could be solved. You are delegating the task to a larger model, like GPT-4, and then this model is orchestrating that for you. It's identifying what would be the best option right now and then doing that.

Bold statement: So here is also, in a sense, one additional threat that can happen to something such as this model hub from Unify, where people decide not to solve this from an API code perspective of using one endpoint, but right from a model perspective. They say, "Hey, I have this master model, and this master model will be responsible for routing my request to the best model that is one of the cheapest and most capable for this task." It can be one direction.

Max Tee: Sorry, I'm just adding to that, right? It's like finding the best possible configuration. All we talk about now is on the model stuff as well. And I believe when we spoke to Daniel, he was also thinking about the different tools, like the physical stuff, right? Like what hardware can you optimize for what kind of models being deployed on them? That could be something interesting. You kind of just tie your online to your offline in a way. Just adding that, yeah.

Final Thoughts and Wrap-up

Rod Rivera: For the commercial side, I keep thinking about these go-to-market strategies. It's also something similar where this starts as a product for a very technical audience, almost those who are at the cutting edge, because these are users who want to use the latest and greatest models, but these models are not available in their favorite frameworks. And now they're offering something where the unique selling proposition, the value, is on the cost savings and efficiency that is rather attractive for more of a business user. So it's a very different positioning, a very different buyer, and a completely different emotion altogether.

So here, what ideas do you see for a company such as Unify to penetrate the market? I mean, hosting hackathons, yes, sure will attract developers who will be happy to develop on their open-source product of this translation from one framework to the other. But that will not necessarily translate into someone who says, "Hey, I want now to use this endpoint that will route to the cheapest provider in the market."

So here, for example, Chris, what would you consider doing if you were in charge of leading the monetization efforts for Unify?

Chris: I think this is a good question. When it comes to monetization of such a product, there are two ways to think about it:

1. Think about other pain points you're also addressing. This goes back to user discovery and pain point discovery, trying to plug more features together to really make a product. That's one way to go, and that's basically the product route.
2. The other way to think about this is perhaps more on the marketing side. Are you an interpreter? Are you a price optimization scheme? Making that clearer to the customer.

There might even be another sub-idea on the marketing front, which is to find the right niche. For example, in industries where the margins are really, really thin, like the restaurant business or the logistics business, any business that really operates on very thin margins, I can see that a product that can help to optimize for price would be very well received.

Now, if we were to plug that into the AI world, the equivalent needs to be: are there any companies that are really operating on thin margins where they need a certain model and they need it to be as cost-efficient as possible? That would be another way to play on the marketing front.

Rod Rivera: And so here as a follow-up, you were active in an industry that has very thin margins, which is the aviation industry. This is also an industry that is looking into digitalization and AI solutions. So for example, if you need to say, "Hey, I want to follow your advice and target these razor-thin margin industries," aviation could be one. From your perspective, how could they penetrate, for example, this industry?

Chris: Actually, the example I had in mind is coming from a very different place. In China, there are these trucks, and normally what they do is they go to a collection place, and then there's a blackboard where they look at, okay, there needs to be a route filled from, let's say, Beijing to Shanghai for this and that price range or for this time. And then there are these truckers that actually bid for that particular route. It's a very manual process, and nowadays, there are these digital platforms that do exactly this.

I believe in situations where you have this auctioning mechanism, and also to my previous image around limit orders, that's the kind of mechanism where I think a business model that Unify is trying to implement could work.

Now, I don't have the answer for what exactly the use case is around that, but I can think of these types of mechanisms or dynamics where you have a lot of players and you're trying to find the lowest-priced player in the market to fulfill the order. That's basically what the logistics example I just gave is, in essence. And I think you see that a lot with these logistics platforms that actually try to find the right spot price between different routes. Obviously, in the trading business, you have similar mechanisms. Maybe even Google AdWords could be another way to think about it. Perhaps that's where the application is best placed, but I don't know, what are your thoughts?

Rod Rivera: Yes, and something else I was thinking about and then I was hearing from the market is that the example you're bringing with the trucks in logistics is a white label, generic business where nobody really cares who is the provider or the company behind, right? They just want to have this job fulfilled at the cheapest and best quality they can get.

However, on this idea of routing for these AI models, what I am hearing is that companies are reluctant to join these efforts because what happens is that they lose the contact, the connection with the potential customers. They lose also their brand, and it becomes just a generic pipe in the end.

Bold statement: And nobody wants that because if you become generic, then you don't have any more premium margins. You just compete on price, and you're in a position where you don't want to be.

The opposite is happening: companies who have the infrastructure are trying to be front and center and as visible as possible for their customers in their branding, trying to see how they can display their logo as much as possible.

And an effort such as Unify goes exactly in the other direction. So one of the risks I see is not only the topic of who are the paying customers for this, but also on the other side of the marketplace, where is the offer coming from, from providers who are interested in joining these efforts?

Max, in the finance industry, there is also a situation of on one side, white labeling, but also on the other side wanting to own the relationship with the customer. How is that balanced?

Max Tee: This is a hard problem in a way. I think a lot of your larger financial institutions already have that brand. Especially, let's talk about the retail market, right? So from a retail perspective, a lot of your larger banks really have that relationship with your end customers, so people already know who they're dealing with. If you're a smaller bank, you normally have some problem, unless you're a Revolut or Monzo, which means people class you differently. In the early days, you also suffered that branding issue.

My take around this is because Unify obviously has their own brand, they really have the front to a lot of the developers. And the question of whether or not someone on the backend wants to then plug in themselves to Unify so that they can help you to optimize those routes, I think that's a good question. I actually don't have the answer for that.

The reason why I think of that is if you look at the mortgage market, I don't know how mortgages work in other parts of the world, but in the UK, you normally go to a mortgage advisor. That mortgage advisor is not 100% affiliated with the banks, right? They just go to the banks and find you the best offer. You basically go to a mortgage advisor, and then the mortgage advisor will go to the first direct bank. Yet mortgage is a big business for the bank. The relationship with the bank will only come after you sign off with your mortgage advisor, and then throughout the other 29 years.

So maybe something like that could be interesting. The models will become like they will see Unify as an acquisition strategy, right? And then because the model is being implemented and embedded in a lot of the solution, the ongoing relationship then will still be with the model itself because you need the model to tweak. Sometimes it's not just the routing, it's just more the model problem.

So I'm just trying to think of different ways to draw analogies to think about this in a clearer manner. In the end, I actually do not know whether or not your providers will want to go on it. I can only go speak to them to figure it out. So far, I haven't been speaking to a lot of model players simply because I just haven't met a lot of them. Should they be all the, I don't know, like OpenAI contenders, then that will be interesting. So at least that's how I'm thinking about it at the moment.

Rod Rivera: If we wrap up all our thoughts, we can say on one side, we're still on the fence about how relevant it is to have a strong brand. On one side, Chris clearly is in the camp of it's not that important; it would be helpful, but not crucial. I, on the other side, think, hey, if someone cannot even find you on Google and is result number six or so, then this might be a problem.

But we also identified this idea that the company should target industries and areas where cost is essential, where they're operating on low margins. As a result, they will be happy to have a provider who enables them to save money.

Then we were also seeing how they can make use of their existing brand and their existing community of developers. Here's again an opportunity in the sense that the team, Daniel Lenton, the founder, has a very strong brand. He's very well known in the community. Plus all the GitHub followers that he has, all the contributors that he has, is a very strong movement. But on the other hand, some of these people may migrate, maybe others not.

We're also thinking of this idea that there are parallels in other industries such as transportation and banking that show that routing and trying to find the cheapest provider is indeed a vital business.

But then again, I would say that from our discussions here, it seems that we are a bit unsure about the perspectives of the Model Hub or Unify going forward. And it seems that we end up with the idea that the most likely outcome could be an acqui-hire, where if they provide a valuable service, it might not be a viable business model or viable company in the end.

Am I missing something as a wrap-up thought, Max, Chris?

Max Tee: I think that's good. I don't have anything to add. We are trying to unify all your messages, right?

Rod Rivera: Fantastic. Well then, that has been all for today. Thank you, Chris and Max, for being here today. Thanks again everyone for listening and joining. Join us next week for another episode where we continue exploring all these models, all these products of the fantastic and still-developing world of AI. Happy to have you here. I'm looking forward to the next time.