Hamza Tahir from ZenML

Hamza and Rod discuss the journey of ZenML, an open-source tool for creating machine learning pipelines. They explore the challenges of transitioning from data science to engineering, the decision to open source ZenML, and the struggles of raising funds. They also discuss the difficulty of monetizing open source products and the importance of finding the right customers for the cloud solution. The conversation touches on marketing strategies and channels, including LinkedIn and YouTube, and the potential of platforms like TikTok and Instagram for professional topics. The conversation explores the adoption of AI in mid-market companies and the impact of generative AI on traditional ML pipelines. It delves into the process of training and deploying generative AI models and discusses the role of ZenML in the generative AI landscape. The debate on fine-tuning models vs. using pre-trained models is examined, along with the challenges in productionizing LLMs. The future of LLMs and the role of ZenML are explored, as well as the potential for JavaScript libraries in LLM development. Advice for students and early-career professionals is shared, and the announcement of a ZenML Cloud free trial is made.

Takeaways

• ZenML focuses on enabling ML engineers to create pipelines and put machine learning processes into production.
• Transitioning from data science to engineering in a small company is a common pattern for those looking to productionize machine learning.
• Open sourcing ZenML and adopting an open source developer-first approach has been more successful for the company.
• Finding the right investors and champions within enterprise companies is crucial for the success of the cloud solution.
• Marketing strategies should be tailored to the ideal customer profile, and platforms like YouTube and LinkedIn offer potential for professional video content. Mid-market companies are adopting AI as part of their strategies, with AI-first or AI-focused companies leading the way.
• Generative AI has disrupted traditional ML pipelines, with a shift towards using pre-trained models rather than training from scratch.
• ZenML plays a crucial role in the generative AI landscape, providing a process-oriented approach to model training and deployment.
• The debate between fine-tuning models and using pre-trained models continues, with both approaches having their advantages and challenges.
• Productionizing LLMs poses challenges related to cost control, engineering robustness, and ensuring stability and security.
• JavaScript libraries have the potential to drive AI development, as web developers increasingly work with AI models.
• Students and early-career professionals should focus on learning how to code and stay updated on emerging technologies like Rust.
• ZenML Cloud offers a free trial, providing a managed ZenML server and resources for learning MLOps and productionizing LLMs.

Episode Transcript

Introduction and Overview of ZenML

Rod Rivera: Thank you, Hamza, for joining us today. For those who don't know, Hamza is the founder and CTO of ZenML. Could you give us a brief overview of what ZenML is?

Hamza Tahir: Certainly. ZenML is an open-source tool that enables ML engineers to create pipelines, facilitating the transition from proof-of-concept notebooks to production-ready systems. What sets ZenML apart from other pipelining tools like Airflow or Kubeflow is its focus on machine learning patterns that emerge when putting things into pipelines.

Rod Rivera: So, it's about repeatability and tracking the entire process?

Hamza Tahir: Exactly. The goal is to repeatably execute the same process with parameter changes while tracking everything. We recommend tidying up and organizing steps into functions early in the development process. This pattern has proven effective for productionizing machine learning, both for training and deployment.

The Evolution of ZenML

Rod Rivera: Your journey with ZenML has been quite long, starting around 2017. How did it all begin?

Hamza Tahir: It started when I met my co-founder, and we decided to focus on applying machine learning in the industry, particularly in Germany. We began with predictive maintenance for trucks and buses, handling large pipeline runs with terabytes of data. As we grew, I transitioned from data science to engineering, which is common in small companies aiming to productionize machine learning.

During this time, I developed relationships in the tooling space and became fascinated with the patterns emerging in ML workflows. We realized the power of our internal process as clients wanted to deploy machine learning on different infrastructure targets.

Pivoting During the Pandemic

Rod Rivera: How did the pandemic affect your journey?

Hamza Tahir: The pandemic hit us hard. Our company wasn't VC-backed but relied on government grants for research. When the funds dried up, we were nearly bankrupt. We decided to shut down the company, but we weren't ready to give up. After some brainstorming, we chose to open-source some of our ideas. We created a new codebase, wound down the old company, and started fresh with an open-source, developer-first approach.

Rod Rivera: Why did you decide to try again instead of getting a job at a big tech company?

Hamza Tahir: It was a mix of naivety and privilege. I felt I had more to give as an entrepreneur and wanted to do engineering work at scale. I took a loan, used some savings, and knew I could always fall back on my family if needed. We set a deadline based on our bank account and kept one employee on board, paying him from our personal funds.

Raising Funds for ZenML

Rod Rivera: How difficult was it to raise money for ZenML?

Hamza Tahir: It was challenging. We had to find the right people who understood that open-source is a viable go-to-market strategy. We were fortunate to connect with Crane Venture Partners, who believed in us. Both our funding rounds were difficult, but I think that's normal for most founders.

Monetization Strategy

Rod Rivera: How do you monetize ZenML, given that it's open-source?

Hamza Tahir: We're still figuring it out. It's challenging to convince people to pay for something they've been using for free. Our approach is to build a managed version of the open-source framework with additional features that are valuable enough for people to pay for at scale. These include things like triggers, integrations, role-based access control, and visualization tools.

The Impact of Generative AI on ZenML

Rod Rivera: How has the rise of generative AI and large language models (LLMs) impacted ZenML?

Hamza Tahir: Contrary to what some might think, I don't see a significant difference in the underlying process. While there's a shift towards using pre-trained models from platforms like Hugging Face or querying OpenAI's GPT, the fundamental pipeline remains similar. You still need to load data, load a model, embed tokens, train or fine-tune, deploy, and consume the model.

Rod Rivera: But isn't the process much shorter now, with less experimentation?

Hamza Tahir: I disagree. What we're seeing now is mostly experimentation, not real production. The process remains the same, especially if you're fine-tuning models. The main difference I see is in the evaluation stage, where benchmarks and loss functions are less clear for LLMs.

ZenML's Role in the LLM Era

Rod Rivera: In a world where people are primarily consuming OpenAI models, how does ZenML help?

Hamza Tahir: ZenML can play a crucial role in both RAG (Retrieval-Augmented Generation) applications and fine-tuning scenarios. While we might add less value in simple RAG setups, our strength lies in providing a repeatable process for training, testing, and scaling models. We're working on features to simplify the fine-tuning process and help users navigate the complexities of hardware selection and cost estimation.

Challenges in Productionizing LLMs

Rod Rivera: What are the top challenges practitioners face when productionizing LLMs?

Hamza Tahir: There are several key challenges:

1. Cost control: Many are surprised by the scaling costs, especially with cloud solutions.
2. Engineering robustness: Implementing best practices like caching and pre-filtering to optimize performance and handle potential outages.
3. Security and input constraints: Preventing prompt injection and jailbreaking attempts.
4. User interface design: Moving beyond simple chat interfaces to more constrained, user-friendly designs.
5. Guardrails and content control: Ensuring the model doesn't produce inappropriate or unintended responses.

The Future of AI Development

Rod Rivera: What trends do you see in AI development tools?

Hamza Tahir: One significant opportunity is in offering JavaScript libraries and SDKs. With AI becoming more front-end focused, there's a growing need for tools that cater to web developers. The web runs on JavaScript, and there are about 40% more JavaScript developers than Python developers. Expanding our toolset to include JavaScript could be a game-changer.

Advice for Aspiring AI Professionals

Rod Rivera: Any advice for students or early career professionals in the field?

Hamza Tahir: It's hard to give general advice as everyone's situation is unique. However, I strongly believe that people should learn how to code, now more than ever. As for programming languages, I'm excited about the potential of Rust, especially seeing the incredible performance gains in tools like Ruff and UV for Python development.

Closing Thoughts

Rod Rivera: Any final words or announcements?

Hamza Tahir: I'd like to invite everyone to try ZenML Cloud. You can get a free trial at ZenML.io/cloud, which includes a managed ZenML server. Check out our documentation, especially the starter and production guide. We're also working on an LLMOps guide to help users productionize generative AI models. If you're interested in these topics, try our quick start and let us know your thoughts on our Slack channel.

Rod Rivera: Thank you so much, Hamza, for this insightful conversation.

Hamza Tahir: Thank you for having me. It's been a pleasure sharing our journey and insights into the evolving world of MLOps and AI.