On Ray and Anyscale. Q&A with Lance Walter
The lack of scalable AI platforms has resulted in 85% of AI projects failing in production, and incurring exorbitant cloud costs.
Q1. Why do you believe companies are currently struggling to get value out of AI?
There are multiple ingredients required for successful AI, and many companies struggle to bring them all together. You need an AI use case that delivers value, a team that can develop it, and the computing power to effectively run the application ongoing.
Beyond talent shortages and prioritizing the right use cases, we see companies struggle to efficiently scale compute-intensive AI workloads without huge cloud bills. We also see challenges in moving from development to production as developers are often forced to change their code and applications to get them to behave and perform properly in production. Finally, we see organizations struggling to “keep up” with incredibly fast-moving AI technology. They try to build their own ML platforms and frameworks and often find them quickly obsolete as AI use cases and available tooling evolves so quickly.
Q2. Anyscale originated from Ray, a free, open source project developed at UC Berkeley. Can you please tell us a bit about Ray and how did it help you create Anyscale?
Our team developed Ray at UC Berkeley while they were developing machine learning models in Python and struggling to scale them. They wanted something that was Python-native, extremely flexible, and scalable to run ML workloads, and that’s how the Ray project was born – to build a unified open source framework for productionizing and scaling ML workloads.
Anyscale was founded to commercialize the Ray project, offering Ray as a managed service with additional features for developer collaboration, performance, and scale.
Q3. So, Ray is an open source compute framework for scaling ML and Python workloads. Why choose Python?
Python is the most popular programming language for machine learning applications. It’s open source with a huge and vibrant community and a big ecosystem of extensions, accelerators, and more. Python gives our users an enormous amount of flexibility without learning a new language.
Q4. What does it mean in practice creating the illusion of what Stoica calls (*) an “infinite laptop” ?
It means making distributed computing effectively “invisible” to the application developer. All s/he should need to worry about is her model. She shouldn’t have to worry about high availability, cluster management, autoscaling, maximizing GPU utilization, etc. The “infinite laptop” is a vision of a developer using the cloud at scale to develop and deploy AI applications with the simplicity and speed of working on a laptop.
Q5. What is the difference between the Anyscale Platform and the Ray Open Source?
Ray is a unified scalable compute framework that eases building and scaling AI apps and Python workloads. Anyscale is a fully-managed Ray platform that also offers additional functionality for developer collaboration, observability, performance, and production.
Q6. You listed a number of Success Stories on your web site. How many of these do use the Anyscale Platform?
We’re very proud of the many projects that depend on Ray open source. For example, OpenAI, which is the fastest consumer application to ever reach 100M users, is powered by Ray. Our user conference tends to feature cutting-edge companies from our community like Uber, Cohere, Instacart, Doordash, and more. Most of our current usage is in open source, fairly typical for a commercial open source company at our stage.
Q7 What role did Ray play in enabling OpenAI to train ChatGPT and models like it?
Ray played a fundamental role in training ChatGPT. You can hear it first-hand from their co-founder from Ray Summit 2022. Training an ML model at the scale of OpenAI is incredibly compute-intensive. Given the scarcity of GPUs and high cloud platform costs, it made sense for OpenAI to “do their homework” to find the right framework to meet the demanding functional needs of their application while providing extreme scale at manageable cost. We’re thrilled that they and so many other large-scale innovators have chosen Ray.
Q8. What is the mission of your company?
Anyscale is enabling companies to fuel breakthroughs in AI by solving one of the toughest challenges in AI today: building and scaling from projects to production. The lack of scalable AI platforms has resulted in 85% of AI projects failing in production, and incurring exorbitant cloud costs. Anyscale is changing the way that developers and organizations build, train, serve and scale AI workloads and entire end-to-end applications. With a singular application built on the most common ML coding language, Python, Ray grants developers the ability to leverage Ray’s scalable libraries and applications without worrying about infrastructure.
Q9. Lee Fixel’s Addition and Andreessen Horowitz led the Series C round, and injected $100 million into your company. NEA, Intel Capital and Foundation Capital also participated in the round (*). How do you plan to monetize the open source project Ray?
Anyscale uses an “open core” model that has been popular in open source for more than a decade. We provide frictionless access to highly-functional software to drive adoption and user value, and then monetize that with add-on features (both back-end and front-end) and services, including fully managing Ray as a service.
Q10. How does the open source project create a market for the new product?
In general, open source projects create a market through adoption. Open source is a wonderful meritocracy where the projects that deliver value and get developer adoption succeed, and the ones that don’t end up failing. Ray adoption creates a market for the Anyscale Platform because organizations deploying business-critical AI applications that need additional performance and scale as well as a managed environment will move to Anyscale.
Q11. What is your vision ahead for Ray?
We see Ray becoming the “de facto compute substrate” for AI and machine learning. For any ML workload, on any cloud, we see Ray powering a generation of AI applications. Some of the brightest machine learning minds in the world, from companies including Spotify, DoorDash, Uber, Instacart, Netflix, are using Ray and helping us make it even better every day, so this vision feels achievable.
Q12. Who qualifies to use your beta version product?
We don’t do a lot of traditional “beta” programs, but we offer multiple libraries and documentation in open source to any curious user. We always want developers trying our stuff and helping us make it better. That’s how it got to where it is. Yes, we’re proud of our team but it’s organizations like OpenAI, Pinterest, IBM and others that have helped us make Ray what it is today.
…………………………………………..
Lance Walter leads Marketing for Anyscale. Lance is an Engineer-turned-Marketer with more than 20 years’ experience in Product Marketing, Product Management, and CMO roles at companies including Oracle, Hyperion Solutions, Business Objects, Pentaho, and Neo4j.