Engineering Serverless Data Platforms: Solving Cold Starts, Multi-Tenancy, and Storage Virtualization at Scale.
Q1. Scaling compute to zero while maintaining persistent database connections is a classic distributed systems challenge. When building serverless database infrastructure, how do you handle the mismatch between ephemeral, high-concurrency client runtimes (like Lambdas or Edge workers) and the stateful, connection-heavy nature of traditional database wire protocols? How does a modern database proxy effectively multiplex these connections at the protocol level without bottlenecking query throughput?
Prisma Postgres runs a full and unmodified Postgres process for each customer. Each database runs in a fully isolated Firecracker MicroVM environment, and we make use of Firecrackers memory snapshotting functionality to suspend the VM to disk when there is no activity. We suspend the database 5 seconds after the last query is finished processing, and start it back up again in 10ms on the next request. Databases with sustained load are never suspended. When the database vm is suspended, we keep the tcp connection alive in a networking layer outside the database vm, enabling applications to reuse connections in a pool, even when the Db is suspended.
Q2. Enforcing strict resource isolation for thousands of micro-databases on shared bare metal is notoriously difficult. What are the major engineering trade-offs when choosing an isolation strategy for serverless database multi-tenancy? Specifically, how do you weigh the performance overhead and cold-start penalties of lightweight virtualization (like microVMs) against the security and “noisy neighbor” risks of OS-level process isolation and cgroups on shared storage drives?
For us, strict isolation is a non-negotiable foundation. So we started with a hardened MicroVM architecture, and worked from there to find a setup enabling high hardware utilisation and low performance overhead. Databases on our platform fall on a long tail distribution with a small number of highly active databases, and a large number of databases with much more sporadic activity.
By suspending inactive databases, we are able to operate upwards of 100k databases on a single bare metal server. It is our goal that customers shouldn’t be able to tell the difference between a running database, and one that had to be resumed from disk. Through a series of optimisations, we are able to resume a database vm in around 10 ms. The most important optimisation is to write a snapshot of the vm memory to disk, which can be loaded directly to ram and resume the VM without an expensive boot process. The speed of modern NVME disks mean that this step alone is enough to be able to resume a VM in less than 100ms. Additional optimisations, including shrinking the kernel, sharing static parts of the VM image between VMs, and utilising incremental snapshot restore enabled us to reach 10ms.
Q3. The industry is moving toward instant database branching where developers get isolated data environments without full data duplication. From a storage engineering perspective, how do you implement zero-copy database branching at scale? How do modern cloud-native filesystems or custom block-stores handle Page-level Copy-on-Write (CoW) to ensure that diverging branches remain highly performant without corrupting or degrading the performance of the parent database?
We have found that customers ask for database branching in one particular scenario: To generate consistent databases for development, testing and staging. These databases are really small, and do not require Copy-on-Write. We implement branching by simply copying the VM. Copy-on-Write is a really cool technology that enable you to instantly branch a very large database. But it’s also very complex, adds overhead on the read path, and introduces bloat and risks into the system. Copy-on-Write would be relevant if customers wanted to branch their large production database, but that’s simply not something our customers are asking for, so we have decided to not take on the additional complexity.
Q4. Waking up an idle database on-demand often introduces a noticeable latency penalty (cold start). What are the most effective architectural patterns for minimizing the P95 latency of a database cold start? How can upstream network proxies intelligently intercept raw database driver handshakes, keep the client-side TCP connection alive, and spin up the underlying database engine fast enough to avoid query timeouts?
We touched on this already. The most naive way to cold-start a database is to boot a fresh vm, then mount the database data files, and start the database engine on it. This is roughly what happens when you restart an AWS RDS database, and the process can take several minutes. It’ simply not something you can do while a customer application is waiting for a response to a query. A step up from there is to have a warm pool of VMs ready to go. But even then, you need to mount the customer data, start the database engine and accept the incoming connection – we are talking tens of seconds before the first query receives an answer. The only realistic option is to keep a running database engine for every customer workload. Now, there are a number of ways you can do that.
The traditional approach is to literally keep a fully spun up database server for each customer workload. This is what AWS RDS does. This is what Supabase does. And this is what Planetscale does. It works, is reliable and well understood, but it comes with a minimum monthly fee per database that makes this inappropriate for workloads that require a very large number of databases with intermittent load. A more ambitious approach is to keep a warm pool of not only the VM, but also the database engine. Warm pools are effective because you can share a small number of instances in a pool of ready-to-go servers between a large number of intermittent customer workloads.
The challenge is that traditional RDBMS servers like Postgres aren’t designed to support loading customer data after they are started. So to make this approach work, you need to operate a forked version of the database engine, which adds complexity and risk. And finally, there is the approach we are using in Prisma Postgres. We operate a fully booted database engine per customer workload – just like AWS RDS – but we have made it so cheap that the platform can absorb the cost of idle databases, and only charge for active requests. As discussed above, we achieve this by taking an incremental memory snapshot of the complete VM, which costs us only around 100MB of NVME space. We do not pay for memory, provisioned CPU or any other shared resources an actively running database engine would consume from the host. And we bring that memory snapshot up and serve the first request in around 10ms. To further optimise cost, we progressively age out the memory snapshot to cheaper media, culminating with a compressed image on object storage, which increases our restore time to a couple hundred ms for databases that haven’t been used in weeks.
Q5. Extensions like pgvector introduce heavy CPU and memory spikes during HNSW index builds, which can easily starve traditional relational queries. How do you design a database control plane to prevent analytical or machine-learning workloads (like vector similarity searches and heavy index compilation) from exhausting the system resources needed by standard, low-latency transactional (OLTP) queries when they share the same underlying engine?
This is a complex question that we mostly sidestep entirely. In traditional hosted database offerings, you provision and pay for a dedicated amount of compute resources, and you need to make sure these resources are sufficient for the workload you are running. If you have spikes, you need to provision capacity to handle those. And if you don’t want to over provision by that much, you might try to throttle certain workloads that don’t have direct impact on customer requests.
On Prisma Postgres, we scale resources based on your needs. If you need to rebuild an HNSW index, we let you dramatically increase your resource consumption during this time, such that you can finish this operation as fast as possible, while continuing to serve customer requests. We manage this on the backend by having over provisioned bare metal servers, and rebalance them to ensure there is always enough headroom for individual workloads.
Q6. Modern cloud databases are increasingly decoupling the execution layer from the storage layer. When architecting a next-generation serverless database platform, what are the primary engineering hurdles of completely separating compute from storage? Specifically, how do you handle continuous Write-Ahead Log (WAL) streaming and cache-invalidation across ephemeral compute nodes without introducing unacceptable latency overhead?
Interestingly, Prisma Postgres is a modern cloud database, but we went the exact opposite direction. Instead of operating the database engine on ephemeral compute nodes, we place customer workloads on fast bare metal servers with a large amount of directly attached NVME storage. We gain maximum density by taking memory snapshots of the database engine, which we can resume in around 10ms. By keeping storage and compute close together, we eliminate the performance overhead and complexity of networked storage. And our internal firecracker setup enables us to utilise Compute much more effectively than an EC2 or Kubernetes based architecture with warm pools. We operate Firecracker in combination with traditional wal based replication for backup and redundancy, while taking full advantage of the performance of modern hardware during normal operation.
………………………………………………..

Søren Bramer Schmidt
Søren is a co-founder of Prisma and previously co-founded Graphcool, the GraphQL backend platform that grew into Prisma. Before founding companies he worked as a software architect at Trustpilot, and he has spent well over a decade building tools that simplify how developers work with data. He writes about databases, developer tooling, and the direction of modern application infrastructure.