On Redis’ Architecture Principles. Q&A with Yossi Gottlieb
“Redis 3.0 with Redis Cluster was released almost eight years ago and these days we’re busy thinking about a significant overhaul, a.k.a Cluster V2. In these past eight years, Redis has become an even more critical infrastructure component and its users keep raising their expectations around ease of use and operational guarantees. The Cluster V2 effort aims to build upon all the lessons learned and deliver to these expectations.”
Q1. What is the architecture of Redis?
The Redis architecture consists of a lightweight, single-threaded server process with asynchronous I/O and a few dedicated auxiliary threads.
When deployed as a cluster, these processes form a shared-nothing architecture that scales uniformly horizontally (more instances) and vertically (more CPU cores).
Q2. What are the main benefits of a primary-replica architecture ?
The main goal of replication is to achieve high availability and data redundancy. Some users also use multiple replicas to scale read performance, although it is rarely necessary because Redis is very fast anyway.
Q3. What is the difference between Redis and Redis cluster?
Redis Cluster is how Redis scales performance and dataset size beyond a single core and instance. For developers, Redis and Redis Cluster appear very similar, with a few exceptions. For example, Redis Cluster supports only a single logical database and does not permit multi-key commands to operate across hash slots. It is also necessary to use a cluster-compatible Redis client, but luckily this is no longer an issue, and most major clients are compatible with Redis Cluster.
Q4. Tell us about Redis cluster data distribution algorithm and the hash slot algorithm
Redis Cluster divides its keyspace into 16K hash slots, which are dynamically assigned to nodes in the cluster. Information about the sharding topology is generally hidden from users and negotiated between the Redis client and server. Users can influence sharding by extending key names with tags that hint about related keys that should map to the same hash slot.
Q5. What is the advantage of using hash slot?
The main advantage of is that they’re very simple. They’re easy to implement by clients, and they simplify operations and troubleshooting. Changing the assignment of hash slots to nodes makes it possible to tune and balance real-world workloads when required. And having 16K slots means scaling is practically unlimited.
Q6. Are the architecture principles the same for Redis Enterprise?
Redis Enterprise clustering was built before Redis 3.0 and the open-source cluster was available, so their internal architecture is slightly different. Redis Enterprise involves a few more moving parts and provides more out-of-the-box automation, a more enterprise-oriented approach to control-plane and management, etc. Open-source Redis Cluster is more bare-bones and requires users to be more familiar with it to deploy and operate it successfully.
These differences are internal and do not affect the application interface, which is mostly the same. Redis Enterprise also supports transparent server-side sharding, so applications and clients that don’t support Redis Cluster can use it and still benefit from clustering. But Redis Cluster compatible applications can seamlessly use both.
Q7. Anything else you wish to add?
Redis 3.0 with Redis Cluster was released almost eight years ago and these days we’re busy thinking about a significant overhaul, a.k.a Cluster V2. In these past eight years, Redis has become an even more critical infrastructure component and its users keep raising their expectations around ease of use and operational guarantees. The Cluster V2 effort aims to build upon all the lessons learned and deliver to these expectations.
Yossi Gottlieb is Chief Architect at Redis, Inc and co-lead of the open source project. Yossi is an experienced tech leader with diverse software engineering experience in various fields. He has served as an advisor and contractor working with top software and cybersecurity companies for over a decade. Before that, Yossi was the CTO of a system integration firm and earlier, a cybersecurity researcher and software developer.
Sponsored by Redis, Inc