On Real Time NoSQL. Interview with Brian Bulkowski.
“The benefits of hybrid storage are lower cost, and more varied price/performance options. We are seeing SSD pricing range down to $3/GB for enterprise drives, and less than $1/GB for consumer drives, and we also support using standard rotational drives for persistence. Our goal is in-memory performance with the price of ‘storage’ – which makes multi-terabyte analytics caches possible” –Brian Bulkowski.
I have interviewed Brian Bulkowski, Founder & CTO of Aerospike.
Q1. What is Aerospike?
Brian Bulkowski: Aerospike is a high speed and highly available NoSQL database. We have proven levels of performance and reliability on commodity hardware that go far beyond any other database – and have the years of production experience to back up our claims.
Q2. How does it compare with other NoSQL or NewSQL databases?
Brian Bulkowski: Aerospike has specialized in areas where downtime is not an option – and at the highest levels of performance. Another area of specialization is applications where low latencies dramatically simplify application development.
As a row-based store with very high levels of read and write concurrency, we perform well where behavior and interactions are concerned – whether those interactions are user-oriented, such as advertising and gaming; or machine to machine, such as capital markets or the emerging Internet of Things.
Q3. What are the typical applications for which Aerospike is currently in use?
Brian Bulkowski: A number of our customers use Aerospike for real-time audience and behavior analysis. A common requirement is to hold profile information for several billion cookies, with a rich schema-free ability to add new data and types. In this use case, a cost-effective multi-terabyte store changes what a business can achieve.
Other use cases include massive multi-player gaming, where a shared game state (at scale) is key to a great user experience, and retaining peak capacity is critical.
Q4. Could you give us some detail on Aerospike`s hybrid memory architecture?
Brian Bulkowski: One of the greatest challenges with databases is keeping indexes in sync with data, and inserting and deleting index entries while under read load. In this area, in-memory techniques shine. An in-memory index can handle a high level of parallelism, and support simultaneous read and write loads. Indexes can be made very small – Aerospike has optimized index entries to one CPU cache line.
Aerospike also uses the technique of allowing a process to restart and retain its index, by using the shared memory facility of Linux. An index needs to be recalculated only when a machine is restarting, and in those cases it can be best to simply use the promoted replica during the short time the index is being re-created.
Q5. What is the main advantage of using a hybrid memory architecture?
Brian Bulkowski: The benefits of hybrid storage are lower cost, and more varied price/performance options. We are seeing SSD pricing range down to $3/GB for enterprise drives, and less than $1/GB for consumer drives, and we also support using standard rotational drives for persistence. Our goal is in-memory performance with the price of ‘storage’ – which makes multi-terabyte analytics caches possible.
Q6. How do you ensure scale up and scale out?
Brian Bulkowski: Scale up is maintained through short individual code paths, and single-server performance benchmarks. Our architecture focuses on enabling multiple cores, deployment configurations with appropriate interrupt routing rules, and classic database techniques like asynchronous IO.
Scale out is ensured through a number of techniques, such as enforced randomness through provably random storage distribution, a minimal and deterministic number of nodes participating in a transaction, and client techniques which limit the effect of a single slow node on other nodes.
Q7. Do you have any performance measures to share with us?
Brian Bulkowski: Please see this performance benchmark report from Thumbtack Technology, demonstrating how Aerospike is 10 times faster than MongoDB or Cassandra.
Please also see Aerospike Principal Architect Russ Sullivan’s High Scalability blog post on achieving 1M TPS on commodity hardware using Aerospike. This use case involves more system tuning than the Thumbtack tests.
Q8. How do you handle real-time big data analytics?
Brian Bulkowski: Many customers use Aerospike as a persistent part of their real-data analytics processing chain. Their application logic needs to know counts of recent visits, recent searches, preferences, preferences of friends, and also store batch analytics results. For example, you might calculate that a certain user is part of an audience, then keep up-to-date per audience counters.
By placing the calculations in the hands of the developer – with extreme low latency – we find developers and architects are able to scale increasingly powerful real-time computations.
Q9. You claim to provide 17x better TCO than other NoSQL databases. Could you please provide some evidence for this claim?
Brian Bulkowski: A great benefit of Aerospike is the ability to store data in Flash, and indexes in DRAM. This plays to the strengths of both SSD and DRAM, as memory accelerates the indexes and finding of the data, and then only one device data read is required to fetch (or write) the data.
Most NoSQL databases require all the data in memory in order to achieve high performance. A more detailed implementation is outlined in David Floyer’s Dec 12, 2012 Wikibon article.
Q10. How do you manage replication in Aerospike?
Brian Bulkowski: We commit each write to memory, and reserve space for the data to be written to disk. The writes are aggregated into large blocks to allow the streaming writes that are optimal for both SSDs and standard rotational disks, which passes through a short disk queue (often committing in less than a few milliseconds).
Each row has a current master server within the cluster. This mapping changes as servers are added and removed, and one or more replicas are chosen randomly from the remaining nodes.
Q11. What about data consistency when updating the data?
Brian Bulkowski: By applying the change to all relevant machines in the cluster, we achieve the important application requirement of a write taking effect immediately. By applying the write to multiple machines in-memory, we achieve very high reliability, because multiple machines must fail in the same millisecond to have uncommitted transactions. When the application code reads, they will always see the updated data by default – with no performance impact.
Q12. Could you please explain how cross data center synchronization works for Aerospike?
Brian Bulkowski: Our cross data center synchronization is a no-single-point of failure asynchronous replication mechanism. Asynchronous replication is important because a network outage or bandwidth issues impedes long haul links, but a data center can still serve requests based on its local cluster. The system can be operated both with a ‘star topology’, where a single master cluster is replicated to other more local data centers, or in a ‘ring topology’, where every cluster takes writes and those writes are forwarded around the ring. Conflicts can happen in a ring deployment if a row is modified in different locations at the same time.
This requires a conflict resolution system – like Aerospike’s versions – that retains conflicts for later resolution, or a heuristic – retaining a row with the most writes and a later timestamp.
Q13. What is the contribution of Don Haderle to Aerospike?
Brian Bulkowski: Don first exposed us to DB2‘s historical strategy of building very fast and reliable single-row operations, then adding higher level primitives. Primary key database operations are similar to storage operations in a file store, and have extraordinary business value. By deploying with many paying customers, we’ve proven our reliable and fast primary key layer – unlike other new, clustered databases which are still seeking large production proof.
Brian Bulkowski, Co-Founder & CTO Aerospike
Brian has 20+ years experience designing, developing and tuning networking systems and high performance high scale infrastructures. He founded Aerospike after learning first hand, the scaling limitations of sharded MySQL systems at Aggregated Knowledge. As Director of Performance at this Media Intelligence SaaS company, Brian’s team built and operated a clustered recommendation engine. Prior to Aggregate Knowledge, Brian was a founding member of the digital TV team at Navio Communications and Chief Architect of Cable Solutions at Liberate Technologies where he built the high performance embedded networking stack and the internet scale broadcast server infrastructure. Before Liberate, Brian was a Lead Engineer at Novell, where he was responsible for the AppleTalk stack for Netware 3 and 4. Brian holds a B.S. from Brown University in Mathematics/Computer Science.
NoSQL Failover Characteristics: Aerospike, Cassandra, Couchbase, and MongoDB.
Ben Engber, CEO, Thumbtack Technology
Thumbtack’s latest whitepaper focuses on how NoSQL databases perform during hardware failures. This paper follows our previous study of NoSQL performance and explores how different consistency and durability models affect hardware planning and application design.
Paper | Advanced| English |DOWNLOAD (PDF)| MARCH 2013
Ultra-High Performance NoSQL Benchmarking: Analyzing Durability and Performance Tradeoffs.
Ben Engber, CEO, Thumbtack Technology, JANUARY 2013
One question that comes up a lot in our discussions with customers is “Tell me which NoSQL database is best.” Of course, the answer to this depends on the use cases and the transactional needs of the specific customer. We regularly go through this analysis with customers, but one thing we find as we start is often the basic questions of how much consistency is needed and what are the tradeoffs involved that have not been answered. Ultra-High Performance NoSQL Benchmarking: Analyzing Durability and Performance Tradeoffs is a benchmark report created to help organizations answer exactly those questions pertaining to NoSQL databases. Our goal was to create a benchmark that addressed a specific class of real-world problems with big data, unlike other published reports that are out there. We focused on how the durability and consistency needs affected raw performance.
Paper | Advanced| English | DOWNLOAD (PDF)| 2013|
– Don Haderle, Father of IBM DB2, Talks about the Information Explosion:
** Follow ODBMS.org on Twitter:@odbmsorg