Database Persistence, Without The Performance Penalty

Database Persistence, Without The Performance Penalty
Benchmarking McObject’s In-Memory Database System With AgigA Tech’s Non-Volatile DIMM Technology
McObject
Copyright 2014, McObject LLC

In the past decade, in-memory database systems (IMDSs) have emerged as the database solution for many real-time applications. An IMDS stores data entirely in main memory (DRAM), leveraging memory’s superior read/write speed while eliminating the file I/O, cache management, data transfer and other overhead inherent in traditional database management systems (DBMSs) that are designed to store records on persistent media. As a result, in- memory database systems perform an order of magnitude faster than their disk-based counterparts.
A disadvantage to DRAM-based storage is volatility. What happens to records stored in main memory if power disappears? Without some kind of backup or log, they’re gone – and some applications require a higher level of data durability and recoverability. As a solution, IMDS vendors offer transaction logging, which keeps a record of changes to the database. But this logging requires writing to persistent storage. Will re-introducing these “database writes” eliminate IMDSs’ performance advantage over disk-based DBMSs?
Another solution is to deploy an IMDS using DRAM that is backed up by battery power. However, disadvantages of this “battery-backed RAM” or non-volatile RAM (NVRAM) include batteries’ restrictive temperature requirements and the risk of leaking corrosive and toxic fluids. Battery-backed RAM also suffers from limited length of storage (data typically must be transferred from the chip within 72 hours of failover to battery), long re-charge cycles, finite battery shelf life, and overall high cost-of-ownership.
To get around these limitations, AgigA Tech, a Cypress Semiconductor subsidiary, has introduced its AGIGARAM® Non-Volatile DIMM (NVDIMM) solution, which removes the battery from battery-backed RAM by combining DRAM with NAND flash and an ultracapacitor power source. When a system is operating normally, AGIGARAM NVDIMMs function as normal, high speed DRAM. But in the event of unexpected power loss, the ultracapacitor provides a burst of electricity that is used to write the contents of main memory to the NAND flash chip. This data can be held there virtually indefinitely, with AGIGARAM restoring the FLASH contents to DRAM upon system recovery.
Managing data with the speed of DRAM, but with instant, reliable persistence in the event of system failure, and none of the drawbacks of batteries – it sounds like IMDS nirvana, at least for that sub-set of real-time applications that require data durability. And happily, McObject can confirm that it works exactly as described, having tested its eXtremeDB IMDS with AGIGARAM in a self-induced power-off, power-on scenario. The next question likely to come from real-time system designers: is using AGIGARAM with an IMDS as fast as working with standard DRAM? An IMDS may gain persistence through AGIGARAM, but does it give up performance?
To answer that question, McObject and AgigA Tech collaborated on benchmark tests comparing IMDS performance using AGIGARAM for main memory storage compared to using conventional DRAM. The benchmark tests also addressed the other question raised above, namely, to what extent will an IMDS with transaction logging retain its performance advantage over a disk-based DBMS? For these latter tests involving persistent storage (of the IMDS’s transaction log, and the
entire database in the case of the on-disk DBMS) the storage “device” consisted of a RAM-disk configured using the AGIGARAM NVDIMM. The reasons for using a RAM-disk instead of a conventional hard disk drive or solid state drive are described below.

DOWNLOAD PAPER (.PDF): IMDS-NVDIMM-paper

Sponsored by McObject