eXtremeDB Industry-Standard, Audited STAC-M3 Benchmark Tests

The Securities Technology Analysis Center (STAC) provides the STAC-M3 benchmarks to evaluate time-series data management (tick database) solutions involving complex queries designed to reflect real-world capital markets computing demands. STAC oversees the tests and audits the results, to enable comparisons between vendors’ technologies.

In multiple audited STAC-M3 benchmark tests, McObject’s eXtremeDB database management system has broken earlier records for the best (lowest) mean response times and for lowest standard deviation (highest predictability/lowest “jitter”) of test results. eXtremeDB set STAC-M3 records using the SQL database programming language. For a discussion of why this matters, see McObject CEO Steve Graves’ TabbFORUM article, “Is SQL Fast Enough for Tick Data?”  

Learn more about eXtremeDB for HPC in financial systems pipelining for blazingly fast analytics of time series data.

McObject & E8 Storage STAC-M3

All the queries used eXtremeDB v.8.0 for HPC and were programmed using Python and SQL, running on a Supermicro SYS-2028TP-HTTR, connected to an E8 Storage E8-X24 high-performance shared NVMe (Intel Optane™) storage appliance.

The results include:

• Faster mean response times than all previously published best results for 5 of the 17 operations.
• More consistent response times (lower standard deviation) than all previous best results for 7 of the 17 operations.
• 17.0x the performance of the previously published best result for the write benchmark (STAC-M3.β1.1T.WRITE.TIME) using a non-McObject database.
• 9.1x the performance of the previously published best result for the NBBO benchmark (STAC-M3.β1.1T.NBBO.TIME) using a non-McObject database.

The full STAC Report™ is available here.

McObject & IBM POWER8 STAC-M3 Kanaga

McObject worked closely with IBM on three STAC-M3 implementations on the POWER8 processor architecture, with the latest utilizing a 2-socket IBM Power System S824L server (12 cores per socket, 24 POWER8 cores total) running Redhat Linux, eXtremeDB version 7.0 and 57TB IBM FlashSystem 900. Results included new speed and jitter records for all 21 of the benchmark queries that were more than:

• 21,139% and 19,875% faster than the best previously published result for the 10 thread Years 3 & 4 Market Snapshots, respectively (10T.YR3.MKTSNAP and 10T.YR4.MKTSNAP)
• 2,742% faster for the 10 thread Year 1 Market Snapshot (10T.YR1.MKTSNAP)
• 887% and 936% faster for the 100 thread Years 1 & 3 Volume Weighted Bid over 12 days, respectively (100T.YR1VWAB-12D-HO and 100T.YR3VWAB-12D-HO)
• Better than 500% faster than the best previously recorded time to complete all 21 benchmark tests

Get the audited benchmark report.

Results of STAC-M3 benchmark using eXtremeDB on IBM POWER8 hardware

The combination of McObject’s SQL-based eXtremeDB database management system (DBMS), running on an IBM POWER8 S824L Linux server, achieved new records in a set of 17 STAC-M3 tests, including the best-ever results for jitter. Adding together the mean response times for these 17 tests,eXtremeDB on POWER8 took 44% less time than the nearest competitive solution while using just one-third the number of cores. This solution provides a significant cost/performance benefit to clients in the financial industry.

These new STAC-M3 results:

• Bested standing performance records in 6 of the 17 tests
• Set records for lowest standard deviation of results (lowest jitter) in 5 of 17 tests
• The results build on previous record-setting STAC-M3 implementations by McObject and IBM that, when combined, hold 16 of the 17 test performance records when compared to all other solutions using non-clustered 2-socket servers. These results demonstrate the value ofeXtremeDB and the IBM POWER8 system for analytics and big data workloads in capital markets, where low-latency and low-jitter are key attributes.

Read McObject’s announcement of the STAC-M3 results. Download the STAC report,  STAC-M3 Kanaga Audited Report eXtremeDB on IBM Power8.

Results of cloud-based STAC-M3 using eXtremeDB on Lucera Compute

McObject worked with Lucera  to deliver a STAC-M3 benchmark using eXtremeDB on the Lucera Compute high-performance, collocated cloud computing infrastructure for financial institutions.

Together, the DBMS and cloud-based infrastructure beat standing records in five of the 17 benchmark tests. Compared to previous STAC-M3 benchmarks using a different DBMS, the McObject-Lucera pairing delivered performance that on average is

• More than three times (3x) as fast as the best previously published result for the Multi-User Intervalized Statistics – 50 Threads benchmark (STAC-M3.β1.50T.STATS-UI.TIME)
• More than twice as fast for the Multi-User Intervalized Statistics – 100 Threads benchmark (STAC-M3.β1.100T.STATS-UI.TIME)
• More than twice as fast for the National Best Bid and Offer benchmark (STAC-M3.β1.1T.NBBO.TIME)

Read the announcement of the STAC-M3 Benchmark test results. Download the report, STAC-M3 eXtremeDB Financial Edition 6.0 on Lucera.

STAC-M3 Results for eXtremeDB on IBM POWER8 Hardware

eXtremeDB paired with IBM POWER8 hardware achieved new speed records for nine of the 17 benchmark tests, including performance that on average is

• More than four times (4x) as fast as the best previously published result for the “Volume Curves” benchmark
• More than four times (4x) as fast for the “Multi-User Intervalized” Statistics algorithm
• More than twice as fast for the “Market Snapshot” algorithm
• More than twice as fast for the “National Best Bid and Offer” (NBBO) benchmark

In these tests, eXtremeDB running on POWER8 hardware achieved a sum of mean benchmark times 1.6x faster than the best previously published STAC-M3 results (66 seconds vs. 106 seconds).

The tests marked the first published STAC-M3 results using industry-standard SQL as the database interface language. They utilized several recently introduced eXtremeDB features, including distributed query processing (the database was partitioned horizontally across 72 shards) and the ability to pipeline vector-based statistical functions from within SQL.

Download the STAC report, STAC-M3 eXtremeDB Financial Edition 6.0 on IBM

STAC-M3 Results for eXtremeDB with Kove, Dell and Mellanox Hardware

Highlights of these STAC-M3 benchmark tests– which examined eXtremeDB with hardware from Dell, Kove and Mellanox – include:

• 9x the previously published best result on tests using the “Theoretical P&L” algorithm
• 7x the best previously published result for the “Year High Bid” benchmark
• Double the previously published best results for both “National Best Bid and Offer” (NBBO) and “Market Snapshot” algorithms
• Lower standard deviations of latency than the previously best results for 13 of the 17 operations

The last result, record-setting low standard deviation of latency, means that financial calculations will be predictably fast. Capital markets developers can expect tighter bounds on application performance (performance will not swing wildly) and have confidence that processing will not bog down at critical times.

Read McObject’s announcement of the STAC M3 Benchmark test results. Download  the STAC report.

McObject’s eXtremeDB DBMS Performance Comparison

In its own testing, McObject benchmarked eXtremeDB against a competing database management system used in real-time financial applications. Results are presented in the table below.

• The tests measured performance on financial calculations executing over tick streams
• Data volume: 20 million ticks for a single symbol
• Operations are expressed in quasi-SQL in the chart above
• The eXtremeDB implementation used its native API to its vector-based statistical function library
• The alternative DBMS used its proprietary development and querying language

Why is eXtremeDB faster?

What explains eXtremeDB’s faster performance? Differentiators include the following:

• eXtremeDB’s efficient row- and column-based data layout ensures the maximum amount of relevant market data is brought into CPU cache and avoids flooding the cache with unneeded data.
• Pipelining vector-based statistical functions keeps needed data near CPU cores, eliminating latency-inducing handoffs of interim results between main memory and CPU cache.  Watch a video about pipelining.
• With distributed query processing, eXtremeDB partitions a database and distributes query processing across multiple servers, CPUs and/or CPU cores. Performance is accelerated via parallel execution of database operations and by harnessing the capabilities of many host computers rather than just one.
• With a code size of as little as 200K, eXtremeDB’s code path is very short. Calculations are performed using fewer CPU instructions, shaving milliseconds from execution time.  Vector-based statistical functions are created with a short code path to keep the code in CPU cache.

Sponsored by McObject