ODBMS.org

Operational Database Management Systems

S-Store-a distributed main-memory OLTP system

by · Published · Updated

About

Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. In the past, these two modes of operation were found only in separate, stove-piped systems. However, with the creation of NewSQL OLTP systems, it becomes possible to perform scalable real-time operations without sacrificing transactional support. Enter S-Store, the world’s first transactional streaming database system.

Architecture

 

Architecture

 

 

S-Store is built on H-Store, a distributed main-memory OLTP system, taking advantage of its ability to process transactions with low latency and high throughput. We add support for push-based processing, including two layers of triggering mechanisms. We also introduce native data structures for streams and windows.

 

Transaction Model

 

 

S-Store models its streaming workflow as DAGs of streaming transactions. A streaming transaction is considered to be a stored procedure (SP) operating on an input batch of tuples. Due to data and processing dependencies, transaction executions (TEs) cannot be performed in an arbitrary order, and additional isolation rules are necessary to preserve ACID guarantees.
Transaction Model
 Applications

S-Store is designed to meet the needs of a number of transactional applications driven by high-velocity streaming inputs. As a demonstration, we have implemented a system that models a bicycle share program, known as BikeShare. S-Store is also applicable to systems such as online gaming, real-time marketing, Internet of Things, voting leaderboards, and real-time ETL frontends.

Applications

S-Store is integrated with BigDAWG, a reference implementation of a polystore database. S-Store has been demonstrated live within the context of BigDAWG, running a MIMIC II application for ICU research.
https://youtu.be/iL7DmM5Ikv0
A second demo showing S-Store’s role in BigDAWG, this time in a the context of analyzing ocean metagenomics.
https://youtu.be/M8C4QW-gweQ

Documentation

Full HTML documentation for S-Store can be found at ReadTheDocs: HTML DOCUMENTATION.

Full PDF documentation can be found here: PDF DOCUMENTATION

People

S-Store is a collaboration between Brown University, Intel Labs, MIT, Portland State University, and Carnegie Mellon University, funded by the Intel Science and Technology Center for Big Data.

Download

S-Store is publicly available on Github, released under a GPL License. It can be downloaded here.The S-Store codebase is provided as-is. Please see the license for more information.

Publications

Data Ingestion for the Connected World
John Meehan, Cansu Aslantas, Jiang Du, Nesime Tatbul, Stan Zdonik
CIDR 2017, Jan 2017
Integrating Real-Time and Batch Processing in a Polystore
John Meehan, Stan Zdonik, Shaobo Tian, Yulong Tian, Nesime Tatbul, Adam Dziedzic, Aaron Elmore
IEEE-HPEC 2016, Sept 2016
Rethinking Streaming: Correct State Matters!
Nesime Tatbul, Kristin Tufte, Stan Zdonik
ISTC Blog Post, June 2016
A Demonstration of the BigDAWG Polystore System
A. Elmore, J. Duggan, M. Stonebraker, M. Balazinska, U. Cetintemel, V. Gadepally, J. Heer, B. Howe, J. Kepner, T. Kraska, S. Madden, D. Maier, T. Mattson, S.Papadopoulos, J. Parkhurst, N. Tatbul, M. Vartak, S. Zdonik
PVLDB 8(12), 2015
Handling Shared, Mutable State in Stream Processing with Correctness Guarantees
Nesime Tatbul, Stan Zdonik, John Meehan, Cansu Aslantas, Michael Stonebraker, Kristin Tufte, Chris Giossi, Hong Quach
IEEE Data Engineering Bulletin, Special Issue on Next-Generation Stream Processing, 38(4), Dec 2015
S-Store: Streaming Meets Transaction Processing
John Meehan, Nesime Tatbul, Stan Zdonik, Cansu Aslantas, Ugur Cetintemel, Jiang Du, Tim Kraska, Samuel Madden, David Maier, Andrew Pavlo, Michael Stonebraker, Kristin Tufte, Hao Wang
PVLDB 8(13), Sept 2015
S-Store: A Streaming NewSQL System for Big Velocity Applications (Demo)
Ugur Cetintemel, Jiang Du, Tim Kraska, Samuel Madden, David Maier, John Meehan, Andrew Pavlo, Michael Stonebraker, Erik Sutherland, Nesime Tatbul, Kristin Tufte, Hao Wang, Stanley Zdonik
VLDB 2014 (Poster)
S-Store: A Big-Velocity Database System
John Meehan, ISTC Blog Post, Dec 2014
S-Store: Real-Time Analytics Meets Transaction Processing
Nesime Tatbul, ISTC Blog Post, Feb 2014

Contact Us

For questions or comments about the S-Store research project, please e-mail:
sstore-feedback@lists.cs.brown.edu.

Check back for more information, including new publications and software releases!

LINK
© 2017 S-Store

 

You may also like...