MarketStore: Database server optimized for financial timeseries data
MarketStore is a database server optimized for financial timeseries data.
You can think of it as an extensible DataFrame service that is accessible from anywhere in your system, at higher scalability.
It is designed from the ground up to address scalability issues around handling large amounts of financial market data used in algorithmic trading backtesting, charting, and analyzing price history with data spanning many years, including tick-level for the all US equities or the exploding crypto currencies space. If you are struggling with managing lots of HDF5 files, this is perfect solution to your problem.
The batteries are included so you can start pulling crypto price data from GDAX right after you install MarketStore. Then you can query DataFrame content over the network at as low latency as your local HDF5 files from disk, and appending new data to the end is two orders of magnitude faster than DataFrame would be. This is because the storage format is optimized for the type of data and use cases as well as for modern filesystem/hardware characteristics.