Comments on: Objects in Space vs. Friends in Facebook. Trends and Information on Big Data, New Data Management Technologies, Data Science and Innovation. Tue, 20 Dec 2016 11:47:00 +0000 hourly 1 By: Reinhold Thurner Sat, 16 Apr 2011 06:25:14 +0000 “in the middle there is always SQL” – SQL is used to manage and organize data about data stored in whatever we have to store this amount of data.
In the middle of the middle there is a model – (ER I think) to describe the meaning of the tables and their relationships. With thousands of databases in use (which is quite common in a large organization) also this “model” represents quite a large complex volume of data – and it is dynamic, multiuser and hopefully ACID compliant. SAP reportedly consists of a collection of 90000 tables (guess the number of rows and relationships!). The obvious conclusion is that there must be another layer to control this structure and to reduce the span of control.
I think the real challenge is to build a hiearchical layer of data structures which reduce the amount of data to manageable sets of information, which provide an insight into the relevant meaning of the PBs of raw data.