Nambiar’s Law of IO
Nambiar’s Law of IO
Two of of the most popular questions asked in the storage performance world are:
– How do you estimate the IO bandwidth requirements?
– How do you estimate the IOPS requirements?
In other words how much IO can be processed by today’s data management and datastore platforms?
Here is my simple rule of thumb, let me call it the “Nambiar’s Law of IO”, #NambiarsLawofIO:
- 100 Mbytes/sec per processor core for every 7 years of software maturity for IO bandwidth intensive workloads. Examples: enterprise data warehouse systems, decision support systems, distributed file system based data processing systems etc.
- 10,000 IOPS per core for every 7 years of software maturity for IOPS intensive workloads. Examples: transaction processing, key-value store, transactional NoSQL, etc.
for up to 21 years, beyond which there is no significant improvement, that’s what we have seen historically.
Examples
Platforms in the >21 years category:
Traditional relational database management systems (Oracle database, Microsoft SQLServer, etc):
– Data warehouse: 24 (number of cores in today’s popular 2 socket x86 servers) x 100 (IO bandwidth that can be processed the by software) *3 (>21 years) = 7.2 GBytes/sec
– Transactional: 24 (number of cores in today’s popular 2 socket x86 servers) x 10,000 (IOPS that can be processed by the software) *3 (>21 years) = 720K IOPS (small block random r/w)
Platforms in the < 7 years category:
New generation distributed datastore systems:
– Hadoop: 24 (number of cores in today’s popular 2 socket x86 servers) x 100 (IO bandwidth that can be processed by the the software) *1 (<7years) = 2.4 GBytes/sec.
– Cassandra: 24 (number of cores in today’s popular 2 socket x86 servers) x 10,000 (IOPS that can be processed by the software) *1 (<7years) = 240K IOPS