{"id":4172,"date":"2016-07-06T09:00:29","date_gmt":"2016-07-06T09:00:29","guid":{"rendered":"http:\/\/www.odbms.org\/blog\/?p=4172"},"modified":"2016-07-06T09:09:14","modified_gmt":"2016-07-06T09:09:14","slug":"on-the-challenges-and-opportunities-of-iot-interview-with-steve-graves","status":"publish","type":"post","link":"https:\/\/www.odbms.org\/blog\/2016\/07\/on-the-challenges-and-opportunities-of-iot-interview-with-steve-graves\/","title":{"rendered":"On the Challenges and Opportunities of IoT. Interview with Steve Graves"},"content":{"rendered":"

“Assembling a team with the wide range of skills needed for a successful IoT project presents an entirely different set of challenges. The skills needed to build a \u2018thing\u2019 are markedly different than the skills needed to implement the data analytics in the cloud.”–Steve Graves.<\/strong><\/p><\/blockquote>\n

I have interviewed, Steve Graves<\/a><\/strong>, co-founder and CEO of McObject<\/a>.\u00a0Main topic of the interview is\u00a0the Internet of Things and how it relates to databases.<\/p>\n

RVZ<\/p>\n

Q1. What are in your opinion the main Challenges and Opportunities of the Internet of Things (IoT) seen from the perspective of a database vendor?<\/strong><\/p>\n

Steve Graves: <\/strong>Let\u2019s start with the opportunities.<\/p>\n

When we started McObject in 2001, we chose \u201ceXtremeDB, the embedded database for intelligent, connected devices<\/em>\u201d as our tagline. eXtremeDB<\/a> was designed from the get-go to live in the \u201cthings\u201d comprising what the industry now calls the Internet of Things<\/a>. The popularization of this term has created a lot of visibility and, more importantly, excitement and buzz for what was previously viewed as the relatively boring \u201cembedded systems.\u201d And that creates a lot of opportunities.<\/p>\n

A lot of really smart, creative people are thinking of innovative ways to improve our health, our workplace, our environment, our infrastructure, and more. That means new opportunities for vendors of every component of the technology stack.
\nThe challenges are manifold, and I can\u2019t begin to address all of them. The media is largely fixated on security<\/a>, which itself is multi-dimensional.
\nWe can talk about protecting IoT-enabled devices<\/a><\/em> (e.g. your car) from being hacked. We can talk about protecting the privacy of your data at rest<\/a><\/em>. And we can talk about protecting the privacy of data in motion<\/a><\/em>.
\nEvery vendor needs recognize the importance of security. But, it isn\u2019t enough for a vendor, like McObject, to provide the features to secure the target system; the developer that assembles the stack along with their own proprietary technology to create an IoT solution needs to use available security features, and use them correctly.<\/p>\n

After security, scaling<\/a> IoT systems is the next big challenge. It\u2019s easy enough to prototype something.
\nBut careful planning is needed to leap from prototype to full-blown deployment. Obvious decisions have to be made about connectivity and necessary bandwidth, how many things per gateway, one tier of gateways or more, and how much compute capacity is needed in the cloud. Beyond that, there are less obvious decisions to be made that will affect scalability, like making sure the DBMS used on devices and\/or gateways is able to handle the workload (e.g. that the gateway DBMS can scale from 10 input streams to 100 input streams); determining how to divide the analytics workload between gateways and the cloud; and ensuring that the gateway, its DBMS and its communication stack can stream data to the cloud while simultaneously processing its own input streams and analytics.
\nAssembling a team with the wide range of skills needed for a successful IoT project presents an entirely different set of challenges. The skills needed to build a \u2018thing\u2019 are markedly different than the skills needed to implement the data analytics in the cloud. In fact, \u2018things\u2019 are usually very much like good ol\u2019 embedded systems, and system engineers that know their way around real-time\/embedded operating systems, JTAG debuggers, and so on, have always been at a premium.<\/p>\n

Q2. Data management for the IoT: What are the main differences between data management in field-deployed devices and at aggregation points?<\/strong><\/p>\n

Steve Graves: <\/strong>Quite simply: scale. A field-deployed device<\/a> (or a gateway to field-deployed devices<\/a> that do not, themselves, have any data management need or capability) has to manage a modest amount of data. But an aggregation point (the cloud<\/a> being the most obvious example) has to manage many times more data \u2013 possibly orders of magnitude more.
\nAt the same time, I have to say that they might not be all that different. Some IoT systems are going to be closed<\/em>, meaning the nature of the things making up the system is known, and these won\u2019t require much scaling. For example, a building automation system for a small- to mid-size building would have perhaps 100s of sensors<\/a> and 10s of gateways<\/a>, and may (or may not) push data up to a central aggregation point. If there are just 10s of gateways, we can create a UI<\/a> that connects to the database on each gateway where each database is one shard of a single logical database, and execute analytics against that logical database without any need of a central aggregation point. We can extend this hypothetical case to a campus of buildings, or to a landlord with many buildings in a metropolitan area, and then a central aggregation point makes sense.<\/p>\n

But the database system would not necessarily be different, only the organization of the physical and logical databases.
\nThe gateways of each building would stream to a database server in the cloud. In the case of 10 buildings, we could have 10 database servers in the cloud that represent 10 shards of that logical database in the cloud. This architecture allows for great scalability. The landlord acquires another building? Great, stand up another database server and the UI connects to 11 shards instead of 10. In this scenario, database servers are software, not hardware. For the numbers we\u2019re talking about (10 or 11 buildings), it could easily be handled by a single hardware server of modest ability.<\/p>\n

At the other end of the scale (pun intended) are IoT systems that are wide open<\/em>. By that, I mean the creators are not able to anticipate the universe of \u201cthings\u201d that could be connected, or their quantity. In the first case, the database system should be able to ingest data that was heretofore unknown. This argues for a NoSQL<\/a> database system, i.e. a database system that is schema-less. In this scenario, the database system on field-deployed devices is probably radically different from the database system in the cloud. Field-deployed devices are purpose-specific, so A) they don\u2019t need and wouldn\u2019t benefit from a NoSQL database system, and B) most NoSQL database systems are too resource-hungry to reside on embedded device nodes.<\/p>\n

Q3. If we look at the characteristics of a database system for managing device-based data in the IoT, how do they differ from the characteristics of a database system (typically deployed on a server) for analyzing the “big data” generated by myriad devices?<\/strong><\/p>\n

Steve Graves:\u00a0<\/strong>Again, let\u2019s recognize that field-deployed devices in the IoT are classic embedded systems<\/a>. In practical terms, that means relatively modest hardware like an ARM<\/a>, MIPS<\/a>, PowerPC<\/a> or Atom processor<\/a> running at 100s of megahertz, or perhaps 1 ghz if we\u2019re lucky, and with only enough memory to perform its function. Further, it may require a real-time operating system, or at least an embedded operating system that is less resource hungry than a full-on Linux distro. So, for a database system to run in this environment, it will need to have been designed to run in this environment. It isn\u2019t practical to try to shoehorn in a database system that was written on the assumption that CPU cycles and memory are abundant. It may also be the case that the device has little-to-no persistent storage, which mandates an in-memory database.<\/p>\n

So a database system for a field-deployed device is going to
\n1. have a small code size
\n2. use little stack
\n3. preferably, allocate no heap memory
\n4. have no, or minimal, external dependencies (e.g. not link in an extra 1 MB of code from the C run-time library)
\n5. have built-in ability to replicate data (to a gateway or directly to the cloud)
\na. Replication should be \u201copen\u201d, meaning be able to replicate to a different database system
\n6. Have built-in security features<\/p>\n

7. Nice to have:
\na. built-in analytics to aggregate data prior to replicating it
\nb. ability to define the schema
\nc. ability to operate entirely in memory<\/p>\n

A database system for the cloud might benefit from being schema-less, as described previously. It should certainly have pretty elastic scalability<\/a>. Servers in the cloud are going to have ample resources and robust operating systems. So a database system for the cloud doesn\u2019t need to have a small code size, use a small amount of stack memory, or worry about external dependencies such as the C run-time library<\/a>. On the contrary, a database system for the cloud is expected to do much more (handle data at scale, execute analytics, etc.) and will, therefore, need ample resources. In fact, this database system should be able to take maximum advantage of the resources available, including being able to scale horizontally (across cores, CPUs, and servers).
\nIn summary, the edge<\/a> (device-based) DBMS needs to operate in a constrained environment. A cloud DBMS<\/a> needs to be able to effectively and efficiently utilize the ample resources available to it.<\/p>\n

Q4. Why is the ability to define a database schema important (versus a schema-less DBMS, aka NoSQL) for field-deployed devices?<\/strong><\/p>\n

Steve Graves: <\/strong>Field-deployed devices will normally perform a few specific functions (sometimes, just one function). For example, a building automation system manages HVAC<\/a>, lighting, etc. A livestock management system manages feed, output, and so on. In such systems, the data requirements are well known. The hallmark NoSQL advantage of being able to store data without predefining its structure is unwarranted. The other purported hallmark of NoSQL is horizontal scalability<\/a>, but this is not a need for field-deployed devices.
\nWalking away from the relational database model (and its implicit use of a database schema) has serious implications.
\nA great deal of scientific knowledge has been amassed around the relational database model over the last few decades, and without it developers are completely on their own with respect to enforcing sound data management practices.<\/p>\n

In the NoSQL sphere, there is nothing comparable to the relational model (e.g. E.F. Codd<\/a>\u2019s work) and the mathematical foundation (relational calculus<\/a>) underpinning it.
\nThere should be overwhelming justification for a decision to not use relational.
\nIn my experience, that justification is absent for data management of field-deployed devices.
\nA database system that \u201cknows\u201d the data design (via a schema) can more intelligently manage the data. For example, it can manage constraints, domain dependencies, events and much more. And some of the purported inflexibility imposed by a schema can be eliminated if the DBMS supports dynamic DDL (see more details on this in the answer to question Q6, below).<\/p>\n

Q5. In your opinion, do IoT aggregation points resemble data lakes?<\/strong><\/p>\n

Steve Graves: <\/strong>The term data lake<\/a> was originally conceived in the context of Hadoop <\/a>and map-reduce<\/a> functionality. In more recent times, the meaning of the term has morphed to become synonymous with big data<\/a>, and that is how I use the term. Insofar as a gateway can also be an aggregation point, I would not say \u2018aggregation points resemble data lakes\u2019 because gateway aggregation points, in all likelihood, will not manage Big Data.<\/p>\n

Q6. What are the main technical challenges for database systems used to accommodate new and unforeseen data, for example when a new type of device begins streaming data?<\/strong><\/p>\n

Steve Graves: <\/strong>The obvious challenges are
\n1. The ability to ingest new data that has a previously unknown structure
\n2. The ability to execute analytics on #1
\n3. The ability to integrate analytics on #1 with analytics on previously known data<\/p>\n

#1 is handled well by NoSQL DBMSs. But, it might also be handled well by an RDBMS via \u201cdynamic DDL\u201d (dynamic data definition language<\/a>), e.g. the ability to execute CREATE TABLE, ALTER TABLE, and\/or CREATE INDEX statements against an existing database.
\nTo efficiently execute analytics against any data, the structure of the data must eventually be understood.
\nRDBMS handle this through the database dictionary<\/a> (the binary equivalent of the data definition language).
\nBut some NoSQL DBMSs handle this through different meta data. For example, the MarkLogic DBMS<\/a> uses JSON<\/a> metadata to understand the structure of documents in its document store.
\nNoSQL DBMSs with no meta data whatsoever put the entire burden on the developers. In other words, since the data is opaque to the DBMS, the application code must read and interpret the content.<\/p>\n

Q7. Client\/server DBMS architecture vs. in-process DBMSs: which one is more suitable for IoT?<\/strong><\/p>\n

Steve Graves: <\/strong>For edge DBMSs (on constrained devices), an in-process architecture<\/a> will be more suitable. It requires fewer resources than client\/server<\/a> architecture, and imposes less latency through elimination of inter-process communication.\u00a0For cloud DBMSs, a client\/server architecture will be more suitable. In the cloud environment, resources are not scarce, and the the advantage of being able to scale horizontally will outweigh the added latency associated with client\/server.<\/p>\n

Qx Anything else you wish to add?<\/strong><\/p>\n

Steve Graves: <\/strong>We feel that eXtremeDB is uniquely positioned for the Internet of Things. Not only have devices and gateways been in eXtremeDB\u2019s wheelhouse for 15 years with over 25 million real world deployments, but the scalability, time series data management, and analytics built into the eXtremeDB server (big data) offering make it an attractive cloud database solution as well. Being able to leverage a single DBMS across devices, gateways and the cloud has obvious synergistic advantages.<\/p>\n

———————
\nSteve Graves<\/strong> is co-founder and CEO of McObject, a company specializing in embedded Database Management System (DBMS) software. Prior to McObject, Steve was president and chairman of Centura Solutions Corporation and vice president of worldwide consulting for Centura Software Corporation.<\/em><\/p>\n

Resources<\/strong><\/p>\n

Big Data, Analytics, and the Internet of Things<\/a>, by Mohak Shah,\u00a0analytics leader and research scientist at Bosch Research, USA.<\/em>ODBMS.org\u00a0APRIL 6, 2015<\/p>\n

\u00a0Privacy considerations & responsibilities in the era of Big Data & Internet of Things<\/a>, by\u00a0Ramkumar Ravichandran,\u00a0Director, Analytics, Visa Inc. <\/em>ODBMS.org January 8, 2015.<\/p>\n

\u00a0Securing Your Largest USB-Connected Device: Your Car<\/a>,BY\u00a0Shomit Ghose,\u00a0General Partner,\u00a0ONSET Ventures, <\/em>ODBMs.org\u00a0MARCH 31, 2016.<\/p>\n

\u00a0e<\/em>Xtreme<\/em>DB Financial Edition DBMS Sweeps Records in Big Data Benchmark<\/a>,ODBMS.org\u00a0JULY 2, 2016<\/p>\n

\u00a0eXtremeDB in-memory database<\/a><\/p>\n

\u00a0User Experience Design for the Internet of Things<\/a><\/p>\n

Related Posts<\/strong><\/p>\n

On the Internet of Things. Interview with Colin Mahony<\/a>,\u00a0ODBMS Industry Watch,\u00a0<\/span>Published on 2016-03-14<\/span><\/p>\n

A Grand Tour of Big Data. Interview with Alan Morrison<\/a>,\u00a0ODBMS Industry Watch,\u00a0<\/span>Published on 2016-02-25<\/span><\/p>\n

On the Industrial Internet of Things. Interview with Leon Guzenda<\/a>, ODBMS Industry Watch,\u00a0\u00a0January 28, 2016<\/p>\n

Follow us on Twitter: @odbmsorg<\/a><\/strong><\/p><\/blockquote>\n

##<\/p>\n","protected":false},"excerpt":{"rendered":"

“Assembling a team with the wide range of skills needed for a successful IoT project presents an entirely different set of challenges. The skills needed to build a \u2018thing\u2019 are markedly different than the skills needed to implement the data analytics in the cloud.”–Steve Graves. I have interviewed, Steve Graves, co-founder and CEO of McObject.\u00a0Main […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[35,66,97,102,966,139,965,192,239,286,772,355,358,370,412,499,964,855,556],"_links":{"self":[{"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/posts\/4172"}],"collection":[{"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/comments?post=4172"}],"version-history":[{"count":7,"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/posts\/4172\/revisions"}],"predecessor-version":[{"id":4179,"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/posts\/4172\/revisions\/4179"}],"wp:attachment":[{"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/media?parent=4172"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/categories?post=4172"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.odbms.org\/blog\/wp-json\/wp\/v2\/tags?post=4172"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}