On NoSQL Databases and Data Security. Q&A with Keshav Murthy and Ramesh Chitor
Q1. Keshav, you are Vice President, Couchbase R&D. What are your current projects?
Keshav Murthy : This summer we released Couchbase Server 7 with major innovations to help customers embrace modern databases easily: Improved data organization, SQL transactions, cost based optimizer, User defined functions in Javascript & Python, and better data organization on the Couchbase NoSQL database. Of course, with every release, we strive for better performance and improved security.
Going forward:
- The query optimizer makes SQL easy to use and enterprise ready. Couchbase 7.0 optimizer does index selection and join-type selection based on cost. We’re adding cost based join reordering.
- The key difference between the relational model and JSON model is arrays. Language constructs to manipulate, index, and cost arrays and array predicates are the keys to query processing on JSON. We’ve had array indexing in 2016; We’re now making it even simpler to create and use.
- Dealing with numbers, dates, and strings is good for applications, but humans think better with visualization. We’re working on better integration with Tableau and other BI/DataViz tools. In 7.0.2, we’re shipping built-in charts. We’re starting with simple charts and expect to evolve quickly.
- We’re enriching our stored procedure support to make the transition to a modern database easier.
- Security requirements come in regularly and these days it’s table stakes.
- We continually improve Tooling, TCO, and developer experience with every release.
Q2. Couchbase became a publicly traded company. What does it mean for your R&D work?
Keshav Murthy : Going public is a point of celebration, and it’s the result of Couchbase helping customers modernize their database layer. Our mission is to build the best modern databases for enterprises. That mission continues. Now, we can invest even more to bring our database innovations to the Cloud and to the developers.
Q3. You are quoted saying “In the database industry, the Incumbents regarded NoSQL with the same derision or claimed they had SQL-less databases a long time ago! ” Could you please elaborate more on this?
Keshav Murthy : Incumbents measure their success differently than disruptors. Incumbents justify ignoring the threat of new entrants to the market by relying on entry and exit barriers. It doesn’t turn out well for the incumbents. The Big 3 ignored Toyota and Honda, Blackberry ignored iPhone and Oracle, and IBM ignored the Cloud. Ironically, Toyota and Honda ignored their disruptor, Tesla. Nobody is immune to the incumbent disease.
The relational database incumbents like Oracle and IBM reacted predictably to NoSQL. In 2015, Oracle said that NoSQL databases have limited capability. Others said, “We developed databases since the sixties without SQL – before SQL was invented(1974). So, we’ve had NoSQL all along!“. The incumbents missed the main points of NoSQL – scale, flexibility, and availability. That’s why NoSQL databases were developed by database users like Amazon and Facebook, and not the incumbent database vendors. Now, the NoSQL database industry is growing with already three public companies: MongoDB, Elastic, and Couchbase. Oracle, IBM, and Microsoft have added NoSQL offerings, but their heart isn’t really in it.
It’s not that the Amazons and Apples of the world couldn’t afford to pay Oracle. Oracle database did not meet their scale-out performance, elasticity, and availability requirements. Creating a general-purpose database with SQL support with scale-out and high availability was a tall order. Hence, the NoSQL companies implemented a highly available, scale-out database, but a simple data access API. This addressed the pain points of the modern workload, even if the developers had to work around the limitations.
SQL is easy to use, but difficult to implement. It took the NoSQL industry time to implement SQL. Couchbase introduced SQL for JSON in 2015, adding joins, window functions, and even transactions. Today, it’s used by thousands of enterprise applications in every vertical. Last year, Amazon DynamoDB introduced PartiQL – a SQL-inspired, SQL-compatible language. MongoDB is still sitting this one out.
The NoSQL industry has evolved from the humble beginnings of key-value API, adding SQL, search, and analytics. NoSQL runs the modern enterprise workloads. The incumbents missed the modern database bus. That was my point.
Q4. The competition between the NoSQL systems to provide better SQL is fierce. Any thoughts on this?
Keshav Murthy : SQL is vast, well developed, and deep. For developers, SQL support should include three things: SQL language, transactions that can execute these statements, and stored procedures. You’ll need a rich set of access methods (indexes, join implementations) and a great query optimizer to run enterprise workloads. Implementing all these for a distributed database is not for the faint of heart.
A couple of years ago, I studied SQL support in Cassandra, Couchbase, CosmosDB, and MongoDB. Cassandra, Couchbase, and CosmosDB have various forms of SQL. MongoDB’s MQL syntax is different but supports the basic SQL operations.
Cassandra has had CQL for a while, but they seem to be focused on improving their data infrastructure. Similarly, CosmosDB has SQL even when they were called “DocumentDB”, but they don’t seem to have done much with it. DynamoDB added support for PartiQL last year. They’ve started with a basic set and even added transaction support for them. I’d expect the language and capabilities will grow over time as they learn to implement efficiently on DynamoDB.
MongoDB MQL’s rich query support comes from its aggregation pipeline. The pipeline design makes it procedural although it has elements of an optimizer: e.g. index selection. MQL added full transaction support in 2019, search integration last year, and window functions this year. Couchbase N1QL added Window functions and search integration a couple of years ago and added SQL transactions. We also added a cost-based optimizer to N1QL. Both MQL and N1QL added Javascript functions this year. So, there’s a good tug of war to make the developer’s life easier.
SQL is 50 years young. There are more SQL projects and products than ever. May the best SQL win.
Q5.Do you have any role models you follow in your work?
Keshav Murthy : Of course. Isn’t life alchemy of what you learn from others and those rare insights? I was lucky to have many role models. I learned a great deal from Fred Ho at IBM – about databases and leadership. He had worked at Tandem, Redbrick — pioneers in databases. I learned transforming ideas in the head to a product in the customer’s hand requires diligence, hard work, and leadership. Getting the product to customers is just the end of the beginning of a process. Keeping the customer’s success in focus helps you improve every day.
Q6. Ramesh, in your role as GTM, Strategic Alliances lead, what do you see as key initiatives?
Ramesh Chitor: Customers, more than ever before, want to keep their data safe. If one can characterize data in 3 states – Data at rest, Data in transit and Data in use, it becomes easier then to break down the requirements to secure data at each of those states. In other words, data storage, network and compute have different needs and characteristics.
Q7. What are the main data challenges that your customers and partners are currently facing?
Ramesh Chitor: The biggest challenge my customers and partners face today is to understand sophisticated levels of attacks on their data. The volume, velocity and variety of attacks that we need to safeguard our data from is massive modern day challenge. This is where Zero Trust Data Management is so important.
Q8. Let’s talk about data security. What do you suggest to your customers to do about it?
Ramesh Chitor: Data security is the #1 imperative of every digital enterprise. Securing digital transformation is what every IT leader is thinking about and prioritizing. The approach to protect your data has to be holistic and cannot be an afterthought. Cybersecurity insurance costs and the rise of ransomware has made every CISO and CXO adopt a proactive strategy to deal with this 21st century threat.
Q9. If we consider the data management hierarchy to formulate your strategyWhere do companies start? Compliance? Governance?
Ramesh Chitor: This really depends on where the companies are in their life cycle. Even just a few years ago, companies started with backing up data, and then branch into compliance, followed by mobility and then security. Now, you have a massive shift, where this is now a Security led discussion. You can see the convergence of SecOps and traditional IT ops in terms of solving your ransomware issues.
Q10: How do you see Security Response units evolve their responses? What role do leading or lagging indicators play in your experience?
Ramesh Chitor: As the volume and frequency of these sophisticated attacks increase, incident response teams focus on how to know whether those we do business with, and their data are secure. This is a hard exercise to effectively assess the risk and can consume a lot of time, and to come up with foolproof methodologies. One has to take into consideration the security of your suppliers, alliance partner products and adjacent technologies that are embedded in the stack. To do this, we need to look at the leading and lagging indicators you can check for. Some of them could be audit reviews, governance check-ups, and on-site reviews. If you cannot do all of those, for lack of time or skills, how would you go about prioritizing these indicators for each situation?
For any security solution to be effective, most of the KPIs of a CISO need to be met. These executives are indisputably on the hook for their own internal indicators of health of their security system. Plans have to be made and documented on how these leaders then, take the learnings to other departments internally. I can see the CISO role being morphed into a Chief Risk Officer in the near term. As the organization grows, it is worth assessing whether they can be an active participant in the community at large, via online communities, through their customers and partners, or funding group of researchers to conquer the next big frontier. Some of these activities are bottoms-up driven by motivated employees that feed directly into the security architecture.
The absence of these leading indicators is a powerful signal that security systems, architectures and response frameworks in place need an overhaul.
……………………………………….
Keshav Murthy is a vice president of Couchbase R&D. Previously, he worked at MapR, IBM, Informix, Sybase. At IBM, he led the SQL and NoSQL Informix R&D teams. Keshav holds twelve US patents. He received two President’s Club awards at Couchbase, two Outstanding Technical Achievement Awards at IBM. He blogs on databases.
Ramesh Chitor is the Senior Director of Strategic Alliances at Rubrik. Bringing over 20 years of experience in the technology industry, Ramesh works with Rubrik alliance partners on driving outcomes and scaling go-to-market programs.