Look ahead to 2020 in-memory computing
Computing predictions for 2020: Comments from GridGain’s CEO Abe Kleinfeld and from GridGain’s founder and CTO Nikita Ivanov.
In-Memory re: historical/operational data:
In 2020, the adoption of in-memory technologies will continue to soar as digital transformation drives companies toward real-time data analysis and decision-making at massive scale. For example, many companies move older data in their operational database to a data lake. There, data scientists can analyze the data on this separate analytics infrastructure. However, this architecture doesn’t work for use cases that require real-time analytics across a subset of operational and historical data. Let’s say you’re collecting real-time data from sensors on a fleet of airplanes to monitor performance and you want to develop a predictive maintenance capability for individual engines. Now you must compare anomalous readings in the real-time data stream with the historical data for a particular engine stored in the data lake. Currently, the only cost-effective way to do this is with an in-memory “data integration hub,” based on an in-memory computing platform like Apache Ignite that integrates Apache Spark, Apache Kafka and data lake stores like Hadoop. This combination of solutions streams the live data into a transactional database and in-memory computing platform, ingests the relevant subset of historical data from the data lake, and maintains the combined data in memory, where the analytics solution can perform federated, real-time queries across the merged dataset. In addition to predictive maintenance, this platform enables a variety of automated business decision making that instantly reacts to real-time changes in the environment, whether in financial services, healthcare, supply chain and more. 2020 promises to be a pivotal year in the adoption of in-memory computing as data integration hubs continue to expand in enterprises.
– Abe Kleinfeld, CEO, GridGain
In-Memory and HTAP and Real-Time Business Processes:
In 2020, the recognition that mature in-memory computing platforms can deliver up to a 1,000x performance improvement over disk-based databases will lead to the acceleration of two critical trends. First, organizations can migrate from their bifurcated architectures that rely on separate transactional and analytical databases to unified hybrid transactional/analytical processing (HTAP) architecture that enables simultaneous transaction and analytics processing on the same dataset. By eliminating time-consuming ETL processes, HTAP powers real-time digital business models and IoT applications across a range of verticals, including financial services, software, e-commerce, retail, online business services, healthcare, telecom, transportation and other major sectors. The second trend is the adoption of in-memory computing solutions for mainframe computing. Many Fortune 100 companies rely on mainframe computing for their most mission-critical, high-value transactions, paired with a separate data lake deployed off the mainframe, typically on Hadoop. With the announcement that the GridGain in-memory computing platform has been optimized for the IBM z/OS operating system, these businesses can now deploy analytics data hubs which allow them to run real-time analytics across combined transactional and historical datasets. This new capability allows them to accelerate the digital transformation of core systems and power real-time business processes by creating comprehensive views across their data. For example, banks can now create comprehensive 360-degree customer views by leveraging their transactional and data lake data to power upsell and cross sell opportunities. As a result of these two trends, 2020 will be the year when organizations of any size can finally run analytics across their entire data estate, leveraging existing infrastructure investments (including mainframes), to power their real-time business processes.
– Nikita Ivanov, CTO and founder, GridGain
In-Memory re: Intel Optane:
High-performance memory technologies that enable digital transformation will continue to grow in relevance and adoption in 2020. The general availability of Intel Optane in 2019 marks the first step towards enabling a broad range of use cases that achieve in-memory processing speeds at lower total costs with faster recovery times. As a persistent memory technology, Optane can ensure that data is not lost when servers restart and RAM would normally be flushed. Using Optane in App Direct Mode (as byte-addressable RAM) means the data is immediately available for processing at in-memory speeds following a reboot. Further, Optane has a higher density than RAM, so IT professionals can put more memory in a single server, enabling in-memory speed across much larger datasets. Many technologies are gearing up to take advantage of Intel Optane. Apache Ignite, the leading open source in-memory computing solution, is already optimized for Optane when used in storage modes or in block addressable memory mode. In addition, full support for Optane App Direct Mode will be available in 2020.
– Nikita Ivanov, CTO and Founder, GridGain
In-Memory for Data Integration Hubs:
Gartner defines a Data Integration Hub (DIH) as “…an advanced application architecture that aggregates multiple back-end system-of-record data sources into a low-latency and scale-out, high-performance data store. A DIH typically supports access to data via an API services layer. The high-performance data store is synchronized with the back-end sources via some combination of event-based, request-based, and batch integration patterns.” Data Integration Hubs allow users to implement high-throughput, large-scale, low-latency front-end services. In-memory data grids have proven to be a powerful building block upon which Data Integration Hubs can be built. These DIHs can span multiple source databases and cloud-based systems while supporting the operation of large numbers of customer-facing business applications. Hundreds of companies have implemented DIHs and this trend will accelerate in 2020 as companies seek methods to reduce API calls on mission-critical, cloud-based solutions such as billing systems. They can also simplify complexity by creating a single data access layer from which their myriad applications can pull data, and they can decouple front-end applications from back-end data stores.
– Abe Kleinfeld, CEO, GridGain
Sponsored by GridGain