Racing toward zero latency and distributed scale: Lessons learned from capital markets
By Irfan Khan, CTO SAP, GCO SAP.
In business, the ability to predict, with certainty, what your customers may enjoy or are interested in buying is possible and many businesses are gaining an upper hand on their competition as a result. Matching the data with machine learning algorithms is essential to predict customer behavior, but being able to analyze the data with lightning speed is equally important in a world where Big Data, IoT, and mobile technologies are a part of our everyday lives. Another consideration is the speed by which you can facilitate a transaction. Will your customer know right away whether a product they purchased is available or on back order? If you are in the financial services industry, can you provide clients with near instantaneous reports upon request?
Predicting what’s next in business and then allowing the transaction to take place with near immediacy, offers great advantages. Take for example the success of online streaming services. After a long day of work, you arrive home and tune into your streaming service. The software running the system predicts what you may want to see, based on what you watched in the past. This is an example of the power of machine learning, where a system is trained to peruse vast amounts of data and identify characteristics to predict what a user will do next. Another component of the service allows you, the customer, to order your movie at the very moment you decide you want to watch it and then it ensures that the product is available.
The ability to predict what customers want – the holy grail for companies in many industries including retail and financial services – hinges on the speed by which they can execute transactions. So let’s go back in time a little, to the disastrous market meltdown of 2008, where capital markets entered the race toward zero latency—the amount of time it takes to process any discrete transaction.
Removing delays in financial transactions
Driven by competition and increased regulatory pressure – one of the many consequences of the 2008 market meltdown – financial firms acquired an insatiable desire to remove any delay from the execution of every trading transaction. In 2011, the industry conducted a wholesale evaluation and refreshed their technology by purging systems that ran too slow. And they reaped the benefits. A major brokerage firm could achieve a $100 million a year advantage by executing trade transactions milliseconds or nanoseconds faster. But even milliseconds were not fast enough for the financial landscape of the time, and the drive to zero latency began. Five years later, capital markets are leading the way in driving down processing time.
One only has to tune into CNBC or log onto their online brokerage account to understand that literally every second counts and that capital markets have made huge strides in reaching near-zero latency. A busy investor trying to decide whether to buy or sell now has tools available to not only analyze data, but to do so at lightning speed.
That said, it was a learning process for capital markets. The goal of achieving zero latency was accompanied with the aim of distributed scale, because expanding physical resources added to the complexity of infrastructure. Soon after the events of 2008, instead of finding ways to make data centers more efficient, hardware was added atop existing infrastructure. This worked for a time, but more and more resources were added into the data center to ensure critical applications didn’t run out of bandwidth. New investments in software were made to meet growing business demands. But applications became large and complex, and modifications were made to satisfy changes without much thought on how it impacted the underlying architecture. Eventually, ever expanding hardware resources and cumbersome business software forced IT to spend most of their time integrating technologies and dealing with breakdowns.
With the introduction of new digital platforms and the accompanying processes that run parallel from end-to-end, trimming report times from days to seconds is not only possible, but necessary to compete in today’s financial landscape.
Not a decision, but a requirement
Near-zero latency offers capital markets agility and immeasurable value because it allows them to conceive, analyze, and transact in new and exciting ways. Just as mobile technology is here to stay, data access is moving towards zero latency. In industries such as retail, the drive to achieve near zero latency is no longer a decision to be made; it is a business imperative. Competitors will potentially achieve an unfair advantage if you are slower in executing transactions. But it is not just about speed. It is also about making better decisions – predicting what’s next and then acting upon it with assurance.
Considering all the data we have at our disposal, the speed at which you can analyze, identify trends, and predict customer actions offers an immense competitive advantage. For example, many financial firms extract information automatically from structured and unstructured data to identify trends and outliers to enable them to make improved trading decisions for their clients.
But achieving zero latency means placing an immense burden on existing infrastructure. Take for example, a large brokerage firm that deals with hedge funds; these types of organizations don’t have the luxury of IT. They are not interested in owning the infrastructure, which is why the cloud is such an attractive option. The cloud can offer firms a simpler approach to entering the race to zero latency with faster go-lives, lower total cost of ownership, and minimized risk. Of course, not just any cloud platform will do. At its core, real-time architecture and an open cloud platform are essential for running modern mission critical applications and data analytics with near-zero latency.
Winning against the clock and beating the competition
So how can other industries apply lessons learned from FSI capital markets to their goal of achieving zero latency?
Businesses are constantly racing against time and a good measure of preparation is necessary to overcome the competition. In the age where instant gratification and seamless customer experience is the expectation, businesses must evolve to instantly explore and analyze data in near-real time. Those that do will be able to gain a competitive edge because they can make better decisions faster, take advantage of favorable market conditions, identify customer trends, and even predict customer actions.
The main lesson from capital markets is that a smart investment is one where a perfectly developed modern infrastructure enables low latency and distributed scale to be achieved. And to take advantage of machine learning algorithms and enhance your business with predictive capabilities you need autonomous and continuous processing even when connections are not available.
Most significantly, if you rely on traditional technologies with add-ons, you’re creating more problems for yourself in the long-term.