On Modernising the Data Architecture Stack. Q&A with Matt Newman.
” It can be difficult to work with legacy architectures and multiple multi-year migration projects that are in-flight at the same time.”
Q1. What are you currently working on at Adobe?
I am involved with integrating Adobe Workfront data and systems into various other Adobe products. I am helping with Adobe GenStudio, which is an end-to-end solution to accelerate and simplify your content supply chain with generative AI and intelligent automation. I am also working on a customer-facing, ad-hoc reporting solution, replacing the legacy version of this with a new modern one that will democratize reporting for all users.
Q2. What are the main challenges you face when modernising the data architecture stack at Adobe?
It can be difficult to work with legacy architectures and multiple multi-year migration projects that are in-flight at the same time. We migrated from Oracle to PostgreSQL, changed messaging technology from NATS to Kafka, disaggregated a monolith into microservices and onboarded onto the Adobe Business Platform (for authentication) all at the same time. It is also a challenge being an acquired company and sorting out which technologies we will adopt broadly, and which ones we will continue to use that we are using before the acquisition. The timing of all of these projects can be overwhelming to sort out.
Q3. How do you handle legacy systems?
We generally start communicating that a technology or system is deprecated, and then encourage everyone to build any new systems on the new technology. We accept that we will likely need to run parallel systems for quarters or even years, and we plan that in. Usually at the end, when we are 80% there, we start aggressively pushing to the new solution and decommissioning the old systems. The projects are particularly difficult when they require customer interaction.
Q4. What are the benefits for your customers?
As we modernize our architecture and technology, our customers get a better, faster experience. They are able to more efficiently plan and execute large-scale projects across their enterprises, and then analyze the results of their work. We are helping companies manage all the complexities and interdependencies of their work.
Q5. What are the lessons you learned so far in using generative AI?
We have learned a lot with generative AI, both in terms of training our own models (e.g., Firefly), and also using pre-trained LLMs from OpenAI, Anthropic and Google. We also have been playing with many open-source projects like Llama and Mistral.
There has been a lot of hype, and sometimes it takes a while for the real business value to come in, but it does come in slow and steady. It will take a while for us to find the ways that generative AI really benefits customers, and then monetize that, but once we do, it will be very strategic for our future. We have all kinds of uses for generative AI on the creative side with generating images, videos, 3D models, etc. Our customers have loved generating content within Photoshop and other tools, and as we enable them to generate content for marketing campaigns and automate even more of their marketing workflows, they will be delighted.
Q6. Why SingleStore’s latest release (SingleStore Pro Max) is of interest for you?
I am excited about SingleStore Pro Max. Features like better database branching and Smart DR will enable us to efficiently scale our deployments. Kai will be interesting for us as we are heavy users of MongoDB, and it will be really nice to do analytical queries using the same MongoDB queries we already run. I also love the free shared tier because it will help other Adobe groups to get exposure to using SingleStore without having to commit a lot upfront.
Q7. Specifically, which of new SingleStore Pro Max features could help you and your customers to take full advantage of generative Al capabilities? Why?
The improved vector search will enable us to use SingleStore in more cases for Retrieval Augmented Generation (RAG). We also use it for similarity and semantic search use cases.
Q8. Could you give some examples of how your customers are currently using AI?
Our customers use Firefly to generate images for marketing campaigns. They also use generative fill and expand features in Photoshop. On the Adobe Experience Cloud side, we have features to generate content (images, marketing copy, etc.) for tasks, projects or even entire marketing campaigns. In Workfront, when a user is assigning a task to someone in our application, we now have a “Smart Assignment” feature that recommends users most likely to be assigned based on similarity search on vectors.
Q9. Anything else you want to add?
I think SingleStore is a great data platform for building real-time analytics. The fast columnar storage enables us to ingest data and query it all in under five seconds. It also has all the enterprise features that we need as a larger company.
…………………………………
Matt Newman, Principal Data Architect at Adobe
I currently am a Principal Data Architect at Adobe. We are changing the world through digital experiences. I personally am working on defining architectural patterns for source databases (MongoDB and PostgreSQL), eventing (Apache Kafka), and destinations (Elasticsearch, Snowflake, Azure Data Lake, etc.). This enables our customers to operate projects and campaigns more efficiently, and gain insights from joining Workfront and Adobe together.
Previously, I have built data pipelines at Cox Automotive (Dealertrack), Dsco (supply chain), and Taulia (fintech company that does e-invoicing and supply chain finance). I have used technologies including MySQL, Amazon Redshift, Amazon Kinesis, Amazon Lambda, Python, etc. Prior to that, I built out the data and analytics platform at Kuali, an edtech startup based in Lehi, Utah. I also managed the database team at the LDS Church, which provides database solutions that power most of the applications of the Church.
Resources
SingleStore Announces Real-time Data Platform to Further Accelerate AI, Analytics and Application Development. SAN FRANCISCO — January 24, 20
Sponsored by SingleStore.