On managing legacy at the Financial Times. Q&A with Rhys Evans.
Q1. What is your role at the Financial Times?
I co-lead a fairly new team which builds tools to help the rest of the teams at the FT build and operate reliable technology. Ultimately this is because almost everything the company does — even surprising things such as distributing a paper newspaper — has a big dependency on technology that runs smoothly.
Q2. What is Legacy for you?
I quite like this quote from the rugged software manifesto : “I recognize that my code will be used in ways I cannot anticipate, in ways it was not designed, and for longer than it was ever intended.” So legacy is all the stuff that the company and the people in it hoped would be replaced by something better by now. I’d also add that it doesn’t have to be old technology; once its funding stream has dried up, even shiny new tech can degenerate surprisingly quickly into an unreliable burden.
Q3. Is Legacy a problem for the Financial Times?
Yes, but we’re far from unique. From banks relying on cobol mainframes to the NHS failing to patch windows machines and being hit by malware, or startups struggling to scale because of the poorly architected systems they built in their early days, the impact of legacy code is everywhere.
Q4. How can you ensure systems are connected to the right people?
The reason you would want to connect systems to the right people is because, by and large, it’s people that know the answers to questions when something goes wrong. Our approach to maintain this network of information is to connect systems to teams which are, in turn, connected to the technical directors that oversee them, and the people who make up that team. By doing this, rather than connecting systems directly to people, we have a knowledge graph about our systems that is far more resilient to change, and that has a direct chain of responsibility up to a senior staff member.
Q5. You have been using the Neo4j graph database. Why not a relational database?
The relationships I described above already involve a few different types of connection between things. Once you start mapping dependencies between systems, connections to infrastructure etc., you end up with thousands of records, all connected in an ever-growing variety of ways. Neo4j scales well to this in two different ways; firstly, it’s schema-less, so the model can easily grow to represent things we haven’t considered yet, and secondly, querying highly connected data is more intuitive and performant using a database that is optimised for graph data.
Q6. You have been using GraphQL: What are the pros and cons?
The biggest pro by far is that it gets rid of the question ‘What should the read API look like?‘. GraphQL is a widely used API standard that is constantly growing in popularity. The Neo4j-graphql-js library that Neo4j publish means it’s trivial to layer this on top of a Neo4j database. Now we have an API that gives all our users access to all the data — including traversing its complex web of connections — and they don’t need to acquire any specialist knowledge to be able to interact with our data; it’s the same as working with any other GraphQL API.
It’s hard to believe, but I don’t really see a downside to using GraphQL on top of Neo4j. It doesn’t solve ALL our problems, and there are interesting things around caching and access to complex queries that we still have to solve… but those problems would be there even without GraphQL.
Q7. What is the software stack you have been using?
Mainly nodejs, running a lot of services in AWS Lambda and some on Heroku. The most interesting part of the stack by far is the Neo4j + GraphQL data layer. We’re hoping to adopt GraphQL federation soon too in order to combine our Neo4j + GraphQL API with lots of other internal data sources.
————————-
Rhys Evans
Rhys has worked in technology for over 10 years, for most of that period working as a Front-end web developer. For the last 5 years he has worked at the Financial Times, initially working on user-facing features for www.ft.com , before pursuing an interest in performance and stability. He now co-leads the Reliability Engineering team, building tools to support the operation of the FT’s 1000+ systems. He spends his spare time birdwatching, hiking and playing a number of musical instruments poorly.