Go Green Stay Green. Q&A with Greg Law

Dr. Greg Law just released an eBook, called Go Green Stay Green

Q1. What is Continuous Integration (CI)?

There are many definitions, but the industry is convening on it meaning to running integration tests continually on the shared repository, and developers pushing very frequently (multiple times per day) to that repo. Often this to a branch, and when the appropriate tests have passed the branch is automatically or manually merged with the master repository. Continuous integration tooling may be able to bisect and determine which commit(s) are associated with given test failures.

Q2. A part of CI is the so called Go Green, Stay Green principle. What is it and what is it useful for?

The CI approach can yield significant improvements in both development velocity and product quality. These two things have traditionally been thought of as in opposition to each other – i.e. you can have more of one at the cost of less of the other. But with CI we can improve both at the same time. However, for that promise to be realised, the tests must be reliable, with no false negatives. Otherwise, developers lose confidence in the CI, and on seeing a failure they are tempted to rerun the tests to see if they get lucky next time. In such a world, it is impossible to know whether you just introduced something with a 50% failure rate.

Q3. Is this related to what evolutionary biologists call punctuated equilibrium?

Not CI per se; but there’s definitely a big shift in best (and indeed common) practice among software development teams that CI is part of. From around 1980 to 2010 there was relatively little change in software development practices. Dev and QA were separate, long development cycles had long testing phases, and there were multi-year release periods. Waterfall has always been a bit of a myth (the term was first coined in the 1970’s to describe how software development can be thought of but doesn’t happen in practice); but nonetheless if in 2008 you were a software organisation that had embraced agile, DevOps, CI, or CD, then you were in the vanguard. If hadn’t embraced those best practices by 2018, you were considered a dinosaur.

Q4. Static and dynamic analysis tools can be very effective at finding commonly-occurring errors. What are the pros and cons?

The main benefits are: (1) they can identify the root cause of certain defects very quickly, often taking the developer straight to the offending line of code; (2) they can find problems even if there is no test-case or reproducer. The main two drawbacks are: (1) false positives (particularly with static analysis); (2) they’re good at spotting instances of frequently-made mistakes (e.g. a buffer overrun or a resource not freed on an error return path), but of little use in catching defects that are not one of these.

Q5. You developed a Software Flight Recording Technology. What is it and what is it useful for?

Software flight recording is an approach where you record or capture all the details of a program’s execution, as it runs. Full software flight recording technology allows one to wind back the tape to any instruction that executed and see the full program state at that point. It’s complete visibility into what a program really did. This is as opposed to static analysis which is a prediction of what a program might do. As such software flight recording technology is really the complement of static/dynamic analysis – it’s strong where static and dynamic analysis are weak, and vice versa. In other words, static and dynamic analysis are great at finding the defects you didn’t even know existed, and software flight recording helps you with the problems that are hurting you right now.

Q6. How does it relate to CI and the Go Green, Stay Green principle?

The main impediment to go-green, stay-green is intermittent test failures. Intermittent failures creep through as the CI doesn’t spot them first time out. It’s very common to have an ever-growing backlog of tests that occasionally fail and no-one knows why. And it’s a vicious spiral – the more failing tests you have, the easier it is for them to slip through and the more they accumulate. In the end, the development team loses faith in the CI and the benefits of it are lost. If on the other hand, your CI is configured to use software flight recording to capture the execution of failing tests, it provides the development team with a very powerful weapon in the fight against entropy, and allows the intermittent failures to be fixed much more easily. Particularly for software projects that already have a problem with unreliable CI – that kind of background radiation of intermittently failing tests. It makes the difference between go-green/stay-green being an impractical dream and a reality.

Q7. What are the use cases that used the Go Green, Stay Green principle and your Software Flight Recording Technology?

The SAP HANA team is a great example. That is a product that is highly complex with very high-quality requirements. As such it has very sophisticated testing that puts the database under extreme load. When that load testing finds obscure issues, the SAP developers can turn directly to a recording of the failed test (which represents a 100% reproducible test case) and fix the defect there and then. As a result they have great product development velocity and very high levels of reliability.

Q8. Anything else you wish to add?

Never use sleeping pills, they only cause insomnia.

Resources:

————————-

Greg is the co-founder and CTO of Undo. He is a coder at heart, but likes to bridge the gap between the business and software worlds. (Sadly, these days most of Greg’s coding is done on airplanes)