Software Logic: How Shorter Release Cycles Reduce Stress
Software Logic: How Shorter Release Cycles Reduce Stress
Continuously integrate, build, deploy, and test to minimize the stress within your engineering team.
I’ve been fortunate to have worked with many high-functioning, high-output teams that can produce and release quality software quickly. Each of these teams has had a unique approach to code reviews, pull and merge requests, and software delivery.
In this post I’ll cover the combination that has worked best for our team at CodeLogic and how we are able to create a quality release every two weeks.
Does this release process sound familiar?
Most likely, you’ve worked on a team that releases software in an extended period, often several months to a year. These releases are generally scheduled for weekends, because every member of the team expects the release to be fraught with nail-biting, last-minute changes to deal with an upgrade that did not anticipate an issue, or a migration script that did not account for an undocumented patch made sometime last year.
Dreading each release
It’s not uncommon for teams to begin dreading each release, knowing that they will have to tell their friends and family that they cannot make plans for a weekend (or longer) and that they will survive only on pizza and caffeine. At the end of the release, the entire team is wiped out from late nights, having to make “just one more change” after another, trying to find the correct amount of duct tape to make the release work well enough “so we can just get some sleep.”
Moving in the wrong direction
The next couple of weeks of development are shot: morale is at a new low due to the team’s energy being sapped from pulling all-nighters and managing high stress levels. One of the most amusing outcomes from these unhealthy release efforts is that the result is not to shorten the release timeline so that fewer changes are introduced, but to add more process overhead to the development and QA pipeline, lengthening the time between software releases.
I’ve worked with teams where this was so extreme that the code produced by the development team did not make it to the QA team for over a month because the process for approvals, checks and balances was so onerous — and the code still had issues when released.
Prevent the long running feature branch
I have yet to meet the developer that looks forward to handling merge conflicts for a long-lived feature branch. At CodeLogic we release every two weeks. Our tickets are broken into small, workable chunks that prevent large, long running feature branches.
Each new feature that is developed is created on a branch using a naming convention that allows our CI system to detect and build the branch with each new commit. Once a feature is complete, the rest of the development team is notified of the merge request. This process allows us to develop features in a sustainable way, reducing the risk of merge conflicts down the road.
The code review process
The reviewer is then able to check the CI system to ensure that no new static analysis concerns have been introduced, that all tests have passed, that test coverage is at an acceptable level, that the code builds and runs on more than the developer’s system, and that no CodeLogic governance rules have been triggered. At this point, the reviewer can proceed with confidence that the code meets all baseline requirements and only needs to review syntax for readability and sufficient comments.
Integration branch tests
After a merge request is approved, the code is merged into a common integration branch where the CI system again builds and runs all unit tests. This ensures that the new code works not only on the developer’s branch, but also when merged with any code introduced by other feature branches.
From this point on, all code promotion is automated. After the integration branch tests pass, the code is promoted by the CI system to a QA branch, where both unit tests and integration tests run, and code is deployed into a QA environment where automated analysis occurs.
Master branch - QA process
Again, after all tests pass, code is promoted to a master branch where artifacts are produced and made available for release. This allows us to cut a release at any time, knowing that all parts of the system have been tested and work together as expected. Once we have created a release candidate, it is again deployed into a QA environment where automated analysis is run again and a final, manual QA process occurs.
The takeaway
While no process is perfect, this is the process that works for us and allows us to create and release new features to customers, at a steady and repeatable pace, without burning weekends.