High performing and continuously delivering organizations such as Netflix, Amazon, Google, Wix and others don’t just have pipelines. They don’t just automate stuff and put jobs on it. That’s the easy part.
Many organizations I have consulted for also have pipelines, jenkins jobs, teamcity projects, TFS configurations. But they don’t perform nearly as well fast or consistently (even though they are not happy with their current status)
Places like Netflix and Amazon don’t just have automated pipelines. They are driven by pipelines.
What does that even mean compared to “traditional” pipeline approaches?
It means important decisions are made in the pipelines without very little to no human interventions. These places trust their pipelines to make decisions for them instead of putting humans in the bottleneck.
In many places humans take on the approving things like:
Merges between branches and pull requests
Deployment to environments
Setting up or killing environments
Enabling or disabling features
Pushing a feature to production (or other environments)
Security
Compliance
Configuration
Load balancing, scaling up or down
Tests passing or not
and more..
Companies can make all these decisions by humans and still be fully surrounded by a bunch of pipelines (automated jobs) that they trigger when they decide the time is right.
Here lies the difference between a pipeline driven organization and a traditional one. To be pipeline driven we remove the onus of approval from humans, and give it to pipelines.
This requires several things:
That we trust a pipeline enough to let it make good decisions
This means we need to teach it to make good decisions by adding important steps to it in the form of various automated tests, security tests, compliance tests etc.
That we reframe questions from “did X say it’s OK” to “Did the pipeline pass?”
That the various knowledge silos in the organization shift their work towards mainly supporting and propping up pipelines relating to their knowledge that can make good decisions:
Ops folks will, among other things, help create infrastructure pipelines, infrastructure tests, environment configurations in code that are run in pipelines
Security folks will, among other things, help create security tests and specs that can be checked by automated pipelines and run by each team as needed.
Test folks will help developers learn how to write automated tests at various levels for the pipelines, and help with the test infrastructure
Developers will teach the pipeline (with the help of ops) how to deploy the application, which tests to run, and to do it per commit.
Everyone will write code that will run in a pipeline.
Everyone will slowly reduce their decisions and slowly let the pipeline make more and more of these decisions.
And the end of the day, what separates the Netflix’s of the world from others is a simple truth: They worked really hard to create pipelines that they trust enough to make the hard decisions, and they use them continuously without waiting for humans in the process.