--- title: "Pipeline Driven: Continuous delivery, build patterns and enterprise DevOps" url: "https://pipelinedriven.org/index.md" canonical: "https://pipelinedriven.org/" tags: [] author: "Roy Osherove" description: "Pipeline Driven by Roy Osherove: continuous delivery and DevOps best practices, build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps." usage: "Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original page)." license: "https://creativecommons.org/licenses/by/4.0/" --- # Pipeline Driven > Pipeline Driven is Roy Osherove's site about continuous delivery: build pipelines, pipeline-friendly testing, DevOps metrics and enterprise DevOps. HTML: https://pipelinedriven.org/ · Guide for AI systems: [llms.txt](https://pipelinedriven.org/llms.txt) · Full text: [llms-full.txt](https://pipelinedriven.org/llms-full.txt) ## On the home page Pipeline Driven · by Roy Osherove ### What is a pipeline driven organization? A Pipeline Driven Organization is operating around the idea that **automated pipelines make the important IT decisions**: 1. Commit - Is the code releasable? 2. Build - Are the infrastructure changes OK? - Are we compliant and secure? 3. Test - Is a feature working? 4. Deploy - When is it OK to deploy a new version? - When and how to deploy a new environment 5. Learn - When to rollback (or forward) a new version [Read the blog](https://pipelinedriven.org/article) [Watch the Øredev 2019 talk](https://pipelinedriven.org/article/video-the-pipeline-driven-organization-enabling-true-continuous-delivery-oredev-2019) #### Who makes the call? Traditional organizations In traditional organizations, even though automated pipelines may exist, they do not make the important & continuous IT decisions. **Humans do.** Pipeline driven organizations In a pipeline based organization, experts from various categories of work **spend their time teaching the pipelines** how to make proper decisions (using automated specialized tests). In this way they can trust the pipeline enough to let it make the decisions instead. Diagram #### Co-Ops - A Cooperative Pipeline [![Co-Ops - A Cooperative Pipeline: a diagram of the delivery pipeline with the expert groups that feed decisions into it](https://pipelinedriven.org/img/c/screen-shot-2019-11-21-at-12.47.58-am-0109cf.png)](https://pipelinedriven.org/img/c/screen-shot-2019-11-21-at-12.47.58-am-0109cf.png) Co-Ops - A Cooperative Pipeline. [Read “Co-Ops: Enabling True Continuous Delivery with Cooperative Pipelines & Processes (Title Feedback Needed)”](https://pipelinedriven.org/article/co-ops-enabling-true-continuous-delivery-with-cooperative-pipelines-amp-processes-title-feedback-needed) #### Latest posts Archive · 2014–2022 [More Posts](https://pipelinedriven.org/article) 1. [Metrics](https://pipelinedriven.org/article/line/metrics) 11 July 2022 ### [A Metrics Framework for Continuous Delivery](https://pipelinedriven.org/article/a-metrics-framework-for-continuous-delivery) Here’s a framework I like to start with when I discuss what types of metrics can help or harm you in your journey to continuous delivery: Lagging Indicators At the organizational level, unit tests are usually part of a… 2. [Talks & Events](https://pipelinedriven.org/article/line/talks) 11 July 2022 Video ### [Video: Lies, Damned Lies and Metrics](https://pipelinedriven.org/article/video-lies-damned-lies-and-metrics) I spoke about what types of metrics and KPIs (including OKRs) you can use with continuous delivery, and which ones you might want to stop using, during my talk at GOTO 2019. Also discussed: Leading and Lagging… 3. [Build Patterns](https://pipelinedriven.org/article/line/build) 19 April 2020 ### [If a Build Takes 4 hours, Run It Every 4 Hours](https://pipelinedriven.org/article/if-builds-take-4-hours-run-them-every-4-hours) (Note: You might need to fix your environment bottlenecks first before being able to act on this blog post) Builds (i.e. compile + run tests + deploys + more tests etc…) represent a bottleneck for getting feedback on our… 4. [Enterprise DevOps](https://pipelinedriven.org/article/line/ent) 6 December 2019 ### [Co-Ops: Enabling True Continuous Delivery with Cooperative Pipelines & Processes (Title Feedback Needed)](https://pipelinedriven.org/article/co-ops-enabling-true-continuous-delivery-with-cooperative-pipelines-amp-processes-title-feedback-needed) “Co-Ops” is a term that one of the attendees at a talk I was giving about cooperative pipelines came up with. I wish I knew his name. I asked him to email me afterwards so I can credit him but he never did. Co-Ops… 5. [Talks & Events](https://pipelinedriven.org/article/line/talks) 20 November 2019 Video ### [Video: The Pipeline Driven Organization - Enabling True Continuous Delivery (Oredev 2019)](https://pipelinedriven.org/article/video-the-pipeline-driven-organization-enabling-true-continuous-delivery-oredev-2019) Roy Osherove — The Pipeline Driven Organization｜ Øredev 2019 · YouTube 6. [Talks & Events](https://pipelinedriven.org/article/line/talks) 2 June 2019 ### [Continuous Delivery Israel Meetup - First Meeting June 24 2019](https://pipelinedriven.org/article/continuous-delivery-israel-meetup-first-meeting-june-24-2019) I’m pleased to announce that we will have the first ever meeting Continuous Delivery Israel meetup, in the AWS Floor28 offices in Tel Aviv, June 24th, 2019, at 6PM. The Group’s page is located at CDIsrael.com If you… #### Browse by line Every post sits on one of six topic lines. - ### Build Patterns Fast, maintainable builds: shipping skeletons, script injection, rolling builds. - [If a Build Takes 4 hours, Run It Every 4 Hours](https://pipelinedriven.org/article/if-builds-take-4-hours-run-them-every-4-hours) - [Guidelines for selecting tools for continuous delivery toolchains](https://pipelinedriven.org/article/the-rules-of-selecting-tools-for-continuous-delivery-toolchains) - [The first rule of continuous delivery toolchains](https://pipelinedriven.org/article/the-first-rule-of-continuous-delivery-toochains) [All 17 posts on Build Patterns](https://pipelinedriven.org/article/line/build) - ### Testing Pipeline-friendly test strategies and trustworthy, noise-free pipelines. - [Delivery Pipelines and Discovery Pipelines](https://pipelinedriven.org/article/delivery-pipelines-and-discovery-pipelines) - [Red Pipelines and Build Whisperers: Getting to a Trustworthy Noise Free Test Pipeline](https://pipelinedriven.org/article/red-builds-and-build-whisperers-trustworthy-noise-free-test-pipelines) - [A Pipeline Friendly Layered Testing Strategy & Recipe for DEV and QA](https://pipelinedriven.org/article/a-pipeline-friendly-layered-testing-strategy-amp-recipe-for-dev-and-qa) [All 3 posts on Testing](https://pipelinedriven.org/article/line/test) - ### Metrics What to measure on the way to continuous delivery, and what not to. - [A Metrics Framework for Continuous Delivery](https://pipelinedriven.org/article/a-metrics-framework-for-continuous-delivery) - [Ten Devops & Agility Metrics to Check at the Team Level](https://pipelinedriven.org/article/ten-ideas-for-things-you-can-measure-as-a-team-on-your-devops-journey) - [DevOps Metric: Amount of Defects and Issues: definition.](https://pipelinedriven.org/article/devops-metric-amount-of-defects-and-issues-definition) [All 7 posts on Metrics](https://pipelinedriven.org/article/line/metrics) - ### Environments Ephemeral environments and the cost of static ones. - [Ephemeral Environments for DevOps: Why, What and How?](https://pipelinedriven.org/article/ephemeral-environment-why-what-how-and-where) - [The four biggest issues with having static environments](https://pipelinedriven.org/article/the-four-biggest-issues-with-having-static-environments) - [Parallel Firehose](https://pipelinedriven.org/article/2014/9/22/parallel-firehose) [All 3 posts on Environments](https://pipelinedriven.org/article/line/env) - ### Enterprise DevOps DevOps values at scale: approvals, monoliths, compliance, culture. - [Co-Ops: Enabling True Continuous Delivery with Cooperative Pipelines & Processes (Title Feedback Needed)](https://pipelinedriven.org/article/co-ops-enabling-true-continuous-delivery-with-cooperative-pipelines-amp-processes-title-feedback-needed) - ["Pipeline Driven" vs. "We have Jenkins Jobs"](https://pipelinedriven.org/article/pipeline-driven-vs-we-have-jenkins-jobs) - [Always start with a "Why" when transforming to DevOps.](https://pipelinedriven.org/article/always-start-with-a-why-when-transforming-to-devops) [All 7 posts on Enterprise DevOps](https://pipelinedriven.org/article/line/ent) - ### Talks & Events Conference talks, interviews and meetups. - [Video: Lies, Damned Lies and Metrics](https://pipelinedriven.org/article/video-lies-damned-lies-and-metrics) - [Video: The Pipeline Driven Organization - Enabling True Continuous Delivery (Oredev 2019)](https://pipelinedriven.org/article/video-the-pipeline-driven-organization-enabling-true-continuous-delivery-oredev-2019) - [Continuous Delivery Israel Meetup - First Meeting June 24 2019](https://pipelinedriven.org/article/continuous-delivery-israel-meetup-first-meeting-june-24-2019) [All 4 posts on Talks & Events](https://pipelinedriven.org/article/line/talks) #### Resources - [WatchTalks and videosConference talks on pipelines, enterprise DevOps and testing.](https://pipelinedriven.org/videos) - [LibraryBeautiful Builds8 short videos and the book on build patterns.](https://pipelinedriven.org/library) - [ReadRecommended DevOps booksRoy’s shortlist, from Continuous Delivery to The Goal.](https://pipelinedriven.org/enterprise-devops-books) - [ToolsFeature toggle frameworksFeature flag libraries by language.](https://pipelinedriven.org/feature-toggle-frameworks-list) ![Roy Osherove](https://pipelinedriven.org/img/author/roy-osherove-2026-face-480.jpg) #### By Roy Osherove Roy Osherove helps software organizations deliver continuously. He is the author of [The Art of Unit Testing](https://www.artofunittesting.com/), [Elastic Leadership](https://www.elasticleadership.com/) and [Enterprise DevOps](https://pipelinedriven.org/books-by-roy-osherove), and speaks about pipelines, testing and team leadership at conferences such as Øredev. [Get in touch](https://forms.gle/jQSx5KoXvrVA1JCK7) ## Pages - [What is Enterprise DevOps?](https://pipelinedriven.org/enterprise-devops.md): What is Enterprise DevOps? The application of DevOps values in organizations with interdependent systems, monoliths, approval gates and compliance requirements. - [Videos](https://pipelinedriven.org/videos.md): Conference talks by Roy Osherove on the pipeline driven organization, enterprise DevOps and testing, plus the Beautiful Builds video library. - [Feature Toggles](https://pipelinedriven.org/feature-toggle-frameworks-list.md): A list of feature toggle (feature flag, feature bit) frameworks for Java, Python, .NET, Ruby and more. - [Books by Roy Osherove](https://pipelinedriven.org/books-by-roy-osherove.md): Books by Roy Osherove: The Art of Unit Testing, Elastic Leadership and Enterprise DevOps. - [Enterprise DevOps Books](https://pipelinedriven.org/enterprise-devops-books.md): Roy Osherove's favourite books about DevOps, enterprise DevOps, continuous delivery and lean. - [Beautiful Builds library](https://pipelinedriven.org/library.md): The Beautiful Builds library: short videos by Roy Osherove on build patterns, continuous integration and deployment automation. - [DevOps & Enterprise DevOps Recommended Books](https://pipelinedriven.org/enterprise-devops-books-list.md): Pipeline Driven by Roy Osherove: continuous delivery and DevOps best practices, build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps. - [Books by Roy Osherove](https://pipelinedriven.org/agility-devops-related-books.md): Pipeline Driven by Roy Osherove: continuous delivery and DevOps best practices, build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps. - [Contact](https://pipelinedriven.org/contact.md): Contact Roy Osherove about continuous delivery, enterprise DevOps, consulting or training. ## Builds - [Faster Feedback, Waves of Confidence, and Automatic Triggers](https://pipelinedriven.org/library/2014/2/18/faster-feedback-waves-of-confidence-and-automatic-triggers.md): Faster Feedback, Waves of Confidence, and Automatic Triggers: from the Beautiful Builds library by Roy Osherove. - [Start by Shipping : Shipping Skeleton](https://pipelinedriven.org/library/2014/2/18/start-by-shipping-shipping-skeleton.md): Start by Shipping : Shipping Skeleton: from the Beautiful Builds library by Roy Osherove. - [The Beautiful Builds Book](https://pipelinedriven.org/library/2014/2/18/the-beautiful-builds-book.md): The Beautiful Builds Book: from the Beautiful Builds library by Roy Osherove. - [The Big Difference Betweeen Automated Builds and Continuous Integration](https://pipelinedriven.org/library/2014/2/18/the-big-difference-betweeen-automated-builds-and-continuous-integration.md): The Big Difference Betweeen Automated Builds and Continuous Integration: from the Beautiful Builds library by Roy Osherove. - [Beautiful Builds - Roy Osherove](https://pipelinedriven.org/library/2014/2/18/beautiful-builds-roy-osherove.md): Beautiful Builds - Roy Osherove: from the Beautiful Builds library by Roy Osherove. - [App Deployment and Automation - 20 tools and tips by Roy Osherove](https://pipelinedriven.org/library/2014/2/18/app-deployment-and-automation-20-tools-and-tips-by-roy-osherove.md): App Deployment and Automation - 20 tools and tips by Roy Osherove: from the Beautiful Builds library by Roy Osherove. - [API Artifacts and Inter-Component Dependencies](https://pipelinedriven.org/library/2014/3/2/api-artifacts-and-inter-component-dependencies.md): API Artifacts and Inter-Component Dependencies: from the Beautiful Builds library by Roy Osherove. - [Introduction to Beautiful Builds](https://pipelinedriven.org/library/2014/3/2/introduction-to-beautiful-builds.md): Introduction to Beautiful Builds: from the Beautiful Builds library by Roy Osherove. ## DevOps & Enterprise DevOps Recommended Books - [Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation](https://pipelinedriven.org/enterprise-devops-books-list/svo6zadfkou0uv27kz6pxg0ev1twz6.md): Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation: a book recommended on Pipeline Driven by Roy Osherove. - [The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations](https://pipelinedriven.org/enterprise-devops-books-list/jkivba3nda04ezgyyuqgb0eua47yr5.md): The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations: a book recommended on Pipeline Driven by Roy Osherove. - [Enterprise DevOps: Patterns for Succeeding with DevOps at Scale](https://pipelinedriven.org/enterprise-devops-books-list/g84kk40asdl2q2joacywae5vxsr4c6.md): Enterprise DevOps: Patterns for Succeeding with DevOps at Scale: a book recommended on Pipeline Driven by Roy Osherove. - [Continuous Integration: Improving Software Quality and Reducing Risk](https://pipelinedriven.org/enterprise-devops-books-list/9j1f9feuknuvl5ihh672gapn45tvvu.md): Continuous Integration: Improving Software Quality and Reducing Risk: a book recommended on Pipeline Driven by Roy Osherove. - [The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win Paperback – October 16, 2014](https://pipelinedriven.org/enterprise-devops-books-list/996b9gc085i16jh6htuycp8z0p2kqc.md): The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win Paperback – October 16, 2014: a book recommended on Pipeline Driven by Roy Osherove. - [The Goal: A Process of Ongoing Improvemen](https://pipelinedriven.org/enterprise-devops-books-list/6xm8hxelahmh0b9h9wolmj7ilz7kh1.md): The Goal: A Process of Ongoing Improvemen: a book recommended on Pipeline Driven by Roy Osherove. ## Books by Roy Osherove - [The Art of Unit Testing, 2nd Edition](https://pipelinedriven.org/agility-devops-related-books/art-of-unit-testing.md): The Art of Unit Testing, 2nd Edition: a book recommended on Pipeline Driven by Roy Osherove. - [Elastic Leadership: Growing Self Organizing Teams](https://pipelinedriven.org/agility-devops-related-books/elastic-leadership.md): Elastic Leadership: Growing Self Organizing Teams: a book recommended on Pipeline Driven by Roy Osherove. - [Enterprise DevOps: Patterns for Succeeding with DevOps at Scale](https://pipelinedriven.org/agility-devops-related-books/enterprise-devops.md): Enterprise DevOps: Patterns for Succeeding with DevOps at Scale: a book recommended on Pipeline Driven by Roy Osherove. ## Posts (41) ### Build Patterns - [If a Build Takes 4 hours, Run It Every 4 Hours](https://pipelinedriven.org/article/if-builds-take-4-hours-run-them-every-4-hours.md) (2020-04-19): (Note: You might need to fix your environment bottlenecks first before being able to act on this blog post) Builds (i.e. compile + run tests + deploys +… - [Guidelines for selecting tools for continuous delivery toolchains](https://pipelinedriven.org/article/the-rules-of-selecting-tools-for-continuous-delivery-toolchains.md) (2017-03-10): If the tool you use does not support continuous delivery values , you're going to have a bad time implementing CI/CD with fully automated pipelines. Here… - [The first rule of continuous delivery toolchains](https://pipelinedriven.org/article/the-first-rule-of-continuous-delivery-toochains.md) (2017-03-10): Continuous delivery transformations are hard enough in many areas already: changing people's behavior, changing policies, architecting things differently -… - [BBBCTC - Branching, Binaries, Builds, Coding and Testing Culture](https://pipelinedriven.org/article/2015/5/30/bbbctc-branching-binaries-builds-coding-and-testing-culture.md) (2015-05-30): As part of a major drive to push for faster "time to main" you might need to change things in the following five areas (that end up affecting how long… - [Pipeline Disintegration (Post Build Pipeline)](https://pipelinedriven.org/article/2015/3/29/pipeline-disintegration-post-build-pipeline.md) (2015-03-29): Other Names: Post Build Hand off, Time Capsule, Strangler Application Symptoms: It is very complicated to add features to the current build process, because… - [Split and Parallelize](https://pipelinedriven.org/article/2014/10/3/q5b4rm6abtjxeleaj0xfapo1n3khnl.md) (2014-10-03): Other Names: Split Work Symptoms: The build is taking too long Problem: It could be that one or more tasks or build steps that are comprised of many actions… - [Irrelevant Build](https://pipelinedriven.org/article/2014/10/3/-irrelevant-build.md) (2014-10-03): Symptoms: The team doesn't seem to care that one of the builds is failing A build is red, but might not be failing. Only one person from a different team or… - [Build Pattern: Deploy by Proxy - Added to the book](https://pipelinedriven.org/article/2014/7/26/build-pattern-deploy-by-proxy-added-to-the-book.md) (2014-07-26): It's been a couple months of other work. but there's finally new material in the book and more coming soon. First: I've now grouped the chapters under… - [Use Version Aware Build Scripts](https://pipelinedriven.org/article/2014/3/2/use-version-aware-build-scripts.md) (2014-03-02): I am thinking of writing a small booklet about better builds . this blog post might end up as a part of it . Let me know if you’re interested by taking a… - [Avoid XML Facing Build Tools](https://pipelinedriven.org/article/2014/3/2/avoid-xml-facing-build-tools.md) (2014-03-02): Having your build scripts in XML is one of the worst things you can do for maintainability and readability. Treat your build scripts like you treat your… - [Start Products with an Empty Build and Deploy Pipeline](https://pipelinedriven.org/article/2014/3/2/start-products-with-an-empty-build-and-deploy-pipeline.md) (2014-03-02): Builds can be nasty beasts when It’s finally time to organize and do them. There are so many things you did not think about, that it can take days or weeks… - [Rolling Builds and the Plane of Confidence](https://pipelinedriven.org/article/2014/3/2/rolling-builds-and-the-plane-of-confidence.md) (2014-03-02): When I check in code I have been working on, I feel: 1. I want to wait for the build result 2. I don’t want to waste time waiting for the build results,… - [How to make your builds FAST using DRY and SRP Principles](https://pipelinedriven.org/article/2014/3/2/how-to-make-your-builds-fast-using-dry-and-srp-principles.md) (2014-03-02): (this will become part of my upcoming booklet on beautiful builds ) One of the things that kept frustrating me when I was working on various types of build… - [Pattern: Script Injection](https://pipelinedriven.org/article/2014/3/2/pattern-script-injection.md) (2014-03-02): I am slowly realizing that perhaps my concepts for the build book could be better serving as reusable patterns for creating builds, solving specific… - [Pattern: Shipping Skeleton](https://pipelinedriven.org/article/2014/3/2/pattern-shipping-skeleton.md) (2014-03-02): I am slowly realizing that perhaps my concepts for the build book could be better serving as reusable patterns for creating builds, solving specific… - [Build Pattern: Location Agnostic Script](https://pipelinedriven.org/article/2014/3/2/build-pattern-location-agnostic-script.md) (2014-03-02): Symptoms: Running the build script on a machine where it was never run before requires extra pre-work to make the machine ready, such as mapping drives,… - [Release Compilation](https://pipelinedriven.org/article/2014/3/2/release-compilation.md) (2014-03-02): Imagine you could only compile once a week. Once every two weeks. Once a month. Scary? All that code that you write blindly, not knowing whether it even… ### Enterprise DevOps - [Co-Ops: Enabling True Continuous Delivery with Cooperative Pipelines & Processes (Title Feedback Needed)](https://pipelinedriven.org/article/co-ops-enabling-true-continuous-delivery-with-cooperative-pipelines-amp-processes-title-feedback-needed.md) (2019-12-06): “Co-Ops” is a term that one of the attendees at a talk I was giving about cooperative pipelines came up with. I wish I knew his name. I asked him to email… - ["Pipeline Driven" vs. "We have Jenkins Jobs"](https://pipelinedriven.org/article/pipeline-driven-vs-we-have-jenkins-jobs.md) (2019-05-03): High performing and continuously delivering organizations such as Netflix, Amazon, Google, Wix and others don’t just have pipelines. They don’t just… - [Always start with a "Why" when transforming to DevOps.](https://pipelinedriven.org/article/always-start-with-a-why-when-transforming-to-devops.md) (2018-10-21): I’ve been to quite a few DevOps and Agile transformations, big and small. Some just want to up the coverage of their tests (not recommended!), others want… - [Continuous Delivery Values](https://pipelinedriven.org/article/continuous-delivery-values.md) (2017-03-10): I mentioned in my last post about the toolchain needing to respect the continuous delivery values . What are those values? Many of them derive from lean… - [The four most common root causes that slow down enterprise continuous delivery](https://pipelinedriven.org/article/the-four-most-common-root-causes-that-slow-down-enterprise-continuous-delivery.md) (2017-03-09): In my journey as a developer, architect, team leader, CTO, director, coach and consultant for software development, the two most common "anti patterns" I… - [What is Enterprise DevOps?](https://pipelinedriven.org/article/what-is-enterprise-devops.md) (2016-10-29): To understand Enterprise DevOps, we have to first define what "regular" DevOps is. Here is my current definition of DevOps. Let's call it "DevOps 0.1" What… - [Devops is...](https://pipelinedriven.org/article/2015/10/29/devops-is.md) (2015-10-29): Infrastructure & Code Treated as software, where Software Is created with fast feedback cycles, where Cycles Do not require monkey work by people, where… ### Metrics - [A Metrics Framework for Continuous Delivery](https://pipelinedriven.org/article/a-metrics-framework-for-continuous-delivery.md) (2022-07-11): Here’s a framework I like to start with when I discuss what types of metrics can help or harm you in your journey to continuous delivery: Lagging Indicators… - [Ten Devops & Agility Metrics to Check at the Team Level](https://pipelinedriven.org/article/ten-ideas-for-things-you-can-measure-as-a-team-on-your-devops-journey.md) (2018-03-10): When I coach teams that are getting into the DevOps and Continuous Delivery mindset, a common question that comes up is "What should we measure?" Measuring… - [DevOps Metric: Amount of Defects and Issues: definition.](https://pipelinedriven.org/article/devops-metric-amount-of-defects-and-issues-definition.md) (2016-10-29): Important to : VP of QA, Operations Definition : How many bugs or tickets are found in production/ how many recurring bugs in production. As well as - how… - [DevOps Metric: Frequency of Releases Definition](https://pipelinedriven.org/article/devops-metric-frequency-of-releases-definition.md) (2016-10-29): Important to : Release Manager, CIO Definition : How often an official release is deployed to production, in paying customer's hands How to measure : If no… - [DevOps Metric: Mean time to recovery (MTTR) Definition and reasoning](https://pipelinedriven.org/article/devops-metric-mean-time-to-recovery-mttr-definition-and-reasoning.md) (2016-10-29): Important to: VP Operations, CTO Definition :When a production failure occurs, how long does it take to recover from the issue? How to measure: Different… - [DevOps Metric Definition: Mean time to change (MTTC) (vs change lead time)](https://pipelinedriven.org/article/devops-metric-definition-mean-time-to-change-mttc-vs-change-lead-time.md) (2016-10-29): Also called "time from concept to cash". Important to : Everyone, especially the CEO, CIO and CTO. Definition : How long does it take a new average feature,… - [Time to Main - Measuring Continuous Integration](https://pipelinedriven.org/article/2015/5/15/time-to-main-measuring-continuous-integration.md) (2015-05-15): A.k.a "Time to Trunk". How do you measure your progress on the way to continuous integration(we're not even discussing delivery yet)? How do you know you… ### Talks & Events - [Video: Lies, Damned Lies and Metrics](https://pipelinedriven.org/article/video-lies-damned-lies-and-metrics.md) (2022-07-11): I spoke about what types of metrics and KPIs (including OKRs) you can use with continuous delivery, and which ones you might want to stop using, during my… - [Video: The Pipeline Driven Organization - Enabling True Continuous Delivery (Oredev 2019)](https://pipelinedriven.org/article/video-the-pipeline-driven-organization-enabling-true-continuous-delivery-oredev-2019.md) (2019-11-20): Roy Osherove — The Pipeline Driven Organization｜ Øredev 2019 · YouTube - [Continuous Delivery Israel Meetup - First Meeting June 24 2019](https://pipelinedriven.org/article/continuous-delivery-israel-meetup-first-meeting-june-24-2019.md) (2019-06-02): I’m pleased to announce that we will have the first ever meeting Continuous Delivery Israel meetup, in the AWS Floor28 offices in Tel Aviv, June 24th, 2019,… - [Interview on Devops and Cloud Infrastructure](https://pipelinedriven.org/article/2016/1/7/interview-on-devops-and-cloud-infrastructure.md) (2016-01-07): I've had the pleasure of being part of a panel on Devops and various patterns and anti patterns relating to cloud operations and continuous delivery. You… ### Environments - [Ephemeral Environments for DevOps: Why, What and How?](https://pipelinedriven.org/article/ephemeral-environment-why-what-how-and-where.md) (2017-03-30): In the previous post I discussed the issues with having static environments . In this post I cover one of the solutions for those issues: Ephemeral… - [The four biggest issues with having static environments](https://pipelinedriven.org/article/the-four-biggest-issues-with-having-static-environments.md) (2017-03-30): An environment is a set of one or more servers, configured to host the application we are developing, and with the application already installed on them,… - [Parallel Firehose](https://pipelinedriven.org/article/2014/9/22/parallel-firehose.md) (2014-09-22): Pattern: Parallel Firehose Other Names: Build Environment Per Stakeholder Symptoms: You can't deploy a new version of the application to your test or… ### Testing - [Delivery Pipelines and Discovery Pipelines](https://pipelinedriven.org/article/delivery-pipelines-and-discovery-pipelines.md) (2019-02-20): In a previous post I discussed the notion of having two seperate pipelines. I want to revisit this, with a bit more details. When we’re talking about… - [Red Pipelines and Build Whisperers: Getting to a Trustworthy Noise Free Test Pipeline](https://pipelinedriven.org/article/red-builds-and-build-whisperers-trustworthy-noise-free-test-pipelines.md) (2019-02-18): People wonder why it takes such a long time to transform the way a large organization works. Here’s just a small nugget, a small tip of a small iceberg, in… - [A Pipeline Friendly Layered Testing Strategy & Recipe for DEV and QA](https://pipelinedriven.org/article/a-pipeline-friendly-layered-testing-strategy-amp-recipe-for-dev-and-qa.md) (2019-02-09): The key point of running a pipeline is to get feedback which in turn is supposed to provide us with Confidence. That’s because a pipeline is really just a… --- ## About the author **Roy Osherove**: Senior Solutions Architect at AWS working on AI for production and AI-driven development; author of The Art of Unit Testing and Elastic Leadership, with about 20 years of consulting and training on unit testing, TDD, team leadership and continuous delivery. ### Sites - [RobotPaper](https://robotpaper.ai/): This blog: AI patterns and anti-patterns, AI and coding agents, and engineering leadership. - [osherove.com](https://osherove.com): Roy's personal site. Author of The Art of Unit Testing and Elastic Leadership; consulting and training on unit testing, TDD, team leadership and continuous delivery, plus about 20 years of blog posts. - [The Art of Unit Testing](https://www.artofunittesting.com): The Art of Unit Testing, 3rd Edition (Manning): readable, maintainable, trustworthy unit tests, plus TDD workshops in C#, JavaScript, Java and Kotlin. - [5whys.com](https://www.5whys.com): Roy's writing on software team leadership: self-organizing teams, surviving chaos, learning mode and the Elastic Leadership model. - [Pipeline Driven](https://pipelinedriven.org): Continuous delivery and DevOps practices: build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps. - [Elastic Leadership](https://www.elasticleadership.com): Elastic Leadership, Roy's framework for software team leaders (survival, learning and self-organizing modes). ### Social - X: https://x.com/royosherove - LinkedIn: https://www.linkedin.com/in/osherove/ ### Contact - Contact form: https://forms.gle/jQSx5KoXvrVA1JCK7 ### License Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original page). License: https://creativecommons.org/licenses/by/4.0/