--- title: "Parallel Firehose" url: "https://pipelinedriven.org/article/2014/9/22/parallel-firehose.md" canonical: "https://pipelinedriven.org/article/2014/9/22/parallel-firehose" published: "2014-09-22T05:02:21.704000+00:00" updated: "2014-09-22T05:02:22.280000+00:00" tags: [] author: "Roy Osherove" description: "Pattern: Parallel Firehose Other Names: Build Environment Per Stakeholder Symptoms: You can't deploy a new version of the application to your test or…" usage: "Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original page)." license: "https://creativecommons.org/licenses/by/4.0/" --- # Parallel Firehose 22 September 2014 · 2 min read · Roy Osherove ## Pattern: Parallel Firehose ### Other Names: - Build Environment Per Stakeholder ## Symptoms: You can't deploy a new version of the application to your test or staging environment because it's in use by other people in the organization such as the QA team, Product owners, or customers. So you can only deploy during the night time, or some other constraint. ## Problem: Developers are sharing the same environment (machines, physical or virtual) with other stakeholders. ## Forces: - You want to do continuous delivery, without needing to wait for others. - Other stakeholders want to use or test the application without needing to stop, or be annoyed by your deployments making the application slower or crashing. ## Solution: Create a separate environment for your other stakeholders, that they can use without you interfering with it, and they can pull a new version to at will. ## Example: Assuming you have the following environment: - A Build Machine that runs TeamCity or Jenkins - A Build Agent Machine (Build1) that runs a TeamCity agent, or a Jenkins Worker - A Test Machine (Test1) where BUILD1 deploys your application to for demo purposes and for use by QA - A developer machine (DEV1) that keeps checking in code to the main branch You want to do continuous testing by running unit and integration tests on check in, as well as smoke tests and acceptance tests by deploying the application automatically on every check in. You can't because QA are using the machine during work hours. You can only deploy and run those tests at night, and they might actually break whatever nightly tests the QA folks are running. Here is one way to solve this: Setup the environment with new machines so you end up with an environment like this: - A Build Machine (BUILDMASTER) that runs TeamCity or Jenkins - A Build Agent Machine (Build1) that runs a TeamCity agent, or a Jenkins Worker - _A Build Agent Machine (Build2) that runs a TeamCity agent, or a Jenkins Worker_ - A Test Machine (Test1) that an agent deploys your application to for dev use - _A Test Machine (Test2) that an agent deploys your application to for QA use_ - A developer machine (DEV1) that keeps checking in code to the main branch Then, in your CI server setup the following two workflows: ### workflow 1: Fully automated Pipeline - Triggered on Check in to main branch - compile on a free agent - Run unit tests on a free agent - Run Integration tests on a free agent - Deploy to TEST1 on a free agent - Run acceptance tests only on TEST1 ### workflow 2: Pull Based QA Pipeline - Triggered by QA clicking on a button, or on a set schedule, or both - compile using a free agent - Run unit tests using a free agent - Run Integration tests using a free agent - Deploy to TEST2 using a free agent - run QA tests on TEST2 ## Summary With this approach nobody is stepping on anyone's toes: - Developers can do continuous tests on TEST1 dozens of times a day (yes, some teams actually achieve this). A.k.a it's so fast it's like "drinking from a fire hose" (hence the pattern name) - QA get a nice stable environment they can control and pull versions into. You can repeat this process for new stakeholders. If you're using cloud services, it's even easier to setup such new environments. --- ## About the author **Roy Osherove**: Senior Solutions Architect at AWS working on AI for production and AI-driven development; author of The Art of Unit Testing and Elastic Leadership, with about 20 years of consulting and training on unit testing, TDD, team leadership and continuous delivery. ### Sites - [RobotPaper](https://robotpaper.ai/): This blog: AI patterns and anti-patterns, AI and coding agents, and engineering leadership. - [osherove.com](https://osherove.com): Roy's personal site. Author of The Art of Unit Testing and Elastic Leadership; consulting and training on unit testing, TDD, team leadership and continuous delivery, plus about 20 years of blog posts. - [The Art of Unit Testing](https://www.artofunittesting.com): The Art of Unit Testing, 3rd Edition (Manning): readable, maintainable, trustworthy unit tests, plus TDD workshops in C#, JavaScript, Java and Kotlin. - [5whys.com](https://www.5whys.com): Roy's writing on software team leadership: self-organizing teams, surviving chaos, learning mode and the Elastic Leadership model. - [Pipeline Driven](https://pipelinedriven.org): Continuous delivery and DevOps practices: build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps. - [Elastic Leadership](https://www.elasticleadership.com): Elastic Leadership, Roy's framework for software team leaders (survival, learning and self-organizing modes). ### Social - X: https://x.com/royosherove - LinkedIn: https://www.linkedin.com/in/osherove/ ### Contact - Contact form: https://forms.gle/jQSx5KoXvrVA1JCK7 ### License Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original page). License: https://creativecommons.org/licenses/by/4.0/