Channels and Producer-Consumer Pipelines
Build bounded asynchronous pipelines with Channel, apply backpressure, coordinate completion, and preserve failures across producer and consumer stages.
Before this lesson
Design bounded producer-consumer flows
Coordinate channel completion
Propagate cancellation and stage failures
The short answer
Channel<T> provides an asynchronous queue between producers and consumers. A bounded channel applies backpressure when consumers fall behind. Complete the writer exactly once, let consumers drain remaining items, and propagate stage failures to the pipeline owner.
Build the runtime mental model
A channel separates the rate and lifetime of producing work from consuming it. Writers await capacity in a bounded channel, which prevents unbounded memory growth. Readers can await data without polling or blocking a thread.
Advanced C# work improves when you separate language syntax, runtime behavior, and application policy. Write down which layer owns the guarantee in this lesson. Then identify the observable evidence—a compiler rejection, test result, generated query, trace, or measurement—that would prove the model correct.
Design the boundary deliberately
Choose capacity from measured burst tolerance and processing cost. Single-reader or single-writer options can enable optimizations when true. The pipeline owner starts stages, observes all tasks, cancels siblings on failure, and decides the final outcome.
The starter isolates one part of the mental model so it can run in the browser. The exercise moves the same rule into a current local .NET project where packages, framework hosting, diagnostics, and multi-file tests are available.
using System;
using System.Collections.Generic;
class Program
{
static void Main()
{
Queue<int> work = new Queue<int>();
work.Enqueue(3);
work.Enqueue(5);
while (work.Count > 0) Console.WriteLine(work.Dequeue() * 2);
}
}Expected output
6 10
Diagnose failure and misuse
An unbounded queue converts slow consumers into memory pressure. Forgetting TryComplete leaves readers waiting indefinitely. Completing on the first producer while other producers still write causes lost work and channel-closed failures.
Classify each failure as a contract violation, transient operational failure, permanent dependency response, concurrency conflict, or programmer defect. That classification determines whether to reject, retry, compensate, cancel, or fail fast. A generic catch-and-continue policy destroys the information needed to make that decision.
| Question | Evidence to inspect | Decision |
|---|---|---|
| Is the input valid? | Validation result and boundary examples | Reject with a stable contract |
| Is the failure transient? | Typed status, exception, and policy context | Retry only when bounded and safe |
| Is state still consistent? | Invariant and transaction outcome | Commit, compensate, or abort |
| Is performance acceptable? | Representative latency and allocation data | Keep simple or optimize one cause |
Apply the concept in production
Track queue depth, enqueue wait, processing latency, throughput, and failures. Define whether shutdown drains queued work or abandons it, and test full-capacity cancellation plus consumer failure.
Finish by making the result operable. Add structured diagnostics at the boundary, propagate cancellation, avoid sensitive data, and record SDK and dependency versions. Test the public behavior instead of private implementation details. If a framework or provider performs translation, serialization, concurrency, or I/O, include at least one test against the real production technology.
A senior-level review should be able to answer four questions: what contract is promised, who owns lifetime and cleanup, how failures become visible, and what evidence supports the design. If any answer depends on “the framework probably handles it,” inspect the documentation or runtime behavior and turn the assumption into a checked decision.
Quick knowledge check
Answer before you reveal.
01What problem does a bounded channel solve?
It makes producers wait when capacity is full, limiting memory and applying backpressure when consumers cannot keep up.
02What must happen before adding complexity to this design?
State the requirement, preserve a correct baseline, collect evidence, and explain how the proposed mechanism improves a specific quality.
Exercise
Practice challenge
Create a bounded Channel pipeline that parses, validates, and stores messages with backpressure, cancellation, completion, and failure metrics.
Requirements
- The implementation states its contract and ownership boundary explicitly
- Automated checks cover the successful path and at least two meaningful failures
- Diagnostics expose failure context without secrets or swallowed exceptions
- The project documents required SDK, packages, setup, run, and test commands
Optional extension: Measure or load-test the critical path and record whether the evidence justifies another optimization or abstraction.
Open in C# compilerLesson checkpoint
One small step locks it in
Mark this lesson complete, then keep the momentum going.
Clear up the details
Frequently asked questions
When should I use channels and producer-consumer pipelines?
Use it when its explicit tradeoff solves a measured requirement or clarifies an owned boundary. Keep the simpler design when the additional mechanism does not improve correctness, operability, or changeability.
Does the browser compiler cover the complete production setup?
No. It runs the focused starter program. Framework, package, database, benchmark, and multi-project work requires a current local .NET SDK and the project commands described in the exercise.
What evidence should I keep after the exercise?
Keep the acceptance cases, automated tests, diagnostic or benchmark output where relevant, and a short decision note describing the chosen boundary and rejected alternative.