Background & Parallel Execution

PowerShell Intermediate/Advanced

Chapter 4 · Background & Parallel Execution

Chapter 3's remoting already ran code somewhere other than your own immediate session — this chapter does the same thing on the local machine, two genuinely different ways. Start-Job is heavyweight but simple: a full separate process per job. ForEach-Object -Parallel is lighter and PowerShell 7+ only — but it introduces a real, easy-to-miss gotcha of its own the moment you try to collect results back into an ordinary shared variable.

Start-Job: True Background Execution, One Process Per Job

$job = Start-Job -ScriptBlock { Start-Sleep -Seconds 5; "Done" } $job # returns immediately — State: Running — your prompt is never blocked

Start-Job launches an entirely separate powershell.exe process to run the script block — genuine isolation, but genuine overhead too: every job started this way costs a full new process, not a lightweight thread.

The Job Lifecycle: Get-Job, Receive-Job & the -Keep Gotcha

Get-Job # check State: Running / Completed / Failed Wait-Job $job | Receive-Job # block until finished, then get the output Remove-Job $job
Receive-Job drains the job's own output buffer by default
Calling Receive-Job a second time on the same job returns nothing, even though the job clearly produced output the first time:
Receive-Job $job # "Done" — and the output buffer is now empty Receive-Job $job # nothing — already consumed once Receive-Job $job -Keep # leaves the output in the buffer, safe to read again later
Add -Keep any time you might genuinely need to read a job's output more than once.

Passing Data In: -ArgumentList (and $using:)

A job's script block runs in its own separate process, exactly like Chapter 3's remote script blocks did — it has no automatic access to local variables either:

# The traditional, always-works approach — param() plus -ArgumentList $job = Start-Job -ScriptBlock { param($n) $n * $n } -ArgumentList 5 # The same $using: scope modifier from Chapter 3 works here too, on modern PowerShell $n = 5 $job = Start-Job -ScriptBlock { $using:n * $using:n }

The Cost of a Job: Why Many Small Jobs Get Slow

Because each Start-Job is a genuinely separate powershell.exe process, starting 100 small jobs means starting 100 separate PowerShell processes — real, measurable startup overhead multiplied by however many jobs you launch. Fine for a handful of independent, possibly long-running tasks; a poor fit for "run this tiny operation 500 times, fast."

ForEach-Object -Parallel: Lighter-Weight Concurrency (PowerShell 7+)

1..10 | ForEach-Object -Parallel { $_ * $_ } -ThrottleLimit 5

Instead of a full process per item, each parallel iteration runs in its own lightweight runspace — much cheaper to create than a whole new process, which is why -Parallel scales to hundreds of small items far better than an equivalent pile of Start-Job calls. -ThrottleLimit caps how many run concurrently — a much lower default (5) than remoting's own default of 32, since these compete for local CPU rather than remote machines' resources.

Exactly like Start-Job and Chapter 3's remoting, a local variable isn't automatically visible inside the parallel block — $using: is required here too:

$multiplier = 3 1..5 | ForEach-Object -Parallel { $_ * $using:multiplier }

The Isolated-Runspace Gotcha: Why a Shared Variable Doesn't Update

$results = @() 1..5 | ForEach-Object -Parallel { $results += $_ } $results.Count # 0 — completely empty, even though it clearly ran five times
The central fact this chapter is built on
Every -Parallel iteration runs in its own genuinely isolated runspace — the same kind of scope separation Chapter 3's remoting had, just local instead of across a network. $results inside each iteration isn't the $results from the caller's own scope; without $using:, it's simply undefined inside that isolated runspace, so $results += $_ creates a brand-new, purely local variable in each of the five separate runspaces — five short-lived copies, none of which is ever the original, and all five are discarded the moment their own iteration ends.

The fix requires a genuinely thread-safe collection, not a plain array — one built to handle multiple runspaces writing to it safely at the same time:

$results = [System.Collections.Concurrent.ConcurrentBag[object]]::new() 1..5 | ForEach-Object -Parallel { ($using:results).Add($_ * $_) } $results.Count # 5 — correct, because a ConcurrentBag is genuinely safe to share and write across runspaces

A Brief Word on Runspaces Directly

ForEach-Object -Parallel is itself built on PowerShell's own Runspace API — a runspace pool it manages for you behind the scenes. Managing raw runspaces directly ([runspacefactory]::CreateRunspacePool() and hand-rolling your own thread-safe result collection and synchronization) is real, substantially lower-level territory, genuinely useful for high-performance custom tooling but well beyond what this course builds from scratch — an honest boundary, not a gap to quietly paper over.

Isolation unitTypical use
Start-JobA full separate process — heaviest overhead, strongest isolationA handful of independent, possibly long-running tasks
ForEach-Object -ParallelA lightweight runspace — much cheaper than a process (PowerShell 7+ only)Many short, similar parallel iterations
Raw RunspacesManually managed — most control, most complexityHigh-performance custom tooling; rarely needed directly
A first practical habit
Before reaching for either kind of parallel execution, ask whether the work is genuinely independent — if one iteration's result depends on another's, parallelizing it correctly gets a lot harder than either -ArgumentList or $using: alone can solve.

Hands-On Exercises

Exercise 1

Explain why calling Receive-Job $job twice in a row returns real output the first time and nothing the second time. What parameter fixes it?

📄 View solution
Exercise 2

Explain why $results.Count is 0 after 1..5 | ForEach-Object -Parallel { $results += $_ }, using this chapter's own isolated-runspace explanation. What's the actual fix?

📄 View solution
Exercise 3

You need to process 200 small, independent items in parallel as fast as possible on PowerShell 7. Would you reach for Start-Job or ForEach-Object -Parallel? Justify your answer using this chapter's own weight/overhead comparison.

📄 View solution

Chapter 4 Quick Reference

  • Start-Job — a full separate process per job; heaviest overhead, strongest isolation
  • Receive-Job — drains the job's output buffer by default; use -Keep to read it more than once
  • -ArgumentList / $using: — how to pass local data into a job or parallel block's otherwise-empty scope
  • ForEach-Object -Parallel (PowerShell 7+) — runs each iteration in a lightweight runspace, not a full process; -ThrottleLimit (default 5) caps concurrency
  • Isolated runspaces — a plain shared variable modified inside -Parallel never updates the caller's copy; each iteration gets its own, discarded afterward
  • [System.Collections.Concurrent.ConcurrentBag[object]]::new() — a genuinely thread-safe collection, the real fix for collecting results out of -Parallel
  • Raw Runspaces — what -Parallel is built on; real, advanced territory, an honest boundary for this course
  • Next chapter: Working with .NET Objects & COM Directly