Motivation

Building new systems is extremely energizing. You're making a thousand decisions an hour, unconstrained by history or precedent.

Over time, as software begins to mature, it grows in sometimes strange ways -- this is inevitable and good. Those weird cases and constraints represent early ideas colliding with the real world; we learned something, and the system improved.

Modifying large systems can be intimidating, especially for engineers who weren't "in the room" when each of those decisions occured. At the limit, this will grow to include everyone, as our team expands and as pieces of our product are iterated, decommissioned, rewritten, and repurposed.

When we fear modifying our systems, progress slows as we retreat into overly-narrow incrementalism — and large changes take longer than necessary. One method at our disposal to reduce the fear of making changes is appropriate test coverage. Another method is our realistic local development system. This doc introduces a third method, Shepherds. Many of us already do this in some form, but this adds a name and an official recommendation to promote the practice.

So... what's a shepherd?

A shepherd is an experienced guide capable of nudging complex or cross-cutting efforts toward timely, high-quality completion. Shepherds can be formal or informal. Shepherds are a valuable resource that can be called on to unblock, demystify, advise, simplify, or otherwise expedite initiatives that may otherwise be risky. With the help of a shepherd, team members who are new to an area or skill can take on stretch assignments to accelerate growth more easily.

Responsibilities of a shepherd

1. Possess accurate and complete knowledge about most of the system, including high-level intent, past and future plans for changes, and implementation details, paired with the ability to communicate that knowledge effectively.
2. Get involved early enough to provide thought partnership.
3. Be responsive to the task/initiative lead.
4. Provide thorough, timely, and constructive reviews and advice.
5. Help your teammates level-up. This may be done by passing domain knowledge, improving docs and generally thinking of ways of helping your teammates, so that if they haven’t already, they may shepherd in the future.

Who should shepherd?

Examples:

• You largely wrote the subsystem in question.
• You recently shepherded or led a large related change.
• Your expertise is frequently requested when things go sideways.

Who should use shepherds?

• Everyone, from time to time! Although shepherds can be super helpful when touching unfamiliar components, even experts can benefit from a second pair of eyes and a pre-wired high-context reviewer.

When do I need a shepherd?

(Non-exhaustive)

• When writing a design doc... most of the time a co-author or shepherd can vastly improve initial draft quality by simply asking relevant questions early on in the process. This early alignment can speed up review cycles and iteration, particularly compared to a scenario where a large doc appears "out of the blue".
• Prior to embarking on a large product change, seeking advice and guidance from someone with a lot of knowledge about the user impact, history of and in-progress planned changes to involved components can greatly accelerate designs.

How does it work in practice?

Recognize the need

• You might realize the thing you’re working on could benefit from a shepherd.
• Alternately, someone else (anyone, really — but possibly your manager) might suggest that your current scope of work could benefit from having a shepherd lined up.

The ask

• The first step is always to ask someone if they’re willing to step in as a shepherd.
• The ask should come from the person doing the work.
• The ask should happen early in the process, ideally before writing code or a design doc (or as early as possible.)
• The ask is truly a request, it takes time and effort to be a good shepherd, so you might have to ask a couple people before finding someone with enough knowledge and bandwidth to help out. That said, shepherding is very high leverage so folks in the position to do it well should try to make time.

Working with a shepherd

• Shepherd-ship can be formal (e.g. tagged as a reviewer on a design doc ahead of time) or informal (pinged in slack and handshake agreement to help out.)
• Tag the shepherd in issue comments, PRs, design doc reviews, etc. as needed.
• Consult the shepherd on approaches to problem solving, how to break up the work into increments, getting unstuck, etc.