Hi, I’m Greg 👋! I’ve written over 100 essays on product, data, and process where I noodle on an idea or a future product direction, often data that exists between systems.
“How did we get here?” is a frequent refrain when considering a bug or some software behavior. As a Product Manager, you hope that the way a product behaves is consistent and related to an overall theory of interaction that matches the way things work elsewhere in the product. But that isn’t always the case. And often when you are reviewing work in progress or reviewing a bug based on long-ago decided behavior and discussion, the details of why that decision was made are missing, forgotten, or just nonexistent.
“Look at the PRD (Product Requirements Document)” is everyone’s default, and this is great if the PRD has been updated with every small decision that has been made in the software process from meetings, to emails, to Slack conversations and JIRA tickets. The reality is more often that the PRD represents our best guess at what the product was going to look like when we were designing it, and may not have been updated to match the way the feature behaves now, in the present.
We were promised flying cars
Everyone knows by now that we don’t have flying cars. At least not ones you can easily buy anyway.
The bigger point here is that a lot of decisions get made between the original process of creating and documenting a feature and the realization of that feature into code, testing, acceptance testing, and customer usage.
When you compare the original specification from the Product Requirements Document to the released product, it should match, even though things have changed, discussions have happened, and the features are not exactly the same. So why is it that QA teams (or Engineers, or Product Managers, or Executives) get surprised by the mismatch of expected features when compared to how things behave when they get released?
What we need: a decision log
When you find an inconsistency between the way things behave and the way you expected a product to work based on reading the requirements, what do you expect? Perfect information that would tell you every reason why the result looks a bit different than the original expectation.
If you had a log of all of the decisions that were made by stakeholders and a short write-up of why these decisions were made and the impact they made on the product, you’d be a long way toward understanding the difference between the product that was planned and the product that was released.
It’s easy – there’s even a Confluence Template you can install! But wait a minute.
Even if you know all of the relevant details for a decision, like:
-
What caused a decision point to occur
-
What was discussed
-
Who were the stakeholders and who ultimately made the decision
-
A summary of the discussion and anything that was added to technical debt to decide later
-
What was changed, and how the behavior of the feature or the product changes as a result
There remains a problem: this is partly a technology issue and partly a process issue. Decision logs get created when people realize that there is a decision that is being made and that they need to preserve the results of these decisions so that other people (later, not in the conversation, without context) could understand why this was done.
If the process is not easy or almost automatic, there’s no way the documentation will happen. The goal of the QA engineer to understand if the PRD is up to date is in peril if the organization doesn’t have a standard for when to update the product documentation in process so that everyone knows that something has changed. One software engineer tried to solve this using JIRA because a decision process seems like a lot of other issues and why not use the tools you have to solve the problems with managing process? But JIRA is a tool that works when people commit to using it. Decision logs don’t write themselves.
An ideal decision log
From the perspective of the people discussing something, an ideal decision log is pretty simple:
-
When: Right after a conversation that has happened (in Slack, in a meeting, in a comment about an issue)
-
What: Document the change and explain why. If there is a change to the original specification, also change the original specification and reference the comment
These simple requirements are partly aligned with the information people need later when they find the change and need to interpret it. From the perspective of people reading about a feature, an ideal decision log needs:
-
Clarity: what changed, who changed it, and why
-
Lineage: was this the first change or was it a subsequent change – how can I link this back to previous discussions?
-
Impact: was this a major or minor change? If we don’t know, what did we think at the time so we can measure the congruence of the impact later
-
Consistency: how do I know that this change has a similar amount of detail as another change I’m likely to read about
-
Context: what else was going on in the organization (or in the group) at the same time this decision was being made
The first three items above can be tackled with policy and process. It’s totally reasonable to expect that when we discuss material changes to features we need to update the product requirements documents. It’s also reasonable to capture some of the conversations happening at the time so that we start to have a historical record of what changed. We probably also know the impact of the decision we’re making, at least in an order of magnitude. Changing the color of an item on a screen is much different than changing a pricing model, updating a data retention policy, or removing a backward-compatible change.
Consistency and context are tricky items to solve with process. Ideally, you’d want metadata around the process to be tracked so that humans didn’t need to capture so much information. You would also want to track decisions over time so that you can understand things like how long do they take to resolve; are the decisions often reversed or never reversed; and whether decisions on certain topics derail or empower the team and produce negative or positive effects.
Here’s a simplifying idea. A lot of our conversations start in Slack or in another messaging tool.
If every time we realized we were making a decision, we:
-
Were able to start a process by making a contextual choice in Slack
-
Automatically open a temporary channel where the decision could be discussed in context
-
When completed, let the system know it was completed and write the results to a table where they could be related to a JIRA ticket, document, calendar item or similar
We would be creating consistency and context for our decisions. Here’s how it might look in Slack:
In this model, a /decision command or similar opens a channel where a few prompts structure the metadata, starting with:
-
Who owns (the person who started the decision item)
-
Who is responsible for deciding (optional, if this is going to be different than the recorder)
-
Who else to invite to this temporary area
-
What’s the summary of the item to be discussed (default to a trimmed-down version of the message where the context was added or set a title)
-
Relate a JIRA ticket (optionally)
When this is done, write this summary as a log to the JIRA ticket and also maintain a history of decisions so that we can create a weekly summary or produce the summary for an individual epic or story in a retrospective. Also, this could give you the prompt to update the PRD, closing the loop.
Having a decision log like this would make it much easier to know what changed, when it changed, why it changed, and what was the commentary around that decision. I’m pretty sure it’s easier to build than a flying car, and it’s going to take some incremental process improvements to implement this decision log as a regular part of our product development. For now, I’m going to start with updating the PRD every time I know I’m in a decision process and it ended.
What’s the takeaway? Remote work can be challenging when decisions are made by team members and not recorded in the places where those team members do their work. Why not make it easier, more transparent, and efficient by building a process to record decisions and prompt updates to needed documents inline with the current process? A decision log can make this easier.
