Steps to Reproduce

June 29, 2022

“Intent is the motivation or purpose behind our actions.” But sometimes listening solely to the supposed intention of a user does not paint an accurate picture of what happened and the specific actions the user took. Science proves that, depending on the emotional impact of an event (e.g.frustration at a broken bug), we might remember a set of steps differently. We can fall victim to our emotions and potentially tell a uniquely different story.

Every small detail is crucial when working with developers to solve a problem. Building trust at its simplest form means documenting a clear understanding of the actions you took to run into a specific issue.

Unfortunately for all of us engineers, users don’t actually think about and document each of the steps they take; they just think about their end goal. To make matters more friction-filled, attempting to walk back through the steps exactly to produce a desired result (encountering the issue again) can prove fickle and frustrating for anyone.

With this in mind, we have built a form factor that tells the true story, and allows actual actions to speak louder than words.

A journey of user actions

Capturing user behavior each step of the way

At our foundation, PlayerZero believes that data is something captured with specific intent, it’s not just a commodity to collect by default. In this vein, we built our product similar to what you would see in a Blackbox flight recorder; it continuously captures the steps a user is taking and stores them locally by default (rather than sending them to the cloud automatically). The moment an issue happens, the user passes-off their session information to our servers, giving our users the permission to look back in time, source the precise breakdown of steps taken, and reproduce the issue.

In the PlayerZero report, a step-by-step breakdown of the actions taken during a user journey is presented to the receiver of the issue. Every “click”, “tab”, “fill”, etc is captured; just like a recipe, each and every ingredient and amount is outlined for the developer to follow along. To bring these steps ever further to life, accompanying each step is a visual representation (DOM snapshot) that shows the browser at that specific moment in time.

Steps to reproduce in a PlayerZero report

Designing for readability and intent

Being strategic for how our users make the most out of steps in the shortest amount of time possible was top of our priority when building this feature. 

Clustered by URL

Now that you have steps to reproduce the issue in question, you can start to put together a mental model for what the user was doing. To make it even easier for developers and product team members to quickly understand and communicate about issues in an application, we’ve grouped the steps by page URL.

Spotlights of a specific step in the Activity Feed

Through our users engaging with PlayerZero reports, we realized that steps can get a bit unruly, especially for sessions that run hours. This is why we’ve created a way to tie a specific step to the marker in the Activity Feed; the moment an issue is reported, the step that prompted the report is logged.

Intelligently grouping by step specific intent

To further refine the digestibility of the user journey, we turned our focus to how we defined the steps themselves. Communicating highly specific & detailed steps is a difficult game, but we believe we found a perfect balance point between being overly verbose and being detailed. Take for example, these two unique instances:

1: If you fill the email field and then click sign-in, we create two different steps. 

  • Thinking about the potential variety of options in a form, it is not clear on our end what is encapsulated in the submission process. Maybe it’s just email and password, or maybe it also includes name, date, favorite type of ice cream. We have no way of knowing what to batch together.

2: You interact with a search bar by clicking on it first, type a few letters, key-down a few times and then hit enter. 

  • Semantically, we would consider that to be a single step (searched for something) despite the multiple associated actions. All of these actions happen within a very specific set of barriers, so we can be confident as to the relationship between each action. Therefore, in our session report, you will see these actions grouped as “Click, Fill, ArrowDown on Search” as a single action. It’s a nice balance between verbosity and grouping. 

Using visuals to address existing labeling deficiencies

An additional key challenge we faced when meaningfully defining steps was that many buttons on an application aren’t clearly labeled. Think about the hamburger icon you click to open a menu, or the trash icon you click to delete something; we visually know what they are, but can’t communicate it programatically. It was clear that we needed to stretch outside of text-based indicators into visual queues. This is where the DOM snapshots come in. 

As you hover over the steps, the DOM snapshots show up right next to the step to show you exactly where the cursor was at the moment that step occurred. It gives a visual continuity in the case where words are not enough to communicate steps. 

Conclusion:

Outlining clear steps for reproduction is an efficient way to help anyone within an organization understand the user’s journey. Leaning into the sentiment that debugging is a mixture of science and art, we see the science side shine through when it comes to reproducing a specific journey. Intentions are less important than the actual actions that define them. Understanding and replicating the experience is vital for diagnosing the problem. 

Written by:
Matt Kasner
Additional Articles

Steps to Reproduce

June 29, 2022

“Intent is the motivation or purpose behind our actions.” But sometimes listening solely to the supposed intention of a user does not paint an accurate picture of what happened and the specific actions the user took. Science proves that, depending on the emotional impact of an event (e.g.frustration at a broken bug), we might remember a set of steps differently. We can fall victim to our emotions and potentially tell a uniquely different story.

Every small detail is crucial when working with developers to solve a problem. Building trust at its simplest form means documenting a clear understanding of the actions you took to run into a specific issue.

Unfortunately for all of us engineers, users don’t actually think about and document each of the steps they take; they just think about their end goal. To make matters more friction-filled, attempting to walk back through the steps exactly to produce a desired result (encountering the issue again) can prove fickle and frustrating for anyone.

With this in mind, we have built a form factor that tells the true story, and allows actual actions to speak louder than words.

A journey of user actions

Capturing user behavior each step of the way

At our foundation, PlayerZero believes that data is something captured with specific intent, it’s not just a commodity to collect by default. In this vein, we built our product similar to what you would see in a Blackbox flight recorder; it continuously captures the steps a user is taking and stores them locally by default (rather than sending them to the cloud automatically). The moment an issue happens, the user passes-off their session information to our servers, giving our users the permission to look back in time, source the precise breakdown of steps taken, and reproduce the issue.

In the PlayerZero report, a step-by-step breakdown of the actions taken during a user journey is presented to the receiver of the issue. Every “click”, “tab”, “fill”, etc is captured; just like a recipe, each and every ingredient and amount is outlined for the developer to follow along. To bring these steps ever further to life, accompanying each step is a visual representation (DOM snapshot) that shows the browser at that specific moment in time.

Steps to reproduce in a PlayerZero report

Designing for readability and intent

Being strategic for how our users make the most out of steps in the shortest amount of time possible was top of our priority when building this feature. 

Clustered by URL

Now that you have steps to reproduce the issue in question, you can start to put together a mental model for what the user was doing. To make it even easier for developers and product team members to quickly understand and communicate about issues in an application, we’ve grouped the steps by page URL.

Spotlights of a specific step in the Activity Feed

Through our users engaging with PlayerZero reports, we realized that steps can get a bit unruly, especially for sessions that run hours. This is why we’ve created a way to tie a specific step to the marker in the Activity Feed; the moment an issue is reported, the step that prompted the report is logged.

Intelligently grouping by step specific intent

To further refine the digestibility of the user journey, we turned our focus to how we defined the steps themselves. Communicating highly specific & detailed steps is a difficult game, but we believe we found a perfect balance point between being overly verbose and being detailed. Take for example, these two unique instances:

1: If you fill the email field and then click sign-in, we create two different steps. 

2: You interact with a search bar by clicking on it first, type a few letters, key-down a few times and then hit enter. 

Using visuals to address existing labeling deficiencies

An additional key challenge we faced when meaningfully defining steps was that many buttons on an application aren’t clearly labeled. Think about the hamburger icon you click to open a menu, or the trash icon you click to delete something; we visually know what they are, but can’t communicate it programatically. It was clear that we needed to stretch outside of text-based indicators into visual queues. This is where the DOM snapshots come in. 

As you hover over the steps, the DOM snapshots show up right next to the step to show you exactly where the cursor was at the moment that step occurred. It gives a visual continuity in the case where words are not enough to communicate steps. 

Conclusion:

Outlining clear steps for reproduction is an efficient way to help anyone within an organization understand the user’s journey. Leaning into the sentiment that debugging is a mixture of science and art, we see the science side shine through when it comes to reproducing a specific journey. Intentions are less important than the actual actions that define them. Understanding and replicating the experience is vital for diagnosing the problem. 

Follow us on Twitter
Sign up for our newsletter.
Get the latest articles delivered straight to your inbox.
Share
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Additional Articles