Capturing User Feedback – One thing people do when an app doesn’t seem to work

August 2nd, 2020 · 5 min read

Do you struggle to know what is going on when something went wrong within your app? Do you get feedback from your users about an issue, but it's hard to figure out what happened exactly? I'm not talking about crashes – more like a white screen, getting stuck, or something did not load.

If you can answer one of these with yes, I might have something interesting for you. In this article, I share the learnings on how we managed to develop an effective way of capturing user feedback when something went wrong.

For the past three years, I've been working on an mPOS app used by thousands of store associates at Decathlon, Burton or UNTUCKit. Regularly we received feedback that they had some issues during the day. The usual process for them was to tell one of their store managers that something didn't work. She would then send our support team an email with some details about what happened. The team would follow up to learn more about it. Often we needed to communicate back and forth with the customer to help us reproduce the issue. Sometimes the customer made recordings or screenshots that helped a lot.

This process had many issues, and below there is a list of problems we identified:

• Problems occur at different frequencies but do not get reported immediately. The associate has to talk to someone to be able to give the support team feedback.
• There is no exact reference point for the support team.
• Often there are no logs from the client (something that happened on associates' phone).
• Unnecessary interaction with additional people.
• Due to missing technical details, the cause of the problem is often more based on assumptions.
• A lot of conversation in the Jira-ticket to get all the details of what happened.

Obviously, it's not a very effective way. A lot of investment in time and communication is needed, even identifying the issue. With this circumstance and result mind, we wanted to improve the user experience and change the outcome.

How can we make this better?

First, we set our appetite. We were looking for a solution that can be designed and developed within a sprint or two (2-4 weeks). With this time constraint, we came up with the idea of allowing associates to send us feedback directly through the app. The following user story was added to our backlog.

As an associate, I want to send NewStore feedback when I have an issue with the app to let them know what happened without involving someone else.

We scoped the work into two parts: First, come up with a user flow to capture the feedback, and second find a place to store it and make it available for support. The user flow we designed provided two ways to tell us what happened – a Report an Issue button and Shake to Report. For storing the feedback, the engineers suggested using Sentry to track the issue since it was already in place to identify crashes. At this point, both options for the user flow sounded like a good idea to me. Whenever something doesn't work, they would shake the phone and tell us.

This first internal version did not make us happy. The shaking and the report button required some training, and after having the developed version in our hands, we realized how sensitive the phone reacts to shaking. Often the dialog would appear to report an issue, but there was no problem. That can be pretty annoying. Mhhh... how can we make it better?

While trying to figure that out, we could already store the feedback in Sentry. Every report provides a broad set of information to understand what has happened, such as a breadcrumb with the UI interactions or data that got sent to the back end. Additionally, some metadata like the phone model, iOS version, and app version. In the two screenshots below you can see an overview of feedback reports and an example.

An engineer in the team suggested using the new form also for crashes to provide more insights – I liked the idea! That addition required a change in the user flow since we wanted to ask the user first with a prompt if she even wants to provide feedback. Improving this helped us to find a better way of capturing the user's feedback.

Ask for feedback when an app gets closed and reopened.

When you observe users what they do when an app has a problem, you can recognize a typical behavior: "It looks like the app has a problem. Let me quickly close and reopen it."

This is it 🤩

When the associate is closing the app and immediately reopens, we assume there was an issue, and ask for feedback. The idea for this came from an engineer on the team. After making sure this is actually feasible with iOS, we moved on with the implementation.

The image below shows you the final user flow we ended up shipping.

In addition, I added a little video where I'm browsing the catalog and recognize that there are no items in the release category. Let's assume this is not how it should be and restart the app.

This way of capturing the feedback turned out to be quite useful, and we could observe the following results:

• Problems get reported immediately
• There are exact reference points, logs and technical details available for the support team
• No additional interaction with other people required
• We reduced the time to investigate on the customers and support side

On the flip side, we see a lot of messages coming that need more clarification. For example, "white screen", "not connecting" or similar. We believe that improving the form by guiding the associate when describing the problem will result in more insightful reports.

Besides helping our mobile app team and support with this, it also helps me as a designer. Every Monday morning, I'm checking the latest feedback that came in, categorize it, and try to understand our user's behavior. Where do we see the most issues, where does it make sense to follow up, and so on.

I hope you enjoyed my little story and there was something that you can take away from it. If you have questions or feedback, feel free to send an email to witzel@hey.com – happy to hear from you.

Interested in more?

Follow me on Twitter if you are interested in my experiences about UX design and product management.

Cheers Henning 👋