How remote company Automattic makes peer reviews work

Automattic, the makers of WordPress.com and WooCommerce, continuously deliver customer satisfaction (CSAT) scores of 95 percent and greater. Yet, 56.6 percent of the company’s customer conversations fell under the “Room for Improvement” category in recent internal assessments.

With a sharp focus on the quality of customer interactions, Automattic conducts regular conversation reviews with Zendesk QA. Grigorij Urasov, Happiness Engineer at Automattic, shares insight into the benefits of doing internal assessments in the form of peer reviews, and how to come to terms with giving negative feedback to one’s teammates.

Advantages of peer reviews

Here’s Urasov’s take on conversation reviews in his own words…

Team Orion [the internal name for my team] has been using Zendesk QA for peer reviews for over a year now. Every week, each member of my team reviews at least 10 interactions of another agent – whom we call Happiness Engineers (HE). We usually choose a new HE for reviews each week.

Since peer reviews can feel intimidating, mostly due to the feeling of being graded, I’d like to point out how I managed to get into the right mindset, and how peer reviews have helped me personally.

When we first started out, peer reviews were mostly positive, but according to the latest Feedback Analysis report in Zendesk QA, 56.6 percent of reviewed tickets fell under “Room for Improvement”. I see this as a good thing indicating that the system is working and HE’s are actually learning a lot from each other.

I will not deep dive into the results of the above feedback analysis, but I’d like to focus on the psychological part of this process.

The responsibility that comes with being a judge of someone else’s interaction might be daunting at first. The one thing that helps me relax when giving or receiving the “Thumbs Down” is the realization that without the “Thumbs Down” rating we might continue to make the same mistakes over and over again.

In my personal opinion, giving and receiving peer reviews is especially important for relatively new Happiness Engineers, just as important as pair ticket and chat sessions and other means of learning. This helps them get to the level of other Happiness Engineers more quickly, provide a more consistent customer experience, and learn about our products.

I felt the lack of peer reviews when I started working on the WooCommerce Mobile queue. This happened simply because there were just two of us working on that queue, meaning that in some cases, neither of us knew what the right answer was. Luckily, most of us work on big queues with a lot of peers and we can (and should) be using a peer review process to constantly improve our performance.

When reviewing tickets of seasoned Happiness Engineers, especially those of Team Ohana (they work with complicated, escalated tickets), I almost always have “Today I Learned” moments. Quite often the only thing left to do is to give 4 “Thumbs Up” and add a #great tag.

Giving negative reviews

It does sometimes happen that you have to give a “Thumbs Down” to 4 or 5 tickets (out of the 10 you’re expected to review) of a particular HE. This isn’t easy because the reviewer cannot predict the reaction of the reviewee. Here’s what I’m striving for when giving negative reviews: I imagine that it’s my own interaction that is being reviewed and try to think if I would feel that the feedback was fair.

I’m trying to avoid phrases like “You should have…” or “this is wrong”. Instead, I usually explain how I would have handled this interaction starting with “I usually…” or “On top of that, I’d add…”. Shifting your mindset from “you are wrong, I am right” to “This is how you handle it, this is how I usually handle it, let’s find the perfect answer” helps a lot.

It isn’t about sugar coating versus being straight to the point. It’s more about realizing that everyone has their own approach. Our task is to level it out for the sake of our customers.

For example, some of us might be giving quick replies without going into too much detail. At the same time, another person could be providing answers that cover all the possible scenarios to all users, including the non-paying ones.

It might feel like the latter did a better job, but that’s not necessarily the case. What if the time spent on that free customer could have been spent on a paying customer who needed urgent help? What if the SLA suffered because of repeated situations like this?

Once I was about to give the fifth “Thumbs Down” in a row to a fellow HE when I realized that our styles are so different that I was simply not seeing the right answers in the replies. After a closer look, I saw that the answers were there: short and to the point. After that, I changed four out of the five “Thumbs Down” to “Thumbs Up”.

It’s best not to assume that you’re 100 percent right when doing peer reviews. You’ll enjoy the process a lot more.