Support ticket
tagging system
In 2018 I led a team that was responsible for revamping the system we use to tag support tickets at Shopify. This tagging system is a controlled vocabulary that's used to categorize, retrieve, and reference support tickets. The act of categorization maps support data to different product areas; this data is then used by researchers, data scientists, and others to understand what's happening in the Support organization.
Our goal was to create a system that allowed tickets to be easily tagged by our support staff, but also had a reasonable level of granularity when it came to how we categorized our tickets. This second point was important because we wanted the data to be organized in a way that would allow researchers to do as little sifting as possible when looking at tickets from a given product area.
This service design project involved months of interviews with stakeholders across the support organization, to make sure that the needs of all affected teams were taken into consideration. By the end of this project, my team was made up of representatives from data, operations, research, and from front-line support. This cross-functional team was able to prune our support ticket taxonomy down from over 130 tags to under 100, all while maintaining (and in some cases improving on) the tagging accuracy across key ticket topics.
Much of this work is confidential, but I can speak to the design principles that me and my team came up with to govern any future work on the system. These principles are informed by the research and experiences we had through this project:
Tags should be easy for the user to apply to the ticket.
Front-stage staff are the ones who apply tags to tickets, but they're not necessarily the ones who use the data that the system outputs. We should design the system to be easy for frontstage staff to use, so they can focus on delivering exceptional service to customers (in this case, merchants).
Fewer tags in the system is better than more tags in the system.
A concise set of tags leads to more consistency and higher familiarity with the tagging system, from those who input the data (frontstage staff, support agents) and from end users of the system (researchers, data scientists, etc).
Represent each distinct product with a specific tag.
Distinct products should have their own tag. Concepts or thematic ideas should have a tag for the overarching idea, not for the granular concepts that might be associated with them. It's easier for people to agree on distinct ideas than it is for them to agree on theoretical or conceptual ones.
Tags should represent the broadest concepts possible.
It's hard for people to agree on what to call something. It's even harder for them to agree when they're asked what something is about or what happened in a situation (like in eyewitness accounts). It therefore makes the most sense for tags to represent fairly broad concepts, while also ensuring that clear areas of difference are separated into different categories. Striking this balance is a bit of a moving target.
Tags should be named based on the user's expectations and needs.
Tags should be named colloquially; any jargon or technical names should be avoided. This is because customers will use the colloquial name instead of the technical name, and it's easier for frontstage staff to remember these naming conventions.
Tags should be the first level of data filtering, not the final word on categorization.
Researchers should do additional work to get the data they need from the tagging system. Frontstage staff shouldn't be expected to tag based on the researcher's expectations. Tags are the first layer of filtering to give researchers a head start on their research.
The more granular the tag is, the more difficult it is to apply to the ticket.
Granularity increases the closer you get to representing an idea that can't be subdivided into at least two separate ideas.
As granularity increases, it theoretically becomes easier for researchers to find what they're looking for, and more difficult for frontstage staff to "agree" on which tag to apply in a given situation. This increase in difficulty for frontstage staff leads to lower accuracy in tagging across the system, and less trust in the system as a source of research insights. increases the closer you get to representing an idea that can't be subdivided into at least two separate ideas. So granularity is actually the enemy of accuracy, and the friend of precision.
The more broad the tag is, the more time it takes to gather insights from it.
"Broadness" increases as the tag starts to represent many different potential conversations or ideas.
As broadness increases, it becomes more difficult for researchers to find what they're looking for, and easier for frontstage staff to "agree" on which tag to apply in a given situation. This increase in difficulty for researchers leads to longer research time and more sifting. However, the researcher can be more confident that the information they need is collected under one tag (rather than many) and that there will be a higher level of tagging accuracy across the entire tagging system. Broadness is the enemy of precision, and friend of accuracy.
Conclusion
This project was a great example of the power of cross-functional teams being stronger than teams made up of a single discipline. The tagging system we designed was not perfect, but it suited the needs of the Support organization for some time. The experience of managing stakeholders at different levels of impact in the company helped develop my communication skills. Future teams will be set up for success when they choose to build upon or revamp the tagging system at a later date.