Mystery Shopping Limitations
1: The Retail Industry
Most companies and brands that want to gain insights into their performance and improve the customer service standards have used mystery shopping as a research tool for measurement, evaluation and defining training needs. How a sale, an inquiry, or a simple conversation is handled drives customer satisfaction, loyalty and also the amount of spending, thus is worth to be evaluated.
AQ is a mystery shopping company, and we certainly believe in the many advantages of mystery shopping. However, as with anything, there’s no denying that there are limitations to mystery shopping. Mystery shopping is a great tool to get insights, especially when it is complemented to other research tactics, but today, let’s have look at mystery shopping limitations:
Low Data Volume
Whilst surveys and polls have a sample size that is meaningful, mystery shopping is a one-man job, the smallest of the small samples. Executing a mystery shop takes some time, a shopper needs to be found, assigned, then travel to the shop, execute the shop, and then the mystery shopping company needs to check the quality, follow up with unclarities. This can be costly and results in companies cutting down their mystery visits per store, resulting in fewer data points that can be collected every week or month.
Data Validity
Surveys can be designed to bring results that are as objective as possible, but still, a shopper is not unbiased, has own preferences, habits and shopping behaviour. There are many trained shoppers that understand what customer service should be and how to conduct a mystery shop, however, there are also many shoppers that do not. The mystery shopping industry is fragmented, with not always sufficient quality control. This can skew results and the validity can be questioned as every mystery shopping report becomes questionable. As mystery shopping limitations go, data validity is the least problematic as most companies – AQ included – have strict quality assurance policies in control in order to ensure that clients receive only correct data to work with.
Singular experience
A mystery shopper captures a snapshot of the whole. A moment is captured, and depending on the circumstances, such as staff shortage on an extremely busy day, experiences differ and do not always reflect the regular customer visit. If a mystery shopper had a negative experience in one store, that experience is projected to the whole store and all the employees. Having many data points that would proof so, received from a larger sample set, would make it worth investigation, but with the few data points per shop, this becomes a dangerous assumption.
Segmentation/ Demographics
A mystery shopper has to match a certain profile to provide the best results and to make the interaction credible, depending on criteria, finding shoppers with a specific profile is very difficult. Also then, the mystery shopper is not a real shopper. He or she is trained to look for certain points of attention. Who knows if a real customer notices or is bothered by that dirt pile at the back of that corner the mystery shopper carefully spotted?
So should we even be using mystery shopping?
Mystery shopping offers real data from real experiences of people. In the words of AQ’s Business Development Director for Asia, Andrew Yeoh: “Mystery shopping is not based on opinion, it is based on customer experience.” It certainly is a great tool to get insights into customer experiences that are hard to gain in other ways. However, the data can be biased, thus mystery shopping should not be used as mass data collection tool that stands alone but for testing and improving the customer experience in combination with other research.
“Mystery shopping is not based on opinion, it is based on customer experience.” – Andrew Yeoh, AQ’s Business Development Director, Asia
Being a mystery shopping specialist, we know that there are certain things mystery shopping can’t do, and it’s important for any company to be aware of its capacity. Mystery shopping limitations are a reality, but it’s how you make that reality work that really makes a difference. Companies with experience know their limits and knowing them means they can avoid the pitfalls that come with them.
