How wholistic research approach helped airline company to boost customer satisfaction in the skies and to bury high research budgets under the ground despite higher sample sizes.
Pegasus, the largest private airline company of Turkey, had been carrying out a Customer Satisfaction Research between 2011-2016 using Computer Assisted Telephone Interview (CATI) with a different research agency. Business issue of Pegasus was that the research didn’t yield an efficient and sufficient insight necessary for action. As of 2016, Pegasus contacted Yöntem Research Consultancy to find a solution for the issue with a new, efficient and effective research design.
Yöntem’s team evaluated the business problem with a 360° approach, defining the issues and improvement areas for each phase and then developing unique solutions for each issue. The outcome was “priceless!”:
- Sample size increased by 21 times, from a monthly average of n=245 to n=5.178
- Monthly standard error decreased from ±6,26% to ±1,36% while quarterly standard error was lowered to ±0,79%, a really difficult-to-reach level below ±1,00%
- Research expenditures decreased by 80%
- Statistically significant improvements were achieved in almost all customer satisfaction attributes.
All these brought additional advantages far beyond business objectives such as not only a more detailed process, period and geography-based analysis but also analysis based on a much more specific criteria such as flight route, flight number, arrival airport etc.
As a final reward, this research model received the Golden Owl award in Satisfaction Research category in 7th Owl Awards of the Turkish Researchers’ Association in 2019.
Let’s look at the challenges we were facing:
1. Changes in Legal Regulations:
- Loss of efficiency in customer lists provided to the previous research agency due to General Data Protection Regulation (GDPR) taking effect on March 24th 2016.
- Related bias in representing the universe.
2. Research Design:
- Lack of cost-effectiveness as considered the CATI method and sample size, was preventing detailed process-based or attribute-based analysis and applying advanced statistical analysis.
- Due to changes in legal regulations mentioned above, customer lists shrank down, and significant ineffectiveness was experienced, which then caused much smaller sample sizes in each wave as well as related bias.
3. Scope of Customer Satisfaction Measurement:
- As the sample was selected from a general customer portfolio, and not based on specific flights, the collected data didn’t reflect process-based experiences, but only perceptual evaluations. Therefore, a real customer experience gauge was not achieved.
- Foreign customers, as a strategically important target group constituting 10,4% of the total customer universe, were not included in the sample due to barriers in achieving English CATI interviews, and therefore the whole target group couldn’t be represented.
- As such, customer satisfaction couldn’t be seen as a 360° picture and satisfaction measurements couldn’t be based on a dynamic, sustainable and improvable structure. Therefore, insights that all Pegasus departments could benefit from, and use the as a base for strategic decisions, couldn’t be gathered.
What did we do then?
We decided to start with a holistic approach and strategy:
- With a desk research, Yöntem clarified legal barriers in line with research objectives
- Yöntem research team worked on switching to alternative interviewing methods to overcome legal barriers while also improving other aspects such as research budget on client-side, sample size and thus more detailed analysis and insights. We concluded that Computer Assisted Web Interview (CAWI) would be the most cost-effective solution which could also increase the sample size and facilitate further detailed statistical analysis, not based on perceptions but real customer experiences.
- Yöntem IT team worked on technical aspects of the CAWI method, again in line with new legal regulations, and developed a programme with encryption technique. The goal was to establish an e-mailing system where customers’ information was not transferred to Yöntem, but where e-mails were sent to the customers directly by Pegasus, with the guarantee of an anonymous feedback collection. In the questionnaire introduction, the respondents were informed that the third party involved is GDPR compliant. So, we managed to solve the issue without violating any privacy regulations and codes, and bias in representation of the universe was minimised significantly.
- To apply the changes, we organized a collective workshop where the social media agency, digital direct mailing agency, Pegasus CRM team and IT teams of both Pegasus and Yöntem worked collectively and accomplished the technical infrastructure of the new research design.
- Yöntem also suggested to Pegasus to launch the new questionnaire in rotation with the old one to understand and manage the effect of the changes on findings for a certain period.
Achievements with the new approach
The benefits of the new research model were as follows:
- A wholistic “Customer Satisfaction Tracking Programme”, covering all processes and facilitating the analyse of the research findings together with the data provided by Pegasus, was formed.
- By shifting to CAWI, it became possible to gauge satisfaction in all phases of the “customer journey”, and a real customer experience measurement was possible with real time instant satisfaction measurement for each process, instead of a rough, perception-based satisfaction measurement.
- E-mails were sent to all customers who had a flight with Pegasus 4 days before. Flight date, flight number, departure and destination airports and if a registered customer, customer-specific CRM-IDs were embedded in the e-mails. Hence, without a need for sample selection, it was possible to send survey links to all customers.
- As CAWI provides the advantage of completing the survey in the most appropriate time for the respondent, response rates and participation level increased, and this made it possible to collect more detailed information and thus more detailed, specific process-based analysis.
- Strong and weak aspects of the brand were analyzed real-time and dynamically on process, period and geography basis.
- In the following periods, satisfaction results were not only analyzed together with weekly delay reports received from Pegasus on a demographic basis, but also other variable bases such as flight route, flight number, arrival airport etc.
- The questionnaire was revisited and improved with open-ended questions to collect instant customer experience evaluations, expectations and complaints in more detailed ways.
- Correlation and Confirmatory Factor Analysis were applied to past data, and the questionnaire was shortened and optimised accordingly by eliminating the service factors which don’t affect satisfaction and NPS scores.
- An English questionnaire was also prepared, aiming to gauge the satisfaction level of foreign customers. Thus, the representation problem in the previous research design was eliminated.
- Sending and completion dates of the survey were analysed, together with response rates and termination levels, and e-mail sending times and frequencies were determined accordingly.
Improvements were observed not only in terms of these aspects, but also in financial and competitive performance of Pegasus, as well as in all strategic Key Performance Indicators. Some examples for these performance improvements are as follows:
- Tremendous increases in NPS scores and satisfaction levels (see Graph 2)
- 39% improvement in satisfaction with delay announcement (see Graph 3)
- Claimed delay by customers decreased from 58% to 27% (see Graph 3)
- 28% increase in satisfaction with space between seats (see Graph 4)
- Significant decrease in almost all customer complaints (see Graph 5)
Lessons learned from the research
One of the main lessons we learned from this research was that a wholistic approach was very beneficial to overcome either technical barriers or even legal barriers while remaining in line with professional and/or ethical rules.
Teamwork and a collective approach with all units of both companies as well as other stakeholders of Pegasus yielded very positive results.
Research showed that improvement isn’t always achieved at the expense of something. The change in research methodology had its positive effects in all aspects such as financial, technical and statistical without renouncing any previous advantages.
