Remote usability testing
British Airways
The digital team at British Airways (BA) was working to overhaul their flight purchase flow on ba.com. The online flight selling platform contributes a significant proportion of overall revenue for ba.com and so it is essential that customers can book their journey as easily as possible.
The team wanted to ensure that the updated flight purchase journey had an optimal user experience by iterating the design based on user feedback throughout the process, whilst not impacting the speed of the overall delivery. This was a very complex task. BA customers can start their journeys anywhere in the world and could be any nationality. They would have a range of experience from first-time flyer to highly experienced business customers. There was an overlapping dimension of the Executive Club where the top tier represented very frequent flyers and the bottom tier could have people who had only flown once – or not at all. The nature of the business added complexity – short-haul and long-haul flights had different travel classes with differing benefits that would appeal to different target segments, and pricing could change for a given seat/product over the course of a year. There could be single travellers, groups of families or groups of non-families.
I had to be aware of all these dimensions, to focus on priorities and immediate design questions.
I proposed a series of 6 remote usability sessions (with 5 tests per session) over a crucial 3 month period in the project. This would mean regular feedback from users, at intervals which fitted with their agile lifecycle.
How might we incorporate user feedback into our 2 week agile sprint lifecycle to ensure that the updated flight purchase flow has an optimal user experience, whilst not slowing the overall project delivery down?
Users & audience
British Airways passengers could be almost anyone in the world. There are frequent flyers, who are most likely members of the Executive Club (the BA frequent flyer loyalty program) but also people who may never have flown before or who have never used ba.com to book flights before.
Roles & responsibilities
For each of the 5 delivery streams, there was at least one UX design lead. My responsibility was to work with these design leads to establish what they needed to get out of the testing, which audience they wanted to focus on and then to execute the testing and present the findings.
Process
The development teams were working in 2 week agile sprints, so it made sense for the research to follow the same pattern.
Recruitment
I recruited 50 participants over 3 months to take part in the usability testing sessions. These were a mix of BA Executive Club members and non members. Recruitment involved reaching out to potential participants, conducting a screener and then scheduling them into the appropriate research slots.
Planning
For each round of usability testing, I worked with the UX lead to establish their specific objectives for the test and then created a facilitation guide to use during the testing to make sure that these objectives would be met.
Executing the research
I used screen sharing software & conference call facilities to moderate each 1 hour, remote usability testing session. This meant that we were not constrained by location and could work across time zones. It also meant that the UX design leads (and other relevant stakeholders) could dial in and observe the sessions.
Analysis & presenting the findings
Following the testing, I analysed each of the recordings and pulled out key findings which I presented back to the BA design leads. These insights were used to by the design team to iteratively work on the designs for the flight purchase flow on ba.com.
Outcomes & lessons
The frequency of the research sessions worked really well to enable the findings to be integrated into the teams regular sprints.
Whilst my team was designing the new flight selling process I was looking for options to supplement our research capability and Rebecca had time available to work remotely. I was delighted that we were able to re-engage her as I had every confidence that her familiarity with the topic and her great research skills would yield valuable insights and she would work well with us. This expectation was fully justified and her contribution was important to the success of the project.
Need some remote usability testing?
Get in touch with me and we’ll come up with a plan.
