Usability is defined by ISO 9241 as the extent to which a product can be used by specified users to achieve specified goals.
Prior to H2 2019, our primary mechanism for usability measurement was collecting task completion benchmarks via quarterly vendor research. The way we benchmarked usability was costly, slow and made it impossible for us to run in synergy with on-platform experiments.
Therefore, we introduced Usability Benchmarking Tool (UBT), to make the benchmarking process cheaper, faster, and enable us to measure specific experiments.
Usability Benchmarking Tool (UBT)
UBT is an unmoderated usability testing tool on the Facebook App, which enables us to measure a person's ability to complete a given task with varying degrees of effectiveness, efficiency and satisfaction.
New UBT designs and content strategy prevent users from high social consequence tasks to some extent and meet needs of our partner teams to measure tasks with high social consequence.
Make ubt more usable
What need to be addressed to make UBT more usable:
Most research participants didn't have the right mental model of UBT. They misunderstood UBT as a tutorial of advertisement of Facebook's new features.
After our participants marked the activity as done, the instruction on the bottom sheet still asked them to conduct the activity. This also confused them because they were asked to do what they had already done.
Here's the new design:
The new design makes UBT better meet users' expectation, thus improves the usability. This helps to:
Increase the number of people who complete the UBT flow in its entirety.
The data quality is improved because the user experience is less of the obstacle for completing the task.
3. Ship android support
Building UBT on Android increases our survey pool eligibility by 5X. This addresses partner team and core team needs to understand usability on Android. We also need to enable targeting multiple platforms (both iOS and Android) in the same task run.
4. Enable translations
Enable translations and targeting by language increases our survey pool eligibility by 2X. This addresses partner teams and core team needs to understand usability in non-English markets.