Reintroduction to digital experimentation in HMRC
Read our blog from Product Manager, Yvie, who explains the work behind the digital experimentation team at HMRC.
Following on from our previous blog, Introduction to digital experimentation, we want to show you what experimentation can do and the effectiveness of this approach.
Within days of the ‘Working From Home Service’ being released, many users were unable to find the service. This resulted in 20% of all digital contact in the personal tax account being about this issue. The business and scrum teams needed to know where they should link users to within the working from home service that would get the most users to complete their task, reduce digital contact and not negatively impact other users.
If we: create an appropriate way for users to access the working from home service
We will see: a decrease in contact, an increase in online completion of the working from home service and we will avoid negatively impacting other users in the personal tax account.
By exploring the existing data, we found most of the contact came from the pay as you earn service. This helped designers identify three locations suitable for us to test.
This winning variant was 366% better at reducing contact than the leading idea before the experiment was run. It was also the only variant that did not have any negative impact on other user journeys. It took a day to rapidly design and build these variants and 5 days to collect over 100,000 users worth of data. Overall, it took 7 days to rapidly design, test and analyse these options, allowing us to inform the team with over 95% certainty of which option would have the overall best impact for the users and HMRC.
If any of this interests you or to find out what we can do to support you, please email firstname.lastname@example.org.