How to Make the Most of User Acceptance Testing (UAT)

So the project you’ve commissioned has completed development.  It’s been tested by QA and is now being handed over to you for UAT.  What happens next? 

For a lot of the clients we work with, our initial project with them may be the first time they’ve ever been involved in this type of endeavor and they’re not quite sure what is needed of them.  What does the UAT process entail?  What are the expectations?  Why is it even necessary in the first place and why should it be made a priority?  To begin to answer these questions, let’s start by defining what UAT is.

What is UAT?

How is this different from QA?  Quality assurance is testing to make sure that the product is functional, that it meets written requirements and specifications, and that as many defects as possible are identified and corrected before it is handed off to the client.  We’re making sure that you have a quality product.  On the surface, UAT and QA sound similar, and to some extent they are.  The key differences are in the goals of testing and the strategies for going about them.

How UAT Differs from QA: Purpose & Approach

As a QA engineer, when I’m testing a component I have documentation telling me what that component should do and how it should behave.  My job is to go line by line through that specification and make sure that everything it says should happen, does happen, and that nothing happens that shouldn’t happen.  I often tell the solutions architects who create the specs that if it’s written down, I’ll test it.  If it’s not written down, I might not know to.  I’ll check that every option on that component works.  I’ll look at it across a variety of browsers and devices.  I’ll look at as many configuration combinations as I can.  I’ll get creative and try to do something unexpected to see if it’s handled gracefully.  By the time I’ve passed a component, you can be well assured that it’s working properly.

Real-World Use Cases

The first gap that UAT covers that QA cannot is all of the real world uses for a given component or piece of functionality.  Sure, the specifications may have a user story or other explanation about the intention of the component but it’s not likely to spell out exactly how and where the component will be used.  However, as the client, you are most familiar with your target audiences and should have a pretty good idea of this.  If you know that a component will primarily be used on one specific page, prioritize testing it there.  If most of your users work on a particular web browser, use that for your testing. Basically, you want to use the component the way that you expect it to be used when live.

It’s All About the 'Right' Content

The most important advice I can give you for UAT is to know your content.  That’s the other major area where QA coverage can only go so far.  It’s important that you use realistic, production-level content in your testing as far as possible.  Here’s an example to illustrate.  Let’s say we have a featured page component.  You select a page to display and it will show the title, an image, and a summary with the first few lines of text.  When I test, I’m probably using dummy data.  I’ll make sure the component is populating properly and the display is correct.  But maybe the actual titles that will be used are longer than what I used for testing, and there’s a problem with their display.  Maybe the images you’re likely to use are a different aspect ratio than those that I used, and they don’t quite fit.  If my test page used lorem ipsum text (filler text that’s commonly used in these situations), it may be hard for me to spot if the displayed summary makes sense, whereas you’d notice quickly with real data. These are all issues that QA could potentially miss that UAT would easily be able to catch.

Knowing your content is also important if you’re dealing with dynamically pulled content.  This could be in the form of a feed component or a search results listing.  Even if I’m working with real content, there’s only so much I can verify with regard to how accurately these components are behaving.  I can adjust filters or search facets and verify that I’m getting different results.  However, you’re in a much better position to determine that you’re getting the exact results that you’re expecting. As the subject matter expert, you will have a keener sense for what content should be prioritized in the results and we can then make changes to the search algorithms to reflect this.

"The most important advice I can give you for UAT is to know your content. "

A Quick Guide for Conducting UAT Effectively

Preparation

Now that we’ve covered some tips on the types of things to be looking out for during UAT, let’s look at how to go about doing it.  First off, you’ll want to be sure that you’re prepared for UAT before it begins.  It’s important that you take UAT seriously and devote the necessary time and resources to it, so that you can surface any issues while there’s still time available to fix them. 

Identify who is going to be doing your acceptance testing.  Be sure to schedule the necessary training for all involved, especially if you are going to be testing a product that’s brand new to you.  When components are demoed, ask questions and be sure you understand how they work. Ensure that your entire UAT team is on the same page about what constitutes an issue/bug and define a process for how issues will be communicated to your development partner.

Execution

Once it’s time to sit down and start testing, I’d recommend using the entire solution like you would in production.  Start putting together some realistic pages and populating components.  Go through the steps you expect your website visitors to follow.  Remember, QA will have focused  on the specific details for how various functionality works, so you should focus on the bigger picture of how the solution comes together.  Prioritize looking at things that will be most widely used first. 

A lot of our clients choose to align their content entry with the UAT period. While you still need to make sure that you’re testing the full solution, this is a great way to optimize your time spent and ensure that you’re using real world examples for UAT.  If this is the route you want to take, be upfront about it from the start of the project.  This will allow the development team to prioritize building functionality that will be most important for content entry.  There may also be a need to coordinate migrating content between environments, so you’ll want to have timelines planned out in advance.

If you’d like additional  guidance, you can ask to see the QA team’s test cases or mind maps. These can be helpful in letting you know what has been covered in QA and can also give you some ideas on where you should focus your testing.  However, if you do have access to QA’s test cases, you don’t want to retrace their steps and run their exact tests.  There is little value gained in duplicating efforts, especially when QA has narrowed in on the details.  You’ll want to be sure you have time to cover things that QA wouldn’t have hit - such as the business use cases and content issues discussed earlier.

A Final Note on Scope

Now that we’ve gone over what should be covered during UAT, there are a few things to keep in mind about what UAT isn’t.  Simply put, this is not the time to be changing things.  While you absolutely want to raise any issues where things aren’t working correctly, you want to avoid introducing any significant new work.  Small tweaks such as adjusting font or adding some padding may be okay.  Larger adjustments such as completely redesigning certain areas or adding new components usually cannot be accommodated without compromising your timeline or other functionality that has not been completed. 

Yes, UAT can be a little overwhelming, but it is a critical part of ensuring your project is successful.  As long as you’ve made an effort to plan out your strategy and are mindful in how you approach it, you can be sure that your UAT period will be effective and valuable.