Usability Testing

Usability Testing

When I started out in testing it was commonly referred to as the Insurance of IT and not in a good way. Insurance was referred to the “necessary evil” which meant that testing was looked upon negatively. Yet testing or Quality assurance is the best way to ensure that customers get the best possible service that they pay for. Our reputation depends on ensuring that we give our customers the best possible experience.

We are all consumers of some sort of technology. It’s everywhere we go and 82% of people in the western world use some sort of technology.

Now lets look at some scenarios were usability testing has not been done and the impact that that has on the reputation of that product.

Scenario 1 = A company that produces smartphones and doesn’t do usability testing (or does poor usability testing) before they send out a large release.

Action

Impact

Applications on the phone don’t work.

Users cannot use the applications, limiting them in their day to day lives.

The update takes more than hour to update

An hour is very valuable to most people. To lose an hour is not acceptable.

The phone crashes and has be either restored by a technician or else replaced.

In today’s world, everyone relies on their smartphone. The impact of not having this is that the user feels like they don’t have connection with their world. Imagine not being able to listening to music on the train?

User has to restore the last version on the phone in order to make it work.

Time will be lost doing this and if the user has not backed up their information in a while then they will lose that information.

The biggest impact is that the user is more likely to psychologically have a bad image of that company which in turn means that if a small incident happens on that smart phone then the user is more likely to move to another provider.

Scenario 2 – An online retail store thinks that usability testing is to expensive and opt not to do it. Mid year or End of year Sales arrive.

Action

Impact

The website crashes when a lot of users try to buy items on sale.

The store looses millions of dollars due to users being unable to buy.

The website is slow.

Users will go to a different website to buy their products

Images are not loading.

Users cannot see what they are buying.

Users cannot pay for their purchases due to checkout problems.

Users will go elsewhere.

In the above scenario the retail store that thought that usability testing was to expensive is now re thinking their testing strategies. They lost millions of dollars in one 12 hour period.

Scenario 3 – An insurance company thinks that testing is “easy” and decides to let their developers do testing and let go all of their key testing team.

Action

Impact

The developers develop the code and do unit testing only.

All defects that would have been found during Usability testing are missed.

When the developers are asked to test they do so whenever they can find the time.

During their testing, the developers do not do any usability testing due to time constraints.

Developers use their skills to develop code. This mindset can ensure that they code is developed really well.

Testers have a different mindset and are known for thinking outside the box. This helps them find defects that users would find.

The developers didn’t know/ didn’t have time to write test cases.

Audits and traceability were hard to find. The developers were blamed for poor testing.

In the above scenario the insurance company lost over 300 customers (again the cost and reputation of the company was unable to defined in dollar amounts) however the internal impact was much bigger. They lost over one third of their developers due to unhappiness, stress, working long hours, no gratitude and the inability to do what they were hired to do.

As testers we hear all sorts of statements around why testing (in particular Usability testing) should/could not be done. Before I list some of those statements below, first lets review those scenarios above. All companies lost millions of dollars, their reputations were tainted with the customers, their internal staff were not happy, they spent millions of dollars repairing not only the defect but also their reputation. Reputation in a markets that flooded with competition is undeniably the best way to lose and gain customers. Companies cannot afford to have negative feedback. Imagine losing 2 million dollars, a companies reputation, staff turnover and long hours all over one defect compared to spending 200 thousand dollars on testing.

Let’s look at the statements.

1. Testing is to expensive.

2. What is the point of doing testing when we have other staff that can do it?

3. Why do we have to add another two weeks just to do testing?

4. We need to meet project deadlines which means we will have to cut testing.

5. In the Agile world everyone does testing.

6. Testing is not a skill, anyone can do it.

7. Why can’t developers just automate everything?

8. Testers are too expensive.

9. A user would never do that.

10. Who invented testing any way?

The cost of Usability testing goes way beyond missing a defect. I have listed 3 scenarios, this is the tip of the iceberg. Let me then ask you about the impact of Usability testing and why it shouldn’t be done?