Quality Assurance Tips

This is important stuff...

Releasing a web application for public use – without performing a complete quality review – will impact your business a little bit now. More importantly, productivity on future application development will suffer as well.

Over time, as more and more slightly flawed applications are released, your developers' time will be increasingly consumed with fixing the impact of mistakes, locating and correcting their core cause. Left unchecked, your geeks will eventually be virtually immobilized by bugs, unable to devote quality time to your new projects. Death by a thousand cuts...

In a perfect world, quality review should be performed by paid QA professionals. They have experience in creating test plans that insure every possible combination of data entry and user action is uniquely applied; and can create issue reports that assist developers in isolating and fixing bugs permanently.

Lacking a perfect world and QA budget, business users and project managers can perform testing, but must try to avoid the natural application of their business model experience. In other words, “act like a simple user”. It's a well-established fact that the developers can not reliably test their own work, much as talent show judges could not be expected to reliably score their own children as contestants.

Here are a few testing procedures that should be applied, where appropriate, before releasing any web application. Note any exceptions that may turn up.

Multiple browsers

Check the application with more than one browser. It’s also a good idea to test on more than one device, even from multiple locations.

Display scripting errors

Insure that your browser is set to display scripting errors. In IE this is Options | Advanced | Display a notification about every script error

Entering data

Special characters In form fields that ask for text input, provide values that include any of these characters:

• & ampersand
• ' “ single AND double quote
• , comma
• < less than sign

See if the values and code survive the storage, reporting and retrieval process. Single quotes affect JavaScript & SQL. Commas can cripple reports.

Numeric fields

Enter text in a field that requests a number. Try 29.9 as an age, or 10.0E as a price.

Losing your place

With a complex web form designed to edit or update lots of stuff, check for a fixed visual indicator of what it is that you are changing. Note any page in which a user who was distracted from the page for some period of time could forget what record was being changed.

Leaving data out: Empty fields

Attempt to submit the form without entering anything. For any page in which a pop-up launches to add additional required settings, attempt to proceed anyway (ignore or close it). Be really dumb...

Data formats

Test any field which suggests that a user is responsible for formatting the data (for example, "Enter the date in this format"). Reject it as non-compliant if it accepts anything but the required format without correction, or breaks.

Dates

In any form field that accepts a date, see if it is possible to enter a date prior to 1752 and check the application's behavior thereafter. If you can’t go that far back, try anything before 1970.

Note: 1752 is the year the English-speaking world adopted the Gregorian calendar. Entries before that year often hang a system completely. 1970 is the “epoch” date for many applications, especially if they’re java-based.

Query strings

If you notice a web page that was loaded by use of a query string in this or similar format :

remove one of the "=value" parts from the query string. Hit Enter to load the page again.

Words vs. numbers

If one of the values is a number, replace it with a neutral word like "one" and hit Enter to load the page again.
If any of the values is a recognizable word, insert the character "&" or “<” into
the middle of the word, and hit Enter to load the page again.

PCI Compliance

Normally required when your site is handling money, consider testing for PCI Compliance even when you're just logging in users for non-financial reasons. All the basic requirements (against cross-site scripting, data injection attacks, etc), are really just darn good ideas in running a tight virtual ship. Consider that someone using your site may have (foolishly) used the same password that they also bank with.

When you find something: What, Where and How?

Explain the nature of the problem in detail. Simple statements like “it didn't work” or “it displayed an error” don't help too much. What did the first line or two of the error say? How did you know that it didn't work? Copy the URL from the top of the browser. Grab a “screen capture” if you know how.

We will need to recreate the issue in order to fix it, so please list the specific steps – from the top – that we would follow to see the same thing as you. If possible, ask a co-worker to try the same steps on their computer and note the results if different.

Too many variables?

A common problem in defining an issue comes from changing too many things between each attempt to recreate it. For example, you may see an error on your computer and ask a co-worker to reproduce it. But he or she may log in as a different type of user and might try a different brand of browser and complete the form differently than you did. Try to keep as many things exactly the same as you can. Then, you'll be able to find the specific text, setting, thing or things that all conspired to reveal that elusive bug!

Is it really breaking?
Is it really working?

Don't confuse a bug or system error with a warning message or intentional design. If in doubt, check the design specification or ask the developers.

Finally, the system could hum along beautifully, take all the information correctly and produce a beautiful report with no errors; all the while placing the totally wrong sales tax or shipping amount in the database for all but the most observant to overlook. Be sure to audit all math!