Testing a Solution
We need to check that our solution works fully, even if the user enters invalid information.
Validation of user input can help, but we need to test all the solution as well, just in case there are other problems.
User Reactions to Problems
If a solution crashes or gives incorrect results, users may…
- make costly and embarrassing mistakes
- become frustrated with the solution and its developer
- stop using it, switching to a rival product
- complain, leave bad reviews, demand refunds, or compensation
Test Data
When testing user input, we should use valid, extreme, and invalid data.
Only valid and extreme data should be accepted. Invalid data should be rejected without crashing the system, and give the user a helpful message to assist them in correcting their input.
Types of Test Data
Valid Data
- Data meets the requirements
- It will not cause errors
- It should be accepted by the system
Extreme Data
- Data meets the requirements, but is right on the edge of being acceptable
- It will not cause errors
- It should be accepted by the system
Invalid Data
- Data does not meet the requirements
- It will cause errors
- It should not be accepted by the system
Pros and Cons of Testing
Advantages
- Finds errors early: prevents small mistakes from becoming bigger problems, which can be more expensive to fix
- Ensures requirements are met: makes sure the final product does what the client/user wants
- Improves user experience and protects reputation: fewer embarrassing failures or bad reviews
- Improves security: hackers may exploit bugs in software
Disadvantages
- Time consuming: thorough testing can take a long time, which can delay the product’s release
- Expensive: requires resources like specialist testers, test environments, and tools
- Not always perfect: impossible to test every single scenario, so some errors may slip through
Types of Testing
Technical Testing
- Carried out by developers / programmers
- Checking the system works correctly and efficiently
- Examples: measuring performance, checking inputs and outputs
- Imagine building a car. Technical testing ensures the engine runs properly, indicators blink, and there are no warnings on the dashboard.
Pros and Cons of Technical Testing
Advantages
- Detects bugs and errors before users see them
- Ensures the system works under different conditions (e.g. stress tests)
- Improves system reliability and performance
- Can be automated, saving time on repetitive checks
Disadvantages
- May not identify usability issues (e.g. confusing layouts)
- Can be time-consuming and costly to run thoroughly
- Requires specialist knowledge / tools
- Developers may "test what they built" and miss hidden problems
User Testing
- Carried out by end users (the customers)
- Checks usability and whether the system meets user needs
- Examples: testing for ease of use, accessibility, layout/design feedback
- Imagine building a car. User testing may ensure the vehicle is comfortable and easy to drive
Pros and Cons of User Testing
Advantages
- Reveals real-world usability issues (navigation, clarity, accessibility)
- Provides direct feedback from the target audience
- Helps ensure the product meets user / client needs
- Can be automated, saving time on repetitive checks
- Increases customer satisfaction and acceptance
Disadvantages
- Time-consuming to organise (recruiting users, running sessions)
- Feedback can be subjective or inconsistent
- May not identify hidden technical bugs
- Can add cost (e.g. incentives, user testing labs)