A-Level Computer Science / Unit 12: Software Design, Testing and Evolution

12.3.2 Test Strategy, Plans Data

🔒 Lesson slides are available to signed-in users. Sign in

12.3.2 Test Strategy, Plans & Data

Testing should be organised deliberately rather than carried out as a collection of unrelated checks. Different testing methods provide different kinds of evidence, while a test strategy and test plan ensure that important requirements, risks and data conditions are covered systematically.

This section brings together the main testing methods and then focuses on how to plan tests, choose suitable test data and record results clearly.

By the end of this section, you should be able to:

  • Explain the purpose of the main testing methods used during development.
  • Distinguish between a test strategy, a test plan and a test case.
  • Describe likely contents of a test strategy and test plan.
  • Select suitable normal, abnormal and extreme/boundary test data.
  • State expected results before tests are executed.
  • Record actual results, failures and retests clearly.

Testing Methods: What Does Each One Check?

Method Main purpose Typical focus
Dry run Follow an algorithm manually using selected data. Variable values, decisions, loops and outputs
Walkthrough Review an algorithm or program with other people. Logic, assumptions, requirements and possible faults
White-box testing Select tests using knowledge of the internal code. Statements, branches, loops and paths
Stub testing Replace an unfinished module temporarily. Calls, parameters and expected temporary responses
Integration testing Check connected modules working together. Interfaces, parameter passing, return values and call sequence
Black-box testing Check visible behaviour against requirements. Inputs, outputs, messages and user-visible results
Alpha testing Test a near-complete system internally. Major faults before external release
Beta testing Allow selected external users to try a pre-release version. Realistic devices, environments and user behaviour
Acceptance testing Check that the delivered system meets agreed requirements. Customer approval or sign-off
Common mistake: These methods are not all direct alternatives. For example, an acceptance test may use black-box techniques, while a dry run may be used before an executable program exists.

Dry Runs and Walkthroughs

Dry run: manually following an algorithm step by step using chosen data and recording how values change.
Walkthrough: a structured review in which the author and other reviewers examine the logic and compare it with the intended requirements.
Feature Dry run Walkthrough
Usually involves One person manually executing chosen data Author plus one or more reviewers
Main evidence Changing values and control flow Questions, observations and identified issues
Useful for Incorrect updates, conditions and loop behaviour Missing requirements, unclear logic and incorrect assumptions
A dry run should use actual test values. A walkthrough is broader and may examine design decisions even when no complete trace is performed.

Testing Code and Modules

White-box testing

White-box testing uses the internal structure of the code to choose tests. A tester may deliberately select data that makes a condition true, makes it false, skips a loop or executes several loop iterations.

White-box testing: testing in which knowledge of statements, conditions, loops and code paths is used to select test cases.

Stub testing

A stub temporarily replaces a module that is unavailable. It should have the same interface as the intended module and produce predictable behaviour so the surrounding code can be tested.

Stub: a temporary replacement for an unfinished or unavailable module.

Integration testing

Modules that work individually can still fail when connected. Integration testing checks whether the correct values are passed between modules, returned values are interpreted correctly and calls occur in the intended order.

Integration testing: testing the communication and behaviour of connected modules.
A module passing its own tests does not prove that the complete system will use that module correctly.

Testing Visible Behaviour and Release Versions

Black-box testing

Black-box testing starts from the requirements rather than the source code. The tester supplies an input or performs an action, predicts the required response and compares it with what the system actually does.

Black-box testing: checking externally visible behaviour without using the internal code to choose the test.
Release stage Who typically performs it? Main purpose
Alpha Internal developers, testers or staff Find major faults before external testing.
Beta Selected external users Find practical problems in realistic environments.
Acceptance Customer or customer representatives Check agreed requirements before approval.
Remember: alpha, beta and acceptance describe the context and stage of testing. Black-box describes how a test is designed.

Test Strategy

Test strategy: the overall approach explaining what will be tested, which methods will be used, where testing will take place and how priorities will be set.

A strategy is high-level. It does not contain every individual test value. Instead, it sets the direction for the complete testing process.

Strategy area Typical question
Scope Which features and components are included?
Risk and priority Which failures would cause the greatest problems?
Methods Which testing methods are appropriate?
Stages When will module, integration and release testing occur?
Environment Which devices, operating systems or configurations are needed?
Responsibilities Who prepares, performs and reviews the tests?
Completion conditions What must be true before testing can be considered complete?
Retesting How will corrected faults and related features be checked again?
“Test the system thoroughly” is not a strategy. A useful strategy identifies priorities, methods, environments and responsibilities.

Test Plan and Test Cases

Test plan: an organised collection of detailed tests for a system.
Test case: one repeatable test containing exact conditions, data, actions and an expected result.
Test-case field Purpose
Test ID Provides a unique reference.
Requirement / feature Shows why the test exists.
Purpose States the exact behaviour being checked.
Preconditions Defines the required starting state.
Test data Gives exact values or actions.
Expected result States what should happen if the program is correct.
Actual result Records what happened when the test was executed.
Outcome Records pass or fail.
Fault / retest reference Links failures to corrections and later retesting.
The expected result should be written before the test is run and should come from the requirement, not from the program’s current behaviour.

Choosing Test Data

Category Meaning Example for an allowed quantity of 4–20
Normal A typical valid value. 12
Abnormal An invalid value or invalid form of data. "twelve" or 7.5
Extreme A valid value at the lowest or highest accepted limit. 4 and 20
Boundary Values at and immediately around a limit. 3, 4, 5, 19, 20, 21
A value may belong to more than one useful category. For example, 3 is invalid and also tests the lower boundary. State the reason for choosing the value.

Selecting Good Boundary Tests

Boundary values are chosen around the point where the expected behaviour changes. For an inclusive integer range, useful values normally include the limit itself and the nearest valid and invalid values around it.

Example: locker code length

Suppose a locker code must contain exactly six characters.

Length Purpose Expected result
5 characters Immediately below the required length Rejected
6 characters Exact accepted length Accepted if the format is also valid
7 characters Immediately above the required length Rejected

Keep one main purpose per test

If a code must contain six uppercase letters or digits, the value ab#1 is too short, lowercase and contains a prohibited symbol. A failure would not reveal which rule was responsible. Separate tests give clearer evidence.

Worked Example: Makerspace Equipment Reservation

A school makerspace uses a reservation system for specialist equipment. A student may reserve between 1 and 4 machines. A reservation can be made only when enough machines remain available. A booking code must contain exactly six uppercase letters or digits.

Selected requirements

ID Requirement
R1 Quantity must be a whole number from 1 to 4 inclusive.
R2 The quantity requested must not exceed the number available.
R3 The booking code must contain exactly six uppercase letters or digits.

Example test cases

ID Requirement Purpose Data / precondition Expected result
MS-01 R1 Typical valid quantity Quantity = 2 Accepted
MS-02 R1 Minimum accepted quantity Quantity = 1 Accepted
MS-03 R1 Immediately above maximum Quantity = 5 Rejected
MS-04 R1 Wrong data form Quantity = "two" Rejected
MS-05 R2 Exact remaining capacity 2 machines available; request 2 Reservation created; 0 remain
MS-06 R2 Request exceeds capacity 2 machines available; request 3 No reservation created; 2 remain
MS-07 R3 Valid booking code LAB7Q2 Accepted
MS-08 R3 Length immediately below required LAB7Q Rejected
MS-09 R3 Invalid character LAB#Q2 Rejected
Notice that functional tests such as capacity checks need a known starting state. That starting state is a precondition.

Recording Results and Retesting

After each test is executed, the actual result is compared with the expected result. A failed test should provide enough information for another tester to repeat the problem.

Record Example
Expected result Request rejected; two machines remain available.
Actual result Reservation created; available count became −1.
Outcome Fail
Fault reference DEF-014
Retest Repeat the failed capacity test after correction.
Regression tests Repeat related successful-capacity cases to check they still work.
Do not change the expected result simply because the program produced something different. The expectation should remain based on the agreed requirement.

Common Mistakes

  • Confusing the strategy and the plan. The strategy is high-level; the plan contains detailed tests.
  • Giving test data without a reason. State which requirement or boundary the value checks.
  • Omitting the expected result. A test cannot be judged consistently without one.
  • Using only normal values. Invalid and boundary cases may expose different faults.
  • Calling every invalid value abnormal and ignoring its boundary role. A value can be invalid and also useful as a boundary test.
  • Forgetting preconditions. Some functional tests depend on a known starting state.
  • Assuming successful testing proves perfection. Testing increases confidence but cannot prove that no faults remain.

Practice

Core questions

  1. Explain the purpose of a test strategy.
  2. State five likely contents of a test strategy.
  3. Distinguish between a test strategy and a test plan.
  4. State five likely fields in a test case.
  5. Define normal test data.
  6. Define abnormal test data.
  7. Explain the difference between extreme and boundary data.
  8. Explain why expected results should be written before execution.
  9. Explain why integration testing is still required after module testing.
  10. Distinguish alpha, beta and acceptance testing.

Scenario: Media Studio Booking

A media studio allows students to reserve from 1 to 6 recording booths. A reservation may not exceed the number currently available. A booking identifier must contain exactly seven characters: two uppercase letters followed by five digits.

  1. Give one normal quantity value.
  2. Give the minimum and maximum extreme quantity values.
  3. Give suitable boundary values around both limits.
  4. Give two abnormal values that test different invalid conditions.
  5. Write one capacity test including its precondition.
  6. Write three tests for the booking-identifier format.
  7. State the expected result for every test.

Review

Concept Key idea
Test strategy Overall scope, priorities, methods, environments and responsibilities
Test plan An organised collection of detailed, repeatable tests
Test case Exact conditions, data, actions and expected result for one test
Normal data A representative valid value
Abnormal data An invalid value or invalid form of data
Extreme data The lowest or highest valid value
Boundary data Values at and immediately around a change in expected behaviour
White-box Tests chosen from the internal code structure
Black-box Tests chosen from requirements and visible behaviour
Integration Checks communication between connected modules
Alpha / beta / acceptance Internal pre-release / external pre-release / customer approval
Final exam tip: Use the chain requirement → test purpose → exact data → expected result → actual result → outcome.