A-Level Computer Science / Unit 12: Software Design, Testing and Evolution

12.3.4 Testing from the User’s Perspective

🔒 Lesson slides are available to signed-in users. Sign in

12.3.4 Testing from the User’s Perspective

Users experience a program through its inputs, outputs, messages and responses. They do not need to know which statements, branches or modules produced those results.

Black-box testing checks this externally visible behaviour against the requirements. As a product moves towards release, alpha, beta and acceptance testing involve different groups of testers and provide different kinds of evidence.

By the end of this section, you should be able to:

  • Explain how black-box testing uses requirements rather than internal code.
  • Derive tests from externally visible system behaviour.
  • Compare expected and actual results.
  • Explain the purposes of alpha, beta and acceptance testing.
  • Distinguish the people, environment and purpose involved in each method.
  • Select a suitable testing method for a development scenario.
  • Explain the limitations of user-facing and release testing.

Testing Externally Visible Behaviour

From a user’s perspective, a system receives an action or input and produces an observable response.

Externally visible behaviour: the inputs, outputs, messages, screens and responses that can be observed without examining the program’s internal implementation.
User action or condition Visible behaviour to check
A valid permit request is submitted. A booking reference and confirmation are displayed.
A trail session has no remaining spaces. The request is rejected with a clear capacity message.
A required field is left empty. The user is told which information must be supplied.
A confirmed permit is cancelled. The booking status changes and the released place becomes available.
The user repeats a completed submission. The system avoids creating an unintended duplicate permit.

The tester may know nothing about the internal algorithm. The test is based on what the system is required to do.

Common misconception: Testing from the user’s perspective does not mean that only end users can perform it. Developers and professional testers can also use black-box techniques.

Black-box Testing

Black-box testing: testing externally visible behaviour by supplying inputs and comparing the actual results with results derived from the specification, without using the internal code to select the test.

The program is treated like a closed box:

Input → system → observable output

The tester concentrates on questions such as:

  • Does the system accept valid user actions?
  • Does it reject invalid or incomplete requests appropriately?
  • Are calculations and displayed values correct?
  • Are the correct records created, changed or removed?
  • Do messages explain what the user should do next?
  • Does the system behave according to the written requirements?

What black-box testing does not require

  • knowledge of the program’s variables;
  • knowledge of its branches or loops;
  • access to its source code;
  • knowledge of its internal modules.
Exam tip: A strong definition should include the specification, supplied inputs, expected results and actual results, together with the absence of reliance on the internal code.

Deriving Tests from Requirements

A black-box tester begins with a requirement and asks what observable result would demonstrate that the requirement has been met.

Example requirements for a coastal trail permit portal

Requirement Test action Expected visible result
A visitor can reserve up to four places while capacity remains. Request three places for an available session. The permit is created for three visitors.
A request must not exceed the remaining session capacity. Request three places when only two remain. The request is rejected and no permit is created.
An email address is required. Submit the form without an email address. A clear validation message is displayed.
A confirmed permit can be cancelled before the visit date. Cancel an existing future permit. The permit becomes cancelled and its places are released.
A previously cancelled permit cannot be cancelled again. Submit a second cancellation for the same reference. The user is told that the permit is already cancelled.

The detailed classification of normal, abnormal and boundary data belongs in Section 12.3.5. On this page, the emphasis is on connecting user actions to required outcomes.

Common mistake: Do not invent the expected result from what the current program happens to display. Derive it independently from the requirement.

Expected and Actual Results

Expected result: the result predicted from the requirement before the test is run.
Actual result: the behaviour observed when the test is performed.

A test passes when the actual result agrees with the expected result. A difference provides evidence of a fault, a misunderstood requirement or an unsuitable test environment.

Test action Expected result Actual result Outcome
Reserve two places when six remain. Permit created; four spaces remain. Permit created; four spaces remain. Pass
Reserve three places when two remain. Request rejected; two spaces remain. Permit created; remaining capacity becomes −1. Fail
Cancel a future permit. Status becomes cancelled and places are restored. Status becomes cancelled but places are not restored. Fail

A failed result should be recorded clearly enough for the development team to repeat the problem.

Exam tip: Avoid writing only “the test failed.” State the expected result, the actual result and the difference between them.

Testing Complete User Journeys

User journey: a sequence of user actions carried out to complete a meaningful task.

Testing one screen at a time may miss faults that occur across a longer sequence. A user journey checks whether connected visible actions form a complete task.

Permit-booking journey

  1. Search for an available visit date.
  2. Select a trail session.
  3. Enter visitor details.
  4. Confirm the number of places.
  5. Submit the request.
  6. Receive a permit reference.
  7. Open the confirmation message.
  8. Use the reference to view or cancel the permit.
Journey checkpoint Evidence to observe
Session selection The chosen date, time and trail remain visible.
Submission Only one permit is created.
Confirmation The reference matches the stored booking.
Later retrieval The correct permit is shown for the reference.
Cancellation Status and remaining capacity are both updated.
Common misconception: A screen can work correctly in isolation while the complete user journey still fails because information is lost between steps.

Alpha Testing

Alpha testing: testing a near-complete system within the developing organisation before it is released to external users.

Alpha testing is normally carried out in a controlled environment by developers, internal testers or other staff from the organisation producing the software.

Possible aims

  • exercise the main user journeys;
  • find serious faults before external release;
  • check installation and configuration procedures;
  • check that visible messages and screens are understandable;
  • confirm that the build is stable enough for external testing;
  • record faults that must be corrected before beta testing.

Trail-portal example

Staff from the software company test the portal on supported browsers using prepared permit scenarios. They deliberately try incomplete forms, full sessions, repeated submissions and cancellations.

Exam tip: Associate alpha testing with an internal, controlled stage before the software is made available to external users.

Beta Testing

Beta testing: releasing a pre-release version to a limited group of external users so that it can be tried in realistic environments before general release.

Beta users may use different devices, browsers, network conditions and patterns of behaviour from those used during internal testing.

Possible aims

  • discover faults that did not appear in the controlled alpha environment;
  • observe how real users interpret the interface;
  • check behaviour across varied devices and configurations;
  • collect feedback about confusing workflows;
  • identify practical issues before a wider public release.

Trail-portal example

A selected group of volunteer walkers and visitor-centre staff use the portal on their own phones and computers. They report problems through a structured feedback form.

Common misconception: Beta software is not simply an unfinished prototype shown to anyone. It should be sufficiently complete and stable for limited external use.

Acceptance Testing

Acceptance testing: testing carried out by or on behalf of the customer to determine whether the delivered system meets the agreed requirements and is suitable for acceptance.

Acceptance testing is especially important for software commissioned for a specific organisation. The tests are derived from agreed requirements or acceptance criteria.

Trail-portal example

The parks authority checks the completed portal against its agreed requirements, including:

  • capacity rules;
  • permit creation and cancellation;
  • required confirmation information;
  • staff access to daily visitor lists;
  • supported devices and browsers;
  • required reports.

When the agreed acceptance criteria are satisfied, the customer may formally approve or sign off the system.

Common misconception: Acceptance testing is not the same as beta testing. Acceptance focuses on whether an agreed customer solution meets its requirements; beta testing collects evidence from selected external users before general release.
Exam tip: In an acceptance-testing answer, mention the customer, agreed requirements and approval or sign-off.

Comparing Black-box, Alpha, Beta and Acceptance Testing

Method Who may perform it? Environment Main basis or purpose
Black-box Developers, testers, customers or users Any suitable testing environment Compare visible behaviour with requirements without relying on internal code.
Alpha Internal developers, testers or staff Inside the developing organisation Find faults before external release.
Beta Selected external users Realistic user environments Find practical and environmental issues before general release.
Acceptance Customer or customer representatives Agreed customer test environment Confirm that the delivered system meets agreed requirements.
Important relationship: alpha, beta and acceptance tests can all use black-box techniques because their testers may judge externally visible behaviour against expected results.
Common mistake: Black-box describes how tests are designed. Alpha, beta and acceptance describe the context, participants or stage of testing. These categories are not direct alternatives in every situation.

Testing as Software Moves Towards Release

A possible progression is:

  1. Internal system testing. Developers and testers check the integrated system.
  2. Alpha testing. The near-complete product is tested inside the developing organisation.
  3. Beta testing. A controlled pre-release version is used by selected external users.
  4. Corrections and regression testing. Reported faults are corrected and relevant tests are repeated.
  5. Acceptance testing. Where a customer has commissioned the system, agreed criteria are checked before approval.
  6. General release or deployment. The approved version becomes available to its intended users.
Important: This is a useful teaching sequence, not a rule that every project follows identically. A mass-market product may have no single customer acceptance stage, while bespoke software may require formal acceptance before deployment.

Collecting Useful User Feedback

A report such as “the portal does not work” is difficult to reproduce. User feedback should describe the circumstances clearly.

Useful report field Example
Tester or user group Volunteer walker group B
Device and environment Android phone using a mobile connection
Action sequence Select session, enter two visitors, submit once
Input or relevant data Trail T4, 16 August, 09:30 session
Expected result One permit and one confirmation
Actual result Two permits were created after one visible tap
Supporting evidence Two references and the approximate time of the submission
Repeatability Occurred twice under a weak network connection

Developers can use this evidence to reproduce the problem, correct it and run regression tests.

Exam tip: Explain how external testing can reveal faults caused by varied devices, environments and user behaviour that were not reproduced internally.

Worked Example: Coastal Trail Permit Portal

A parks authority commissions an online portal. Each visit session has a fixed capacity. A visitor may reserve one to four places, receive a reference and cancel a future permit.

Selected requirements

ID Requirement
R1 A visitor can reserve between one and four places.
R2 A reservation cannot exceed the session’s remaining capacity.
R3 A successful request produces one unique permit reference.
R4 Cancelling a future permit restores its reserved places.
R5 A previously cancelled permit cannot be cancelled again.

Black-box tests

Test Requirement Initial condition and action Expected result
B1 R1, R3 Request two places when eight remain. One permit for two visitors is created; six places remain.
B2 R1 Request five places. The request is rejected with an explanatory message.
B3 R2 Request three places when only two remain. No permit is created and capacity remains two.
B4 R4 Cancel a future permit containing three places. The status becomes cancelled and three places are restored.
B5 R5 Cancel the same permit again. A message states that the permit is already cancelled.

Alpha stage

Internal testers execute the complete booking and cancellation journeys on the supported browsers. They discover that pressing the submit button twice quickly can create two permits.

Beta stage

Selected walkers use the corrected portal on their own devices. Several report that the session-capacity message is difficult to understand on a small screen. One user also reports that a confirmation is delayed on a weak mobile connection.

Acceptance stage

The parks authority checks the agreed requirements. During acceptance testing it finds that cancelling a permit changes its status but does not restore the places. Requirement R4 therefore fails and acceptance cannot yet be completed.

Correction and retesting

The capacity update is corrected. Tests B4 and B5 are repeated, followed by earlier reservation tests to confirm that the change has not affected permit creation.

Exam tip: Notice that each stage provides different evidence: internal stability during alpha, realistic-use feedback during beta and formal requirement checking during acceptance.

Choosing the Appropriate Testing Method

Scenario clue Likely method Reason
The source code is not used; outputs are checked against requirements. Black-box The test is based on visible behaviour.
The near-complete application is being tested by the developer’s own staff. Alpha The testing remains internal before external release.
A limited pre-release version is used on external users’ own devices. Beta Realistic environments and user behaviour are being investigated.
The commissioning organisation checks agreed requirements before approval. Acceptance The customer is deciding whether the delivered system is acceptable.

A method-selection process

  1. Identify who is carrying out the test.
  2. Identify whether the testers are internal or external.
  3. Determine whether the system is pre-release or being considered for approval.
  4. Identify whether the test comes from code structure or external requirements.
  5. State what evidence the method is expected to provide.
  6. Link the chosen method directly to the scenario.
Exam tip: Testing-method questions usually require more than a definition. Name the method and explain why its participants, environment and purpose fit the scenario.

Limitations of User-facing Testing

Method Limitation
Black-box A failure shows that visible behaviour is wrong but may not reveal the internal statement responsible.
Alpha Internal testers may use familiar workflows and controlled environments that differ from real use.
Beta Feedback quality varies, some scenarios may remain untested and exposing a pre-release version carries risk.
Acceptance Passing agreed criteria does not prove that every possible user situation has been tested.
Common misconception: Passing alpha, beta or acceptance testing does not prove that no faults remain. It provides evidence about the tested version, environments, requirements and user activities.

Interactive: User and Release Testing Lab

Read each scenario and choose the most suitable method. The widget explains which clues identify black-box, alpha, beta or acceptance testing and builds a release evidence record.

Scenario

Check visible capacity behaviour

A tester uses the permit requirements to predict what should happen when three places are requested but only two remain. The source code is not used.

Scenario 1 of 8

Choose the most suitable method

Focus on what information is used to design the test.

Clues: specification · visible output · no source-code knowledge
Scenario Correct method Key evidence

Common Mistakes and Misconceptions

  • Defining black-box testing as “testing by a user.” It is defined by using external behaviour and requirements rather than internal code.
  • Choosing black-box tests from branches in the source code. Tests chosen from internal paths are white-box tests.
  • Using the program’s output as the expected result. The expected result must be derived independently.
  • Calling all pre-release testing beta testing. Internal pre-release testing is normally alpha testing.
  • Confusing beta with acceptance testing. Beta involves selected external users; acceptance involves customer approval against agreed requirements.
  • Assuming alpha testing uses only developers. Other internal testers or staff may also participate.
  • Assuming beta users perform formal sign-off. They normally provide real-world fault reports and feedback.
  • Testing isolated screens only. Complete user journeys may reveal lost or inconsistent information.
  • Recording vague feedback. A useful report includes actions, data, environment, expected and actual results.
  • Claiming that successful release testing proves perfection. Testing increases confidence but cannot demonstrate that no faults remain.

Practice

Core questions

  1. Define black-box testing.
  2. Explain how a black-box test is derived from a requirement.
  3. Distinguish between an expected result and an actual result.
  4. Explain why a complete user journey may reveal faults that isolated screens do not.
  5. Define alpha testing.
  6. Define beta testing.
  7. Define acceptance testing.
  8. State two differences between alpha and beta testing.
  9. State two differences between beta and acceptance testing.
  10. Explain why passing release testing does not prove that no faults remain.

Scenario A: Community Equipment Portal

A portal lets residents reserve tools from a community workshop. A requirement states that a tool already reserved for a time slot must not be allocated to another user.

  1. Write a black-box test action.
  2. State the expected result.
  3. State one possible failing actual result.
  4. Explain what evidence should be recorded.
  5. Explain why the source code is unnecessary for this test.

Scenario B: Language-learning App

A near-complete app is first tested by the developer’s quality team. It is then supplied to 150 invited learners who use it on their own devices before its public launch.

  1. Name the first testing stage.
  2. Name the second testing stage.
  3. Explain the purpose of each stage.
  4. Give one issue the second stage may reveal that the first may miss.
  5. Explain what information an invited learner should include in a fault report.

Scenario C: Custom Archive System

A museum commissioned a digital archive system. Before approving delivery, museum staff check import, search, editing, access control and report generation against the agreed requirements.

  1. Identify the testing method.
  2. Explain why it is not beta testing.
  3. State the evidence needed before approval.
  4. Explain what should happen when one agreed criterion fails.
  5. Explain what sign-off represents.
Challenge: Design a release sequence containing internal alpha testing, selected external beta testing and customer acceptance. State the distinct evidence expected from each stage.

Review

Prompt A strong response should include
Black-box testing Specification, externally visible behaviour, supplied inputs and expected versus actual results.
Expected result The result predicted independently from a requirement.
Alpha testing Internal testing of a near-complete system before external release.
Beta testing A limited pre-release version used by selected external users in realistic environments.
Acceptance testing Customer checking against agreed requirements before approval or sign-off.
Useful user report Environment, actions, input, expected result, actual result and supporting evidence.
Testing limitation Passing tests increases confidence but does not prove that no faults remain.
Final exam tip: Use the chain tester → environment → basis for the test → expected result → purpose.