Testing as requirements validation method
In the context of airborne system design assurance and of the application of ARP-4754A, the question of the difference between validation and verification is very often raised, especially in the case of how testing can contribute to validation.
According to the ARP-4754A definitions of validation and verification:
- Validation includes the activities necessary to demonstrate that the item requirements are complete, correct and consistent with regard to the system needs and architecture
- Verification includes the activities necessary to demonstrates that the item complies with its requirements
In the context of ARP-4754A compliant development, requirements are elaborated for the aircraft, then for each system and finally for the items (equipment, hardware, software) that compose each system. Activities to be performed for validating requirements include primarily checking the traceability of the requirements to upper level requirements and performing engineering reviews
The difficulty of requirements validation
Depending on the nature of the requirements and their allocation from upper-level as defined by the aircraft or system architecture, achieving the demonstration of correctness with traceability and engineering review may be more or less difficult:
- In some cases, the requirements are fully allocated to the lower level and, therefore, simply copied as lower-level requirements. Here, determining correctness, completeness and consistency of requirements at lower level against upper level is straightforward
- In some other cases, the requirements at upper-level specify a high-level objective, while the corresponding requirements at lower level specify a detailed functional behaviour. The system designer makes hypotheses and assumptions based on engineering judgment and experience to establish the items requirements and the corresponding traceability. Here, in some cases, determining correctness cannot be achieved just by reviewing the requirements at lower level and ticking a box in a checklist
Let’s take an example where a requirement at aircraft level is something like “ensure passenger comfort in case of turbulences” and where, at the lower level (flight control system), engineering identified requirements for characterization of flight control laws. Filtering algorithms would be specified, but wouldn’t this result in oscillations or other aircraft behaviours that could incommode the passenger?
The answer to this question appears to be difficult with requirements validation methods such as modelling and simulation. ARP-4754A proposes also testing as a requirements validation method.
Testing for product verification
Testing imply stimulating a product with input data and signals and checking that the observed behaviour is consistent a specified behaviour.
Testing is widely used for product verification. The product is submitted to test data and the obtained behaviour is checked against expected behaviour defined on the basis of its requirements. The goal of this activity is to demonstrates that the product complies with its requirements. However, this does not provide any evidence of the validity of the product requirements. The test may succeed even if the requirements are wrong (i.e. not valid).
So, how can testing be used for the validation of product requirements?
Testing for requirements validation
The idea is to use the product developed and produced on the basis of the requirements to be validated to integrate it in the upper-level system and to submit the result to tests, the expected results of which are defined on the basis of the traced upper-level requirements.
- Testing as Requirements Validation -
Of course, this assumes that the product is correct with regard to its requirements. This assumption is supported by development reviews and verification testing; however, for verification tests, the expected results of the tests are determined on the basis of the product requirement, while for validation tests, the expected results of the tests are determined on the basis of the upper-level system requirement.
In the above example, the control laws requirements may be validated by producing a prototype flight control computer with a correct implementation of these laws. This prototype would be installed on a flight test aircraft and the perception of comfort of a “Guinea Pig” passenger would be assessed after flight testing with turbulences.
Author: Philippe Robert, System design Assurance & Certification Engineer, PMV Consulting & Services
#SystemEngineering #AircraftCertification