International Defense Security & Technology Your trusted Source for News, Research and Analysis
Related Articles
Design in any discipline – electronics, mechanical, aerospace, etc. – begins with a specification that captures what the end product should do and essentially drives the entire development cycle. Verification is intended to check that a product, service, or system meets a set of design specifications. Validation is intended to ensure a product, service, or system results in a product, service, or system that meets the operational needs of the user
Verification and validation processes are a vital part of all steps of the system engineering V-model. Per the related IEEE standard, Independent Verification and Validation (V&V) processes are used to determine whether the development products of a given activity conform to the requirements of that activity and whether the product satisfies its intended use and user needs. V&V life cycle process requirements are specified for different integrity levels.
The scope of V&V processes encompasses systems, software, and hardware, and it includes their interfaces. This standard applies to systems, software, and hardware being developed, maintained, or reused (legacy, commercial off-the-shelf [COTS], non-developmental items). The term software also includes firmware and microcode, and each of the terms system, software, and hardware includes documentation. V&V processes include the analysis, evaluation, review, inspection, assessment, and testing of products.
Typically they account for 20 to 30 percent of development costs. In the preconcept phase, great companies test customer perception of the planned product in order to optimally design the product to customer needs and to improve probability of market success.
In the early development phases, virtual tools are used to ensure that the different components fit together, that the product can be built, and that all functional requirements are met. In the development phase, verification procedures involve performing special tests to model or simulate a portion, or the entirety, of a product, service, or system, then performing a review or analysis of the modeling results.
In later stages, all kinds of tests are required, for example, to ensure safety and environmental requirements are met, durability is proved, and so on. Finally, close to production, testing reveals whether the product can be built, quality is demonstrated, and final checks before the start of production are made.
In the post-development phase, verification procedures involve regularly repeating tests devised specifically to ensure that the product, service, or system continues to meet the initial design requirements, specifications, and regulations as time progresses.
Hardware verification tests (otherwise known as qualification tests) could include:
- testing hardware under conditions simulating expected real-life conditions, including storage, transportation, operation and maintenance environments
- ensuring the hardware conforms with local environmental requirements, including shelter, space, furnishings and fittings, electrical power supply and relevant extremes of temperature, humidity and pollution
- ensuring appropriate documentation is adequate and complete
- verifying that hardware is capable of performing under expected normal conditions and possible abnormal conditions
- ensuring appropriate security measures are in place and that they conform to appropriate standards
- ensuring that appropriate quality assurance measures are in place
Verification, and validation methods
Model Based Development/Design has become a standard engineering industry operating procedure in CAD/CAE. There are several proven capable and trusted tools for graphical modeling and simulation of commonly engineered systems such as manufacturing, electrical, medical, computational, mechanical, and communications.
Commercial software simulation tools are presently in a highly advanced state of development having long since proven their usefulness and reliability in many engineering fields in the global marketplace. The concept of graphical modeling is simple representation of any physical system by its inputs, a black box containing functional logic and outputs. The approach can be a top-down or bottom-up hierarchical structure within which each a black box may contain multiple subsystems with the lowest level containing the basic logic and arithmetic operations, even down to bit level control if so required.
The architecture of a given physical system is graphically arranged or designed to best simplify conceptualization and understanding of underlying logic. This has tremendous advantages over interpretation of a system by analysis of potentially hundreds of thousands of lines of code.
Hardware-In-The-Loop simulation
Hardware-in-the-loop (HIL) simulation is a type of real-time simulation. You use HIL simulation to test your controller design. HIL simulation shows how your controller responds in real time to realistic virtual stimuli. You can also use HIL to determine if your physical system (plant) model is valid.
In HIL simulation, you use a real-time computer as a virtual representation of your plant model and a real version of your controller.
The desktop computer (development hardware) contains the real-time capable model of the controller and plant. The development hardware also contains an interface with which to control the virtual input to the plant. The controller hardware contains the controller software that is generated from the controller model. The real-time processor (target hardware) contains code for the physical system that is generated from the plant model.
When to use Hardware-In-The-Loop Simulation
Use HIL simulation to test the design of your controller when you are performing Model-Based Design (MBD). The figure shows where HIL simulation fits into the MBD design-to-realization workflow.
Validation involves using actual plant hardware to test your controller in real-life situations or in environmental proxies (for example, a pressure chamber). In HIL simulation, you do not have to use real hardware for your physical system (plant). You also do not have to rely on a naturalistic or environmental test setup. By allowing you to use your model to represent the plant, HIL simulation offers benefits in cost and practicality.
There are several areas in which HIL simulation offers cost savings over validation testing. HIL simulation tends to be less expensive for design changes. You can perform HIL simulation earlier than validation in the MBD workflow so you can identify and redesign for problems relatively early the project. Finding problems early includes these benefits:
- Your team is more likely to approve changes.
- Design changes are less costly to implement.
In terms of scheduling, HIL simulation is less expensive and more practical than validation because you can set it up to run on its own.
HIL simulation is more practical than validation for testing your controller’s response to unusual events. For example, you can model extreme weather conditions like earthquakes or blizzards. You can also test how your controller responds to stimuli that occur in inaccessible environments like deep sea or deep space.
Hardware testing
Hardware testing is usually more detailed and thorough than verification. Testing is needed to ensure that every component of a system is operating as it should, and that the system is performing exactly in accordance with the specific local requirements.
A comprehensive structured testing program is one that ensures that all aspects of a system are tested. This is especially important for key systems such as electronic voting systems. Testing measures that could be followed include:
- developing a set of test criteria
- applying ‘non-operating’ tests to ensure that equipment can stand up to expected levels of physical handling, such as transit drop tests
- examining if appropriate any code ‘hard wired’ in hardware (this code is sometimes known as firmware) to ensure its logical correctness and to ensure that appropriate standards are followed
- applying functional tests to determine whether the test criteria have been met
- applying qualitative assessments to determine whether the test criteria have been met
- conducting tests in both ‘laboratory’ conditions and in a variety of ‘real life’ conditions
- conducting tests over an extended period of time, to ensure systems can perform consistantly
- conducting ‘load tests’, simulating as closely as possible a variety of ‘real life’ conditions and using or exceeding the amounts of data that could be expected in an actual situation
- verifying that ‘what goes in’ is ‘what comes out’, by entering known data and checking that the output agrees with the input
Hardware maintenance
After hardware has been verified, tested and implemented, it must continue to be maintained. Maintenance routines will vary depending on the type and complexity of the technology. Many hardware items will come with a maintenance schedule or program recommended by the manufacturer or supplier. Maintenance could also be provided by the manufacturer or supplier as part of the purchase agreement.
Systems should be maintained to ensure that they continue to perform to the level demonstrated during the testing stage. Ongoing monitoring or testing plans may need to be put in place to ensure that maintenance needs are identified and met when necessary. Where systems are in long-term use, a mechanism could be put in place to monitor feedback from users as another means to determine the need for maintenance and modification.
Where modifications to hardware are made as a result of system maintenance or upgrades, it may be necessary to instigate further rounds of system verification and testing to ensure that standards are still met by the modified system.
