Managing risks related to the obsolescence of safety-critical embedded software verification frameworks

by Pierre-Henri Stanek, Cantata Product Manager
15.05.2020 Unit Testing,Code Coverage

 

Most of us live in a world surrounded by technology. It has been a major source of progress and comfort but… technology becomes obsolete rapidly. The newer the technology is, the more the obsolescence cycles are shortening. This is quite true for a smart phone or the latest TV sets, but can we consider that quick obsolescence is applicable for advanced industrial products like an aircraft, a rocket or even a nuclear plant? Well, these systems require much more time spent in design, development and field deployment… Even though the innovation efforts engaged are significant, they are rather slow compared to consumer devices or enterprise IT systems. Thus, obsolescence cycles are somewhat longer, but still a major source of risk for guaranteeing the proper operation of these industrial systems that need to last for decades and continue to operate safely.

Obsolescence is one of the key elements that can increase the safety hazards for critical industrial systems. Working for a company providing industrial-grade solutions for the software verification of safety-critical systems, I have seen many customers trying to protect themselves from obsolescence in different ways. I must say that the risk related to hardware issues is well understood and managed. Sufficient units of mechanical, optronic and electronic spare parts are stored during many years in proper conditions of temperature and humidity. Beyond hardware spare parts, computers operating critical software are regularly renewed or maintained as-is with the adequate Operating Systems (or at least OS variants and versions compatible with the applications to be operated). But what about software?

 

Software is often seen as “eternal” or at least not decaying. As long as engineers can retrieve a compiler which can generate the machine code for the CPU architecture they feel protected from software obsolescence. Instead the focus is put on the archiving capacities, such as the move we have all experienced in the past years from floppy disks, to hard disks, to CD/DVD, and finally now to cloud-based solutions. However, while software is technically eternal, the tools for maintaining and verifying it are not…

Embedded software that can affect the safety of people or damage the environment is certified in the context of many industrial standards: ISO 26262 (automotive industry), EN 50128 (railway), IEC 60880 (nuclear), DO 178B/C (avionics), IEC 62304 (medical), etc. As we previously mentioned, many of these software units need to operate for several decades with a defect rate close to zero. However, when defects are detected the embedded software needs to be modified, or if the system requires modifications due to an optimization or a must-have feature requested by the end customer. Thus, stable software could undergo evolutionary changes during a maintenance phase and require re-verification efforts.

 

As previously said, de-archiving software source code, finding the proper compiler and regenerating the object code is often not a real problem. Engineers, however, have bigger challenges to face if the tools used to manage the applications lifecycle (requirements management, version control, testing…) are not usable anymore. Indeed, the software vendors may have simply gone out of business, or decommissioned old versions that are not maintained anymore. In addition, old license incompatibilities may prevent development teams from running the tools. For ensuring proper software verification evidence, that may end up being a very critical situation.

 

How can the software verification be maintained? Shall the developers restart testing all the code manually from scratch, or at least with the help of a recent software testing tool? Unfortunately, this is not realistic (time and money wise) for complex systems which have required many man years of verification and validation efforts. Shall they look for a modern verification solution that offers the generation of an automatic test safety net and restart from this basis of passing tests, knowing the software is already “proven in use”? Well, it’s a pragmatic option that may be applicable in some cases, but in many safety cases this will be inadequate for standards compliance certification or enough to convince a Designated Engineering Representative of suitable safety qualification.…

 

QA Systems has a lot of experience over the past 20 years with code parsing for the purpose of (automatic) test generation. Analyzing the source code and test scripts coming from a 3rd party legacy tool in order to generate the equivalent test framework in our Cantata software verification tool, is something we have mastered. We can provide you with a test Converter which automatically:

So, what’s left in terms of migration options? Converting all the existing test scripts and test cases from the legacy testing tool to a modern tool which makes the proper conversion of all test vectors, stubs and coverage probes in order to re-rerun them with your target environment, and thus retrieve identical test evidence used to claim your certification credits? That’s certainly the best option to move away from a risky tool obsolescence situation, but is it possible to implement such a “tool to tool” conversion bridge in a professional and secure way?

 

 

 

  • converts 3rd party test scripts and test cases
  • retains the test environment
  • retains the overall verification scenarios…

 

…while granting the test extensibility for new features and verification evolutions.

 

This approach and technology has been developed and co-experienced with major industrial accounts facing the problem of testing tool obsolescence. While the process of migrating from legacy test tools has cost implications for any organization, the automation of that migration via direct test conversion, offers a significantly easier route to modern unit and integration testing. If you are interested in a concrete use case, please refer to this article published in the French reference magazine “l’Embarqué” (article available both in English and French).

More to consider

 

If you wish to read more, QA Systems has published a Testing Tool Converter feature brief (available both in English and French) on the web site that covers the same topic.

 

If you are interested in trying the Tool to Tool Converter for your own software verification projects, QA Systems team can be contacted by e-mail at info@qa-systems.com.

 

You are also welcome to sign up to the QA-Systems newsletter… You will receive notifications of other useful software development content straight to your inbox.


RELATED RESOURCES
Start
Trial
QA-Systems