• Technical Support
  • Contact
  • Blog
  • Italiano
    • English
    • Français
    • Deutsch
    • Português
    • Español
    • 简体中文
  • Strumenti
    • ANALISI STATICA
      • QA-MISRA
        Analisi approfondita per codice C/C++ su scala industriale
      • Cantata Test Architect
        Comprendere, Definire e Controllare l’Architettura del Software
      • Metriche del Codice Sorgente
        Metriche automatizzate del codice sorgente per C/C++
      • Get a demo
    • TEST del SOFTWARE
      • Cantata
        Test automatizzati di unità e di integrazione per codice C/C++
      • Cantata Team Reporting
        Elemento aggiuntivo per cruscotto di gestione dello stato dei test
      • AdaTEST 95
        Test automatizzati di unità e di integrazione per codice Ada
         
      • Why Pay for Unit Test Tool?
  • Soluzioni
    • SETTORI
      • Automobilistico
      • Energia
      • Dispositivi medicali
      • Automazione industriale
      • Ferroviario
      • Aerospaziale e difesa
      • Critico per l’azienda
      • E
      • E
      • Get a demo
    • NORME DI SICUREZZA
      • ISO 26262
      • IEC 60880
      • IEC 62304
      • IEC 61508
      • EN 50128 / 50657
      • DO-178C
      • MISRA
      • AUTOSAR
      • CERT
      • Why bother to unit test?
  • Accademia
    • Seminari
    • Formazione
    • Webinar
    • Speaker
    • Modulo di registrazione
  • SOCIETÀ
    • Chi siamo
    • Management
    • Partner
    • Newsletter e comunicati stampa
    • Eventi
    • Carriere
    • Get a demo
  • Resources
  • Porsche
  • Italiano
    • Inglese
    • Francese
    • Tedesco
    • Portoghese, Portogallo
    • Spagnolo
    • Cinese semplificato
  • Strumenti
    • Analisi Statica
      • QA-MISRA
      • Cantata Test Architect
      • METRICHE DEL CODICE SORGENTE
    • Software Testing
      • Cantata Team Reporting
      • Cantata
      • Adatest 95
  • Soluzioni
    • Settori
      • Automobilistico
      • Energia
      • Dispositivi medicali
      • Automazione industriale
      • Ferroviario
      • Aerospaziale e difesa
      • Critico per l’azienda
    • NORME DI SICUREZZA
      • ISO 26262
      • IEC 60880
      • IEC 62304
      • IEC 61508
      • EN 50128 / 50657
      • DO-178C
      • MISRA
      • AUTOSAR
      • CERT
  • Accademia
    • Seminari
    • Formazione
    • Webinar
    • SPEAKER
    • Testimonial
  • Societa
    • Chi Siamo
    • Management
    • Partner
    • Eventi
    • Carriere
  • RESOURCES
  • Contact
  • Blog
autotest

AutoTest

Automatic test case generation

autotest

AutoTest

Automatic test case generation

QA Systems > Strumenti > Cantata > Autotest

AutoTest automatically generates passing Cantata unit tests to exercise all the C and C++ source code. It can be used on new or legacy projects, making it easy to:

  • Configure automatic test generation
  • Identify code testability issues
  • Generate tests with full code coverage
  • Plug ‘edge case’ gaps in coverage
  • Create a safety net of baseline regression test
  • Upgrade from legacy test tool to Cantata

What is Cantata AutoTest?

Cantata AutoTest creates complete unit test scripts or adds test cases to existing Cantata scripts. It parses the C or C++ source code to determine all possible paths through the code as required by a structural code coverage metric target:

  • 100% function Entry-points
  • 100% Statements
  • 100% Decisions
  • 100% Unique Cause MC/DC

An algorithm creates test case vectors which exercise all required code paths, using the Cantata powerful white-box capabilities to set data, parameters and control function call interfaces.

The test vectors drive the code, and check the parameters passed between functions, values of accessible global data, order of calls and return values.

AutoTest generated cases, are editable in the same ways as user generated cases, and each test case has a description of what path through the code it was created to exercise, making them easy to maintain.

What does AutoTest help me do?

CONFIGURE AUTOMATIC TEST GENERATION

AutoTest generation options can be easily configured via the GUI or command line:

  • Code coverage required (Rule Set)
  • How function calls are simulated or intercepted
  • Whether to access global static variables
  • Whether to access global static variables
  • What data to check (i.e. global data, function call arguments or return values);
  • Whether global data is modified during function calls to force paths;
  • Generation max limits on time, paths, arrays etc.

Selecting a code coverage Rule Set will determine the depth (i.e. number of test cases needed to achieve the target). The style is determined by use of the AutoTest specific and standard Cantata Workspace code generation options.

Isolation unit tests or cluster integration tests, using black-box or white-box testing approaches, provide flexibility to fit with whatever initial testing thoroughness and test maintainability for code refactoring is required.

IDENTIFY CODE TESTABILITY ISSUES

Where AutoTest is unable to generate tests, or the tests are incomplete, the AutoTest Generation Report provides useful insight into any testability issues within the source code.

Warning messages are written to the AutoTest Generation Report, showing where and why it was not possible to generate test cases, indicating possible problems in the source code or the ability to dynamically test it. It can therefore be a valuable supplement to code review, even if AutoTests are not dynamically run.

GENERATE TESTS WITH FULL CODE COVERAGE

The Cantata AutoTest Generation Report identifies the expected code coverage which will be achieved by executing the generated tests. As AutoTest cases are initially generated to pass, the only failing checks will be where a Coverage Rule Set checks achieved coverage against a target.

Achieving full code coverage is of course only part of the testing objectives. AutoTest generated test cases can be reviewed and where suitable mapped to requirements (see Cantata Requirements Traceability capability for more details). In this case, AutoTest dramatically reduces the developer effort to select test case vectors to verify the requirements.

PLUG EDGE CASE GAPS IN COVERAGE

Where complex code is being tested, even detailed requirements, and thorough robustness tests may leave untested execution paths in the code, because the different combinations of input vectors required for full coverage are difficult to identify. For these ‘edge cases’, AutoTest can generate test cases to plug any gaps in code coverage if the code is not redundant.

AutoTest can be used on just selected functions to generate a focused set of test cases achieving 100% coverage. Cantata automatic code coverage optimisation can at a single click delete/disable any test cases which do not add to the overall code coverage achieved.

CREATE A BASELINE REGRESSION TEST SAFETY NET

Legacy code that relies on system testing is very expensive and inefficient to modify, but writing a comprehensive set of unit tests from scratch is simply not commercially viable. Cantata AutoTest automatically generates a suite of unit tests for all selected code. These tests can act as a baseline safety net against future regression errors as code changes. This reduces reliance on expensive and time consuming system tests as well as being more thorough and pin-pointing errors precisely.

Cantata AutoTest generates a set of Cantata Makefiles, these can be easily integrated into any open source or commercial CI tools to enable fully automated testing during development, at every integration.

UPGRADE FROM LEGACY TEST TOOLS

Even where other unit testing tools are already in use, there can be real advantages in upgrading to the modern powerful test facilities available in Cantata. This especially true where legacy tools are not well supported or are not qualified for safety standards tool certification needs.

Cantata AutoTest provides two highly automated routes for upgrading unit test tools:

For a code base with an accepted level of quality, existing tests can simply be replaced by an AutoTest generated set of Cantata unit tests.

For legacy unit tests written in C or C++ (e.g. xUnit tools), existing tests can be incorporated into Cantata test scripts, retaining investment in legacy test cases and then enhanced with additional Cantata capabilities including AutoTest.

How does AutoTest work?

AutoTest can be invoked using a Test Script Wizard, or it can generate additional test cases in an existing test script in the Test Script Manager view. For larger code-bases it may be more convenient to run AutoTest from the command line.

AutoTest parses the source code and uses Cantata options to generate:

  • AutoTest Generation Report, which provides both summary and detailed information on the tests that have been generated.
  • Cantata test scripts (or additional test cases to existing scripts)
  • Cantata Makefiles to build the test executable

The generated tests are then built and run, producing the standard set of Cantata results and reports.

AUTOTEST GENERATION ALGORITHM

Cantata AutoTest creates complete Cantata unit test scripts from source code using an algorithm that automatically generates test case vectors that exercise code paths, check data, check parameters and record call orders. The code paths used to generate the test cases are determined by selecting the metric types (Entry-point, Statement, Decision or unique cause MC/DC) in a code coverage Rule Set.

Preferences are available to influence the AutoTest generation algorithm, including the required level of code coverage, the method of controlling calls, what types of data to check, how data may be modified to force execution paths, time limits, path limits, and many more. These preferences affect the style of the Cantata tests, and limit the test generation scope.

Tests can be more or less thorough, and can use white-box or black-box approaches. The decision on what style of unit tests to generate with Cantata AutoTest will depend on regression plan objectives and how code changes. For example, allowing flexibility for code refactoring of static functions without breaking black-box tests.

AUTOTEST WARNING AND ERROR MESSAGES

Where Cantata AutoTest generation results in incomplete tests or encounters suspect code quality, warning messages are written to the AutoTest Generation Report. These warning messages indicate possible problems in the source code or its dynamic testability, so they can be of significant value.

Cantata AutoTest will report:

  • unreachable code, where it cannot generate test cases to exercise all parts of the software under test
  • potential crash scenarios such as null pointers, an out-of-bounds array index, decisions using uninitialised variables, and division by zero
  • type truncation, where type casting might result in data loss
  • uninitialised data use, such as a field within a large structure
  • non-constant static data within a function, as its value will be unknown at the start of a test case
  • non-returning functions, where a function does not return control to the caller – although Cantata does have the capability to work around this for testing purposes.
  • implicit function declarations, where a function is called before it has been declared
More information about Cantata Autotest

For more detailed information about Cantata Autotest please see:

Cantata Feature Brief – AutoTest

START TRIAL
GET A DEMO
BACK TO CANTATA

QA Systems GmbH
Roggenstrasse 11
71334 Waiblingen | Germany

© 2021 QA SYSTEMS GMBH

Phone: +49 (0)711 138183 -0
Fax: +49 (0)711 138183 -10
E-Mail: [email protected]

  • Note legali
  • Politica in materia di riservatezza
  • Contact
  • Resources

Newsletter

Get the latest technology news updates, product releases plus tips and tricks:

subscribe to newsletter
  • Segui
  • Segui

Newsletter

Get the latest technology news updates, product releases plus tips and tricks:

subscribe to newsletter
  • Segui
  • Segui

Phone: +49 (0)711 138183 -0
Fax: +49 (0)711 138183 -10
[email protected]

  • Note legali
  • Politica in materia di riservatezza
  • Contact
  • Resources

QA Systems GmbH
Roggenstrasse 11
71334 Waiblingen Stuttgart | Germany

© 2020 QA SYSTEMS GMBH

Start
Trial
×

Login

Hai dimenticato la password?

Or

 

Create an Account

Sign up for access to premium content
Register for full access to our extensive resources library.
  • Questo campo serve per la convalida e dovrebbe essere lasciato inalterato.

×

Autotest

Academy Registration Form

  • Contact details attendee

  • QA Systems arranges a block booking of rooms for seminar participants at the respective hotels, usually with special rates. The participants are responsible for their own reservation, payment, etc. If you wish to receive the hotel information, please tick:
  • Questo campo serve per la convalida e dovrebbe essere lasciato inalterato.