Compiler Validation Service

Phaedrus Systems can supply compiler validation for C compilers.  This is Validation not Certification because at the current time there is no Certification for C compiler as there is/was for Ada or Pascal compilers.

Compiler validation is essentially the highly controlled, repeatable and reproducible testing of a compiler using a validation suite - a recognised set of test programs. The purpose of such testing is to provide a reliable indication of how well a compiler complies with the standard for the language that it implements.

Commercial validation suites are available and Phaedrus Systems distributes what it regards as the leader in the field, SuperTest from Solid Sands. However, validation is complex and, like much in software, benefits from previous experience, which is why Phaedrus Systems has developed a compiler validation service.

The service will be led by Olwen Morgan, who has over 40 years’ experience in software engineering, mostly in high-integrity and critical systems. Her experience in compiler validation dates back to the early 1980s when she worked on the joint BSi/NPL/NCC project to set up BSI's Pascal compiler validation service. She was among the first in the UK to perform compiler validations on microcomputers and prepared the final version of the User's Guide to the Pascal Compiler validation suite. Later becoming involved with C, she oversaw the development of a suite of programs to test the error-handling behaviour of C compilers and has conducted several compiler validations for large organizations and with compiler developers. She was also the original author of the rules on which MISRA C was based. She continues as a consultant advising clients on software quality assurance in general and compiler validation in particular.

You may ask don't compiler companies test their own compilers?

The thoroughness of compiler testing varies widely among compiler vendors. Some have state-of-the-art testing laboratories while others make do with much more limited facilities. Some use only one validation test suite while others use all available suites plus randomly-generated stress tests. Some test on-target, others only on-host. Also, owing to the large number of options in modern cross-compilers, particularly for C and C++, it is very unlikely that the options under which a vendor performs testing and validation will coincide with those used for any particular embedded application.

 

On-Target Compiler Validation

As of 2016 Phaedrus Systems is seeing more of a requirement to test On-Target. That is a validation of the compiler on the actual project target hardware using the compiler switches to be used in the deliverable system. This is a far more complex task that generic or desktop validation. A single incorrect parameter or test can render the whole validation as flawed.

 

Compiler Validation Information

	C Compiler Validation for Embedded Targets		Repeatability and Reproducibility: Why testers sweat the details
	Pre-tests for Compiler Validation		Stress Testing Compilers
	C Compiler test suite selection		Code Generator Validation
What is a compiler? Coming Soon   The mathematician Felix Klein once said that everyone knows what a curve is, until he has studied enough mathematics to become confused through the countless number of possible exceptions. . The situation is similar with compilers	What is a compiler ? Coming Soon		Compiler Benchmarks

 

See Also

Some interesting comments from 2011 on Compiler Validation from Derek Jones who produced one of the the first fully validated C compilers (in 1991) click here

> ISO -C

> JSF++ Standard

> MISRA

> ISO Standards

> SCSC

> Quest

> Engineers Guides

> Compiler Validation