logo slogan
Phaedsys Logo

Embedded Systems Engineering
Standards Column
vol 13.8
Nov/Dec 2006

Happy Christmas:It's a MISRA Test Suite

By Chris Hills

Chris Hills

 

These are my own personal views and not those of my company Phaedrus Systems. The full version of this column resides at www.phaedsys.org, under the Technical Papers button.

 

Well I survived the hordes at ESS and surprisingly very few GCC fans that wanted to lynch me! I will come on to my comments on GCC and the feedback shortly. Also news on the MISRA test suite and cautionary tale for exhibitors on how to get arrested at ESS!

From my point of view ESS was bigger and better this year. I know of half a dozen engineers who have been helped greatly because they discovered things at the show (tools, information, ideas) that they did not know before. In several cases it will save them thousands of pounds. Several found out which major silicon company is about to pull the plug on one of it’s major lines. Now, whilst I can’t promise a trip to ESS will save you thousands or in one case save the project no one said the show was worth not attending! By the way thanks to those who turned up on my stand with doughnuts and chocolate biscuits to go with the coffee!

On a similar line about information for engineers struggling to complete a project I have an unusual tale. I have to phone customers following contacts that come in to me in a variety of ways: an email, a business card, a personal contact or a referral from a colleague on another project. When phoning a company you get a variety of responses. From a direct dial number to the person you want to talk to, usually with it’s own voice mail these days, through to metaphorically grappling with the receptionist.

It is the receptionist syndrome that can be the problem. Sometimes you ask the receptionist for a person and there is a click and without warning you are talking to them. Others ask who you are and which company. I always thought this was so they can tell the victim who is about to talk to them or not. Though quite often I find that they just note it in their log (I assume) and pass the call through without saying who is calling! Other receptionists come back after a pause and ask for more information. The Engineer has the receptionist filtering which is fair enough.

There was one response I got recently which left me stunned. The response was: “Our engineers are too busy!” When I queried what she meant I was told that their developers were not permitted to take to external calls from suppliers. It distracted them from their work. No matter that I was responding to a contact from them.

I thought the time was long gone when engineers were kept like battery hens and not permitted outside contact. Though we are returning to some limits on Internet and web access but this is more for security due to the huge amount of spam, viruses and other nasties that we are plagued with these days.

Though I can understand it if the engineers are constantly plagued by cold calls. Does this still happen? I thought that these days email had replaced the telephone for cold calling. However, after some discussions recently with one sales rep he said you should phone potential customers every week regardless of whether they have said they didn’t need a call for a month or three. Another supplier I know of said that if the customer was not sure which of several solutions they wanted then several of their sales reps would each call the customer and compete for the business. Apparently, the supplier thought, having several sales reps chasing the customer showed commitment! I think it would be more of a nuisance. What do you think.

However the worst “nasty technique” that was described to me recently defies belief. I am told it comes from the 70’s and the person who told me had never used it but did name the company who taught it. This was allegedly from a big company selling high value equipment. The people they were targeting were middle and senior management. It was “The Flat” ploy. Apparently if you were cold calling a named manager but they wouldn’t let you through you could call back a day later and awkwardly decline to give a name or company but insist you had to speak to Mr X. As a “last resort” you would confidentially tell the receptionist it was a “delicate matter regarding the flat he rents” and you are “the landlord”. Back in the 1970’s about 95% of the managers they were targeting were married so this implied it was the flat of his mistress, which is why the landlord was calling the office rather than his home. Apparently his would usually get you through. Though it was reckoned that you had 10 seconds to get the hook in on your pitch. The only worry about the false rumour of a “love nest” was the manager might be moved before the sale was far enough advanced! Marriage wrecking office rumours were not seen as a problem…. and salesmen wonder why they have a bad name.

Back to proper engineering: I bring news of Good Cheer for Christmas though I should state that these are my own views and in no way an official statement from MISRA-C, MISRA or MIRA. The Good News is MISRA may have released the MISRA-C:2004 Exemplar Suite by the time you read this. Well we are doing the final run through and proof check in the second week of December. Several things to note here: It is an EXAMPLE suite NOT a test suite. It is nothing like comprehensive enough to be a test-suite. We do intend to improve and expand it but a full test-suite is a very large and complex thing that is beyond our resources at the moment.

For MISRA-C3 we have discussed that we will do example/test code for each rule as we go. Whether we can keep to that remains to be seen. If we can C3 and the Exemplar suite should be launched at the same time.

The Exemplar suite is a, non-exhaustive, set of examples of what should pass and what should fail a rule. There will be [at least] one file per rule. I think we also decided to have the legendary “this file is intentionally empty” on rules where there is no example. This is so you know you are not missing a file. One of the problems we had is cross talk i.e. to set up some things to fail a rule you have to break other rules. Also whilst there are “Compliant” and “Non compliant” lines marked this does not mean your checking tool will raise a particular flag at precisely that point.

As the Exemplar Suite is source code it is a little difficult to control it’s distribution so it is likely to be available without charge to end users but it will not be GPL or Open-Source and copyright will remain with MISRA.

Following hard on the heels of the MISRA-C:2004 Exemplar Suite, we hope will be the Technical Corrigendum for MISRA-C:2004 this is now in draft stage. We have to get it done as the Exemplar Suite is based on MISRA-C with the TC changes. This we expect to be on free download as a PDF. It will only contain the changes and hopefully will be available in January.

Staying with MISRA-C matters: An interesting thing I have seen from the MISRA forum and talking to some customers are questions on how do I do so and so “in MISRA”. Where what they are asking is clearly bad practice. The worrying thing is they are innocently asking this in public. They clearly do not know or seem to understand the bad practice. In many cases it is ways of getting round the letter of a MISRA-C rule without having to deviate.

This is rather sad that instead of trying to improve their Software Engineering Skills where rules are clearly trying to prohibit something they are more interested in hacking so they can still use it. The answer is: Raise a deviation and show it to some one else. It both you and the other person think you can use that deviation in a years time a court of law, in front of a jury, as a defence then OK. If you have any doubts don’t do it. That, by the way is my personal opinion and not that of MISRA etc.

The other problem I have come across recently is people looking for MISRA-C checkers. What is the problem with that? Well the problem is some of them don’t have static analyzers and they are just looking for checkers to check for the rules and not a static checker that also does the rules. MISRA—C, if you read chapter 4 in the sections: 4.2.3 tool selection and validation, 4.2.4 Source complexity metrics, 4.3.3 formalization within quality system it assumes that static analysis will be used. Section 2.2 Objectives of MISRA-C states: “Great emphasis is placed on the use of static checking tools to enforce compliance with the subset…”

This again saddens me as it means people are looking to “tick the box” rather than improve their software engineering. I am not making this point as some might think because I sell static tools. The ones I sell are low cost with a wafer thin margin, not enough to put shoes on my children’s feet at Christmas etc. besides there are other free static tools out there anyway.

It appears that rather than reading the first five sections of MISRA-C they are just implementing or in many cases trying to subvert the rules in section 6. This seems somewhat perverse. I am not sure why programmers what to spend time getting round rules rather than looking for better solutions.

Well I have better respond to the comments I have had about my column in September regarding GCC. I had several communications on the subject. The responses here are a conglomeration of those responses as they overlapped a fair bit.

One comment was that I overstated the case on rebuilding old compilers. The responses were: Yes, it will produce differences when not built on the original hardware with the original tools but the larger differences will be in the implementation of the language between versions.

This is correct. There are two main reasons for wanting an old version. It is either because the old version is the one that was “certified” for use on the project. Though in these cases they tend to archive the tools with the project. The other reason tends to be for language compatibility. This is often because only a small change is required in a large program and it no one really wants to convert all the code to compile in the latest version of compiler because of changes in the implementation of C. Apart from the effort to modify the code it could modify the behaviour of the program in unexpected ways.

However as I pointed in the first article: all commercial compiler companies I know can, and most to my certain knowledge have, provided original binaries of any version of their compilers. With the commercial compilers as you can get exactly any version of the compiler as originally used. Therefore there is no advantage in having the source and in fact this is a disadvantage. You need the original compiled binary if there are archives of the original binaries of each version of GCC then fine. As I can run most Win95 games on my WinXP box and any Dos program in a command window I can’t see a problem with running an old commercial compiler binary on a modern PC.

The original binary will accept the old code and compile in exactly the same way with exactly the same behaviour. A recompiled GCC will accept the old code but may not behave in exactly the same way as the original. You don’t have that variable with commercial compilers.

One of the comments regarding the source of a compiler was “ Compilers are not that complex, commercial ones maybe (but hell you’ll never know)” Well I can tell you that from several sources the GCC is more than 5 years behind commercial compilers and a lot further behind ones in the 8 and 16 bit area. Modern compilers are complex.

However I have discussed the point of editing the source of compilers with various compiler writers and they maintain that the average user should not be playing about inside a compiler. Compiler writers intimately know their compilers having designed and maintained them. They also have a full suite of regression test tools that the Gcc user does not have. So you can edit the source but you have no real way of knowing if you have cleanly fixed the problem and not introduced another problem. I know Gcc comes with it’s own test suite but full testing of a compiler requires so much more.

The point where I have to agree fully with one respondent is that a lot of it boils down to LEGAL LIABILITY. When you buy a commercial compiler the responsibility is on the compiler vendor. With Gcc it is with YOU the user to do the testing and ensure it is suitable for use. This is both expensive and time consuming. I am not a lawyer but I doubt very much that simply running the GCC test suite will suffice as testing the compiler. So if you use Gcc without doing some rigorous testing you, the user, could be liable for any bugs in the compiler.

The problem of liability comes with bug fixing. With the commercial companies the onus is on them to fix the problem and retest the compiler with Gcc fixes arrive from somewhere but who retests the compiler and takes responsibility? One respondent said that [when fixing bugs] the same situation with GCC (or any open source) is that the bug gets fixed but more importantly everyone else gets to share in that experience, learn from the problem, propagate the fix etc. My comment here is “everyone else” is in the commercial world employed by companies to write applications not fix compilers. Which comes back to my point that there are hidden costs in owning a Gcc compiler. Also I suspect that many are using GCc not for any altruistic or “open source” reason but because it is “free” they are not interested in helping to fix bugs. I would suggest that 99% of project managers will not be interested in setting aside time to help bug fix or patch the tools.

As for bug fixing I do know of a case explained to me recently where a Gcc compiler was used on some hardware under two different OS. The two compilers exhibited an error. When the bug fixes came, one for each compiler the compilers were patched it was found the fixes caused different side effects in two compilers…

Some comments were that modern compilers seem to be concentrating on the IDE and debuggers. Well this is largely in response to customer requests, besides you need to remember that all compiler writers are software engineers too. It was then pointed out that with Gcc you only get the compiler and you have to put together your own packages such as target specific tools, debuggers and boot-loaders etc. Once again you have to spend time [which equals money] to put all this together or, usually, buy a pre-packaged set up. So Gcc is not as free as people think. It appears the solution used by some Gcc users is to compile and run the code on a PC to test it the recompile for the target. I would rather compile in a cross compiler and use a target specific simulator. I suppose it is OK if you are targeting a 32 bit target and using a 32bit PC but where the target is an 8 or 16 bit system things can behave differently.

One defence of gcc was that whilst not the best for any architecture it is constant. That is there is a Gcc for virtually every architecture and buying the best commercial compiler for several architectures will get expensive. However the problem is that Gcc is not really competitive in the 8 and 16 bit architectures.

Interestingly in ALL other trades and professions the suggestion that using a general purpose tool that is “not bad” as opposed to a set of specialised tools would probably get you thrown out of the trade or profession. Ask a carpenter what he thinks of swapping his set of six different saws for a general purpose one. Going back 60 or so years the cost of good carpentry tools would be the same to a carpenter as several compilers are now in real terms.

No one suggested using Gcc for any safety critical work. Though I understand there is one version but this is only supplied and maintained by the company that supplies is as part of a specialised package. Cost would not be a factor in getting this version I understand.

One of the more interesting comments I had about GCC was with regard to a version supplied with some other commercial items. I have removed the names because I don’t think they are intentionally trying to subvert GCC and I can think of quite a few other companies who are doing exactly the same thing, again without malice. What was sent to me was:-

 

Here is an example of a possible infringement of the GNU licence straight off of the ***** CD that I received from ***** last week. These are parts the GNU licence allowing redistribution of the GCC compiler:
"3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: "

"a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, "

"b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, "

"c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) "

So you see there must be source code provided or a written offer to provide it. I can find no source code neither can I find a written offer to provide it. If that is the case then the GNU license is not being complied with. The consequences of non-compliance are described in section 4 of the GNU license:

"4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance."

So if in fact there is no written offer to supply the source code, ***** has terminated its' right to distribute the GCC compiler. Presumably ***** will also have infringed by distributing the CD to me and if you have sent out a similar CD then presumably you will have infringed too! The only organization that can enforce this is the Free Software Foundation who own the copyright of the GCC compiler.

 

The point is that most people who supply GCC as part of a tool chain usually think that as long as you can download it freely from somewhere on their web site they are covered. Well it appears not to be the case. However I am sure the gcc debate will go on for a while.

A cautionary tale from ESS that ALL exhibitors need to read… I discover this some days after ESS from the somewhat embarrassed from the person it happened to. He had a drink or two at the after show party for exhibitors. He then went to his car in the NEC car park. Realising that after a day in a hot show with little food and water the two (or three) drinks had probably put him over the limit he decided to get a hotel room on the NEC site and not drive anywhere. However having sat down for the first time in about 10 hours he decided on a short nap before walking over to one of the hotels on site. About 30 minutes later he was woken by the NEC Security asking if he was OK? So he explained to security the situation and that shortly he would walk to the hotel across the site and was it OK to leave the car there overnight? Security seemingly satisfied went away. He dozed off again.

About 45 minutes later there was another knock on the window. A policeman from Caste Vale Police Station. So the whole story was repeated along with the comment it was probably time to go and book into one of the hotels on site. The Police response was to arrest him for being Drunk In Charge of a Motor Vehicle! So rather than helpfully driving him over to the hotels on the NEC the trip was to a police cell in Castle Vale. Unfortunately this was without the overnight bag that was left in the boot.

The following morning the CPS decided that as no attempt had been made to move the car… apparently the police went though the CCTV of the car park and that the driver had already stated, to NEC security who also said the car had not been moved, that the car would not be even be driven to the hotel on the NEC it was not likely that they could get a conviction. However the Inspector still wanted to push the case despite the PC saying there was no point. We believe this lead to the police being awkward and not releasing him early. He had made it clear they were working on a stand from 09:30.

In the event, without breakfast (or any food the evening before) he was released in to the middle of Castle Vale some miles from the NEC at about mid-day, hungry in need of a shave, shower and a change of clothes. Even in the middle of the day Castle Vale is not a place I would want to wander round. One of the jokes about the place is you can loose your alloy wheels if you slow down at traffic lights! Fortunately another PC took pity and gave him a lift back to the NEC. By the time he got back to the car, in need of a shower and change of clothes there was little point in growing back into the show and onto the stand.

So take care when at the NEC. I am sure that the NEC Security have chapter and verse on their regulations and rules to back them up for calling the Police. I am sure the Police can find, somewhere, some sort of justification. However it just seemed to be a “jobs-worth” attitude and a total waste of resources by the Castle Vale Police. Probably it was done just to get another point on the “crime solved” ratings. I know they have far more pressing matters to deal with and real criminals to chase. The other thing to note is they now keep finger prints and DNA from anyone they bring in, even if they don’t press charges.

Well I hope the last column of 2006 has given you something to think about. As noted in my “Two Weeks in the Summer” columns do have a break at Christmas. The mind needs to rest.

 

 

Author Details and contact

 

Eur Ing Chris Hills BSc CEng MIET MBCS MIEEE  FRGS   FRSA is a Technical Specialist and can be reached at This Contact

 

Copyright Chris A Hills  2003 -2008
The right of Chris A Hills to be identified as the author of this work has been asserted by him in accordance with the Copyright, Designs and Patents Act 1988