Embedded Systems Engineering Standards Column vol 11.8 November/December 2003 Are Free Tools Worth The Money? By Chris Hills

 

I have to give the usual disclaimer that that these are my own personal views and not those of the ESE Editor and publisher. or those of my employer at the time..... I now work for Phaedrus Systems Ltd!

 

There is a 75% cut down version for print in ESE. The full one is on my web site at www.phaedsys.org Which you are now reading.

 

This month I an extending the definition of "standards" a bit because I want to look at the use of "free" development tools and their effect on the market. This is not a plea on behalf of my employer who is a tools vendor but some genuine questions brought about by several unconnected questions and discussions I have had this month. As usual there is a longer version on the ESE web site and on www.phaedsys.com

 

On the face of it "free" tools are seen as a "no loose" solution scenario for the developer. Linux and the PIC development tools are always given as examples

 

However I don't think it is that simple. One [expensive] compiler vendor I know advertises a very inexpensive "competitors" compiler on his web site as an alternative. He does not sell it but links to them. That is why the title of this item is free tools with a question mark. It is a shot across the bows of the tool vendors, distributors, resellers, developers and some subtle arguments the users need to consider.

 

Software Engineering is one of the few, possibly only, profession that is unregulated and where the practitioners do not automatically buy top quality tools. Doctors, dentists, surgeons, pilots, draftsmen and civil engineers buy quality tools. Even automotive mechanics, otherwise Snap On ™ would have gone out of business a long time ago! Your garage mechanic probably does not buy his tools from the local market where the home mechanic does. My son, a student graphic designer buys the "proper" markers at five pounds each rather than the 99p (a set) markers I would buy.

 

Why do all these people buy expensive tools? Partly because the guarantee that the tools will be of a consistent high quality. Partly because the tools tend to be well-designed. They are probably easier, for a trained professional, to use well than cheaper versions. They will have additional "professional" features. They are usually well supported by the manufacturer and any third party distributor. There will also be a professional pride in using the best tools. In most professions one has to be qualified and registered to practice, this tends to create a professional pride in the work.

 

In software there are various "free" tools. Those that are completely free (eg Gnu) shareware, those given away by distributor s and silicon vendors to create a sale of something else and the student and evaluation versions.

 

Starting in reverse order the evaluation and student versions are sensible products designed to let people evaluate the system. The idea is that if the user can try the tool and see that it does what it claims or is suitable for the job then they will buy the tool. There are many evaluation versions of quality tools that flood the universities so that almost every new electronics engineer knows the "industry standard" and "preferred" tools.

 

Some evaluation versions are time limited and some size limited. This is where it starts to get interesting. A time limited version is usually easy for the vendor to do. It is usually a slight change to the license. A size-limited version takes more work.

 

The advantage of the size-limited version is that it lets students (or any one else) use a tool for a term, 2-3 months, or a year or forever. Whereas a 30 or 60 day licence usually falls over the day before you have to do the final build, report, print etc etc.

 

The vendors will say that the time limited version stops users doing projects for free but in my experience this is a red herring. Most commercial projects are larger than the usual size limits I have seen on size limited tools. I have found that most people doing the evaluation of tools for a new project frequently get called off to things on a previous project and often run out of time on a time limited license any way. They usually require a second or third evaluation licence.

 

In any event where there are very small commercial projects that do fit in the small size limit they are unlikely to have bought a full size tool anyway.

 

This does start to highlight part of the problem. The tool vendor has reasons/excuses for the easy option of the time limit rather than the size limit…. "The easy option" time limited licenses it forces the customer to keep in contact with the tool vendor for extension licenses usually when the last thing you need is a sales rep badgering you because time lines have slipped.

 

Tool vendors should take the time to reassess what is better for the users rather than looking at ways of applying pressure to sell rather than service a customer. This is a short term sales orientated view that I think is wrong.

 

This changes where the users want to evaluate a bit more than just the compiler or development environment and what to evaluate some software with the compiler. In this case the size-limited version is on no use. At this point a time-limited version would be required. These cases are less common and usually require special handling anyway. This comes back to service rather than simply pushing for sales.

On the other side I have seen a case recently where a development team have been supplied with back-to-back 60-day evaluation licenses for the last 10 months by a silicon distributor. The silicon distributor is happy to do this as it gets them sales of parts when the project goes into production. That is all they are interested in. Sales of parts nothing else. This is why, in my opinion, silicon distributors should not be permitted to sell tools.

 

I spoke to the developer, at a social event, who said the company is happy as it saves them buying a compiler, at about 3000 pounds, and in any event they would not buy the compiler as it "had bugs" in…

"It has bugs in" is a reason I often here for not buying SW tools. Well, if people are not going to buy the tools the tool vendor will not get the income to fund continual development or the feed back that there is a problem in the first place! Had the developer bought the compiler the chances are that with a call to the vendor the bug would have been fixed. With the compiler having dubious licence they can not do this for the version they have.

 

This is where the sharp practice of some sales people is going to kill off the tools required to develop the products that use their silicon, but they don't care if they make their profit this quarter. They will sell the cheapest tools they can find to give you "a debugger" to keep costs down. If they can minimise the cost of using a part they will but only in terms of pure cost on the numbers. Many of these sales people have no idea or interest in the development process or the engineering problems in the project. They only work on numbers and this quarters figures.

 

The accountants like this, but have you calculated the cost in time to use these tools? The average cost of an engineer in the UK is £50 per hour. OK! I know that is not what they are paying you (or me)! Factor in Employers NI, Pension, the buildings, heating etc. That is the cost per hour (on average) to a company for an engineer.

 

The "cheap" ICE some distributors sell will cost say 1500 pounds. The expensive ICE will cost you say 3500 pounds. The difference: 2000 pounds or about 40 hours. That is 5 days of your time or about two days in the average team where several people have a go at a problem. I have seen engineers reactions when seeing the "expensive" ICE having used the "cheap" ICE or debugger on a project realise that the more expensive tool would have saved them a lot more than 40 man hours in the project and in fact been the cheaper option in the long run. In many cases they highlight other advantages that would have given them a higher level of testing etc that would bring down the maintenance costs as well.

 

Now I know many of you will say that it is fine for me to say that and you all have horror stories of poor support from tool vendors or distributors. However there are two sides to this as well

 

There are cases where some tool vendors have been less than helpful with support. This goes back to the problem that support is seen as a necessary evil by accounts and they do their best to minimise it. The other problem is that anyone can be an "engineer" some companies hire anyone who can just about read the manual…. This has to change. Tool support needs to be taken seriously by the vendors, resellers and distributors. This is often where the paid support comes in. However this does not always guarantee good results but it does usually have a better rate of success. However, you can vote with your money by not renewing the support contract next year if the support is not up to the mark.

 

Some times I have had "global support" where the company concerned had three very good teams of engineers around the world providing high quality 24 hour support. On other occasions it has been 24 hour support from people in a call centre somewhere (inexpensive) in the world.

 

This is a problem. Support. Some times the support is more user education than support for a tool. There are programmers who still use printf to debug real-time embedded code, who use extremely dubious parts of C and refer to K&R like a mantra , programmers who cannot see the problem with using a monitor to debug a real time application.

 

I recently had a report on a student project sent to me where the student had an evaluation version of a good compiler but had used a highly inappropriate free compiler because he had hit the size limit on the commercial compiler. The problem was that, neither the student or more worryingly the lecturer could see any problem with using this free compiler…. The code compiled and ran as expected…. so far.

 

The problem is, having discussed the particular compiler with the author of a compiler used on and validated for, high integrity systems, that under the covers of the free compiler, no, it's not GCC, there were some basic design flaws and some areas that were " over simplistic" and unreliable. The optimisation was non-existent and whilst the author had a reasonable knowledge of C it was far from perfect. Also the author of the free tool had no contact with the silicon vendors for the parts he was writing the compiler. It was also clear that this tool had never been tested with either of the two standard C compiler test suits. This is not surprising as neither are cheap nor is compiler validation testing.

 

Ok, so it was a student project. It was also part of an engine management system that they tested by running in a real car on the road with real people in the car…. Luck was with them: Nothing crashed. AFAIK the vehicle is still on the road. "somewhere" in the UK. The problem is that they believed all compilers are equal. Nothing could be further from the truth.

 

Another horror story I came across was from a programmer who was not going to "ripped off" with expensive commercial software. So he used a free compiler (not Gcc) . The libraries did not work so he "wrote his own" and the system threw up unexplained errors when compiling and linking two files.. So he wrote it as one c file. It "worked" so the customer was happy. As was the programmer who is charging his standard rate. Will programmers lower their "extortionate" rates if they use free tools? Will they not charge for software they have used on a previous project? I doubt it. Free software is "free" to me so I can sell my software to you it seems.

 

This brings me on to the GCC. This is not a bad compiler, if you get the vanilla version. The problem is that it is not that good either. The other problem is that the ports to different targets vary in quality, speed of bug fixes, features and optimisation. The problem I have had mentioned to me is that if you find a bug there may or there may not be a fix for it and it may or may not turn up "some time" in the future…. The other problem is that it can be fixed by A.N. Other programmer "somewhere" in the world who's knowledge and understanding of C may or may not be as good as your own. As for testing it on a recognised test suite…

 

I mentioned this to a highly regarded figure in software testing who said that Gcc was a real problem. He had to test some Sw that was compiled using the GCC. There were, supposedly, two identical compilers that ran on the same HW under two different OS. No, it is not Windows/Linux on x86. These two "identical" compilers behaved differently and both incorrectly. The problem is that no one knows who is going to fix which version when.

 

I have just had the comment from one developer (at 00:30 on Monday morning!) that it has taken him quite some time to put together the GNU and Cygwin for his target… He had to hunt around for the sources. I bet if he costed it out at 50 pounds per hour he would be surprised how much his "free" tool had cost him.

 

The overriding problem is that accountants who only understand numbers, not engineering, are calling the shots. Also there are many programmers out there who do not understand all the ramifications and features of the tools and the problems who will insist on cheap tools and free support on bulletin boards. If it continues there will be no quality tools only the free/cheap ones. There will be no improvement in tools or software.

 

The silicon vendors will discover for some types of project robust, well constructed, and probably validated, tools will be required. See the previous columns on Corporate Manslaughter and Licensing Engineers. Then these tools will have to be re-born. However then you will have to buy the tools from the silicon vendor at the price they set. There will be little or no competition because there will be no creditable 3rd party tools. Several companies have had their own tools but most, in the end, realise that the best tools come from commercial competition.

 

So free tools may cost more to own and use than commercial tools with a higher up front cost. There is a hidden cost in time and some of the silicon distributors are making things worse with a short-term numbers view of the market doing anything to sell the chips. I will conclude by quoting P.J Plauger the columnist in the much missed Byte magazine and C Users Journal, C standards Guru and force behind the Dinkumware libraries "Fortunately for us most commercial companies can not afford to use free software [& tools]"

 

Author Details and contact

 

Eur Ing Chris Hills BSc CEng MIET MBCS MIEEE  FRGS   FRSA is a Technical Specialist and can be reached at This Contact

 

Copyright Chris A Hills  2003 -2008
The right of Chris A Hills to be identified as the author of this work has been asserted by him in accordance with the Copyright, Designs and Patents Act 1988