Coders Write Specifications

Coders write code. Should they not write down specifications like a darzi? Let me describe an example.

I go to a developer and say, "I want a routine that gives me the roots of a quadratic." (user requirement).

In less than an hour I get an executable "quadratic.exe". The developer asks me to run it. I get prompt a prompt: "Please enter a: ". The developer informs me what is "a" and also what the following two inputs "b" & "c" are.

       I enter 0 for "a" and 1 for "b" and "c". I get a divide by zero error. I am expecting the result to be       -1 (x + 1 = 0)
       I enter 1 for a, 0 for b, and 1 for c. I expect to get imaginary 1 (j1). I get a NAN.
       I enter 1 for a, 0 for b, and -2 for c. I expect 1.414. I get 1.414213
       ....

I did not get what I wanted. Should I be disappointed? No. What I got was a prototype.

Prototypes are very useful. They help to refine user requirements and specifications. I do that now. Also, I add that I require to have  a function that will compute the roots and return it in a structure. I then define the structure for the developer.

The developer should now put the requirement and specifications in the function header. And it should be done before making any changes to the source code. As a matter of fact there should have been a header for the prototype code.

Function headers, and file headers are not comments. Comments are post action. Comments are made by Commentators.

Function headers, and file headers are requirements and specifications. They precede the action. Acts need to have a purpose; a desired result. They need to be performed when required preconditions are met.

Cricketing example: Dhoni (or Virat) sees a ball coming at him. It meets the preconditions for a six. He knows the Specifications for the action he has to take. He takes the action. The result follows. The commentators watch the result and pass Comments.

Commentators pass Comments. Coders write Specifications.

Importance Of Geometry

I believe the subject of geometry to be the most important subject taught in schools. It is first rate preparation for the mind for any profession that that requires reasoning. Abraham Lincoln kept a copy of Euclid's Elements of Geometry in his saddlebag, and studied it late at night by lamplight; he related that he said to himself, " … you never can make a lawyer if you do not understand what demonstrate means; and I left my situation in Springfield, went home to my father's house, and stayed there till I could give any proposition in the six books of Euclid at sight. I then found out what demonstrate means, and went back to my law studies" (see The Abraham Lincoln connection.)

A very readable translation of the 13 books of Elements is available here.

Euclid built geometry on ten postulates (axioms). In addition to the axioms, each of the books contains definitions of the geometric terms used in that book and following books. No symbols (as in current geometry textbooks) are used. This make the proofs verbose. The proofs are all graphical as Euclid did not have symbolic algebra that we use today. What he had was Geometrical Algebra. The proof of Pythagoras’s Theorem is an example.

Application of Euclid’s Method to Software

Software should be provably correct. The propositions in Euclid’s books rest on propositions proved earlier. The initial propositions of Book 1 depend just on the definitions and axioms. They are like the leaf functions in a program. I believe that software constructed in the same way will result in provable software.

What a function should do (including any side-effects) should be clearly documented like a proposition. The function body itself is equivalent to a sequence of geometric constructions.

I have found that writing unit tests drives the process of re-factoring code into simpler functions. If unit tests are getting messy, the function under test is messy. The function must be re-factored.

Unit tests and re-factoring are the initial stages. These must be followed by proof of correctness of the construction. The process is not linear; it is cyclical. If the proof is difficult, decompose the function yet again.

A proof answers the question, “Why will this do what it is meant to do?” It should be part of the function documentation. Correctness of the proof must be attested to in the same manner as mathematical proofs. It must be accepted by peers who understand what is being coded.

Assertions, Preconditions, and Post-conditions are akin to lemmas. They are simpler to prove. In addition they have the added advantage of being executable.

Of course, proofs, despite peer reviews, can be defective. That does not mean they should not be attempted. We should learn from defective proofs.

Beautiful Code: Documenting Source Code

An article, authored by Adam Kolawa, in the book Beautiful Code, uses the subroutine SGBSV from the CERN mathematical library, written in Fortran, as example. The function comment block covers almost 2 pages of the book. The executable lines of code take up barely half a page! As Adam says, "One of the first things to notice in the code for the SGBSV routine is that it starts with a long comment that describes the routines purpose and use. In fact, the comment is exactly the same as the manual page for that routine".

Two statements that specially resonated with me:

• "If you cannot tell what the code does by glancing at the naming convention and several code lines, then the code is too complicated".
• "I hate reading code that was written to show off the developer's knowledge of the language, and I shouldn't need to go through 25 files before I can really understand what a piece is really doing".

Needed "Renovation"
Personally, I consider SGBSV - or any of the other names in the function - fails the test mentioned in the first quote above. But I suppose it makes sense to people who use the CERN mathematical library. Considering Fortran 90 permits names with 31 characters, I think the routines need to be "renovated".

User Interface

What is the user interface if you are writing C code?

As people who have worked with me know, I am a fanatic (I suppose evangelist and fanatic are synonyms) for source code documentation. In all the time that I evangelized, I do not think that I ever mentioned header files.

And it is in the header file that the user interface of a C module resides.

I recently "shipped" some re-factored code to one of our teams. I shipped a second version after about a week. The revised version. changed the behaviour to make it more in line with what would be expected behaviour in the real world. Prompt came the response that no change was noticed in the files.

Going through the .c files, to locate the changes would have been difficult. WinDiff of course would have showed the difference. But it would still be confusing. What was required of my customer was to go thru the .h files. There was only one .h file that was modified. In that the signature of just one function was altered. The comments in the .h file explained the WHAT and the WHY of the change.

Unfortunately, header files are not viewed as the user interface. It should contain all the information required for the" customer" of module to use the module. The "customer" should not have to go thru the .c file. That is for maintainers.

Was I blameless? No. I missed out on a standard practice - a Change Log.

A super set of my experience, described above, is described in API Documentation.

What Keeps Defects Hidden?

My top five:
1. No daily build and smoke - failure to get the bad news early
2. No unit tests
3. No reviews
4. Code that is hard to understand (follow Linus Thorvald's guidelines)
5. Aggressive scheduling leading to Muri

So when code is not subjected to these it is like work-in-progress widgets inventory piling up at a machine. It is inventory piling up. 

Okay the code works fine, passes all the tests, but there is no user manual. Or it is hard to modify because it is a Bhoot Bangla (going slightly off topic see an oil rig that was one).

When we say we have shipped, did we ship a finished product or was it work-in-progress?

Source Code Comments

Some weeks ago my son who works for one of the iconic software companies rang up and asked, "What is the definition of Standard Deviation?"

I was rather taken aback. He has a good degree in Mathematics and his college days are not all that long ago - at least compared to mine. So was it a trick question? Since at the moment I was having lunch, I used that as an excuse to get back later with the answer.

When I did get back with the definition (without having had to google it :-), I got the full story.

He was going through some code where the comments described  a wrong computation for Standard Deviation. Yet, given the halo around the people who generated the code he doubted his own understanding. As it turned out the code was correct. The comments were wrong.

Imagine if a maintainer decided to change the code in accordance with the comment.

Here are the mistakes made by the author of the comments:

Describe the WHAT. In this case it is sufficient to say, "The following computes the Standard deviation of ....". If required a reference to a standard text or a Wikipedia link can be provided.
Describe the WHY? What use is going to be made of the Standard Deviation? Why is it required to be computed?

The HOW should be in the form of an algorithm. Again, in many cases the algorithm would be available in standard texts, or in Wikipedia; provide a link.

And use Intentional Naming. Use StdDev, or Sigma; not S.

Code Comments, Process Worth

Yesterday I was leafing through Guy Kawasaki's, "Reality Check". I had read it a year ago. There were many statements that resonated with me. Here are two of them.

Code Comments
"Luckily the lack of comments usually doesn't matter, because the code is so crappy that a total rewrite is necessary in year."

Lack of comments is a bad smell.

Process
Quoting Stanford psychology profesor Carol Dweck, Guy says, "Instead of praising children's intelligence or talent, focus on the processes they used."

And at the end of the chapter: " ... focus on the process worth ethic, not the inherent brilliance, of your employees."

At acmet's Tools BU we certainly did that.

Hacker Culture & acmet's Tools BU

A few weeks ago GGanesh, sent me an email about what he considered were the strengths and weaknesses of acmet's tools BU. In that he mentioned:
"But an acmet thing which I remember most, is the way we encouraged getting the bad news early. We definitely had the hacker's attitude. We played with the code and got the bad news early."

Did we really have a hacker attitude? I wondered if I would ever have been comfortable working with a bunch of hackers.

GGanesh's mail led me to Paul Graham's essay on hackers.

One paragraph in the essay says:
"The latest intellectual property laws impose unprecedented restrictions on the sort of poking around that leads to new ideas. In the past, a competitor might use patents to prevent you from selling a copy of something they made, but they couldn't prevent you from taking one apart to see how it worked. The latest laws make this a crime. How are we to develop new technology if we can't study current technology to figure out how to improve it?"

If it is good for progress to look inside the competitor's product, should practices that facilitate poking around inside's one's own product not be instituted?  Poking around is reading (reviewing) code, poking values into variables while stepping through code, writing tests to see what breaks the code... Should not the product be constructed in a manner that makes future improvements easier? (The tag line for the Tools BU was "Software Maintenance as RENOVATION".)

So, yes I agree with GGanesh. And though the attitude was not strongly evident, it was getting stronger

Hindering Poking Around
This is my partial list of what hinders poking around.
* Inscrutable names of modules, functions, structures and data elements
* Prolific use of globals in the name of efficiency (a wolf in a sheep's clothes)
* Magic numbers
* Source code documentation that does not explain what sections of the code do and why it is required to be done, and what are the likely pitfalls.
* Undocumented pre-coditions and post-conditions
* Strong coupling between modules and weak cohesion within a module
* Cliques that discourage new generation of developers from getting to know the current code

Good Hackers
To make code easy to poke around, to bring up a new generation of developers, needs self-confidence and humility. Be critical of code, or ideas; not persons. I think most software companies would want such hackers. I definitely would have wanted more of them. That is the kind of hackers you want for a sustainable software process.

What do Embedded Software Engineers Really Do?

A great post by Colin Walls.

What *** really *** resonated with me is what he says about maintenance, “efficient” (quote marks are Collin's) code, keeping it simple, writing code that is readable by non-expert humans, commenting (and then commenting some more).

“Efficient” code is another wolf in a sheep's clothing.

Documenting Source Code - How It Was Driven Home to Me

This happened almost two decades ago. It was my first startup. We had built a radar Moving Target Detector using four DSP processors, plus another DSP as a controller. I had taken it from proof of concept, to field trial, to ruggedized prototype. Over that time the team size had grown from one (me) to three.

The ruggedized prototype was now to be put through acceptance checks. We gave a final check and then packed it for transporting to the customer's site. When we assembled it, and put it into operation, the ouput would not settle down as it was meant to. That got me into a flap. I was sure that during transportation some damage had been done to the hardware. I had visions of doing a messy hardware debug on the customer's premises. Luckily, I decided to sleep on the problem.

The next day I asked one of my engineers if anything had been done to the code that was in the EEPROMs. He said that there was a long sequence of code which did nothing but write zeroes into the RAM. He had taken that out. The data memory was no longer initialized.

So was my engineer at fault? No, I was. This was code that I had written almost at the start of the project. The purpose of the code was clear to me. But I was no longer "in contact" with the code. And I had failed to document it.

I have since then, been rather fanatical about documenting the intent of any block of code. Documenting the WHAT [is intended to be acheived], and the WHY [it is important that it be acheived]. The HOW is not that important; the code should say that - unless of course the code is convoluted. But then one should not be writing convoluted code in the first place:-)

This incident led me to formulate my extension to Murphy's Law:

If anything can go wrong, it will - and it will happen in the presence of the customer!