Tutorials » Testing and Process Recommendations for Software Engineering

Go to Part Two

Overview
Many small software engineering organizations have problems cooperating and communicating on projects. A primary reason for this is that these small organizations often grow out of the individual efforts of the one or two initial programmers. Projects undertaken by an individual programmer are small enough for that one person to understand the requirements, write the code, find the bugs, produce the deliverable, and provide the ongoing support, often without really being aware that these are separate tasks. As a result, comments in the code and user documentation are usually sparse. Other engineering processes and artifacts - such as repeatable (documented and/or automated) tests, requirement and specification documents, and records of bugs that were found and fixed - are often extremely informal or completely absent.
 
Reasons why this happens include:

• Many software developers have not had appropriate education in software processes. Many students have had courses in the details of specific languages and general programming constructs (linked lists, hash tables, and so on), but few have had specific instruction in how to write a requirements document, how to think about developing unit tests, etc.
• Small projects can and do succeed without these extra documents and processes, so the need for those processes are not perceived. An individual engineer who is working on a project small enough to keep in his or her head, may reach a state that he or she can call "deliverable" much more quickly without producing requirements documents, unit tests, etc. He or she may not perceive post-delivery support as part of the engineering cost and may not keep careful track of how much time is spent on that part of the project. (This is especially true if the end users come to expect and accept low-quality work and don’t bother the engineer with reports of annoying but non-critical bugs.) The engineer may therefore not believe in the value of these additional processes and may in fact be openly disdainful.

As a software engineering organization achieves successes, it will tend to tackle more ambitious projects. Soon there are two features of the projects that were extremely minimal or absent earlier on:

• The organization is working with customers that require some useful estimate of when the project will be completed and how much it will cost. Too many inaccurate estimates will ruin the organization’s reputation and could bankrupt it.
• Several engineers (and sometimes a newly-promoted or newly-hired engineering team leader or manager) will have to cooperate closely on facets of the project, and must therefore be able to communicate openly, freely, and in high detail.

These two factors drive the need for improved engineering processes in a small project group.
 
The question then is: what processes will provide the payoffs (better cost and time estimates and efficient cooperation among engineers) without requiring too much overhead (thereby spending more money and time than will be saved)?
 
It is recommended that a small organization should implement lightweight, informal versions of the following processes:

• Record user requirements, the engineering organization’s later questions about those requirements, and subsequent resolutions of those requirements.
• Provide source control of tools, deliverables, and documentation.
• Record and track bugs.
• Record the tests that are to be run against the deliverable.
• Record the details of each test run.
• Establish standardized coding and commenting guidelines.
• Conduct peer code review.

This is enough process to show quick payoff without too much time expenditure. Attempting to implement more process than this too early in the maturation of an engineering organization is likely to fail. Individual engineers may be confused and annoyed at the imposition of too much "overhead", and engineering leadership may not have the time to educate and enforce additional process. Attempting to implement more processes than can be taught and enforced is a sure road to failure.
 
Let’s look at each of those recommendations in a little more detail. Each recommendation will be kept as separate as possible. This way, an organization that quickly comes to agreement about how to implement any of these suggestions can do so without having to implement them all at the same time. This is extremely important! The goal is to help the organization be more productive through slow and steady change, and not to spend all your time implementing and arguing about engineering process improvements.
 
Requirements Documentation
One problem, which is not unique to small organizations, but which tends to affect them much more than individual programmers, is incomplete communication and understanding of customer requirements. One person talks to the customer and finds out what they want, and other people end up writing the actual code. This results not only in cases where the first person wasn’t clearly communicating with the customer, but cases where the engineers within the organization weren’t clearly communicating with each other.
 
The first step towards solving this problem is to maintain a written users requirements document. This means more than just the initial document that the first customer contact person writes while discussing the project with the customer! All subsequent questions and amendments should be recorded directly in the same document. There are two reasons to do this.
 
First, at some point in your organization’s growth and maturation, you will want to be sure that the customer cannot continually withhold payment with claims of "That wasn’t what I wanted." Maintaining a customer requirements document means that, at some point, you can easily derive a contract from the users requirements document, by adding some language like "...and anything that is not specified above can be implemented in any way we like."
 
Second, by recording the questions and amendments to customers’ requirements document, you build a body of data that can be reviewed before future customer contacts, so that your people know what issues to clarify in subsequent projects.
 
Specific detail about any particular format for these customers’ requirements document will not be discussed. It is much more important that the document be written down in full, and that subsequent questions and amendments be written into the same document, than that the document follow any particular format.
 
It is strongly recommend that you combine this detailed tracking of customer requirements with source control, as discussed in the next section.
 
Source Control of Tools, Deliverables, and Documentation
One problem that crops up when multiple people cooperate on a project is that they are simultaneously changing the files that make up the project. This may result in cases where engineers lose each other’s changes, nobody knows which of multiple versions of the project are the "current" one, etc. This problem affects not only the actual deliverable product, but also internal documents and tools used to build, test, and describe that product.
 
It is recommend that a small organization use a simple source code control tool - say SourceSafe for Windows or RCS for Unix - to allow engineers to more easily find (and discuss) the versions of these files.
 
The organization should keep a central version of the project on a server. This central version should include all source files, internal tools, documentation, and the fully built version of the product that corresponds exactly to the versions of those files.
 
Each engineer should keep a local copy of the entire project on his or her own machine. When the engineer performs some identifiable chunk of work, the engineer should build the project on his or her own machine and check for problems.
 
Then he or she should warn the organization that he or she is going to be changing the central repository. It is very important that only one engineer have the right to update the central repository at one time! There should be a note on a central whiteboard, or a toy that gets passed around indicating the right to change the central repository, or some such physical representation of this right. Once the engineer gains the right to change the repository, he or she should then update their local copy of the project with any changes that happened to the central repository while he was doing this work, and build again and check for problems. Then he or she should submit the changes to the central repository, build again there, and check for problems. Once the central repository builds with no problems, the engineer should tell the organization that he or she is done with this batch of work and relinquish the right to change the repository.
 
Note that in this section there has been no discussion on what exactly the engineer should do to check for problems. That is a separate issue. It is strongly recommend that a small engineering organization be sure to separate the discussion of source code control from the discussion of how to check for problems!
 
The project should be arranged so those tools used to build the project and the documentation about the project are parallel to the source files for the project. For example, a simple project that has source files, database definition file(s) for a standard commercial database, and some documentation, might be arranged with the following directories:
 
MyProject/src:  a directory containing all the source files (this directory may of course have additional structure inside it)
 
MyProject/bin:   a directory that starts out empty, into which the executables are placed when the project is built. (Opinions vary on whether intermediate files - object files etc. - should go into this directory or into the source directory as the project is being built.)
 
MyProject/doc:  a directory that contains documentation relating to the project. This directory might have further structure to it - for example, if you have some documentation intended for internal use only, and other documentation intended for end users, you might create a MyProject/doc/internal directory and a MyProject/doc/external directory. If you are recording user requirements with the rigor recommended in the previous section, there should be a MyProject/doc/requirements directory as well.
 
MyProject/database:  a directory containing the database-related file(s).
 
MyProject/tools:  a directory containing tools relating to the project (test helper programs, documentation readers, etc.)
 
MyProject/tests:  a directory containing the tests.
 
Someone in the organization should understand the source code control system well enough to "own" problems with it, provide information and tools that speed up everyone’s ability to get fresh versions of the project and build them, etc. One tool that this person should provide is a single command that will retrieve a clean copy of any version of the product and copy it into a clean local area on the developer’s machine. This command might first just completely blow away anything in that local area on the developer’s machine, or to save time it might simply blow away any built pieces (object files, executables, etc.) and retrieve source files only. The source code control system owner will be in the best position to decide how to do this.
 
Go to Part Two