Scope Creep and Action Reports (SCARs) in Software Projects

Sarma Danturthi, PhD, PMP– May 18, 2016

There is a strange quote that says, “Two things in life are very easy to do. One, walking on water and two, developing software, if the water and functional/user requirements of the software are frozen.” Funny? Right, but those in the software development field know this statement is an absolute truth. What causes the scope creep?

In information technology software development projects (after submitting the initial requirements), a customer is very likely to change and keep changing the requirements—much to the annoyance of the development team. A team writing software is also often negligent in creating the proper documentation or following a coding standard. This article discusses the scope creep that may sneak into software projects and why the creation of project documentation should be made mandatory, whether the project methodology is agile, waterfall, or any other.

Introduction
Jim Davis, a project manager, just started working on a new software development project that’s very similar to the project his team completed last year. A month after submitting the requirements, the customer is now busy adding more changes to the original requirements and expanding the scope of project. To recall how his team dealt with this kind of scope creep, Jim wants to look into the lessons learned document from last year’s project, but he cannot remember where it is. The team’s technical lead left the company and none of the other team members have any idea about what Jim is asking about. Though Jim remembers some of the pitfalls his team had in the last project, he shudders to think that the team has to go through the same route again. A lessons learned document would have helped Jim save hundreds of work hours, but the document is not available. Worse yet, it was probably never created. Jim is now at a crossroads as to what to do with the scope creep he is experiencing and how his team will sail through all the troubles again.

Does the story of Jim sound familiar? This is the story we may discover again and again in the software development life cycle, even though every project manager has heard the rule a dozen times—in every phase of a project, create the action reports and lessons learned documentation, whether all the project management principles are followed to the core or in part.

Let us face it. First, while developing software, every project manager has more or less experienced the dreaded scope creep at one stage or the other, where the requirements keep changing with regular customer feedback even if the project methodology follows an agile, waterfall, or any other model. Second, it is probably an open secret that a majority of the project managers and developers in the field of information technology often avoid creating action reports or lessons-learned documentation, or for that matter, any other technical documentation, unless absolutely necessary. There are several reasons—either valid or invalid—for the lack of (or willingness to create) these documents. This paper discusses these two factors—combined as an acronym SCARs—and tries to reason as to why scope creep may sneak into software projects and why action reports should be made mandatory, however tough a project manager’s stance is. In this discussion, by using the term “action reports,” we should keep in mind that they are to be created and updated throughout the project and do not replace any other existing or required project documentation. Instead, they are helpful in augmenting a project’s success and will prove to be valuable assets en route to continuous process improvement.

Scope Creep
There is a strange quote that says, “Two things in life are very easy to do. One, walking on water and two, developing software, if the water and functional/user requirements of the software are frozen.” Funny? Right, but those in the software development field know this statement is an absolute truth. What causes the scope creep? To find the answer to this question, let us dig a bit deeper into a software company’s infrastructure. A software development firm usually has three different environments—development, test, and production. If we leave out the mom and pop and bed and breakfast software shops (which actually develop software of high quality), a project manager is familiar with these environments. At best, all three environments are on different servers—both front and back ends. But there are also some companies that have only the back end on different environments, keeping the front end intact on only one machine.

The first option is ideal and obvious—to keep everything detached in every environment. Each environment has its own front and back ends, operating system, software, libraries, objects, and so on and so forth.  A great deal of systems administration, maintenance, and coordination among the teams is necessary to keep these three environments functioning. With three environments and a change control board (CCB) in place, the design and development is done first on the development machine and, when complete, the customer is informed about the approaching test on the testing machine. But during and at every stage of the software development project life cycle, the customer is in direct contact with both the project manager and developers and can give feedback with one or more change requests that may pass through CCB.

In the second option of one front end and three different back ends without any CCB in place, the developer just changes his connections to a different machine while developing, testing, or deploying on a public-facing production server. The bed and breakfast software companies do not have the resources or manpower for three separate environments and take their risks with only one. Their deployment may usually be on a paid hosting website or server. This method is a trial and error and is prone to risk, but these shops feel comfortable with the risk they are taking to deploy or rollback their changes as they deem fit.

Except for a large software development firm, it is very common not to have any CCB to approve or reject changes requested by the customer. The reason for not having a CCB is simple—lack of money for the resources, either human or otherwise. In the absence of CCB, the customer and the project manager are in direct agreement on what changes to make. It is also quite common that these requests from the customer to the project manager sometimes are verbal and/or happen on a mere phone call. Once the project manager starts implementing those verbal changes, the customer gets a feeling of being able to wave the magic wand whenever he wants because the project manager has allowed the customer to play the cards.

As surprising as it may sound, the changes are often for inserting a check box, adding or removing a text box, changing text in a message box, modifying what is displayed, how it is displayed, adding a new column to the database table, adding new functionality to the application, conforming a web page to a standard such as ADA/EOE or W3 Standards, and so on and so forth. The project manager, after allowing the customer to have his way, will soon notice that the project is expanding beyond the original requirements and scope creep is becoming a regular pain. This is because before a software package is created, the customer has only a jagged idea of how the software should look, but really cannot tell what exactly it should look like. This is contrary to a civil engineering project—such as construction of a bridge—where everything is finalized at the design stage. More often, even the most experienced software developer and project manager cannot tell ahead of time how beautiful the final software product will be when complete, because one control (e.g., check box) can be programmed to run like another (e.g., radio button) or a developer can design his own user control with a different look, feel, and functionality.

Let us consider an example. A few days before final deployment while testing the code in action, the customer thinks he now needs an additional text box on a web page, in which he can type a note and record it. It is a very simple request that will not take more than fifteen minutes of code changes. At least, that’s what the customer assumes. After receiving this request from the customer, the project manager calls an open dialog in the war room and discusses this change with his programmers. A text box can be created easily, but now the team raises some valid questions. What does the customer plan to type in the box? A free entry text box on a web page can be a land mine for SQL injection attacks. So, the text box requires a validator for data. If we somehow avoided that land mine with a validator, the next question is, where do we store the data typed in that text box? Do we already have sufficient table space on the back-end database for it? If not, how much time and space is required on the back end? Also, if the customer is typing some free text into the text box, when he retrieves the data, should the entered data show up on a different page? Questions galore—and what is seemingly a very simple text box for the customer, quickly becomes a complex scope creep problem for the project team to implement.

The project manager and the development team may start feeling bitter and hostile toward the customer for these ever-changing, unpredictable requirements. On the other end of spectrum, the customer may also have negative feelings toward the project team for not getting the changes he/she wants. Suddenly, every stakeholder recognizes that this scope creep now will have an effect on budget, timelines, etc. The problem in such a situation is lack of communication, since the customer thinks the changes are very simple; whereas the project manager and his team know that these requirements were not mentioned at the start of the project. However, an experienced project manager should be able to handle these things with discretion, dexterity, and diplomacy.

Even in large software development corporations with a fully functioning CCB, this kind of scope creep can happen. When the project manager informs the customer about the possible scope creep and additional costs, if the customer insists on a change at any stage of development with proper change requests through CCB, the project manager will have to budge for various reasons. First and foremost being the fear of losing the business and the goodwill of the customer. Second, the reputation of the company could be at stake. Even if both of these reasons are invalid, we know that a customer can force the project manager to make changes when the customer thinks he “should have his say” in the project. The project manager can find ways to avoid or limit the scope creep by having a boundary for the original requirements and by stating new completion dates, additional costs, pushing the project to another phase, etc. Also, the scope changes can be controllable if done the right way with a certain knack and diplomacy from the project manager. But, as it so happens in software projects, if the customer agrees to the new dates and costs, the project manager has to budge again.

In addition to projecting a smiling face, there are two rules that are repeatedly drilled into the minds of all those working in software development. One, never say no to the customer and two, do not display technical arrogance. Saying no to the customer will lose your business and may cost you your job. Secondly, as a developer or project manager, you may be the smartest person who knows everything from Alpha, Lambda, and Sigma to the most sophisticated rocket science used to send an unmanned spacecraft to Mars, but the customer does not have to know those things. All you need to do is “listen to the customer and comply with what he wants.” There is no need show the customer your abilities to design a space shuttle. In other words, “do what is needed and do not volunteer to do what is not required,” is the official song. These are some of the reasons IT staff always complies and why scope creep may happen—sometimes, at an alarming regularity when dealing with a specific customer.

So, there we have the undocumented first rule—in every software project, be prepared in advance for scope creep to happen. Once recognized and ready—if scope creep happens a few times even after completing several projects successfully—the project manager and his team are at ease to work with a customer and on future projects.

Action Reports/Lessons Learned Documentation
Our next discussion is the creation of action reports or lessons learned documents and adhering to coding standards. As project management practitioners, we all know the real importance of lessons learned documentation and their role in continuous process improvement. We are all brainwashed thoroughly to create lessons learned documents and/or action reports at every stage of the project—not just at the end of the project. But, if we randomly pick a software guy and ask how much time he/she really spends creating any documentation, we know the answer. Software development teams, in general, are very negligent in creating those documents. Large organizations have built-in software, such as Visual Studio’s Team Foundation Server (TFS), to process documentation, but what about the mom and pop, and bed and breakfast joints that cannot afford to buy software for bug tracking, documentation, etc.? Except for the user and operating manuals, there is really no other documentation that is created by those shops. Even after a successful test run, did we actually record the number of bugs found and how they were corrected? Did we document how a difficult bug was eliminated after several days of research and mental toil? If not, once we fixed the bug and have lost track of the process, only the person who did the original work may remember those bugs and their resolutions—and maybe only for a short period of time. If the original coder has moved on to another project or company, the entire knowledge base goes with him or her, just out of the door and into the oblivion. Think of the number of hours one can save and the amount of mental stress one can avoid if only such an experience were properly documented.

Why doesn’t the software team or project manager care much for creating the action reports? There are dozens of reasons. Among them are laziness, lack of time, ever-changing technologies and their versions, the absence of any project management methodology in the project, staff changes, programming language changes, migration to a different environment/framework, or various other financial and social obligations. Yes, the “my-job-is-holier-than-document-creation” factor comes in too.

The other side of the documentation coin is purely technical, much more distressing, and actually happens inside a perfectly functioning piece of code. If we randomly pick a software development business and look carefully within its infrastructure, we find that there is no proper standard that is usually followed for coding—either by the team lead or by those who work with the team lead. Even large organizations give very little importance to coding standards because their focus is mainly on a version of working code rather than properly written working code. There is quite a bit of difference between “working code” and “properly written working code.” A properly written working code is readable, has plenty of comments, follows a pre-defined format, encourages other coders to follow the format, gives insight to the application, class, or objects being created, and makes the life of a coder/reader easy. It also compiles without bugs and runs successfully. On the other hand, a “working code” may function and can, apparently solve the problem, but once a bug is found, anyone other than the original coder will find it increasingly irritating to understand the functioning of the code in order to trace or debug. Even the original coder—after not seeing the code for six months or more—will find his own code very annoying when he comes back to debug. A properly formatted and well-documented or commented code saves time, energy, and avoids frustrations. But when it comes to coding, do we follow a format or a standard? If we do, how good is the format and how often do we document the code? Those are the most important questions for everyone in the software development field. Lame excuses like lack of time or interest for not writing proper comments or for not following a coding format should not be entertained by the project manager. If writing dry code takes a few hours, with a little practice, writing properly written working code can be efficiently done in the same number of hours.

Documenting is a lucid write-up of what we know, what we did, and how well we understand the process. By documentation, it is neither a dry Microsoft Word document nor a lengthy novel, but one that saves our lives. In one sentence, documentation is the life blood of the software development process and is an essential component of successful project management. But, there is a general disregard and alarming indifference to creating documentation for software projects. In his book, Memoir of the Craft – On Writing, fiction writer Stephen King asks all the people who do or want to write, “If God gives you something you can do, why in the name of God wouldn’t you do it?”

Why, indeed! The question applies equally to all of us working in the software field who refuse to create the needed documentation and do not follow any formatting or coding standards. Above all, we need to remember that the greatest advantage of action reports or lessons learned documentation is that they help us in continuous process improvement and in shaping the success of current and future projects.

Conclusion
Those of us working in the software field need to accept the fact that scope creep can happen sooner or later. Even if we have worked in the field for a year, ten years, or thirty years, a customer request will probably push us beyond the original scope of the project. And the best action we can take for creeping scope is to expect it and be prepared for it. Secondly, any action reports created at any stage of software development life cycle—design, development, code, test, or deployment—are going to be life savers, both to the original coder and the successors. The more documentation we have about a project, the better our lives become because these documents provide a mental map of the project and can save us embarrassment and hundreds of hours of time. Documentation also includes having a coding standard and religiously sticking to it while coding. If we conscientiously avoid the documentation for whatever reasons, we know we have an accident waiting to happen—an imminent wound, and an ugly scar when the wound heals. The best way ahead to avoid all these is to remember the acronym—SCARs. We need to be prepared in advance for possible scope creep and action reports. In addition to promoting success in current and future projects, they can expertly guide us in continuous process improvement.

About the Author
R. Sarma Danturthi has a PhD in engineering from the University Of Memphis, Memphis, TN, USA and has taught graduate-level courses in computer science. He has been working in the IT field for several years. His experience includes design, coding, leading project teams, and successful project management. He has also published papers in peer-reviewed journals and has written book chapters on software interfaces, modeling, and simulation. His interests include cloud computing, intelligent interfaces, and mobile application development, among others. He can be contacted at d_danturthi@hotmail.com