Mind the Gap

For anyone who has visited London, the title immediately brings to mind that continual reminder that the London subway system was built many years ago and there are sometimes some very significant gaps between the platform and the train. An unwary traveler can easily step into that gap and find themselves embarrassed at a minimum or even seriously injured as they fall halfway to the tracks. While it is drummed into your sub-consciousness in your time spent in their "tubes", I don't recall ever hearing that message on a continuous loop. But here in a airport in Pittsburg, built at least a century after the earliest London line, I hear a comparable warning on a continuous loop to no effect except to annoy people waiting nearby. I find this a sadly common situation in American society and design and I would like to find a name for it.

An innovation in airports and other transportation centers is the addition of moving walkways. It is hard to believe that there is still an American citizen left who isn't familiar with them by now. Yet the walkway near my gate has speakers at the end of each walkway that plays the loop "Caution...moving walk is nearing its end. Please watch your step." with only a two second pause between repetitions. Now when someone is on the walkway near its end, it makes perfect sense. However this system has no sensor so the loop plays even though there are no walkers, and in fact have not been for more than 10 minutes. I can just imagine why this has come to be and it is that image that keeps me at a slow burn with each repetition.

A continual theme from the political right is the nanny state. While I disagree on where this came from,

The Failure of Reductionist Thinking in the Creation of Software Intensive Systems

My career was dedicated to helping large organizations build software intensive systems that would solve their business needs. This was an honorable calling and one that had many moments of intellectual joy. With each level achieved, I perceived that the projects were often doomed to sub-optimal achievement by decisions made earlier in the process. I was drawn inexorably to these higher levels like a moth to a light. Since my capabilities allowed me to ascend toward management ranks I was allowed to see how these decisions were arrived at and to make my attempt to avoid the pitfalls I had witnessed in earlier projects. The culmination for me was to be project manager for several large, complex, high-risk projects, work that I eventually found soul crushing and caused an existential crisis that I am only now recovering from. But this experience has left me with a clarity of vision on how management and the engineering staffs they employ come from such different cultures that to find they achieve anything at all is a testament to the underlying unity of human society.

Projects are funded to solve business problems. This goal directed nature of these efforts gives them a vitality that is lost if the goals cannot be succinctly expressed or are not shared by the stakeholders. This deep understanding of the goals is the sole province of the leaders of that organization. Many projects fail at this level do to the annunciation of commandments or the articulation of such laudable, but ultimately vacuous goals as to be useless for the guidance to a satisfactory conclusion. At this level, at a minimum, a project must be able to articulate how a successful conclusion will be recognized. There is no great harm in allowing untestable or abstract statements. However management must recognize that their job is not complete until the project can articulate a set of concrete business objectives that will justify the capital and expense of this project. It must also accept that this is a contract with the project team and the project stakeholders that constrain them to accept this as a stated end position and only change it with the full cooperation of the project team. After all, things change and knowledge is learned during the articulation of the goals. But too many projects lose focus and support when the vague goals they started with are never reified into something tangible enough to drive decisions and the most energetic members of management move on to other more alluring goals leaving the project with the heavy lifting of ensuring at least some reasonable goal is achieved with the resources expended.

Traditional waterfall methodologies inherited from the success of very large well disciplined organizations to manage large complex projects may never have worked very well but where they were employed with thought, consistency with well trained and stable staffs, they worked better than anything else that had previously been tried. The dominance of the model of project process led most managements to view the creation of software systems as more akin to the creation of an automobile on the assembly line than the creation of a work of art with inscrutable processes and unpredictable outcomes. Yet the experience of the past 40 years suggests that the latter may be a closer model of systems creation than the former.

The most recent understandings of project failures involve the consequences of the earliest design decisions on some of the most valuable properties of the system to be created, the emergent qualities of the complete ensemble. This view is supported by the thesis of SEI which posits that most, if not all, "non-functional" qualities of a software system derive from the was the system is decomposed and constituted during the design process. If is oft stated that you can't tack on (take your pick here: usability, security, performance, etc) on the end of an already existing product. It must be a design goal from the beginning due to the cross cutting concerns. That is to say that qualities are orthogonal to the functional needs. If you think about it, it cannot be otherwise since we know we can refactor code in any number of ways without making any change in its dynamic behavior. Yet one or two inappropriate decompositions can make the achievement of some desirable qualities difficult or impossible. Yet these earliest design decisions are made long before there is any significant understanding of even the problem-domain, no less the solution-domain for this project. This reality puts architecture-centric techniques into a tension with Agile techniques which are embraced by managements for their fast delivery of tangible results. To balance these two competing forces requires a management that can weigh short-term gain against long-term investment; that can participate in the technological vision of an architect and create a safe-space in which difficult concepts can be given enough time to mature without engaging in a blind trust that time and money will necessarily result in a better product.

Another danger faced by management at these early stages of the project is the lack of maturity both within management and within the engineering community to properly analyze the problem-space. From an engineering perspective, the requirements engineering phase is at its best if it results in a set of models which express views of the problem-space in a way that enables the designer to project a new and different version that can be understood by both the business and the development organization(s). Yet no matter how powerful, these models are often the source of a great deal of damage. At this point I always have Magritte's famous "pipe" painting in mind which proclaims "this is not a pipe." The point made is that the painting is a depiction of a pipe, not an actual pipe. In software even more than in art, the error of reification is so easy to make that it is almost impossible to avoid. No model of human or organizational behavior can ever capture the true richness of behavior that it is capable of. But rather than embrace, or even acknowledge this truth, management will attempt to use a model to constrain behavior in an attempt to achieve uniformity, uniformity that may be driven by a desire to reduce the skill set needed by the worker, to ensure that different organizations create an identical product or merely to believe that they are achieving some business goal that is never expressed. This reduction of human work to a standardized process has been studied since the dawn of the assembly line and doesn't need to be rehashed. But what is important here is how this mentality can cause the project to mistakenly believe that the model of the organization into which a system will be crafted really expresses the behavior of the people and the jobs they do rather than allow for the extra-system work that in most cases allows the system adapt and remain resilient to unexpected input. The model is not the process.

The biggest dangers in systems development are seen when the team believes they understand the problem in sufficient detail to fully articulate the solution specification. This is the realm of design. This design exists on a continuum from the selection of previous designs that satisfy this requirements, as if the problem is a burnt out bulb and the solution is the replacement with a functionally equivalent one, to creation of a novel and innovative solution to a problem that had never been tackled before, such as the creation of a spacecraft to support colonization on Mars. Too often a project is funded as if the problem space is burnt out light bulb while the stakeholders want to go on a mission to Mars. Project management, with support from senior management, must ensure that everyone on the project is guided to the same place on this continuum and that this scope is normalized to the budget it is given.

Another pitfall during this phase of the project is the abdication of management decision making. Design is all about tradeoffs. I haven't met a business manager yet that will not quip when asked if they want it fast or cheap will not answer "yes." It is not unreasonable for business managers to seek a quality product that is instantly available, does everything they could ever think of for that product and for it to be free. But we all know that it is impossible. Decisions must be made and if the business will not make them, the designers will. For the business to not engage in this decision making (assuming that the designers do not usurp that responsibility) is a clear abdication of their responsibility. While a talented designer may actually make more astute decisions than the business, the business is left poorer for its ownership of those decisions and it leaves a flaw in the crystalline connections of forces that connect the problem to the solution via a line of empowered managers. What is sought is a product that once delivered is known to represent the best decisions possible from this organization and one that cannot be disowned by those managers.

This brings me to what I believe is perhaps the most intractable danger in systems development; the achievement of emergent properties in the system. The drumbeat for "quality software" has been growing over the years and will surely increase as we attempt ever more complex systems in ever more demanding areas like health and transportation. Yet what we are grappling with is an engineering approach to achieving properties that have more in common with chaos and stochastic processes than they do with Newtonian mechanics. The Alexandrian pattern languages will enable us to evolve solutions through trial and error. Yet to an engineer this is an unsatisfactory place to be. By what principles, by what mathematics can be envision some emergent property and then back into the simple rules that will allow this property to be expressed? Kurtzweil suggests we will see singularity before 2050 but even if we do, I don't think it will do us much good. Reaching the tipping point where our manufactured logic machines have comparable statistics to the wetware in our heads does not guarantee that they will become HAL-like. Unless and until we know more about the brain than the number of neurons, dendrites, axons or even their wiring pattern, I don't think the hardware achievement will mean much beyond the new economics of computation. There is still a new mathematics waiting to be created out there, one that does not fall prey to reductionist thinking.

Are They Really Requirements?

The paradigm we get from the waterfall model is that a client states their requirements and they lead unidirectionally to the specification for the system. We already know from the popularity of the Agile methods that waterfall is not the correct model, that for many projects it is more productive to approach it as an artistic product and make many iterations that can be shared with the client. The benefits of this approach have been well documented but it still leaves the idea that there are some set of requirements that just need to be uncovered and refined. I now believe this is a fatal flaw in this part of even the Agile methods.

Iterative methods gain their power from the inclusion of the client in the design process. In the older waterfall approaches, after the requirements were documented and signed off, the client would often see little progress (except bills and excuses why the project was behind schedule) until an advanced level of testing allowed the client to see the complete system as it is intended to work. The pain this caused when it was realized that what was built was not an acceptable solution to the client's problem was always significant since so much had already been sunk into the wrong solution. Agile at least boxes the risk into a single iteration and if they are small enough, the damage can be limited. One good trait of all these Agile methods is that they support faster failure of dysfunctional projects.

Clients have always had misgivings about requirements signoffs. While they could not articulate it, they felt they were being setup for failure. Their business expertise did not prepare them for the task of directly specifying the product they needed nor is the average business analyst prepared to completely understand the business function that is within the scope of development. Neither is generally prepared for the design tradeoffs that are often done without any direct client involvement, even if the team is articulate enough to explain how a particular design decision impacts the various emergent properties of the system under construction. The current methodologies simply don't work during the requirements elicitation and analysis phases of even the most Agile methods. Instead they substitute what I like to call the waving of the hands form of requirements where everyone simply talks about what is needed, very often in "I'll recognize it when I see it" kind of terms. The design team does their best to interpret these statements, goes away and comes back with something that can be critiqued. The progressive elaboration of models and prototypes will allow an experienced team to eventually drive to a solution.

So what is wrong with this approach? After all, it does produce a solution. Is it optimal? Who knows? Is there any traceability to the design decisions made? Probably not. Does this sound like engineering? Emphatically no. We can do better.

What this approach lacks is the top-sight and planning that enable an architecture centric approach. Many solutions do not depend upon an architecture centric approach to achieve business success. When the solution is self-contained, highly derivative of a prior effort or does not have exceptional quality requirements, the emergent properties of the system will probably not be difficult to achieve when there is little or no "big analysis up front." But this is not the forefront of software engineering today and the solution to these challenges has not yet been found.

Emergent properties of a software system do not derive from one or even a small number of design decisions made when designing the product. Rather they emerge from a collection of many, most or sometimes all of the elements that comprise the system that have cross-cutting concerns addressed in a manner that allows the ensemble to achieve the collective goal. When the top-level decomposition of responsibilities has been properly done, these emergent properties can be achieved by independent developers working on sub-systems within the larger ensemble. But too often poor choices made in the first few design decisions can block the achievement of an emergent behavior even when all purely functional properties exist in the finished product. If the emergent properties that exist do not satisfy the business need, the product is rejected. But if the tradeoffs provide a product that "satifices", the product will be accepted and since the knowledge of what might have been is at best speculative (at worst, vindictive), the missed opportunity will never be known.

This can be improved by recognizing the key decision making role the client plays throughout the design process. The key challenge faced in trying to include the client, though, can be easily seen in a near universal scenario. The client has established a timeline and budget for the project. Then the requirements gathering begins. At some point, an astute engineer will observe that it is possible to get the needed functions and qualities a, b, and c at levels of x, y, and z and this combination is judged unacceptable. The engineer turns to the client and says, which are you willing to sacrafice. The client says "none. I want them all." What is really being said here is that the client is abrogating their responsiblity to be a decision maker in the project. If the engineer is correct that the qualities cannot all be achieved simultaneously (time and budget being two of those qualities), then the decision is foisted upon a team member to make a decision for the client. Just as the client felt cornered when asked to sign off on the requirements document, so too does the designer feel abandoned just when he needs the client the most. This is, after all, a business decision, a business decision that can have real business consequences. What is needed is for the client to engage in a form of negotiation to explore the possibilities, ensure that it really is impossible to achieve all goals simultaneously and then to take responsibility for the ultimate responsibility with the help of the business analyst, architect or whoever is in the lead design role.

So you can see that it is not as if requirements are completely elaborated at the time the project team ordinarily asks the business to sign off on them. This is an important first step and one that cannot be taken lightly. But neither can this be seen as the final word for business input. For an architecture centric approach, it must include sufficient elaboration of the qualities that can only be achieved with cross-cutting concerns that span a large portion of the code base to be developed. For any development it must at least be accurate even if it is not complete. But the impact of the aspirational goals cannot be known until the design progresses to a lower level of design and only then can the implications of that set of aspirational goals be known. As soon as possible thereafter the client must be brought back into the design process to negotiate the tradeoffs that should be made.

This process is much closer to advocacy than it is to specification. As leverage points appear there is someone to argue for the most advantageous tradeoffs that achieve their client's goals. For the client representative(s), it is to collaborate with the chief designer. For the project team, it is to do the most professional job possible in predicting the properties that will be present in the final product extended from this design.

from

Tim Bender to me

show details Sep 10 (2 days ago)

In discussing a large software project with an artist, I conjured the analogy that a software product is like a painting. Each engineer is an artist with their own style and they must paint a small portion of a large masterpiece, often without ever being able to see the whole thing. Recently, I watched a movie which made me recall this analogy and I pondered it further as a way of explaining in quite a simple way some of the rather complex interactions that occur in software engineering.

The idea centers around giving groups the common task of creating the simple image of a house with a lawn, a small family, the sun, a bird, and a flower. The image would need to be simple enough to be easily reproduced by an individual, but complex enough to offer varying entry points for concept learning opportunities.

Some of these concepts are weak and need some fleshing out.
1. Requirement solicitation:
Scenario: Give a team a sheet of paper and some coloring pencils. Express to them the importance of this drawing and that it must look exactly like what is being requested. Tell them to draw "a house with a lawn, a small family, the sun, and perhaps a bird and flower or something". Being purposefully vague and leaving them to either create only the initial description, or go ask follow-up questions.
Challenge: Requesting help and/or more information.

2. Specifying interfaces:
Scenario: Give a team the exact image they are to produce and a collection of transparencies attached to construction paper (making them non-transparent). Tell the team that each person must draw something on their transparency. Inform that the transparencies will be stacked to create the complete image.
Challenge: Specifying interfaces clearly to minimize integration failures.

3. Organizing for a task:

Scenario: Give a team the exact image they are to produce and a blank canvas with some pens. Tell them that each of them must contribute something and that they will be asked to state their contribution. From there, let them self-organize.

Challenge: Skill auditing. Some members will be excellent at sketching/drawing. Some may be terrible. Those that are not gifted in this area could take on a small portion of the image or provide administrative support. A variation might include purposefully leaving out supplies that a team member must requisition.

I was curious if you would have any thoughts on this.

***********************

Me? Have opinions? Perish the thought!

This is an interesting metaphor but I probably don't see it the same way you do.

On my visit to Vienna I had a curated tour of the Kusthistoriches (Art History) Museum. In the pre-20th century tradition of large art studios, it was common for the studio to accept commissions in very must the way you describe and in a way that does have some interesting parallels to software engineering. Each studio is headed by a master who is the creative genius that gave the studio its fame and provides the marketing effort to keep it employed. But given the output of the studio, it was not possible for the master to personally paint all the canvases. Raphael ran such a studio and we spent time talking about how to tell a Raphael from his hand versus one that he may have never touched.

In the studio system the various artists had their individual talents. The less talented might be relegated to painting landscape backgrounds, another buildings, etc. Only the most talented would paint the main subject which was always a human being. The ability to achieve life-like form and tone was prized and only the very best could capture the "essence" of a human subject. For historical paintings that had many figures, the master might paint a few of the figures but would allow the studio to fill in the remainder. This form or organization seems to be very similar to how a software project develops. There is someone with experience who is responsible for the early launch and concept. But what is different is that unless it is an architecture driven development, there is no chief designer who will oversee the design from start to finish. With each phase transition it is the project manager who oversees the progress toward the goal and not an architect. A project manager is usually more focused on the more mundane aspects of the project such as time and budget rather than the less externally visible properties of the system under development.

There is a fundamental difference between art and software. For art, the effect of the completed product is what is valued and not the qualities of the individual pieces, no matter how good. So it is with a software system. A property like security is not solely dependent upon any single component, although that cross-cutting concern will be seen in many of the modules. The lack of attention to even one point in the design can doom the security of the entire system. To one extent or another, this can be true of all the quality (non-functional) requirements placed upon the system. Performance is famously lost through a weak-link in the processing as is availability; modifiability by the inappropriate decomposition of the modules; usability by the lack of a facility for an un-do in the transactions. These qualities can be reduced to an engineering model that provides quantitative, or at least, testable properties. This is not true of art which is often said to lie in the eye of the beholder; not a good quality in an engineering project.

Ironically, I have seen many project proceed as if they were art project. The project management, the client, the business analysts, and the coders all proceed with the functional requirements only and wing it with the crap that is often offered as "non-functional requirements." This forces the team to make decisions for the client as to what qualities and what relative priorities to give to those qualities. This causes the client to not perceive the lack of some quality until they see an early prototype of the product, or worse, the finished product. Then, and only then, does the performance/availability/fault tolerance/usability/security get the attention that it deserved often with disastrous consequences to the design concepts that had guided the many hands that crafted the code.

The number 2 scenario sounds exactly like the way cell animation is done. I think the key difference here is that the transparencies are opaque and therefore block what is behind. This makes the "interface" irrelevant since there is no need for coordination between Bambi and the forest trees behind her. As long as there is no interleaving between two transparencies, there is no need for the careful coordination that is suggested with interfaces. Software, of course, is very different since most any decomposition of a function into smaller modules will create some form of interface that must be spec'd.

For any product that will be created by many hands I cannot believe that self-organization, as it is often understood, can work. I think most people take self-organization to be some form of egalitarian 'let's all get along' I do not believe this is what the Agile manifesto suggests. Rather I believe the Agile manifesto suggests that experienced and skilled professionals are more effective of organizing themselves for the task without an explicit plan from some "manager" who is not as experienced with the technology. However all the same roles will be filled. Group dynamics ensures that a leader will be selected and that the group will go through the steps of forming, storming, norming and performing. Self-organization will also do nothing to prevent the group dysfunctions that affect traditional forms of management as well.

Going back to your example, if a group of artists are given a commission and left to self-organize, the leader that emerges is not guaranteed to be the best artist. It is often true in engineering circles as well. Arrogance, self-confidence, domineering personalities are the most likely to be the group leaders. When this is also the most talented individual and has the leadership skills to bring the team along, the results can be striking. But it can just as easily lead to disaster or a mediocre product too.

Availability versus Fault Tolerant

From Quora...

What is the difference between a highly fault tolerant and a highly available system?

Edit

Add Question Details

Add Comment • Flag Question

3 Answers • Create Answer Wiki

Edmond Lau, Quora Engineer

26 votes by Todd Lipcon, Kimberly Wang, Scott Nyce, (more)

While availability and fault tolerance are sometimes conflated to mean the same concept, the two terms actually refer to different requirements. Designing for high availability is a stricter requirement than designing for high fault tolerance.

Availability is a measure of a system's uptime -- the percentage of time that a system is actually operational and providing its intended service. Service companies, when offering service level agreements (SLAs) to their customers, usually quantify their availability in nines of availability. Carrier-grade telecommunication networks claim "five nines" of availability [1, 2], meaning that the network should be up 99.999% of time and experience no more than 5.26 minutes of downtime per year. Amazon's S3 covers three nines of availability (99.9% uptime) in its SLA [3] and offers a service credit if it is down for more than 43.2 minutes per month.

Fault tolerance refers to a system's ability to continue operating, perhaps gracefully degrading in performance when components of the system fail. RAID 1, for example, by mirroring data across multiple disks, provides fault tolerance from disk failures [4]. Running a hot MySQL slave that can be promoted to a master if the master fails, or eliminating Hadoop's NameNode as a single point of failure [5] are other examples of making a system more fault tolerant.

Making individual components more reliable and more fault tolerant are steps toward making an overall system more highly available; however, a system can be fault tolerant and not be highly available. An analytics system based on Cassandra, for example, where there are no single points of failure might be considered fault tolerant, but if application-level data migrations, software upgrades, or configuration changes take an hour or more of downtime to complete, then the system is not highly available.

--------
[1] http://en.wikipedia.org/wiki/Car...
[2] http://www.windriver.com/announc...
[3] http://aws.amazon.com/s3-sla/
[4] http://en.wikipedia.org/wiki/RAID
[5] http://www.cloudera.com/blog/201...Suggest Edits

What is a Software Product?

I have just completed another seminar at SEI and I want to return to the subject of the prior post. However I am going to recast it into an answer to the question "What is a software product?"

I think most people are likely to take the question to refer to a consumer product like Word or Windows 7. In this case, the product primarily consists of an executable module but no source code. In addition, to make it fit for its intended use, there are several other assumed artifacts associated or imbedded into the product. One is either an installer application or instructions on how the product is to be placed into the intended environment. In addition, there is an implied deliverable which will instruct the end-user how to perform the tasks for which the product intended using the product. These days that is often left implied more than explicit as many products are designed to be self-evident. This means that the intended end-user has sufficient experience with similar user interfaces that the use can be learned with some exploration and no explicit training. Of course this is not always the case and products may either embed the user manual into the product or provide it as a separate artifact.

There is another sense of software product though. If you consider the automobile factory that turns out cars, the cars are products to be sold to consumers. However the factory itself is an asset that can be sold to another auto maker and retooled for their use. In a similar way, source code is a major component in a product that allows a software vendor to create software products. This "software factory" is a tangible asset and is recognized as that through intellectual property laws and common sense. (To avoid some awkward language, from here on, when I refer to a software product, I am using this second sense of the term and will ignore products that only include executables. )But is the source code the entire asset? If we were to sell our intellectual property to another, what can reasonably be considered part of the asset?

In the rush to market, many software vendors are willing to sacrifice the quality of the allied artifacts of the complete system. While this helps to achieve the end goal of creating a working system, it creates technical debt in the deferred maintenance on the artifact with a measurable increase in cost to the total cost of ownership. If the product will never be modified, the complete lack of supporting documentation for the product is reasonable. Even if some was produced during the initial construction of the product, there is no reason to keep it since it will never be read. But how many software products are created that are never intended to be modified? The correction of latent defects, changes in the environment, new requirements, are just a few of the many reasons why the maintenance phase of a software product life cycle is typically the costliest. To ignore the needs of this phase of the product life-cycle are foolish and self-defeating. Therefore any technical debt incurred from the initial development will eventually need to be paid if the consequences of that debt are to be avoided.

The platonic ideal that is sought is some form of self-documenting code. While local comments, when done well, or even well written code without comments, can often be understood without additional commentary, large software systems cannot be self-documenting since many issues transcend individual modules. A software engineer may be able to reproduce the information needed from the source code alone, or at least find a way to insert the modification without it, but this is most likely a more expensive fix than it would have been had the recreation work not been needed. The lack of adequate support for the code base increases maintenance cost. Further, until a code base automatically includes self-documenting ways to explain design choices that transcend individual module, this kind of documentation will continue to exist outside of the source code itself.

This highlights a weakness in the current practice of software engineering management. The true value of this artifact is discounted by management. Since the increased support cost can never be quantified, there is no self-correction of management attitudes. The continued ratio of maintenance cost to total life-cycle cost will remain the same. Worse, there are two compounding effects from this. First, the best information about the design of the system often only exists in the minds of the developers. When they leave the project, it is usually lost for good. Even if they remain in the same organization, the fidelity of the information is lost over time. Second, without an adequate understanding of the design principles used in constructing the product, a maintainer is likely to undermine some of the design clarity that requires restraint on the part of the coder due to ignorance. It is as if the architect leaves no plans for a building behind and the maintenance engineer tries to knock out a wall only to discover an unexpected support post or beam. Even worse would be to not recognize the structural nature of the element during the remodeling and remove it, thus weakening the structure increasing the risk of structural failure.

As Agile methods gain wider use in industry, some inexperienced developers are likely to believe that producing good design documentation is just a bother. A lack of professional development, the desire to just get on with their career, an ignorance of how to best document the design, or some combination of all three likely contributes to the poor quality of this product artifact. Often professionals do not even consider the design artifact to be part of the product itself but view it as some form of construction artifact that serves no purpose after delivery. Certainly many management teams do not appreciate, nor know how to evaluate, the quality of these allied artifacts.

The lack of attention given to the maintenance phase infrastructure and staff needs is a significant blind spot. It has always been a function that has suffered from lower status than the initial construction. As the product life cycles of these software products has gotten longer, there is likely to be at least some far sighted management teams who will eventually realize that long-term profits can be improved through a reduced maintenance phase in their software assets. Once that begins to happen, the types of design documents that most efficiently support the maintenance needs will become the object of study. Until then, these support staffs will continue to be expected to do their jobs with less than ideal knowledge transfer and the need to continually read the minds of developers who are no longer around to ask.