. 4
( 9)



6a (NO) Purchase Orders were not properly written.

When a Purchase Order is not properly written, it can, and probably will,
affect the budget, the schedule, and the quality of the product.


Initiate or reinitiate the Purchase Order. Ensure that there is a clear trail to
the requirements document (contract) backward through the Requirements
Flow-down Matrix (see Attachment 8) and the Requirements Traceability Ma-
trix (see Attachment 7).
Ensure that the Purchase Order fully describes the products or services to be
delivered. Each product or service must be separately listed.
Each Purchase Order is complete and properly written when it contains:
Reference Number, Order Date, Vendor, Contact Information, Name of Item,
Stock (Catalog) Number, Number of Units, Price, Delivery Schedule, Delivery
Location, Purchaser, and Authorizing Signature.
It is bene¬cial that the information contained in the Purchase Order be com-
plete and properly written for a few reasons. For example, it conveys to the
vendor exactly what is expected. Also, you must know exactly the status of each
Purchase Order because of the impact it has on your schedule and your budget.
Most companies have preprinted Purchase Order forms. If yours does not,
create your own. Even if your Purchase Order form is nothing more than a
memo, at least it is documentation of what has been ordered and provides a
basis for the schedule and for ¬nancial accountability. If your preprinted form
does not contain all the information above, I suggest you add the information
within the body of the Purchase Order.

6b (NO) All vendors are not competent to perform their tasks.

Vendors are not competent to perform their tasks if they have not passed the
criteria set forth in your enterprise standards. If you do not have enterprise
standards, the following should be established as the criteria:

’ Technical Performance
’ Cost Performance
’ Delivery Performance
’ Management Performance
’ Procurement Policies and Plans
’ Quality Assurance Program (see Attachment 6 for Quality Assurance
’ Cost of Quality Position (see glossary)

Each criterion must meet the levels established by the enterprise or, in the
instance that there are no enterprise standards, by your project. The vendors
should have been graded against these criteria before each subcontract was


Each vendor that has been shown to be not competent should be reevaluated
using the format similar to that shown in Figure 4-1 on the following page.
The results of this evaluation will isolate the vendor™s problem area. If you
already know what that problem area is, ¬ne. In that case, this exercise will
document the situation for any future activity found necessary such as a Show
Cause letter, etc.
Once you have isolated the problem area, you can set about determining the
cause of the problem and eliminate or change it.

6c (NO) Purchase Orders are not properly monitored.

Simply stated, a Purchase Order (PO) is not properly monitored if an event,
positive or negative, occurs and you are not aware of its happening.
The PO can be considered to be properly monitored when the vendor™s work
is monitored by an assigned project person (usually the Materials Manager call-
ing upon technical and program personnel as required) using monitoring tech-
niques such as:

Figure 4-1 ” Vendor Evaluation Sheet




National Software

Analog Selction Algorithm

G. Smith

Scale Factor

Item Consideration Rating*

1 Organization 3
2 Management 4
3 Manpower 5
4 Access to Management 5
5 Processes 3
6 Procedures 2
Subtotal** 22
No. of items rated** 6
Average of ratings (Subtotal/No of items)** 3.7
*An evaluated number within the Scale Factor.
**Calculated number.

M-M Form

’ Vendor Progress Reports
’ Vendor Meetings
’ In-Process Reviews

The above include Schedule and, if proper, Budget Reviews.
These reviews must be conducted at regular, frequent, and strategic intervals.
Simply conducting these meetings and reviews does not mean the subcon-
tract is performing properly; it only means that the subcontract is being moni-
tored properly. But, if the subcontract is not being monitored properly, you will
not know if it is performing properly.
Within each of these must be monitoring points or metrics that indicate that
an event is in tolerance or out-of-tolerance.
If schedule is critical, the PO should include an incentive or liquidated dam-
ages clause (see glossary) that is invoked in the event the delivery time is not
POs present a particular problem. Because they are generally of relatively low

economic value, they are frequently issued then left alone. A problem is not in
evidence until the item is not shipped, and then it™s too late. Consequently, you
must monitor a PO as you would a subcontract.

Establish reports, reviews, and meetings such as:

’ Vendor Progress Reports
’ Vendor Meetings
’ In-Process Reviews

These reports, meetings and reviews include schedule progress and, if proper,
These reviews must be conducted at regular, frequent, and strategic intervals.
Within each of these must be monitoring points or metrics that indicate that
an event is in tolerance or out-of-tolerance. When an out-of-tolerance condi-
tion occurs, it must be recti¬ed immediately. Most POs are issued for short-
turn-around items, and time is critical.
The most common problem with POs is that the item purchased is ˜˜bumped

out™™ of its planned place in the production cycle by some other, higher priority
project. If this happens, you must ensure that the item is reinstated on the
production line in time to make your schedule. If schedule is critical, it is a
good idea to have an incentive or liquidated damages clause in the PO that puts
a dollar value on delivery time.

6d (NO) Vendors are not performing properly.

Vendors are not performing properly when all monitored events are not
being performed on schedule, within budget, or are not technically competent.
The method you use in determining this status is to conduct regular and fre-
quent reviews at strategic points in the process to ensure that performance is


Implement regular and frequent reviews at strategic points in the process to
ensure that performance is proper. Such reviews include:

’ Vendor Meetings, including
• Schedule Reviews
• Budget Reviews2
• Technical Interchange Meetings (TIMs)3
’ In-Process Reviews

Because you are only noting that something is wrong with this Cause De-
scription, you may need to refer to other Cause Descriptions to get to the actual
source of the problem. Table 4-15 on the following page will refer you to alter-
native Cause Descriptions.


7a (NO) Each person is not competent to perform the tasks assigned.

What happened to the people we proposed to perform this job in the ¬rst
place? To win the job, the best people in the company are usually bid and their
resumes are placed in the proposal. Unfortunately, every other program wants

Ta b l e 4 - 1 5 ” M e e t i n g s a n d R e v i e w s

Review or Meeting Cause Description Appearance
Schedule Reviews 1f, 5e, 6d
Budget Reviews 1f, 5e
Design Reviews 11a, 51e, 52a, 53
Technical Interchange Meetings 1f, 5d, 5e, 6d
Subcontractor Meetings 5d, 5e
In-Process Reviews 5d
Customer Meetings 5d, 5e

these people too, and another project got there ¬rst! Some customers have rec-
ognized this trait and insist that at least 80 percent of the people bid must be
assigned to the job. This is probably the number one problem with a large
company bidding and running many programs simultaneously using the matrix
management system.
This situation can usually be seen during the Planning Phase and can be used
to predict a problem in the future but how do you rectify this situation? There
are three ways.


First, the best way I know of to get the right people is to have the most
exciting project in the company. The right people will be clamoring to be a part
of it. Or you could be the best project manager in the company, and the right
people will come knocking at your door to be on your project. These are abso-
lutely the best ways to man a project. If either of these conditions is beyond
reality, show management how much will be lost if the right people are not
assigned to this project, and either get the right people assigned from the com-
pany or hire the right people.
Second, training existing people is a good answer, but it is second best.
There™s certainly nothing wrong with training”it™s just the cost of training.
Someone has to pay for training in money or time or both. How is training
paid for in your company? Does the company pay for the training and the
trainer™s time and for the student™s time or does the program pay for any or all
or some of these? The answers to these questions will have a bearing on how
you proceed. Remember, even if the company pays for all the elements, you will
not have the people to perform program tasks while they are in training.

If your company uses the matrix form of personnel assignment, then you
must resolve the competency issue with the functional manager. The chances
are that the reason you have the person assigned is that you are getting what™s
left over. Refer back to Cause Description 7a and use the same techniques to
help you resolve this issue here. If your project personnel are assigned directly
as they are in a projectized organization, then the issue must be resolved with
the human resources department or your own people. Who interviewed this
person in the ¬rst place? These kinds of mistakes are often made and leave the
project and the company with people who are constantly being transferred
around rather than resolving the personnel issues. This issue really should have
been solved during the hiring process! If you have already hired this person or
if there are no other persons available from the function, training is the only
real alternative. To double up or offer unanticipated On the Job Training (OJT)
will likely drive your budget ˜˜off-scale-high.™™
Third, you could ¬re this person (if company policies allow such action) and
hire another person in his place. Just remember, the ¬ring-hiring process in a
large company usually takes a long time and there are frequently legal issues
involved with ¬ring a person in any size company.

7b (NO) Each person is not available when needed.

If each person is not available when needed, it means that you have lost
control of the personnel situation. Occasionally, this will happen when a ¬‚u
epidemic comes roaring through and you are caught in the middle of it. Techni-
cally, that™s not necessarily lack of control, but you must nevertheless recover
the time and the work.
Some of your people may be on vacation or on leave. If you are aware of
that situation and have accommodated their absences, you have control. If you
are not aware that one or more of your people are on leave, you have lost
control. This situation occurs most often when the matrix form of management
is used and the individual coordinates absence with his or her functional super-
visor but you are not aware of the situation. This is bad news.


Establish an understanding with both the people assigned to your project
and with the functional managers that you must be a part of the coordination
loop before anyone takes off on vacation, leave, etc.
Occasionally, a person will encounter a sudden illness or bereavement.

There™s not much you can do about these situations except to try to cover them.
The best way to cover these situations is to look at each position when you are
not under pressure and consider what you will do if this person or that person
is absent for one, two, or three days or even weeks. In one case you may be able
to double up. In another case you may be able to defer the task of the person
missing. In another case you will need a temporary person to ¬ll in. Consider
making up a matrix with the names along the side and the conditions across
the top. At the intersects, enter the action to take. This approach takes a lot of
trauma out of the situation when it occurs, but takes a lot of time on the front
end, and situations may change as the program progresses through design, im-
plementation, test, etc. This technique is worth its weight in gold when your
personnel are represented by a union and there is the possibility of a strike.

7c (NO) Salaries/wages are not equal to or less than those bid.

I don™t think I need to say too much about this event except to say it happens
all the time in the matrix form of management. One of the major reasons it
happens is that, once again, you get a different person than the one that was
bid, or the person that was bid gets a raise (that you didn™t know about) before
being assigned to your project. This person may be a senior person whom you
would love to have, but just can™t afford because his salary is over what was


There are several ways to approach this issue. First, you can change the mix
of personnel to accommodate this person or these persons. I recommend that
you pursue this avenue ¬rst, even if you want to get relief in other ways. This is
the ¬rst question the functional manager or management will ask you. Second,
you can approach the functional manager to assign someone else. After all, the
functional manager was the one who bid this labor rate in the proposal and
then gave you something else. Third, you can attempt to convince the functional
manager to subsidize the delta salary of the individual within the functional
manager™s overhead (care must be taken here, especially on government con-
tracts, which may not allow this action). Finally, you can request pro¬t or cost
relief from management to the extent that this person or these persons are
impacting your budget. Don™t be surprised if you get turned down though.

7d (NO) Interpersonal con¬‚icts do exist.

Interpersonal con¬‚icts do exist when tension, personal problems, or down-
right animosity is exhibited between members of the team.


Interpersonal con¬‚icts are a behavior exhibited for a number of reasons.
Among them are:

1. Personality con¬‚ict between individuals
2. Personality issue of an individual
3. Dif¬cult personal (outside) environment
4. Dif¬cult work environment
5. Lack of understanding of one™s role
6. Rejection of one™s role

This book is not and does not intend to be a treatise on psychological issues
and human behavior. However, there are some layman™s observations and ac-
tions that can be taken by the project manager to preserve the objectives of the
project team. These observations and actions follow the numbering scheme
above. Before employing any of these approaches, you may want to attempt to
rectify the situation yourself. That™s ¬ne if it is just a squabble. If it™s a real
problem, however, don™t get any more involved than letting the people con-
cerned know that you are aware of the situation and that if they don™t ¬x it
themselves, you will ¬x it. Sometimes that solves the problem and sometimes it
moves the problem underground. Know your team members and use good
judgment. You personally should not go too much further with these situations.
If possible, turn the problem over to the personnel (HR) of¬ce or other profes-
sionals in the company who are equipped and chartered to handle these kinds
of situations. Your job is that of project manager.

1. Personality con¬‚ict between individuals. Personality con¬‚icts between individ-
uals occur for many reasons, most of which are not fully understood. You
can recover from this situation in one of two ways. First, you can choose the
person who is most useful to the team and transfer the other one. Second,

you can determine the most likely fomenter of the problem and transfer or
¬re (if you have the authority) that individual.
2. Personality issue of an individual. Individual personality issues occur in a lot
of people. Sometimes you just have to put up with them. There are some
people who are critical to the project and they know it and use that position
to their advantage. How you handle this one is a measure of how you handle
all the personnel issues. The only suggestions I can offer is to take this person
to lunch and try to get next to him. Or, if you have political power, use it.
Or, just put up with him. Or transfer him. Just remember that the good of
the project is the most important factor in your decision.
3. Dif¬cult personal environment. When a dif¬cult personal environment exists,
it sometimes takes a long time to recognize. The situation can manifest itself
in a number of ways”the individual™s work begins to suffer or the individual
becomes cranky or both. The ways to handle these situations can vary widely.
Part of the solution could depend on your company. Does the company have
an Employee Assistance Program (EAP) that could help this individual and
relieve the situation? If so, get the person enrolled. Another approach is to
get the person transferred to a staff position somewhere so that the problems
do not affect others. Still another approach is to ¬re or have the person ¬red.
This is very dif¬cult in a large company these days. It requires a lot of record
keeping and consulting and a lot of time on your part. Use the option that
bene¬ts the project the most (i.e., produces the least impact).
4. Dif¬cult work environment. This situation is more in your ballpark than the
employee™s. Why is the work environment dif¬cult? Is the source of the prob-
lem the customer, the company, policies and procedures, the facility itself
(e.g., a smelly building), the schedule, the budget, or something else? Only
after you determine and analyze the source of the problem can you begin to
take action. The answers to these questions could easily ¬ll another book.
The only suggestion I can make here is to recognize and ¬x the situation if
at all possible because it will deeply affect your project. On the other hand,
you may not be able to rectify the situation at all and simply have to live
with it. This is tough and is one of the main reasons people quit projects and
companies. If you can™t solve this problem directly, try to offer an offsetting
positive that is greater than the negative offered by the problem (i.e., offer a
premium such as more money or a cruise or a vacation, etc., for working
this project). Even if you can™t get the problem solved, it needs to be docu-
mented and forwarded. Certainly it needs to be a part of the ˜˜Lessons
Learned™™ paper you will generate at the end of the project. A word of cau-

tion here. If the ˜˜smelly building™™ is the result of an environmental situation
that could harm your team members (and you), you must take action to
remove the people, and perhaps the equipment, from that environment. To
do otherwise is both foolish and possibly criminal. If you have personal
knowledge of a hazardous condition and choose to ignore it, you could be
legally and morally responsible for the harm done to your people or the
equipment. This is not legal advice”it is common sense.
5. Lack of understanding of one™s role. A true lack of understanding of one™s role
resolves to only two possibilities. First, the individual has not been properly
apprised of what is expected of him and possibly of what he should expect
of those around him. Both these issues can be overcome with training. True
team training covers both these issues and is a valuable asset to any project.
Second, the individual has been apprised of his role and still does not under-
stand it. Try to apprise this person once again. If that fails, neither of us can
resolve the issue”send the person back to his functional manager.
6. Rejection of one™s role. If this occurs, you have a real problem! This is classi-
cally known as a standoff. Who is going to win this one? The ¬rst thing for
you to do is to make sure that the role you are asking this individual to
perform is the correct one. Is this what the project needs in order to be
successful? Or was a mistake made when this role was de¬ned? Was a mistake
made in assigning this individual to this role? If you made the mistake, then
you need to reconsider. The mistake could be that the individual™s talents
were not fully considered. How about adjusting the roles to make the project
run smoothly? Can you do that and make the project work properly and
solve the individual™s problem? If you can, do it. If the other side of this
situation is the problem”that is, this individual will reject the role no matter
what, you have only one choice”one of you has to go. Which one is up
to you.


8a (NO) All personnel have not been adequately trained.
Insuf¬cient training is a sad set of affairs. Usually the reason for not provid-
ing training is that it™s expensive or that time has not been scheduled for it. If
you think training is expensive or time consuming, wait until you see the bill
for not training or not training properly. It will not only affect this task but will
show your customer base that you don™t have the trained people to do this kind
of work.


First, you must discover what training is lacking. Is it basic training”the
individual does not know how to perform the basic job assigned? Is it speci¬c
training”the individual does not know how to perform the team-speci¬c task?
The answer is to provide the proper training needed.
If the answer is basic training, you have a personnel competency issue and
need to return to Cause Description 7a and 7a (NO). If the answer is speci¬c
team training, you must provide or arrange to have provided team orientation
or training.
Even though you won™t have the program people you need while they are
being trained, at least they will be trained when they get back. If you are in the
Implementation Phase of your task and just ¬nd out you need some special
training, you will ¬nd that training can be performed after normal duty hours
or on weekends, or you may be able to bring in others who are trained to
mentor your people on an OJT basis. Where there is a will, there is a way.
For the project, nothing could be worse in the world of training than having
attended the wrong training. Time will have been used that will have to be made
up, and new training will need to be added, which will itself use up even more
time and money. The worst kind of wrong training is the training course that
gives out wrong information that leads to people making the wrong decisions
or taking the wrong actions. Carefully evaluate the content of the courses your
people will attend.

8b (NO) The training program is not economical.

The training program could be too expensive in dollars or in time consumed.
Even if the company pays for training, the project loses the time of its personnel
in attendance. This is particularly true if you are beyond the Implementation
Phase in your project. Look very carefully at training and the real need for it
once the project has started. It could be very expensive indeed. By the way, a
training course may well be too expensive. Usually the training department,
usually a part of human resources, has evaluated the training course before it is
presented, but occasionally something slips through. I have had personal experi-
ence with this situation. It ended up costing three days of the time of two dozen
of the highest paid and most needed persons in the company. Carefully evaluate
the need and the cost of training courses.


First, make yourself aware of the training programs available in the company
and in the marketplace. Concern yourself with project-oriented training courses
and team-oriented training courses. There are many of them available.
Second, if someone else has proposed training for your task team, carefully
evaluate the need for and the cost of the proposed training program. Sometimes
the training department comes up with programs they are fully convinced are
great but are actually a waste of your time and the time of your team members.
Third, if you ¬nd that you have attended a training program that is indeed
too expensive, either in dollar or time terms, there is not too much you can do
to recover on your program. Instead, report the ¬ndings to HR (training) and
to management and, if possible, seek relief from the cost burden. Be sure to list
this situation in your ˜˜Lessons Learned™™ paper (see glossary).


9a (NO) The proper amount of data is not being delivered on time.

The proper amount of data is not being delivered on time when the data
deliveries do not match the data stipulated in the requirements document (con-
tract). Some documents are delivered once (i.e., System Test results) and some
require multiple deliveries (i.e., Monthly Status Reports).
Each line item of deliverable documentation in the requirements document
(contract) should contain a delivery date or schedule of dates, a format, and a
content requirement. If it does not, you should create these requirements. All
requirements are then included in the Data Plan.


Create a Data Plan using a spreadsheet program or a Relational Data Base
(RDB) program with a report similar to Table 4-16.
Usually the ¬rst Data Plan is created with delta dates (i.e., dates relative to
award). Soon after award, the actual dates are used. By using this format and
by using a spreadsheet application, the sort function is of great value.
Ensure that proper resources are allocated to implement the Data Plan and
that the customer™s expectations regarding the scope, content, detail, and format
of the deliveries are clearly understood.
It is usual for data to be created in one part of the organization (engineering,

Ta b l e 4 - 1 6 ” D a t a D e l i v e r y M a t r i x

Doc No Title Resp. Format First Del Frequency
A-0001 Monthly Progress Jones DID 1234 30 days Monthly
Report ARO1
T-0001 System Test Smith DID 2345 System Test minus One time
Package 30 days
T-0002 System Test Harris DID 4567 System Test plus One time
Results 30 days
ARO: After Receipt of Order.

for instance) and forwarded to the Data Manager for formatting and disposi-
tion. Ensure that each person understands what data is expected. This is equally
true for electronic transmissions.
Except in the smallest projects, the person who generates the data is not
usually the person who sends the data to the customer. Without Project Of¬ce
review, cognizance, coordination, and control, the practice of individuals send-
ing data directly to the customer is guaranteed to give you heartache.


10a (NO) The Quality Plan is not thorough, complete, and

The Quality Plan is not thorough, complete, and authorized when it does not
completely address and ¬ll the requirements stipulated for the project or when
it has not been authorized by the proper authority.


Review the requirements stipulated by the requirements document (con-
tract) or by enterprise policies and ful¬ll the requirements. You should have
completed this issue before the project started. It is of little value to write a
Quality Plan after the fact.
Refer to Attachment 6 for a Quality Assurance Plan Outline and a Quality
Control Plan Outline.

Additional References:

MIL-STD- 9858
MIL-STD- 2167
MIL-STD- 2168

10b (NO) Speci¬c quality characteristics that are important to the
project were not identi¬ed.

Speci¬c quality characteristics that are important to the project were not
identi¬ed when these characteristics are not documented and used as a checklist.


There must be documented characteristics of the item being developed
whether that item is a system or a unit or a module. The best place to derive
these characteristics is from the Speci¬cation. If you do not have such a list, use
the following characteristics as a starter and look into the Speci¬cation for simi-
lar items that may be used to characterize the quality of the item in question.

’ General
• Primary purpose of the item
• Interfaces of the item
• Common language within the item
• Life-cycle objectives
• Operations and maintenance plan
’ Speci¬c
• Availability
• Other speci¬c characteristics from Spec

10c (NO) Quality is not measured so that improvement or
degradation is not clear.

Quality is not measured so that improvement or degradation is not clear
when each quality characteristic is not measured and not tracked via metrics.


Return to Cause Description 10b and ensure that each quality factor is recog-
nized and documented. Create a two-column list and enter the quality factors
in the left column. Evaluate each factor and assign or develop a measurement
or metric for each factor that will not only show success or failure but will show
general ˜˜health™™ as well (i.e., an analog that shows direction of quality and can
be compared to a standard or the last reading).


11a (NO) Final delivery was not accepted by the customer without

It is expected that ¬nal delivery will be accepted by the customer without
delay because it is one of the most important steps in the closure process. Occa-
sionally, the customer will not accept the product for one reason or another.
There can be only three reasons that the customer will not accept the product
at ¬nal delivery. These are:

1. You have not completed all the requirements stipulated in the requirements
document (contract).
2. There are circumstances beyond the control of the customer.
3. The customer simply does not want to accept the product.


In the ¬rst situation, ask the customer why the product is not being accepted.
What requirements have not been met? Next, you need to outline the require-
ments and lay out the Requirements Traceability Matrix (RTM), then double-
check the requirements and results. If necessary, make the changes required.
Then call the customer for a conference and make your presentation and show
you have completed all requirements.
In the second case, you still ask the customer the same questions. There are
three possible reasons that the customer cannot accept delivery. These are: force
majeure, an Act of God, or some reason the product honestly cannot be ac-
cepted, such as the fact that transportation services or storage facilities are not
available. Check your contract to determine if the ¬rst two are exceptions to the

contract. If they are, inventory the product, work with the customer to secure
the product, and relieve the team. If the third issue is the reason, inventory the
product, work with the customer to secure the product, and relieve the team.
Go through all closure steps that you can. If the ¬rst two factors are not invoked
or if the third factor is the issue and you have completed all requirements for
delivery, you likely have a claim. You must advise the customer of this situation.
Go through all closure steps that you can to minimize additional costs to the
In the third case, you likely have a political situation on your hands. First,
ensure that you have an open communication channel with the customer. En-
sure that you have completed all requirements necessary for delivery. If at all
possible, negotiate with the customer to make ¬nal delivery. Sometimes a cus-
tomer will use acceptance as a bludgeon to get something he wants. And some-
times it is cheaper in the long run to cave in and give customers what they want.
But, if the wants are expensive and if you are clean and can prove it, and if you
have exhausted all reasonable remedies, inventory the product. Advise manage-
ment, turn the issue over to legal, and advise the customer of the situation in

11b (NO) Third-party or drop shipping is involved.

Third-party or drop shipping is involved whenever the product is not
shipped directly from your facilities to the customer™s facilities.
This may appear to be a strange way to end up the checklist but it is a
situation that can contribute greatly to ¬nalizing a contract or delivery. Drop
shipping (shipping from an associate or subcontractor) can be a useful and
sometimes economical methodology. Just as often however, it can end up being
uneconomical and get you into trouble in that you are depending on another
contractor who does not have the responsibility you have.


Short of not using third-party or drop shipping, I suggest you add an incen-
tive clause to the subcontract with the associate or subcontractor. Such an in-
centive clause would have the effect of more than neutralizing any liability you
might incur for late or bad delivery. The reason I say ˜˜more than™™ is because
your reputation is on the line. You could have run a wonderful program for a
long time and lose it all at the last moment. You need to make the penalty so
severe that the event does not happen. By the way, if you use penalties, it is a

good idea (and sometimes required) that you use positive incentives as well. In
other words, if you penalize a subcontractor for being late, give a bonus for
being early or even on time. There are situations when you want the product to
arrive exactly on time, not sooner, not later. Each of these situations requires
analysis and judgments”it is not possible to develop a checklist to cover all
these situations. Instead, as always, use your good sense.


1. Most Purchase Orders are issued against a catalog number and a catalog price;
therefore a budget review is not called for. However, if there is an add-on or modi¬-
cation using your money, Budget Reviews are proper.

2. Most Purchase Orders are issued against a catalog number and a catalog price;
therefore, a budget review is not called for. However, if there is an add-on or modi¬-
cation using your money, Budget Reviews are proper.

3. You can usually hold TIMs only when there is a change in the standard purchased


5.1 General

You will probably be alerted to having a technical problem by observing that
one or more of your technical measurements or metrics is out-of-tolerance.
Naturally, you will want to discover the cause of this condition, and that is
where this chapter comes in. The Technical Recovery Checklist can be used at
any time, not only for recovery but as an adjunct to planning or checking tech-
nical performance.

5.2 Technical Search Tables

Table 5-1 is the Technical Recovery Checklist. Use this table to ¬nd and isolate
a problem in the technical part of your project. The Technical Performance
Checklist is presented in Chapter 3 of this book.

.......................... 9758$$ $CH5 12-09-02 08:29:58 PS

Ta b l e 5 - 1 ” Te c h n i c a l R e c o v e r y C h e c k l i s t

51 ARCHITECTURE Recovery Yes
51a (NO) All Critical Success Factors (CSFs) such as Mean Time To Repair 112
(MTTR), Mean Time Between Failure (MTBF), etc., have not been
documented and understood
51b (NO) All modules/subsystems are not well de¬ned 113
51c (NO) All key functions such as time, length, weight, performance 114
requirements, and interfaces, etc., listed in the requirements are not
appropriately covered
51d (NO) All major elements (physical and data) are not described and/or not 115
51e (NO) All key aspects of user interfaces are not well de¬ned 116
51f (NO) The Architecture does not hang together conceptually 117
52 DESIGN Recovery Yes
52a (NO) The design process is not correct and/or traceable to enterprise, 117
customer, and standard processes
52b (NO) The design is not correct and not traceable to the requirements 119
52c (NO) The design is not ef¬cient 120
52d (NO) The design does not adequately address issues that were identi¬ed 122
and deferred to design at the architectural level
52e (NO) The design is not partitioned into manageable segments 122
52f (NO) The design does not account for supportability, Life Cycle cost/total 123
cost of ownership, and future expansions
52g (NO) Technical Performance Measures (TPMs) such as data retrieval 124
time, weight, error rate, etc., have not been de¬ned or
53 DESIGN REVIEWS Recovery Yes
53a (NO) All Design Reviews were not completed according to required 125
53b (NO) The customer has not approved each Design Review 127
54a (NO) All required In-Process Reviews were not conducted according to 127
required processes
54b (NO) The appropriate authority has not approved each In-Process 129
55 PROTOTYPES Recovery Yes
55a (NO) The prototypes do not re¬‚ect the requirements 130
55b (NO) Prototypes were not constructed incrementally 132
55c (NO) Prototype changes were not incorporated into the design using the 133
Change Control Process
55d (NO) Each prototype change was not reviewed and accepted by the 133
originator of the requirements

56 SUBCONTRACTS Recovery Yes
56a (NO) The sum of all subcontracts does not re¬‚ect all tasks allocated 133
56b (NO) Each subcontract does not contain all tasks allocated 134
57a (NO) The sum of all Purchase Orders does not re¬‚ect all purchases to be 135
57b (NO) Each Purchase Order is not complete 136
58a (NO) All production/manufacturing processes are not traceable to 138
standard, customer, or enterprise processes
58b (NO) The line(s) were not properly designed and set up for this (these) 139
58c (NO) Shop Orders were not correct or thorough 139
58d (NO) The materials were not proper for the processes and the product(s) 140
and/or did not meet the requirements
59 UNIT TEST Recovery Yes
59a (NO) Each Unit Test does not correctly re¬‚ect the requirement 141
59b (NO) Each design element that applies to the routine/module/subsystem 143
does not have its own test case
59c (NO) Unit Test ¬ndings were not reviewed for completeness and not 144
forwarded to be incorporated into Subsystem Tests and the System
59d (NO) All Problem Test Reports (PTRs) were not captured, dispositioned, 144
or worked off
60 SYSTEM TEST Recovery Yes
60a (NO) The System Test Plan/Procedure was not approved by the customer 146
60b (NO) The System Test is not traceable to the requirements 147
60c (NO) The System Test has not tested all elements of the system 148
60d (NO) The System Test was not performed under appropriate load(s) 149
60e (NO) The System Test was not performed using the same kind of 149
personnel that will be used by the customer
60f (NO) The System Test was not properly documented and did not 150
incorporate the test results of all prior-level tests
61a (NO) The Con¬guration Management Plan (CMP) is not thorough, 150
complete, or authorized
61b (NO) Change requests were not presented and approved by an 152
appropriate level of the Review Board
61c (NO) Version controls are not in place and are not re¬‚ected on (in) the 153
62a (NO) All required System Effectiveness Factors have not been 154
appropriately considered

Starting at 51a, read each assertion in the table. If you can answer YES to the
assertion, check it off and proceed to the next one. If you answer NO to the
assertion, go to the page number listed under the ˜˜Recovery™™ column for that

5.3 Technical Recovery Cause Descriptions

Each assertion listed in the Technical Recovery Checklist is supported by a
Cause Description. Each Cause Description broadens and deepens the under-
standing of the checklist assertion and provides a recovery from the issue raised
by the assertion. Following are explanations of the assertions found in the Tech-
nical Recovery Checklist.


51a (NO) All Critical Success Factors (CSFs) such as Mean Time To
Repair (MTTR), Mean Time Between Failure (MTBF), etc.,
have not been documented and understood.

All Critical Success Factors (CSFs) such as MTTR, MTBF, etc., have not been
documented and understood if they have not been incorporated into the design
and a clear trail does not exist from each CSF to its incorporation into the


The ¬rst step in this process is to create a CSF Checklist. Begin by creating a
matrix as shown in Table 5-2. Notice that the requirement is in the far left
column and the ¬nal proof is in the far right column. Scour the requirements
document (contract) for requirement inputs and complete that column ¬rst.

Ta b l e 5 - 2 ” C r i t i c a l S u c c e s s F a c t o r ( C S F ) M a t r i x

CSF Unit A Unit B Unit C Unit D Final Proof
MTTR 0.5 hrs 0.5 hrs 0.5 hrs 0.5 hrs RMA Analysis
0.5 hrs Para 3.2.1
MTBF 30,000 hrs 30,000 hrs 30,000 hrs 30,000 hrs RMA Analysis
30,000 hrs Para 3.2.2

document each issue and get a full reading to determine its validity and
whether it has been properly de¬ned. Qualify each requirement as manda-
tory, highly desirable, desirable, or nice to have. Keep very careful notes
(minutes) of these sessions with the customer. Turn these notes into Change
Notices and formalize them with the customer. Even if the changes are de-
nied, you have a basis for change if the issue arises later on.

After the project is formalized:

1. This means that you have already signed up to the requirement as it exists
and is documented. Consequently, it may be more dif¬cult to get the re-
quirement incorporated into the SOW or Spec and considered ˜˜in-scope.™™
This is particularly true if you are operating under a ¬xed price contract. Use
the same techniques as above. If you are working on a project instead of a
program, you shouldn™t have to worry about the contractual issues. In either
event, it is best to understand the requirements to the fullest extent possible
as early as possible. The nature of a Research and Development (R&D) proj-
ect or program is that it will probably be under constant change. The same
rules apply, however, and the baseline must be updated constantly.
2. Another technique that can be used after the job has started is to interview
or simply walk through with the people who are performing the job. Some-
times these results are surprising because the performing people are not in a
formal mode or mood and you learn what the real issues are. You should be
performing In-Process Reviews, as described in Cause Description Family 54,
In-Process Reviews.

51c (NO) All key functions such as time, length, weight, performance
requirements, and interfaces, etc., listed in the
requirements are not appropriately covered.

All key functions, performance requirements, and interfaces listed in the re-
quirements are not appropriately covered when all are not listed on the ordinate
(the ˜˜Y™™ axis”up the side) of the Requirements Traceability Matrix (RTM)
(see Attachment 7) and the Requirements Flow-Down Matrix (RFM) (see At-
tachment 8) or the WBS locations are not listed on the abscissa (the ˜˜X™™ axis”
across the top or bottom). The requirements are only appropriately covered
when there is an intersect between all requirements and all the WBS locations
showing dispositions.


Quite clearly, what is needed is to list the requirements and the WBS ele-
ments and ascertain the intersects. If you do not have an RTM or an RFM, use
the references shown above (Attachments 7 and 8) and create the necessary
Immediately after you have created the matrix, assess the actions and re-
sources necessary to accomplish the requirements in the matrix. It is one thing
to list the requirements and quite another to ¬nd the resources to get them
Key functions are usually easy to handle. They are explicit in the speci¬cation
and lend themselves nicely to the RTM, RFM approach.
Performance requirements are not usually quite so straightforward in that
there are frequently different ways and methods to reach a given performance.
Nevertheless, performance requirements are objective and can be treated
squarely with objective methods. If performance is to be distributed to more
than one WBS element, the usual method of allocating and accounting for per-
formance requirements is through the use of budgets. A budget allocates part of
the performance requirement to one WBS element and part of the performance
requirement to another WBS element and so on until the entire budget is satis-
¬ed. Those responsible for each WBS element must meet the budget allocation

as if it were the entire speci¬cation.
Frequently, interfaces are not covered in a speci¬cation but must be inferred
or created. This is particularly true if you have the option of decomposing the

system into your own segments or software modules. In this case, you should
de¬ne the interface and all the parameters of the interface and document them.
Usually, these parameters and characteristics are documented into an Interface
Control Document (ICD). ICDs vary by the interface they portray, and each is
usually different. What must be covered in the ICD is a complete characteriza-
tion of both sides of the interface or the interface each side must meet. Such
characterizations might include voltage levels, accepted interfaces (such as
RS-232, etc.) the segment or module must meet. As you can see, this can go
on ad in¬nitum. Just ensure that the interface is adequately characterized and

51d (NO) All major elements (physical and data) are not described
and/or not justi¬ed.
All major elements (physical and data) are not described and/or not justi¬ed
when the system does not hang together or when there are obvious gaps in or

between components of the system or when the functions or physical attributes
of subelements are not clear.
Do not confuse this assertion with having a system ˜˜A™™ Spec (see Cause
Description 2a) which will only characterize the system at a functional level.


You must meet with the customer to determine the customer™s intent for any
element that is not characterized, no matter whether it is a subsystem or a
module. You must be cautious in this regard. If the customer tells you to charac-
terize it yourself, you can do that. But, do that just once. If you characterize the
elements and take them to the customer and the customer says, ˜˜No, that™s not
quite what I wanted, go back and recharacterize,™™ you are in a process called
˜˜Bring me a rock.™™ In other words, the creator of the requirement does not
know what is really wanted and will tell you to ˜˜Bring me another rock™™ until
the right rock is evidenced. This process will consume you and your team.
You must insist that the customer either characterize the elements and their
interfaces or give you complete latitude to do it, in writing.

51e (NO) All key aspects of user interfaces are not well de¬ned.

All key aspects of user interfaces are not well de¬ned when they do not follow
the accepted standards established for the industry.


If you are at this point, you probably did not get direction from your cus-
tomer (generator of the requirements) regarding what standards to follow.
Here, we are talking about color, size, look and feel, data rates, protocols, loca-
tion, and, to some degree, content.
The ¬rst step is to identify the user interfaces. The second step is to apply
the standards accepted by the industry for those interfaces. Probably 98 percent
of the characteristics of all known user interfaces have been de¬ned in standards
of some sort.
Additionally, to ensure that the interface is thoroughly understood, each side
should simulate the other side of the interface for internal testing before integra-
tion. This should illuminate incompletely speci¬ed aspects of the interface. Both
sides are likely to interpret ambiguous parts of the speci¬cation in different
ways, so comparison will bring to light any problems in the interface de¬nitions.

Once all these steps have been accomplished, it is wise to sit down with the
customer and get agreement regarding the interfaces. The agreements should
then be documented and incorporated into the design requirements, reviewed
at the Design Reviews, and demonstrated and tested at every applicable level.
See also Cause Description 51d.

51f (NO) The Architecture does not hang together conceptually.

The Architecture does not hang together conceptually when the system speci-
¬cation fails to describe the functional components of the system in terms of
their behaviors or to provide component-to-component interfaces.


Architecture recovery is an iterative and interactive process, accomplished in
four steps:

The ¬rst step is the discovery or de¬nition of a set of views that represent
the system™s fundamental structural and behavioral elements.
The second step is a fusion of the extracted views.
The third step is the development of attribute-based relationships among the
system™s components.
The fourth step is revisiting the previous steps with a view to architectural
conformance and targets for reengineering.


52a (NO) The design process is not correct and/or traceable to
enterprise, customer, and standard processes.

Here we are talking about the design process, not the design. To be sure,
incorrect design processes can, and probably will, lead to incorrect design.
Whenever an incorrect design surfaces and is recognized, the design must be
changed. The question though is: ˜˜Why was the design incorrect in the ¬rst
place?™™ That cause is pursued in Cause Description 52c. So, chances are, we got
here by going through Cause Description 52c. Incorrect design processes are
determined by eliminating the other causes.
The design process is not correct and/or traceable to enterprise, customer,

and standard processes whenever the required numbers and types of Design
Reviews and the content of each Design Review are not traceable to the standard,
the customer, and the enterprise processes.
The design process is frequently de¬ned by the customer in the Statement
Of Work. The customer will require that this particular program have a Prelimi-
nary Design Review (PDR), a Critical Design Review (CDR), an In-Process Re-
view (IPR), or any number of other Design Reviews. In the supporting or stan-
dard documents, the customer will de¬ne the content of those reviews. This is
the usual mode of operation. When an enterprise works with a customer or set
of customers, the enterprise generally incorporates the usual customer require-
ments into the enterprise policies, plans, and processes. Probably the best exam-
ple of this is contractors who work with the federal government.


Lay out the appropriate enterprise requirements (see glossary), the customer
requirements (from the requirements document), and the standard require-
ments (see glossary) for the design process. Interrelate all the requirements and
summarize and organize them into a checklist that will drive the design process
paragraphs of the program plan. Retain that information to solidify all data
trails. You should end up with a general matrix that will boil down to an outline
similar to the one below. If it is not possible to accommodate the above steps,
go directly to the outline below and use and update it as necessary.

Paragraph Title
1.0 Organization
1.1 Objectives
1.2 Responsibilities
1.3 Authority
2.0 Decision and Control Process
3.0 Con¬guration Management
3.1 Hardware
3.2 Software
3.3 Documentation
4.0 Review Process
4.1 Design Reviews
4.2 Subcontractor Reviews
4.3 Design Approval and Certi¬cation

5.0 Continuous Acquisition and Logistic Support (CALS)
6.0 System Test Planning
7.0 Technical Performance Management
8.0 Communication Plan
9.0 Action Item Process
10.0 Con¬‚ict Resolution process
11.0 Requirement Management Process
12.0 Development Process (including trades)

52b (NO) The design is not correct and not traceable to the

It is possible, but not likely, that a design is correct and yet not traceable to
the requirements. The primary ingredient in that process is luck. It is not a
desirable position in which to be. It is assumed that if you cannot trace the
design to the requirements, you do not have a Requirements Traceability Matrix
(RTM) or, if you do, the RTM is wrong or the design is wrong.


The ¬rst step in this process is to create an RTM. If you do not have an
RTM, use Table 5-3 as a start. Modify the RTM for your own needs. Just be
sure not to change the concepts of content and ¬‚ow.

Ta b l e 5 - 3 ” R e q u i r e m e n t s Tr a c e a b i l i t y M a t r i x ( R T M )

Unit System
SOW/ WBS S/C SOW/ Test Test
Spec Para Requirement Number Spec Para Number Para Monitor
4.3.1 Security 06-03-02 N/A T-0304 4.4.1 Smith

3.2.1 System weight 02-04-03 3.4.6 T-0045 3.4.1 Jones
shall be less
than 10,000

Once you have concluded that the design is incorrect, you must determine
why it is incorrect. There are four possibilities:

1. The requirement was misinterpreted, resulting in an incorrect design. In
this case, you need to go back to Cause Factor 2a/2a (NO) and ¬nd out
why the requirement was misinterpreted and reconcile the requirement.
2. There are customer requirements or expectations (implicit requirements)
that were not previously revealed.
3. The designer was (is) incompetent. In this case, you need to go to Cause
Factor 7a/7a (NO) and work the problem from there.
4. The Design Review processes were not followed. In this case, you need to
go to Cause Factor 52a/52a (NO) and work the problem from there.

Finally, you need to ensure that the necessary design tasks are included in
the project schedule and are properly monitored and performed.

52c (NO) The design is not ef¬cient.
The design is not ef¬cient when it does not perform as the requirements
document (contract) demands, or does not have the inherent reliability, main-
tainability, or availability demanded by the requirements document (contract),
or does not meet at least the same quali¬cations for these factors for competing
products and is not economical in its design, production, or throughout its life
Notice the use of the conjunction or. If your product does not meet all the
requirements and conditions, it is not ef¬cient. This does not mean that trade-
offs cannot be performed. But, if a trade is made, it must be agreed to by the
customer and documented in the requirements. If the product is a competitive
product being designed to the assumed requirements of the marketplace, man-
agement, as well as the departments of marketing, sales, production, customer
relations, maintenance, and quality must be involved and must accept the trade.
Usually the trade involves cost at the expense of some other factor set such as
reduced maintainability to gain faster production (e.g., using rivets instead of
screws) resulting in lower cost.

The ¬rst step in this process is to create an RTM. If you do not have an
RTM, use Table 5-4 as a start. Modify the RTM for your own needs. Just be
sure not to change the concepts of content and ¬‚ow.

Ta b l e 5 - 4 ” R e q u i r e m e n t s Tr a c e a b i l i t y M a t r i x ( R T M )

Unit System
SOW/ WBS S/C SOW/ Test Test
Spec Para Requirement Number Spec Para Number Para Monitor
4.3.1 Security 06-03-02 N/A T-0304 4.4.1 Smith

3.2.1 System weight 02-04-03 3.4.6 T-0045 3.4.1 Jones
shall be less
than 10,000

Ensure that the RTM contains those factors associated with ef¬ciency. Or, if
this is a subjective evaluation, list those factors that are in question along with
their counterparts. Increasing or decreasing one factor will have a direct impact
on at least one other factor.
Immediately after you have created the matrix, assess the actions and re-
sources necessary to accomplish the requirements in the matrix. It is one thing
to list the requirements and quite another to ¬nd the resources to get them
It must be understood that ef¬ciency requirements in one regime or disci-
pline or product will not be the same as the ef¬ciency requirements in another.
For example, the reliability required for a component of a lawn tractor may not
necessarily be the same as the reliability required for a component of the space
It is a good idea to conduct a Failure Mode Effect Analysis (FMEA also
referred to as FMECA-Failure Mode and Criticality Analysis”see paragraph
8.2.3 in Chapter 8) for components that are expected or required to have a high
reliability, availability, or similar stringent requirement.

Additional Resources:

MIL-STD-1629; automotive standards such the SAE, AIAG or Ford Motor
Relex V x.x; from Relex Software
540 Pellis Road, Greensburg, PA 15601

Phone: 724-836-8800

52d (NO) The design does not adequately address issues that were
identi¬ed and deferred to design at the architectural level.

The design does not adequately address issues that were identi¬ed and de-
ferred to design at the architectural level when those architectural elements have
not been de¬ned or are not traceable to the design. Further, they are not ad-
dressed in the appropriate Design Review or clearly identi¬ed in the design.


In order for the system to be complete, all the requirements must be incorpo-
rated into the architecture and the design, and the ˜˜data trail™™ must be identi¬-
able. Frequently, at the architectural level, issues appear that are too complex to
be addressed at that level or are not de¬ned enough to create an architecture to
drive them. This is particularly true in R&D programs. As the design unfolds,
these deferred items must be kept up with and referred back to the architecture
element from which they were deferred in the ¬rst place. Making additions to
the RTM, in reverse order, is a good way to accomplish this task.

52e (NO) The design is not partitioned into manageable segments.

The design is not partitioned into manageable segments when the segments
are either not logical, cannot be de¬ned, cannot be tested, cannot be scheduled,
or cannot be costed. The purpose in partitioning is to create groupings for the
Work Breakdown Structure (WBS) and thus distribute the workload among the
resources available. If this cannot be accomplished, the task cannot be ef¬ciently
worked”if it can be worked at all.


Decompose the requirements into logical groupings. The groupings can be
by subsystem, physical grouping, functional grouping, time ordering, data ¬‚ow,
control ¬‚ow, or some other criterion. Grouping forced by resources available
can be used but should be the last item on the list, not the driving factor.
Remember that the next step will be the allocation of requirements into the

groupings you have established. In some cases, the groupings may need to be
adjusted to make all the requirements ¬t as in Figure 5-1.

Figure 5-1 ” Requirements Allocation

A major rule for partitioning is to minimize interfaces across partition ele-
ments. These interfaces are the most challenging technical issues on most proj-
ects. Does this box belong in this tree or the other tree? You must answer this
question by establishing a ˜˜clean™™ line of division between both the grouping
criterion and the allocation of requirements. Don™t distribute part of a grouping
in one WBS box and the remainder in another box. Try not to distribute part
of a requirement in one WBS box and the remainder in another. If you must
do this, you must create a ˜˜budget™™ to allocate those parts and to document
where you put them. Even when the lines are clean, you may well still need to
interface one element with another. In this case, ensure that a reasonable and
usable Interface Control Document (ICD) exists or is created. See Cause De-
scription 51c/51c (NO) for more detail regarding ICDs.

52f (NO) The design does not account for supportability, Life Cycle
cost/total cost of ownership, and future expansions.

The design does not account for supportability, Life Cycle Cost (LCC), total
cost of ownership, and future expansions whenever all these factors are not
taken into consideration in the design, production, implementation and opera-
tions, and maintenance alternatives.


If you are at or near the very beginning of your project, you are in a position
to achieve the objectives of LCC, that is: Choosing the most cost-effective ap-
proach to the entire life cycle/total cost of ownership system, product, or unit
within the available resources. Sometimes this can include planned or estimated
future expansions as well. The analysis must cover the entire lifespan of the
system, product, or unit. The LCC process provides a systematic methodology
for evaluating and quantifying the cost impacts of alternative courses of action.

It can be used to support trade-off analyses between several product design
con¬gurations, or the sensitivity of a speci¬c design to changes. The LCC can,
and probably will, affect the distribution of costs between up-front design or
production costs and ¬eld operation and maintenance costs. Care must be taken
to involve the entire life span of the system, product, or unit. Frequently, only
design or production costs are considered, leaving operation and maintenance
costs to be added later. If the speci¬cation calls for a Design To Cost approach,
its result may well be different than the Life Cycle Cost. Be certain to check with
the customer regarding intent. The customer could have intended Life Cycle
Cost but said Design To Cost or some similar term. The results will probably
be different.
If you are at a point other than the very beginning of your project and are
confronted with LCC issues, you are starting a long uphill battle. The LCC
program should be started before design is begun, indeed, before parsing of the
system has begun. Usually, LCC begins with trade-off analyses ¬rst on various
designs, then on various production methods and techniques, and ¬nally on the
planned operations and maintenance approaches.
Nevertheless, you are here and must make the best of what you have. If you
are in the design phase, you could stop and conduct trade-offs and then reenter
the design phase. If you are beyond the design phase and into the production
phase, chances are that it will be too expensive to go back. At this point, it is
probably prudent to stop the project and to concentrate on the production
phase and beyond. Consider alternatives that not only make production more
ef¬cient and less expensive but alternatives that make operations and mainte-
nance more ef¬cient. This is the point at which you need to consider spare
parts. It is much more ef¬cient to turn out spare parts during the production
phase than to retool at a later time and gear up to produce those parts. Remem-
ber the $800.00 hammer? That™s what happened. The proper alternative is to
provide Pre Planned Product Improvements (P3I) during the planning cycle
and phase these improvements in over time.
If you are beyond production, the only alternatives you have are installation,
operation, and maintenance. Here, you simply look at trade-offs for those as-
pects of the project.

52g (NO) Technical Performance Measures (TPMs) such as data
retrieval time, weight, error rate, etc., have not been de¬ned
or accommodated.
Technical Performance Measures (TPMs) such as data retrieval time, weight,
error rate, etc., have not been de¬ned or accommodated whenever the TPMs

are not fully understood or do not appear in the related WBS sections or the
related test procedures.


It will probably be necessary to start with de¬nitions of the TPMs. Likely,
you will need to decompose the TPM into its constituents and de¬ne each of
them. Most TPMs are metrics; that is, they have a measurement (or measure-
ments) related to a goal or are expressed as a goal. For instance, an error rate of
less than 10-6 bits. In this case, you will need to de¬ne the number of bits in a
total transmission and then de¬ne the number of errors of transmission. You
will then need to de¬ne a method of measurement that will verify the TPM”an
error rate counter, for instance.


. 4
( 9)