Report to Congressional Requesters

GAO

WEAPONS

Lessons Learned

Program Generally

Effective but Could Be

Improved and

Expanded

Results in Brief 3

Background 4

Lessons Learned Program Has Made Positive Contributions but

Needs Improvement 6

Sharing of Lessons Learned Could Be Expanded 12

Conclusions 14

Recommendations for Executive Action 15

Agency Comments and Our Evaluation 16

Table 1: Status of the Chemical Stockpile Disposal Project 5

Table 2: May 2000 Agent Release at Tooele Chemical Agent

Disposal Facility 20

Figure 1: Chem-Demil Programmatic Lessons Learned Program

Process 10

Figure 2: Lessons Learned Stakeholders and Process Steps 28

Contents

Figure 3: Chemical Demilitarization Program Organization Chart 34

DFS deactivation furnace system

DOD Department of Defense

GAO General Accounting Office

LIC liquid incinerator

SDS spent decontamination solution

September 10, 2002

The Honorable Jeff Sessions

The Honorable Gordon Smith

The Honorable Ron Wyden

United States Senate

The Honorable James V. Hansen

The Honorable Duncan Hunter

The Honorable Bob Riley

House of Representatives

The Army has been tasked to destroy about 31,500 tons of highly toxic

chemical agents by April 2007, the deadline set by an international treaty

for the elimination of all chemical weapon stockpiles. Until they are

destroyed, the chemical agents will continue to pose a threat to the

thousands of people living and working near the disposal facilities where

the agents are being stored. To destroy the weapons, the Department of

Defense (DOD) established the Army�s Chemical Demilitarization (or

Chem-Demil) Program. The Army has destroyed over one-quarter (8,044

tons) of the U.S stockpile as of March 2002.

Originally, the Chem-Demil Program consisted only of the Chemical

Stockpile Disposal Project, also known as the baseline incineration

project, which was initiated in 1988 to incinerate chemical weapons at

nine storage sites. Then, in response to public concern about incineration,

Congress established the Alternative Technologies and Approaches

Project in 1994 to investigate alternatives to the baseline incineration

process. In 1997, Congress established the Assembled Chemical Weapons

Assessment Program to identify and test additional technologies as

alternatives to incineration. Today, five of the nine storage sites use

incineration; three others will use or plan to use alternative technologies.

The technology choice for the final site has yet to be determined.

The Chemical Stockpile Disposal Project operates a Programmatic

Lessons Learned Program whose aim is to enhance safety, reduce or avoid

unnecessary costs, and maintain the incineration schedule. A lesson

learned is a set of rules or principles that summarizes past experiences to

help people better perform future tasks. The project�s goal is to capture

and share lessons learned from experience so that stakeholders�

engineers, contractors, and program managers�working in similar

situations on new facilities can apply the knowledge. A lesson learned is

thus the product of a process through which lessons are captured and

shared with stakeholders.

facilities in May 2000, some Members of Congress and state and local

communities near disposal sites became increasingly concerned about the

2000, you requested that we report on the status of the Chemical Stockpile

Emergency Preparedness Program and on the Programmatic Lessons

Learned Program. We issued a report in August 2001 on the Emergency

the Lessons Learned Program has effectively captured and shared lessons

stockpile and (2) identified the extent to which lessons learned have been

shared and areas where sharing could be improved. You also asked us to

provide additional information on incidents at three sites and the

corrective actions taken following the incidents. The information is in

appendix I.

In performing our analysis, we used the underlying principles of

criteria for assessing the program systems that capture and share lessons

Knowledge management includes four fundamental principles: leadership

guidance), processes (including performance measurements) to turn

vision into reality, technology that allows implementation of goals and

supports the processes, and a culture of knowledge sharing and reuse.

Together they create an environment in which a lessons learned program

can successfully function.

We conducted our review from October 2001 to May 2002 in accordance

with generally accepted government auditing standards. See appendix II

for a description of our scope and methodology.

framework for sharing knowledge. The Army uses knowledge management principles in its

The Lessons Learned Program has successfully supported the incineration

has captured and shared many lessons from past experiences and

incidents. It has leadership that communicates the importance of the

processes for capturing and sharing lessons, and a technology to facilitate

and support the program. It also has developed a culture that promotes

using lessons to foster safe operations. However, the Lessons Learned

Program does not fully apply generally accepted knowledge management

principles and lessons sharing best practices, thereby limiting its

effectiveness.

senior managers to help them in decision making or daily operations.

Guidance is needed especially if managers decide not to implement a

lesson learned. In at least one case, this resulted in cost avoidance

prevailing with serious safety, cost, and schedule consequences. The

program also does not define performance measures or provide

incentives for participation.

or validate whether a corrective action has been effective in resolving a

deficiency.

prioritize lessons. These shortcomings not only make it difficult to

verify or validate corrective actions but also may discourage some from

using the database, with potentially serious consequences.

The Lessons Learned Program has been effective in sharing knowledge

among the different stakeholders within the Chemical Stockpile Disposal

Project. However, as new components were created to destroy the

stockpile, the scope of the Lessons Learned Program remained primarily

limited to the incineration project. No policies or procedures were

established to ensure that lessons sharing would expand to all

components of the Chem-Demil Program. As a result, some components

that could greatly benefit from timely and full sharing of lessons learned

with the incineration project are not doing so. This can lead to higher risk

and costly duplication and delays. The Assembled Chemical Weapons

Assessment Program and the Alternative Technology and Approaches

Project in particular could find full participation in the program useful

because the majority of the processes they use are the same as those used

by the incineration project. Lessons sharing best practices would dictate

Results in Brief

that all Chem-Demil Program components share important information

such as lessons learned because they are all part of the same program with

a common objective.

We are making recommendations to help improve the operation and

overall usefulness of the Programmatic Lessons Learned Program.

In 1985, Congress required the Department of the Defense to destroy the

U.S. stockpile of chemical agents and munitions and to establish an

organization within the Army to manage the agent destruction program.

Later, Congress also directed DOD to research and develop technological

alternatives to incineration for disposing of chemical agents and

munitions. These activities evolved into the Chem-Demil Program. The

Chem-Demil Program includes the Chemical Stockpile Emergency

Preparedness Program, created in 1988, to enhance the emergency

management and response capabilities of communities near the storage

was added in 1993 to destroy any chemical weapons or materiel not

included in the stockpile disposal program.

The Chemical Stockpile Disposal Project has or plans to use incineration

to destroy chemical agents at five sites: Johnston Atoll in the Pacific

Ocean; Anniston, Alabama; Pine Bluff, Arkansas; Umatilla, Oregon; and

Tooele, Utah. Tooele is the only site with a facility currently operating. The

three other stateside facilities are scheduled to begin operations in fiscal

years 2002-2003. The Johnston Atoll facility has finished destroying its

stockpile and is being closed. The Alternative Technologies and

Approaches Project will use non-incineration methods (such as agent

neutralization by chemical treatment) to destroy agents in bulk containers

Weapons Assessment Program is also researching alternative methods to

destroy agents in weapons at Pueblo, Colorado, and Blue Grass, Kentucky.

This would include the Chemical Stockpile Disposal Project.

a portion of the agent stored at Pine Bluff by using an alternative method under the

Alternative Technologies and Approaches Project. No decision on whether an alternative

technology will be used at the Pine Bluff site has been reached.

Background

The Office of the Secretary of Defense and the Department of the Army

share management roles and responsibilities in the Chem-Demil Program.

The Program Manager of the Assembled Chemical Weapons Assessment

Program reports to the Under Secretary of Defense for Acquisition,

Technology, and Logistics. Thus, it is independent of the Program Manager

for Chemical Demilitarization, who reports to the Assistant Secretary of

the Army (Installations and Environment).

treaty committing member nations to dispose of selected chemical agents

and materiel by April 29, 2007. In September 2001, the Army updated the

life cycle cost estimate for the Chem-Demil Program from $15 billion to

$24 billion. The new cost estimate extended the agent destruction

schedule at four of the eight stateside sites beyond the initial target date of

and Umatilla, Oregon, among others, the incineration program has

successfully destroyed over 25 percent of the original stockpile (see

table 1).

Percent of total stockpile

destroyed 6 19 0 0 0

Start of operations June 1990 Aug. 1996 4th quarter FY02 4th quarter FY03 4th quarter FY03

End of operations Nov. 2000 4th quarter FY05 3rd quarter FY09 2nd quarter FY09 3rd quarter FY09

Legend

FY = fiscal year

Source: Program Manager for Chemical Demilitarization.

The Lessons Learned Program was created in part because many different

contractors were involved in the incineration program, and a system was

Development, Production, Stockpiling and the Use of Chemical Weapons and on Their

Destruction (known as the Chemical Weapons Convention) in April 1997.

2007 deadline will be requested if and when necessary.

needed to collect and preserve the institutional knowledge and acquired

and share (implement) lessons learned during the different phases of the

chemical stockpile demilitarization process. It collects two different kinds

emergency response, and public outreach. As criteria for assessing the

knowledge management processes used by the Lessons Learned Program,

we identified four of a number of federal organizations that practice

knowledge management and operate lessons learned programs. In making

our selections, we reviewed literature and spoke with knowledge

management experts to find organizations recognized for their ability to

share lessons or effectively manage knowledge. We identified the

following organizations: the Center for Army Lessons Learned, the

Department of Energy, the U.S. Army Corps of Engineers, and the Federal

Transit Authority (for more details, see appendix II).

There are two levels of authority involved in developing lessons learned

from proposed engineering changes. A Configuration Control Board

composed of headquarters staff in the Office of the Program Manager for

Chemical Demilitarization has authority to approve, reject, or defer

engineering change proposals that involve costs above a set limit or

affecting multiple sites. The Field Configuration Control Boards have

authority over changes at their sites involving lower costs. In September

2001, the Lessons Review Team (consisting of headquarters staff) was

established to screen all lessons and engineering changes and provide the

information needed to determine which lessons require a response from

sites. For more information on the lessons learned process, see appendix

III.

The Lessons Learned Program has made valuable contributions in support

chemical stockpile. It has generally operated consistently with knowledge

management principles and lessons sharing best practices and has

successfully captured and shared thousands of lessons. However, the

program does not apply or incorporate all knowledge management

principles and lessons sharing best practices. For example, the program

does not provide needed guidance for senior managers; it does not have

systems.

Lessons Learned

Program Has Made

Positive Contributions

but Needs

Improvement

formal a validation procedure to determine whether a problem has been

fixed; and the database of lessons learned needs improvement.

The Lessons Learned Program has contributed to the Chem-Demil

promoting safety, maintaining schedule, and saving or avoiding costs. We

encourages headquarters, field staff, and contractor personnel in the

incineration program to use the Lessons Learned Program. It has provided

funding and has established processes to capture, evaluate, store, and

share lessons. It is committed to continuous improvement and has

provided the technology needed to support the lessons learned process.

Finally, it fosters a culture in which knowledge sharing is an important

element of day-to-day operations.

While it is difficult to quantify the benefits of each lesson, available data

indicate that lessons learned have generally helped avoid on-the-job

injuries (by using government-furnished-approved tools that are better

suited to specific tasks), reduce costs (by improving the containers used to

transport weapons), or maintain schedules (by improving the design of a

socket to disassemble weapons). We also found that lessons from

accidental releases of chemical agents at Johnston Atoll and Tooele, Utah,

were implemented at other incineration sites under construction, thus

incorporating improvements into the design of those new facilities.

The Lessons Learned Program does not have guidance explaining how

senior managers (at headquarters) should use it in support of their

decision making process. Specifically, there is no guidance that defines the

procedures to be followed when an alternative to a lesson is chosen or

when a lesson is not implemented. Lessons learned guidance for another

federal government agency recommends that lessons be used to optimize

management decision making and to interact with other management tools

We reviewed documentation of lessons learned from incidents at the

Johnston Atoll and Tooele, Utah facilities, and found that three other

(DOE-STD-7501-99).

Important Program

Contributions

Program Lacks Guidance

Decision Making

lesson that had evolved from problems with pipes in the pollution

expensive material (hastelloy) to fix their problem than the material used

the three sites primarily because it would have involved higher initial

costs, and delayed the schedule. In February 2002, pipes at Anniston had

failures similar to those experienced at the first two sites. This raised

safety concerns and resulted in a 4-week delay to replace the pipes with

hastelloy. It is too early to determine whether the material used at the

Umatilla and Pine Bluff sites will have the same problems. Although they

need flexibility to manage the program, senior managers also need

guidance to help make decisions that allow them to consider the potential

impact of not implementing lessons learned. This process would include

safety and risk analyses that can provide criteria should they decide not to

adopt a lesson learned.

There is no formal procedure to ensure that the lessons or corrective

actions that have been implemented have fully addressed a deficiency.

Chem-Demil Program guidance for engineering change proposals does

require that changes be tracked and reported after implementation, but

there is no similar requirement in the guidance for the Lessons Learned

Program (which includes programmatic lessons). Both contractor and

incineration project officials also confirmed that there are no procedures

for monitoring the effectiveness of corrective actions. As a result, a

problem could reoccur and affect safety and costs.

As shown in figure 1, the Lessons Learned Program process does not

contain the final validation stage (dashed line), which most knowledge

air (gases produced during incineration) before it is released into the environment.

lower cost material. Later, however, both the Tooele and Johnston Atoll sites informed the

Lessons Learned Program that a more expensive material (hastelloy) was the appropriate

solution to address the piping failures.

decision to continue using the lower cost material in part of the pollution abatement piping

systems at the three future sites; the recommended lesson emerged from a repeated

problem. Implementing the recommended lesson would have cost the Chem-Demil

Program more than $750,000 and involve multiple sites.

Program Lacks a

Procedure to Validate

Lessons Implemented

management systems and Army guidance consider as a necessary step. As

we previously reported, Army guidance states that lessons learned

programs should have a means for testing or validating whether a

be made to evaluate improvements or to identify positive or negative

trends. The standard also states that corrective actions associated with

lessons learned should be evaluated for effect and prioritized. Without

such a validation procedure in the architecture of the Lessons Learned

Program, there is little assurance that problems have been resolved, and

the possibility of repeating past mistakes remains.

(DOE-STD-7501-99). According to the standard, the development process includes

identification, documentation, validation, and dissemination. The utilization and

incorporation process includes identification of applicable lessons, distribution to

appropriate personnel, identification of actions that will be taken as a result of the lessons,

and follow-up to ensure that appropriate actions were taken.

Note: PLL (Programmatic Lessons Learned) is referred to in this report as the Lessons Learned

Program; in the figure, engineering change proposal is referred to as ECP.

Source: GAO analysis, based on data from PLL.

The lessons learned database includes about 3,400 issues, 3,055

engineering change proposals, and 2,198 lessons. But it is not easy to

obtain fast and ready access to relevant information. Furthermore, the

lessons in the database are not prioritized, making it difficult to identify

which lessons are most important and which need to be verified and

validated.

It is important that an organization employ appropriate technology to

support the participants of a lessons learned program. Having a

technology be available does not automatically guarantee its use or

content structure so that items may be found easily and retrieved quickly,

and (3) deliver only relevant information from all possible sources.

According to database users we interviewed and surveyed, it is difficult to

Database Is Difficult to

Use, Lessons Are Not

Prioritized

find lessons because the search tool requires very specific key words or

phrases, involves multiple menus, and does not link lessons to specific

events. As a result, some users are reluctant to use the database and thus

Many users who responded to our survey stated that they experienced

difficulties in searching the database, and some we interviewed described

specific problems with searches. One described the database as

lessons linked to specific incidents.

Users we interviewed made a number of suggestions to improve the

Furthermore, because the database does not prioritize lessons, managers

may be unaware of some important areas or issues that need to be

monitored or lessons that need to be reviewed and validated. By contrast,

lessons learned processes used by the selected federal agencies include

periodic reviews of the usefulness of lessons and the archiving of

information that is no longer pertinent or necessary. The processes also

include prioritizing lessons by risk, immediacy, and urgency. In 1998, the

Army Audit Agency recommended that the database be purged or archived

of obsolete items and that current and future lessons be prioritized. In

September 2001, the Chem-Demil Program created a Lessons Review

(Washington, D.C., Jan. 30, 2002).

But the team is not prioritizing lessons.

Several other areas also did not adhere to knowledge management

principles and lessons sharing best practices. For example, the Chem-

Learned Program is to achieve its goals or define performance measures to

assess effectiveness. Knowledge management principles stress the

importance of leaders articulating how knowledge sharing will be used to

support organizational goals. Furthermore, the Chem-Demil Program does

not provide incentives to encourage involvement in the Lessons Learned

Program. Lessons sharing best practices and knowledge management

principles prescribe developing and using performance measures to

determine the effectiveness of a program. In addition, the Lessons Learned

Program currently surveys employees after workshops to measure their

satisfaction; however, these surveys are not sufficient to assess the overall

effectiveness of the program. The program is attempting to identify ways

to measure the cost and benefits derived from lessons learned. Knowledge

management principles also encourage using performance evaluation,

compensation, awards, and recognition as incentives for participation in

lessons learned programs. The lack of incentives in the Lessons Learned

Program may lead to missed opportunities for the identification and

sharing of lessons learned.

The Lessons Learned Program has shared thousands of lessons among the

five incineration sites through the different phases of construction, testing,

and destruction of chemical agents. However, as the Chem-Demil Program

evolved through the 1990s, and as the components using alternative

technologies were added, the scope of the Lessons Learned Program did

not expand to share lessons with the new components (see app. IV for a

Program remained primarily focused on the five incineration sites. At the

same time, each stockpile destruction component developed its own

separate lessons learned, but without any program wide policies or

procedures in place to ensure coordination or sharing of information

Some Knowledge

Management Principles

Are Not Applied

Sharing of Lessons

Learned Could Be

Expanded

of the Chem-Demil Program was being hindered by a complex

barriers to sharing.

Today, the four sites that are likely to use alternative technologies are not

full participants in the lessons learned effort:

participate in the lessons learned process or activities. In at least one

instance, the Assembled Chemical Weapons Assessment Program

requested (from the Program Manager for Chemical Demilitarization),

a package of data including lessons on the pollution abatement system

filters, mustard thaw, and cost estimates. The data were eventually

provided, but they were too late to be used during a DOD cost data

cost data for the review because it was unable to obtain information

from the incineration project in a timely manner.

develop its own separate database that it will share with the Lessons

however, could be very valuable to other components of the Chem-

Demil Program, especially the Assembled Chemical Weapons Program,

which also researches alternative technologies. This plan could lead to

lost opportunities and duplication of efforts.

Many of the lessons learned by the incineration project could be used by

the other components of the Chem-Demil Program to promote safe, cost-

Emergency Preparedness Program maintains its own best practices on an Internet site,

shares lessons at national meetings, and does coordinate with the Lessons Learned

Program, especially for outreach and public relations efforts.

lessons and cost data through the Deputy Assistant to the Secretary of Defense

(Chemical/Biological Defense) and the Deputy Assistant Secretary of the Army (Chemical

Demilitarization).

effective, and on-time operations. Many of the technical processes

(storing, transporting, unloading, and disassembling weapons) and

programmatic processes (regulatory compliance, management, public

relations practices) used by the Chemical Stockpile Disposal Project are

very similar to those used by the other programs. This is also the case for

processes used to develop operating destruction, or throughput, rates and

cost and schedule projections. In fact, the majority of processes at

incineration facilities are the same as those used by the Assembled

Chemical Weapons Assessment Program and the Alternative Technologies

and Approaches Project. Under these circumstances, promoting a culture

of knowledge sharing would enable all components to capture and use

organizational knowledge.

Furthermore, there is the possibility that the Pueblo, Colorado, site (and

possibly the Blue Grass, Kentucky, site) now managed by the Assembled

Chemical Weapons Assessment Program, which now reports to a DOD

transfer of responsibilities does take place, it would be important for the

two programs to be already sharing information fully and seamlessly. Even

if the transfer does not take place, knowledge management principles and

lessons sharing best practices both dictate that components of the same

program should share information, especially if they all have a common

goal.

The Lessons Learned Program has made important contributions to the

that the program generally adheres to knowledge management principles

has not been realized. The program needs guidance to help senior

managers make decisions that allow them to weigh the potential impact of

not implementing lessons learned. This guidance would be a set of

procedures, including safety and risk analyses, to be followed before

deciding to counter a lesson learned. Without such guidance, decision

makers, in at least one case, chose lower cost over safety and schedule,

ultimately at the expense of all three. Also, the Lessons Learned Program

lacks procedures to validate the effectiveness of implemented lessons. The

lack of a validation step partially defeats the purpose of the lessons

learned process, which relies on the confirmed effectiveness of solutions

emerging from knowledge and experience. If the effectiveness of a lesson

cannot be validated over time, problems may emerge again, with a

negative impact on safety, costs, and schedule.

Conclusions

Further, the information in the lessons learned database is not easily

accessible or prioritized. These drawbacks have frustrated users and may

discourage them from using the database. This could lead to wrong or

misinformed decisions that could affect safety. In addition, there is no

overarching coordination or sharing of information across all the

components of the Chem-Demil Program, which grew and evolved over

time without policies or procedures to ensure that knowledge would be

captured and communicated fully. As a result, fragmented or duplicative

efforts continue today, and the Assembled Chemical Weapons Assessment

Program in particular lacks access to important data maintained by the

Chemical Stockpile Disposal Project and the Alternative Technologies and

Approaches Project. In the case of the Chem-Demil Program, the absence

of policies and procedures promoting and facilitating the broadest

dissemination of lessons learned places the safety, cost effectiveness, and

schedule of the chemical weapons destruction at risk.

To improve the effectiveness and usefulness of the Chemical

the Secretary of Defense direct the Secretary of the Army to

making exceptions to lessons learned,

learned to ensure corrective actions fully address deficiencies

identified as the most significant, and

easily find information and develop criteria to prioritize lessons in the

database.

We also recommend that the Secretary of Defense direct the Secretary of

the Army to develop policies and procedures for capturing and sharing

lessons on an ongoing basis with the Alternative Technology and

Approaches Project and in consultation with the Under Secretary of

Defense (Acquisition, Technology, and Logistics) develop policies and

procedures for capturing and sharing lessons on an ongoing basis with the

Assembled Chemical Weapons Assessment Program.

Recommendations for

Executive Action

The Army concurred with our five recommendations and provided

explanatory comments for each one. However, these comments do not

address the full intent of our recommendations. With regard to our

recommendation that it provide guidance to assist managers when

deciding to make an exception to a lesson, the Army stated that the

Lessons Review Team has guidance for characterizing the severity level of

lessons learned. However, as our report clearly points out, this guidance is

for site officials and is insufficient in assisting senior managers at

headquarters on important decisions involving costly lessons that could

potentially impact several sites. We believe that good management

practices require that senior managers make decisions based on risk,

safety, and cost analyses and that guidance should be developed to

support this decision-making process as we recommended.

In concurring with our recommendation to develop procedures to ensure

corrective actions fully address deficiencies, the Army stated that it is

Learned Program currently does not validate the results of corrective

actions. Contracting this important function will require monitoring by the

Chem-Demil program to ensure that validation is properly conducted as

we recommended.

The Army stated that it has improved the Lessons Learned database to

make it easier to locate information. Converting the database to an

Internet-based program should also improve its accessibility and utility.

address all related user issues identified in our report in order to improve

the benefits of the database.

The Army concurred with our recommendation to develop policies and

procedures to capture and share lessons with the two alternative

technology programs. It stated that progress had been made toward

sharing lessons between the Alternative Technologies and Approaches

Project and the Lessons Learned Program at key milestones. The Army

also said it has shared the lessons database with the Assembled Chemical

Weapons Assessment Program. However, the Army should require, as we

recommended, that policies and procedures for capturing and sharing

lessons on an ongoing basis be established, instead of sharing at key

milestones and on a one-way basis. This approach would ensure that both

alternative technology programs fully participate in the Lessons Learned

Program and that the database is constantly enriched to enhance safety,

Agency Comments

and Our Evaluation

cost, and schedule based decisions for all components of the Chem-Demil

program.

technical comments, which we incorporated where appropriate.

We are sending copies of this report to interested congressional

committees, the Secretaries of Defense and of the Army; the Assistant

Secretary of the Army (Installations and Environment); the Under

Secretary of Defense (Acquisitions, Logistics, and Technology); the

Director, Federal Emergency Management Agency; and the Director,

Office of Management and Budget. We will also make copies available to

others upon request. In addition, the report will be available at no charge

on the GAO Web site at http://www.gao.gov

Please contact me at (202) 512-6020 if you or your staff have any questions

regarding this report. Key contributors to this report were Donald Snyder,

Bonita Oden, Pamela Valentine, Steve Boyles, and Stefano Petrucci.

Raymond J. Decker

Director, Defense Capabilities

and Management

There have been three releases of agent from operating incineration

facilities and one incident during construction that have generated several

lessons learned. The incineration process and the releases and

construction incidents are described below.

A baseline incineration process uses a reverse-assembly procedure that

drains the chemical agent from the weapons and containers and takes

apart the weapons in the reverse order of assembly. Once disassembled,

the chemical agent and weapon parts are incinerated in separate furnaces

and the gaseous and solid waste is treated in a separate process. Liquid

brine resulting from the treatment of exhaust gases in the pollution

abatement system is dried to reduce the volume and transported to a

commercial hazardous waste management facility.

The path to weapons disposal, in general includes six major steps.

1. Chemical weapons are stored in earth-covered, concrete-and steel

buildings called igloos. These igloos are guarded and monitored for

any signs of leaking weapons by the U.S. Soldier and Biological

Chemical Command.

2. Chemical weapons are taken from the igloos and transported to a

disposal plant in sealed on-site containers by the U.S. Soldier and

Biological Chemical Command. The sealed containers are resistant to

fire and impact.

3. When the on-site containers arrive at the disposal plant, workers check

them for leaking weapons before opening them. Chem-Demil crews

then load the weapons onto conveyors that carry the weapons through

the disposal process. When the weapons are loaded onto the conveyor,

the U.S. Soldier and Biological Chemical Command no longer has

responsibility for them.

4. From this point on, workers manage the disposal process from an

enclosed control room using advanced robotics, computer technology,

and video monitoring equipment. Automatic, robotic equipment drains

the chemical agent from the weapon and takes the weapons apart in

explosive proof rooms.

5. Once dismantled and drained, the individual weapon parts travel to

different furnaces in the plant, each designed for a specific purpose.

The liquid incinerator destroys the chemical agent, the deactivation

Appendix I: Information on the Incineration

Process and Incidents at Three Sites

Incineration Process

furnace destroys explosive materials, and the metal parts furnace

heats shell casings and other heavy metal parts to destroy any

remaining agent contamination.

6. The pollution abatement system cleans the air before it is released into

the environment.

The Tooele Chemical Agent Disposal Facility (Tooele plant) is located on

Deseret Chemical Depot in Tooele, Utah. The facility is designed to

weapons. Tooele plant is the first chemical weapons disposal facility built

within the continental United States. Construction of the Tooele plant

began in October 1989 and disposal operations began in August 1996.

Operations at Tooele plant should be completed in 2008. The Tooele plant

incorporates systems originally tested and used at the Chemical Agent

Munitions Disposal System, also located at the depot. These systems were

Agent Disposal System (Johnston Atoll plant) in the Pacific Ocean. The

Johnston Atoll plant was the first integrated facility built to dispose of

chemical weapons.

The sequence of events described in table 3 is based on documents from

Hazardous Waste, U.S. Army Safety Center, Department of Health and

program contractor. On May 8, 2000, the day shift was processing rockets

in the deactivation furnace system. The deactivation furnace system lower

tipping gate (used to control the feed of munitions to the furnace) did not

close properly and munitions/agent processing was terminated. Workers in

protective gear began to clean and repair the gate and a strainer. A bag

from the strainer, contaminated with GB (nerve) agent, was left on top of

the gate. This is believed to be the source of the agent that was released.

Vapors were drawn from the bag through the furnace system.

During the initial attempt to re-light the afterburners following the

cleaning procedure, the agent monitoring equipment alarmed. During a

second attempt to re-light these burners another agent monitor alarmed. In

summary, a small amount of agent escaped through the common stack

during attempts to relight the furnace. (See table 2.)

The several corrective actions taken were based on 105 investigation

findings involving operations, training, and equipment. Lessons learned

Agent Release at

Tooele

from this incident include (1) modifying feed chute clean out procedures,

(2) providing operator refresher training, (3) installing a deactivation

furnace remote operated valve to isolate the deactivation furnace during

afterburner re-lights, and (4) redesigning deactivation furnace feed chute.

spent decontamination solution (SDS) in the liquid incinerator (LIC) #1.

sticking.

the jam in the lower tipping feed gate.

through the system. The major problem was that the pressure began to oscillate significantly.

incinerator and the pollution abatement system.

problem. This water hose malfunctioned.

droplight did not work. The entrants left, retrieved a working droplight, and returned for a third time to the

explosive containment room to complete the tipping feed gate maintenance.

hose.

feed gate. The sock contained about one pound of agent-contaminated fiberglass fragments.

pull air through the DFS incinerator and pollution abatement system.)

saturated with liquid during the entrants� maintenance operation on the tipping feed gate.

afterburner.

left by the entrants).

safest course of action for preventing a release of agent.

decision was made to stop trying to re-light the burners.

the temperature in the DFS afterburner was approximately 1,250 F lower than the requirement.

emergency responders until nearly 4 hours after the first alarm at 3:34 a.m. on May 9, 2000.

order to slow the loss of temperature to in the DFS.

May 9, 2000

analysis confirmed the presence of agent.

Source: GAO analysis based on information provided by the Program Manager for Chemical

Demilitarization.

In addition to reviewing lessons from Tooele incidents, we were briefed on

two incidents that occurred at Johnston Atoll, and we reviewed relevant

investigation reports for these incidents. Both incidents resulted in

corrective actions and generated several lessons learned.

On March 22, 1994, the liquid agent gun purge process began. The next day

workers dressed in protective gear removed the liquid agent gun, and

three lines had to be disconnected and capped (sealed). These three lines

to the liquid agent gun are the atomizing air, fuel oil, and the agent line.

During the disconnecting of the agent line, the liquid incineration room

agent monitoring system alarmed. Also, the agent monitors in the common

stack began to alarm. Operators turned off the induction fan to divert

room air out through plant exhaust to the carbon filters.

Lessons learned from this incident include (1) replacing the fuel oil purge

system flow meter with an instrument that could be read in the control

room; an investigation found that the flow meter on the agent purge line

was not functioning (2) directing room air away from the pollution

Agent Releases at

Johnston Atoll

abatement system to prevent contaminated air from escaping through the

duct work without going through the furnace and (3) counseling workers

on the importance of following approved standard operating procedures.

On December 8, 1990, a laboratory analysis confirmed emission of

chemical agent from the common stack following a purging (flushing) of

the agent line. It was determined that the probable cause of the release

was that a quantity of agent GB (nerve) leaked from the agent gun or feed

line into the primary chamber of the liquid incineration furnace, and the

agent was swept downstream by the induced draft fan (used to draw air

through the plant) while the furnace was in a cool-down cycle. It appears

that the agent that leaked into the incinerator and ultimately discharged to

the atmosphere was from either valves in the agent feed line to the

primary chamber that were not totally sealed or the agent remained in the

agent line after it was purged and was aspirated into the incinerator and

subsequently the atmosphere. During the incident, and due to a

malfunctioning agent-sampling probe, the agent-monitoring equipment in

the common stack did not detect agent.

Lessons learned from this incident include (1) improving the process to

purge (flush) chemical agent from the feed line by adding a fuel oil purge

and increasing the purge cycle to ensure a complete purge, (2) modifying

the alarm system in the common stack to provide redundancy and test the

alarms more frequently, and (3) closing all four valves after the agent line

is purged and process activities involving the liquid incineration feed

system when the furnace is cooling down to the charcoal filters.

On September 15, 1999, more than 30 construction workers were affected

by an irritating vapor in the air while working in the munitions

demilitarization building. This incident caused many workers to

experience respiratory irritation, sending them to the local hospital where

they were examined and released. Later that day, all construction work

stopped and approximately 800 contracted workers were sent home.

Investigations and analyses lead to the determination that chemical agent

was not involved; instead this was determined to be a construction

incident.

may not have been adequately ventilated. The building ventilation system

was not designed to control contaminants during construction; it was only

intended to control a release of chemical agent when construction was

complete and operations had begun. The release of 800 contracted-

Construction Incident

at Umatilla

construction workers without informing them of the situation that no

chemical agents were involved, coupled with the slow release of

information to the press, eventually heightened public concern.

Lessons learned from this incident include (1) enhancing local ventilation

in the munitions demilitarization building, (2) establishing and posting

evacuation routes and response procedures throughout the site, (3)

installing a temporary public address system at the construction site, and

(4) ensuring there is adequate communications between the site and any

off-site facilities particularly in the event of an incident.

On July 15, 2002, at the time we were drafting this report, an individual

working at the incineration facility in Tooele, Utah, experienced a

confirmed accidental chemical agent exposure. This individual was

performing maintenance on an agent purge line valve in the liquid

incinerator room and was exposed to residual agent present in the agent

purge line. The worker exhibited symptoms of chemical agent exposure.

are conducting investigations looking into the events associated with the

accidental exposure, it is too early to report on lessons resulting from this

incident. The Program Manager for Chemical Demilitarization is awaiting

the investigation reports and will incorporate the corrective actions into

lessons learned. According to the Army, agent operations will not

commence until all corrective actions have been taken and the plant is

deemed safe to operate.

Agent Exposure at

Tooele

To assess the Lessons Learned Program, we reviewed literature on the

principles of knowledge management and our previous reports on lessons

sharing best practices.

interviewed Chem-Demil Program managers, personnel, and the

contractor staff who manage the Lessons Learned Program. We also

reviewed management documents describing the program and we

managers (headquarters and field level) and systems contractor staff at

three sites (Aberdeen, Maryland; Anniston, Alabama; and Tooele, Utah)

to determine how clearly management articulated its expectations

about using lessons learned. We did not select a statistical sample of

database users; therefore, our survey results cannot be generalized to

all Lessons Learned Program database users.

relevant to the lessons learned process. We also interviewed personnel

from the office of the Program Manager for Chemical Demilitarization,

the Anniston, Alabama, site, and the contractor responsible for

managing the Lessons Learned Program.

reviewed the lessons learned process and identified the methods used

to gather, consolidate, and share information with stakeholders. We

technology tools support the lessons learned process.

knowledge sharing and use, we talked to program managers for each

Chem-Demil Program components, headquarters staff, and personnel

from the lessons learned contractor staff to determine how lessons are

shared and whether employees are encouraged to participate in the

program. We also asked the staff we surveyed how frequently they

submitted information to the program, whether they used the lessons,

and whether there were incentives to encourage participation.

general interviews.

Appendix II: Scope and Methodology

To determine whether lessons learned contributed to the goals of the

destruction program; we documented and reviewed several important

lessons that program staff identified. We also traced several lessons from

incidents at Johnston Atoll and Tooele to verify that they had been shared

and implemented at the Anniston facility. We used unverified Army data

to assess whether the Lessons Learned Program achieved its aim of

reducing or avoiding unnecessary costs. To determine if the Lessons

processes we identified four of a number of federal organizations that

practice knowledge management and operate lessons learned programs. In

making our selections, we reviewed literature and spoke with knowledge

management experts to find organizations recognized for their ability to

share lessons learned or effectively manage knowledge. We obtained

information from the Center for Army Lessons Learned, the Department of

Energy, the U.S. Army Corps of Engineers, and the Federal Transit

Authority. We interviewed representatives from each organization about

the processes they used for identifying, collecting, disseminating,

implementing, and validating lessons learned information. We reviewed

their lessons learned program guidance to compare and contrast their

process. We also interviewed an expert familiar with the program about

the management of the lessons learned process. To assess the search,

linkage, and prioritization of the database, we obtained documentation

and interviewed the contractor staff about the information in the database.

We tested the search feature of the database, including accessing menus,

keyword and category listings, and analyzed several lessons learned we

had obtained from our searches. We obtained opinions from the staff we

surveyed on the effectiveness of the lessons learned database and their

suggested areas of improvement. The respondents included managers and

others with an average of 9 years experience in the Chem-Demil Program.

The staff we surveyed routinely search the database for lessons learned

information. We did not select a statistical sample of database users;

therefore, our survey results cannot be generalized to all Lessons Learned

Program database users.

To assess the extent to which lessons learned have been shared, we

interviewed the Program Manager for Chemical Demilitarization and the

contractor responsible for operating the Lessons Learned Program. We

also attended status briefings for each Chem-Demil component. We

focused our work primarily on the stockpile destruction

projects/programs. We conducted interviews with officials from the

Alternative Technologies and Approaches Project, the Assembled

Chemical Weapons Assessment Program, and the Chemical Stockpile

Disposal Project to gather evidence on the commonality the alternative

technology components have with the incineration program and the extent

to which they share lessons learned information. To determine whether

each component participated in the Lessons Learned Program by either

sharing or receiving lessons learned information, we reviewed workshop

minutes from calendar years 2000 and 2001.

To describe the incidents at three sites, we attended briefings on the

incidents provided by officials from the incineration program, and

reviewed incident investigation reports and entries in the Lessons Learned

database. We identified key lessons from these sources and toured the

Anniston Chemical Disposal Facility, to determine whether lessons

learned had been shared and implemented. During our visit, we observed

that several lessons from the Tooele incident, among others, were

implemented.

The Lessons Learned Program was established to collect and share lessons

learned within the incineration program. The Programmatic Lessons

Learned Program uses various methods to identify, review, document, and

disseminate lessons learned information among government and

contractor personnel. The program uses facilitated workshops to

introduce lessons and also takes lessons from engineering change

proposals. The Lessons Review Team reviews issues and determines

specific lessons to be implemented. These issues, engineering changes,

and lessons are stored in a database.

The program uses five distinct steps to develop lessons learned, as shown

in figure 2.

headquarters and site personnel), critical document reviews (of

changes to program documents), engineering change proposals

(technical changes at one or more sites), quick reacts (immediate

action), and express submittals (information from a site.)

workshop, an assessment (a study to support a management

recommendation for change), engineering change proposal review

process (a team at each site reviews changes at other sites), and

directed actions (requests for information on actions a site has taken.)

lessons review team (headquarters activity to segregate lessons into

response required or not required.)

personnel, through access to the database, technical bulletins (a

quarterly publication with information of general interest to multiple

sites), programmatic planning documents (containing policies,

guidelines, management approaches, and minimum requirements), and

site document comparisons (new documents with baseline

documents.)

Four primary elements of these steps are discussed below.

Appendix III: Lessons Learned Process

Note: PLL (Programmatic Lessons Learned) is referred to in this report as Lessons Learned Program;

in the figure engineering change proposal is referred to as ECP.

Source: Project Manager for Chemical Stockpile Disposal, Programmatic Lessons Learned Program

Plan, Revision 3, April 2002.

Facilitated workshops are the primary method for introducing lessons

learned into the Lessons Learned Program. Facilitated workshops are

meetings that offer an environment conducive for site and headquarters

personnel to speak openly about experiences. The intent of the workshops

is to allow program personnel familiar with particular subjects to hold

detailed discussions of issues relative to specific subjects. All issues

discussed in the workshops are entered into the database and later

reviewed to determine if the issues should become lessons learned.

The facilitated workshop process begins with a memorandum that

requests site personnel from the Lessons Learned Program team to

identify topics they want to discuss in workshops. These topics are

generally divided into three basic categories: (1) valuable information

Lessons Learned

Program Facilitated

Workshops

provided to other sites, (2) challenging issues and discussion of issues

with other sites in anticipation of possible recommendations, and (3)

general topics to discuss different approaches to a problem. After each

workshop, a feedback survey is sent to participants to determine user

satisfaction with workshops.

Engineering change proposals are the primary method of approving and

documenting design changes at the sites. Members of the Configuration

Control Board and the Field Configuration Control Boards are responsible

for reviewing and approving engineering change proposals within certain

dollar limits. The Configuration Control Board, consisting of members

from headquarters, is also responsible for managing changes to items or

products identified for configuration control, such as facilities and

equipment in order to maintain or enhance reliability, safety,

standardization, performance, or operability. Each Field Configuration

Control Board consists of members from a site, and is responsible for

controlling engineering changes during construction, systemization,

operations, and closure of facilities. Engineering change proposals are

discussed during bi-weekly teleconferences where the sites can ask the

originating site questions about the proposed engineering change.

The Field Configuration Board is responsible for approving engineering

change proposals with an estimated cost of $200,000 or less. The

Configuration Control Board is responsible for approving proposals with

an estimated cost of $200,001 to $750,000. Proposals over $750,000 are sent

to the Project Manager for Chemical Stockpile Disposal for approval. After

approval, the engineering change proposals are reviewed and input into

the database and sent to the Lesson Review Team as part of the review

process.

Engineering changes are the primary source of design-related lessons

learned. Engineering change proposals are approved changes in the design

or performance of an item, a system or a facility. Such changes require

change or revision to specifications, engineering drawing, and/or

supporting documents. Consequently, the Program Manager for Chemical

Demilitarization developed a review process as a method to capture these

lessons in the Lessons Learned Program. The purpose of the Engineering

Change Proposal Review Process is to provide Chemical Demilitarization

sites with more control over lessons learned decisions and incorporate

lessons learned sharing under the Lessons Learned Program. Additionally,

the review process is structured to allow each site the opportunity to

review engineering changes being implemented at other sites and consider

Engineering Change

Proposal Review

Process

the applicability to their site. The review team consists of members from

team, and the U.S. Army Corps of Engineers.

The Lessons Review Team, established in September 2001, is responsible

for reviewing issues discussed in facilitated workshops to determine their

potential impact and to determine if a specific site action is required.

Additionally, the review team reviews engineering change proposals to

determine if they are design-related lessons learned.

interest and significant impact on safety, environmental protection, or

plant operations. The Lessons Review Team designates lessons learned as

mandatory if the method of implementation has been or is directed from

the Program Manager for Chemical Demilitarization headquarters. A

given site must provide information to the home office on the action taken

required to provide information to the headquarters on the action that the

site has taken.

For mandatory lessons, the Lesson Review Team decision makers provide

specific guidance for implementation of lessons. Technical support staff

on the team conducts lesson reviews and provides recommendations to

the decision maker regarding lessons. A team member is responsible for

initial review of lessons and recommended designation, distribution of

materials before the meetings, and facilitation of the meetings.

The Lessons Learned Program database is a repository for (1) issues

generated from facilitated workshops, (2) engineering change proposals,

(3) critical document reviews, (4) quick react/advisory system and other

lessons learned process data, and (5) programmatic and design lessons

learned. As of April 2002, the database contained 3,400 issues, 7,630

directed action, and 3,055 engineering change proposals.

The database was developed as a stand-alone program allowing users to

employ search utilities or category trees to retrieve lessons. The program

opens to the main screen, which consists of a search, categories, and

lessons screens. The lessons screen is a search mechanism that utilizes a

Lessons Review Team

Lessons Learned

Program Database

or subcategories to narrow the search for lessons in a specific area. To

summarize information and identify lessons in the lessons learned

database, the database contains background information to support each

lesson. The background information provides a condensed history, as well

as the status of each lesson at the Chemical Demilitarization site.

The Departments of Defense and the Army made several changes to the

management structure of the Chem-Demil Program, principally in

response to congressional legislation. Originally the Program Manager for

Chemical Demilitarization reported directly to the Assistant Secretary of

the Army (Installations and Environment), who also oversees storage of

the chemical weapons stockpile. The U.S. Soldier and Biological Chemical

Command manages the stockpile. The Command also manages the

loading, delivery, and unloading of chemical weapons at the destruction

facility. After the estimated cost of the program reached a certain dollar

defense acquisition program. To manage this program in the Army

acquisition chain, it was then transferred to the Assistant Secretary of the

Army (Acquisition, Technology, and Logistics). The Program Manager for

Chemical Demilitarization continued executing the program. In 1997, the

Chemical Stockpile Emergency Preparedness Program was removed from

the Program Manager for Chemical Demilitarization and transferred back

to the Assistant Secretary of the Army (Installations and Environment)

where it is currently managed by the U.S. Soldier and Biological Chemical

Command. Also in 1997, the Army and the Federal Emergency

Management Agency signed a new memorandum of agreement to better

manage the on- and off-post emergency response activities, respectively.

Assembled Chemical Weapons Assessment Program be independent of the

Program Manager for Chemical Demilitarization and report directly to the

this legislation was to separate the pilot program from the baseline

incineration activities. Achievement of this goal also meant that two

program offices would share responsibilities associated with disposal

legislation does not specifically state whether or not the Program Manager

for Chemical Demilitarization will manage the assessment program once

the development of technology evaluation criteria, the technology

assessment, the demonstration, and pilot phases end.

Secretary of Defense (Acquisition, Technology, and Logistics).

Appendix IV: Chemical Demilitarization

Program Management Developments,

1997-2001

In May 2000, we reported on the fragmented management structure and

the inadequate coordination and communication within the Chem-Demil

roles and responsibility of program participants and establish procedures

Army, in December 2001, transferred the Chemical Demilitarization

Program to the Assistant Secretary of the Army (Installations and

Environment), bringing all components of the program, except the

Assembled Chemical Weapons Program, under a single Army manager, as

shown in figure 3. Another significant management change occurred in

April 2002 when the Program Manager for Chemical Demilitarization

retired after holding this position for the past 5 years.

40,000 tons of chemical agent stored there, is part of the Chemical Demilitarization

management demonstrated steps that had been taken to address this issue.

Source: Offices of the Chemical Demilitarization and Assembled Chemical Weapons Assessment

Programs.

Appendix V: Comments from the

Department of the Army

The General Accounting Office, the investigative arm of Congress, exists to

support Congress in meeting its constitutional responsibilities and to help

improve the performance and accountability of the federal government for the

American people. GAO examines the use of public funds; evaluates federal

programs and policies; and provides analyses, recommendations, and other

assistance to help Congress make informed oversight, policy, and funding

of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is

files of current reports and testimony and an expanding archive of older

products. The Web site features a search engine to help you locate documents

using key words and phrases. You can print these documents in their entirety,

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and

daily. The list contains links to the full-text document files. To have GAO e-mail

The first copy of each printed report is free. Additional copies are $2 each. A

check or money order should be made out to the Superintendent of Documents.

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a

single address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000

TDD: (202) 512-2537

Fax: (202) 512-6061

Contact:

Automated answering system: (800) 424-5454 or (202) 512-7470

U.S. General Accounting Office, 441 G Street NW, Room 7149

Washington, D.C. 20548

Obtaining Copies of

GAO Reports and

Testimony

Order by Mail or Phone

To Report Fraud,

Waste, and Abuse in

Federal Programs

Public Affairs