International Defense Security & Technology Your trusted Source for News, Research and Analysis
Related Articles
The Department of Defense (DOD) has a well-documented history of taking much longer and spending much more than originally planned to develop and acquire its weapons systems. In particular, as systems transition from development to production, programs experience significant manufacturing problems. Billions of dollars in cost growthoccur as programs transition from development to production, and unit-cost increases are common after production begins. Several factors contribute to these problems including inattention to manufacturing during planning and design, poor supplier management, and a deficit in manufacturing knowledge among the acquisition workforce. Essentially, programs did not identify and resolve manufacturing risks early in development, but carried risks into production where they emerged as significant problems.
It is essential to find better ways of doing business and, in particular, to make sure systems are manufactured on time and cost-effectively. To this end, leading commercial companies have achieved more predictable outcomes from their manufacturing efforts because they understand producibility—the relative ease of producing designs of an item, product, or system economically with available production techniques—and identify manufacturing risks early and manage them effectively throughout a product’s development life cycle. In 1996, GAO reported the practices that world-class commercial organizations had adopted to more efficiently produce quality products, to improve DOD’s quality assurance program. DOD was spending $1.5 billion extra per year on military-unique quality assurance requirements for major acquisitions and billions more on cost and schedule overruns to correct problems. GAO concluded that repeated unstable designs, poor process controls, and poor transition to production caused the manufacturing quality problems. While DOD had taken some actions, its culture was cited as the biggest reason for slow adoption and unimplemented recommendation
While DOD has made some progress over the last two decades in addressing the problem—including policy changes and advocating the use of best practices for product development—GAO’s recent weapon system reviews show that manufacturing problems, among others, continue to hinder acquisition cost, schedule, and performance outcomes. For example,recent Air Force study reports that manufacturing and quality assurrequirements are not included in the contracts to develop weapon systems, which could affect the contractor’s approach to manufacture. Some DOD programs and prime contractors had an inadequate defense manufacturing workforce—both in terms of numbers and experience—to effectively manage and oversee manufacturing efforts, which resulted in schedule delays or cost inefficiencies. The manufacturing workforce includes occupations such as specialists in quality assurance, business, manufacturing engineering, industrial engineering, and production control.
The Exoatmospheric Kill Vehicle program was put on an accelerated development schedule in response to a directive to develop and deploy,the earliest possible date, ballistic missile defense drawing on the best technologies available. According to the contractor, it bypassed some oits normal development-review processes to accelerate delivery of the vehicle, which also resulted in a high acceptance of manufacturing risks without sufficient identification and management of risk-mitigation plans For example, the program went into production without completing qualification testing.
Nearly two-thirds of programs that entered production after 2000 reported more than a 5 percent increase in average unit cost growth, while 32 percent of programs reported average unit cost growth that ranged from 11 percent to more than 15 percent. Over 50 percent of current programs in production have encountered some form of delay after the production decision, when manufacturing processes should be in control. Consequently, warfighters often must operate costly legacy systems longer than expected, find alternatives to fill capability gaps, or go without the capability altogether.
Acquisition programs and projects in many organizations are broadly divided into phases of technology development, product development, production, and operation activities. These phases may be further divided by decision points or stage gates with criteria and activities that should be met or completed before committing additional resources to the project. Passing from one decision point to the next requires evidence and documentation, such as test reports, data analysis, and other assessments to demonstrate that these criteria have been met.
In recognition of the lack of manufacturing knowledge at key decision points and the need to develop more affordable weapon systems, DOD made recent changes to its policy. On December 8, 2008, DOD issued a revised version of its policy instruction on operation of the defense acquisition system that, among other things, recognizes the need to consider manufacturing risks earlier in the acquisition life cycle and assesses risks prior to key decision points. It also developed MRLs as a measure that could strengthen the way the department manages and develops manufacturing-intensive systems.
Several factors contribute to these issues during the planning and design phases. These include the inattention to manufacturing during planning and design, poor supplier management, and lack of a knowledgeable manufacturing workforce. Essentially, some of these programs moved into production without considering manufacturing risks earlier in development. During the material solutions phase, prior to milestone A, the 2008 policy instruction requires the analysis of alternatives to assess “manufacturing feasibility.” During the technology development phase, prior to milestone B, the instruction states the following: Prototype systems or appropriate component-level prototyping shall be employed to “evaluate manufacturing processes.” A successful preliminary design review will “identify remaining design, integration, and manufacturing risks.” A program may exit the technology development phase when “the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment” and “manufacturing risks have been identified.”
After milestone B, one of the purposes of the engineering and manufacturing development phase is to “develop an affordable and executable manufacturing process.” The instruction says that: “the maturity of critical manufacturing processes” is to be described in a post-critical design review assessment; system capability and manufacturing process demonstration shall show “that system production can be supported by demonstrated manufacturing processes;” and the system capability and manufacturing process demonstration effort shall end, among other things, when “manufacturing processes have been effectively demonstrated in a pilot line environment, prior to milestone C.”
Finally, at milestone C, the instruction establishes two entrance criteria for the production and deployment phase, which include “no significant manufacturing risks” and “manufacturing processes [are] under control (if Milestone C is full-rate production).” Low-rate initial production follows in order to ensure an “adequate and efficient manufacturing capability.” In order to receive full-rate production approval, the following must be shown:
1. “demonstrated control of the manufacturing process,”
2. “the collection of statistical process control data,” and
3. “demonstrated control and capability of other critical processes.”
MRLs Have Been Proposed to Improve the Way DOD Identifies and Manages Manufacturing Risk and Readiness
MRLs are a measurement scale designed to provide a common metric and vocabulary for assessing manufacturing maturity and risk. MRL assessments identify the risks and manufacturing readiness of a particular technology, manufacturing process, weapon system, subsystem, or element of a legacy program at key milestones throughout the acquisition life cycle.
An analysis by the working group shows that MRLs address many of the manufacturing issues not covered by DOD’s technical reviews, particularly reviews conducted in the early phases of acquisition. In their development, comprehensive efforts were undertaken to design and develop MRLs from DOD as well as industry resources. The working group also designed MRLs as a structured and disciplined approach for the way manufacturing risk and readiness is expected to be identified and assessed. The working group also developed a set of tools that include a deskbook, checklist, and a website to help managers and users apply MRLs and conduct assessments. In addition, the Army and Air Force report that their use of MRLs on pilot programs contributed to substantial cost benefits on a variety of programs, including major acquisition programs. MRLs, when used in combination with technology readiness levels, are expected to address two key risk areas—immature product technologies and immature manufacturing capability.
The working group also developed a set of elements called “threads” to provide acquisition managers and those conducting assessments an understanding of the manufacturing risk areas. For these threads, desired progress is defined for each MRL, to provide an understanding of risks as readiness levels increase from one MRL to the next. Conceptually, these threads are manufacturing elements that are essential to programs as they plan, prepare for, and manage the activities necessary to develop a product. For example, the materials thread requires an assessment of potential supplier capability by MRL 3 and an assessment of critical first-tier suppliers by MRL 7. Likewise, the manufacturing personnel thread calls for identifying new manufacturing skills by MRL 3 and identifying manufacturing workforce requirements for the pilot line by MRL 7.
MRL levels
MRL 1—Basic Manufacturing Implications Identified
This is the lowest level of manufacturing readiness. The focus is to address manufacturing shortfalls and opportunities needed to achieve program objectives. Basic research (i.e., budget activity 6.1 funds) begins in the form of studies.
MRL 2—Manufacturing Concepts Identified
This level is characterized by describing the application of new manufacturing concepts. Applied research (i.e., budget activity 6.2 funds) translates basic research into solutions for broadly defined military needs. Typically this level of readiness in the science and technology environment includes identification, paper studies, and analysis of material and process approaches. An understanding of manufacturing feasibility and risk is emerging.
MRL 3—Manufacturing Proof of Concept Developed
This level begins the validation of the manufacturing concepts through analytical or laboratory experiments. This level of readiness is typical of technologies in the science and technology funding categories of Applied Research and Advanced Development (i.e., budget activity 6.3 funds). Materials or processes, or both, have been characterized for manufacturability and availability but further evaluation and demonstration is required. Experimental hardware models have been developed in a laboratory environment that may possess limited functionality.
MRL 4—Capability to Produce the Technology in a Laboratory Environment
This level of readiness is typical for science and technology programs in the budget activity 6.2 and 6.3 categories and acts as exit criteria for the materiel solution analysis phase approaching a milestone A decision. Technologies should have matured to at least technology readiness level 4. This level indicates that the technologies are ready for the technology-development phase of acquisition. At this point, required investments, such as manufacturing technology development, have been identified. Processes to ensure manufacturability, producibility, and quality are in place and are sufficient to produce technology demonstrators. Manufacturing risks have been identified for prototype build, and mitigation plans are in place. Target cost objectives have been established and manufacturing cost drivers have been identified. Producibility assessments of design concepts have been completed. Key design performance parameters have been identified as well as any special tooling, facilities, material handling, and skills required.
MRL 5—Capability to Produce Prototype Components in a Production-Relevant Environment
This level of maturity is typical of the midpoint in the technology-development phase of acquisition, or in the case of key technologies, near the midpoint of an advanced technology-demonstration project. Technologies should have matured to at least technology readiness level 5. The industrial base has been assessed to identify potential manufacturing sources. A manufacturing strategy has been refined and integrated with the risk-management plan. Identification of enabling/critical technologies and components is complete. Prototype materials, tooling and test equipment, as well as personnel skills, have been demonstrated on components in a production-relevant environment, but many manufacturing processes and procedures are still in development. Manufacturing technology development efforts have been initiated or are ongoing. Producibility assessments of key technologies and components are ongoing. A cost model has been constructed to assess projected manufacturing cost.
MRL 6—Capability to Produce a Prototype System or Subsystem in a Production-Relevant Environment
This MRL is associated with readiness for a milestone B decision to initiate an acquisition program by entering into the engineering and manufacturing development phase of acquisition. Technologies should have matured to at least technology readiness level 6. It is normally seen as the level of manufacturing readiness that denotes completion of science and technology development and acceptance into a preliminary system design. An initial manufacturing approach has been developed. The majority of manufacturing processes have been defined and characterized, but there are still significant engineering or design changes, or both, in the system itself. However, preliminary design of critical components has been completed and producibility assessments of key technologies are complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on systems or subsystems, or both, in a production-relevant environment. A cost analysis has been performed to assess projected manufacturing cost versus target cost objectives and the program has in place appropriate risk reduction to achieve cost requirements or establish a new baseline. This analysis should include design trades. Producibility considerations have shaped system-development plans. Industrial capabilities assessment for milestone B has been completed. Long-lead and key supply-chain elements have been identified. All subcontractors have been identified.
MRL 7—Capability to Produce Systems, Subsystems, or Components in a Production-Representative Environment
This level of manufacturing readiness is typical for the midpoint of the engineering and manufacturing-development phase leading to the post-critical design review assessment. Technologies should be maturing to at least technology readiness level 7. System detailed design activity is underway. Material specifications have been approved and materials are available to meet the planned pilot-line build schedule. Manufacturing processes and procedures have been demonstrated in a production-representative environment. Detailed producibility trade studies and risk assessments are underway. The cost model has been updated with detailed designs, rolled up to system level, and tracked against allocated targets. Unit-cost reduction efforts have been prioritized and are underway. The supply chain and supplier quality assurance have been assessed and long-lead procurement plans are in place. Production tooling and test equipment design and development have been initiated.
MRL 8—Pilot-Line Capability Demonstrated; Ready to Begin Low-Rate Initial Production
This level is associated with readiness for a milestone C decision, and entry into low-rate initial production. Technologies should have matured to at least technology readiness level 7. Detailed system design is essentially complete and sufficiently stable to enter low-rate production. All materials are available to meet the planned low-rate production schedule. Manufacturing and quality processes and procedures have been proven in a pilot-line environment and are under control and ready for low-rate production. Known producibility risks pose no significant challenges for low-rate production. The engineering cost model is driven by detailed design and has been validated with actual data. The Industrial Capability Assessment for milestone C has been completed and shows that the supply chain is established and stable.
MRL 9—Low-Rate Production Demonstrated; Capability in Place to Begin Full-Rate Production
At this level, the system, component, or item has been previously produced, is in production, or has successfully achieved low-rate initial production. Technologies should have matured to at least technology readiness level 9. This level of readiness is normally associated with readiness for entry into full-rate production. All systems-engineering/design requirements should have been met such that there are minimal system changes. Major system design features are stable and have been proven in test and evaluation. Materials are available to meet planned rate production schedules. Manufacturing process capability in a low-rate production environment is at an appropriate quality level to meet design key-characteristic tolerances. Production risk monitoring is ongoing. Low-rate initial production cost targets have been met, with learning curves validated. The cost model has been developed for the full-rate production environment and reflects the effect of continuous improvement.
MRL 10—Full-Rate Production Demonstrated and Lean Production Practices in Place
This is the highest level of production readiness. Technologies should have matured to at least technology readiness level 9. This level of manufacturing is normally associated with the production or sustainment phases of the acquisition life cycle. Engineering/design changes are few and generally limited to quality and cost improvements. System, components, or items are in full-rate production and meet all engineering, performance, quality, and reliability requirements. Manufacturing process capability is at the appropriate quality level. All materials, tooling, inspection and test equipment, facilities, and manpower are in place and have met full-rate production requirements. Rate production unit costs meet goals, and funding is sufficient for production at required rates. Lean practices are well established and continuous process improvements are ongoing.
Page
