Project+2

=toc= =**SSE 674 Software Risk Management**= Project 2: Part III: RISK MANAGEMENT INFRASTRUCTURE Reference: Managing Risk (methods for software systems Development) by Elaine M. Hall


 * Overview:** The risk management infrastructure includes methods to establish the awareness in developing the policy, conducting training, verifying compliance and improving the process. The goal of this infrastructure is to establish an environment which will support the implementation of risk management in the projects.

**Chapter 9 Develop the Policy**

 * Objective:** To set an organization in motion requires the development of policy, an administrative order which sets the direction of the organization. Policy is determined by the highest level in the organization and it’s a top-down approach which ensures that it is important enough to allocate resources to it. The focus of this chapter is to provide a risk management policy whose goal is to facilitate in developing a risk ethic within the organization.

**9.1 Obtain Commitment**
The importance of a risk ethic must be addressed in an organization since it characterizes a proper risk management philosophy which is based on the idea that one of the responsibility of the organization is to handle its risk appropriately. The following are set of rules for risk ethic and they provide an efficient way to handle risk at work area. 1) Take responsibility for risk. 2) Do not blame people for risk. 3) Communicate risk to the right people. 4) Be proactive in managing risk. 5) Learn from unexpected outcomes.

Commitment is the first step in developing a policy for risk management which is a top-down approach where the administration allocates resources to a task and bottom-up which include employees who provide support to the task. Change can only occur if both top-down and bottom-up can commit and work together in developing a policy. Commitment is based on trust and it strengthens as progress is made toward goals. The following figure 9-1 shows phases of the learning process where institutionalization is the result of learning process that require levels of commitment with time.



**9.2 Allocate Resources**
Any person at any level of the organization can be a leader for instance if there is a need within the organization, there are several ways to let the senior management aware of it by using sign petitions, making telephone calls and sending letters. Resources are very important in the organization and they facilitate in developing policy that sets the organization in motion. Personnel resources need to survey the existing practices, obtain the commitment for the policy, define the policy and communicate it to the organization. When the administration allocates resources to develop policy, it is recognizing the importance of it to the organization.

Budget provide money which can be used in influencing everyone through policy, but it is also important to identify right sponsor for the idea that needs to be applied to the organization. A sponsor control budgets and it is possible to find engineering managers who may support project managers and understand the needs of their projects. Directors can utilize their budget to help several projects and proposals in their business. It is necessary to state the cost and benefits to the sponsor once a source of money is found since the sponsor expect the policy to be successful in the organization once it is implemented. If money cannot be allocated from the budget, then sponsor should be notified immediately and the sponsor needs to place a line item for this task in the next budget.
 * 9.2.1 Apportion the Budget**

In developing a policy and process for software risk management plan, resources should be allocated in terms of budget and schedule. The schedule shows the time for each activity required to achieve the objectives. It is usually dependent on the amount of money received from the sponsor where funding is done in incremental along with activities scheduled to show progress toward the objectives. The activities consist of the documented policy, a standard process definition, training material and a method to improve the practice. The following figure 9-2 shows the sample chart of scheduled activities and it cover the dependencies of the activities and the duration for each activity. It is a good strategy to recruit a volunteer team when it comes to taking action at the organization level. It is important to ensure that volunteers are capable of handling the task and can follow the schedule appropriately. Individual should sign up with their supervisor so that each team member can be fully supported for their efforts. Motivating individual is important also by letting them know the personal benefits in learning new skills and taking a leadership role within the organization. If individuals are motivated, they will be more willing to work in accomplishing the objectives for the organization.
 * 9.2.2 Apportion the Schedule**
 * 9.2.3 Assign the Personnel**

**9.3 Survey Existing Practice**
Surveying is the first task the team should accomplish in building a firm foundation for managing risk and individuals that are from software development side, they need to ensure the practices in business, proposals and other projects are followed appropriately. Depending on the application the policy should be flexible enough to adapt to the changes in the practices.

Risk can be introduced in new business easily since customer demand can create situations which a new business may not know how to handle the situations properly due to inexperience or not enough staff. The following are some questions that should be discussed in new business: 1) Will sales prices have to be changed to reflect unanticipated demand levels for the product? 2) Will organizational downsizing affect time to market? 3) Can the software engineers be trusted to deliver a quality product? 4) Is project management easier than this job in marketing?
 * 9.3.1 New Business Practices**

Marketers have methods for dealing with risky situations and they can be a useful source in developing a policy that accommodates new business practices in software risk management. The following questions can be used to gather information from people in marketing to gain insight on how they accomplish their work. 1) Do they run business models on spreadsheets? 2) Do they use customer surveys? 3) How do they track product features of the competition? 4) How do they manage their uncertainty and make decisions with incomplete information?

There is significant risk in writing proposals and there must be a balance between the resources invested in proposal and the contract. During the proposal phase, the bid-no bid decision is the best decision with the greatest risk since lots of money and years of effort can be saved if prime contractor assess their risk early in the proposal. It is important to handle risk intelligently from using uncertainty to improve the cost model to simulating system performance. To develop a policy that accommodates proposal practices in risk management, it’s a good idea to check with proposal writers since they may have methods for dealing with risks. Also gathering winning proposals write ups and gathering any customer- driven requirement can be helpful in dealing with risks.
 * 9.3.2 Proposal Practices**

Projects usually have persistent problems rather than managed risks since organization normally have some risk management activities in process. Surveying people who are working on projects can be useful since they may discuss risk in terms of potential problems. The following are some questions that can be asked when taking a survey: 1) Do they write down potential problems in weekly status report? 2) Are issues reviewed at technical staff meetings? 3) Does someone coordinate issues that affect several integrated product teams? 4) When is technical interchange meeting scheduled? 5) Does system engineering track technical performance measures at the software level? 6) How does software engineering prevent problems in meeting specifications, cost, and schedule? Knowing the level of risk management practice helps to provide the training material needed to implement the policy and gathering any documented processes, templates, completed forms, or needs to implement risk management practices; realizing if something is missing for example, tools that support risk analysis may be needed in organization and the organization may not be aware of it.
 * 9.3.3 Project Practices**

Internal research and development practices usually have different risk management practices from either proposal or projects practices. Experimentation provides the ideas to be tested and they can be very useful since unexpected outcomes can be handled properly. Treating unexpected outcome as a mistake is a wrong practice that only reduces confidence and inhibit risk taking. To develop a policy that accommodates risk management for laboratory environments, it is important to ask several people in research and development practices. The following are some questions that can used as tools which can assist in developing a policy: 1) Do they have courage to take a risk when they do not know the outcome? 2) Do they understand that the risk of not taking risks is being left behind? 3) Are they able to embrace their mistake? 4) Do they own the mistake, examine it, and learn from it?
 * 9.3.4 Research and Development Practices**

These questions try to determine the attitude toward risk in the part of the organization which is responsible for the growth of their team members. Gathering innovation techniques, experimentation methods can be beneficial in managing uncertainty; for example, people in research and development practices can be benefited by increasing the number of experiments, which in turn can lower the risk by increasing knowledge and the ability to handle the task through trial and error.

**9.4 Define Draft Policy**
Draft policy’s purpose is to enroll the opinion leaders in the change process and let go of the old ways that are no longer effective. People can contribute to shaping the policy by a top-level review of the policy description.

Opinion leaders should be involved so that they can be aware of a draft policy for risk management. They influence others because they are vocal about their beliefs and involving them early in the policy definition process is a good strategy since it shows that their experiences can be utilized effectively in draft policy.
 * 9.4.1 Involve the Opinion Leaders**

The following outline organizes the policy contents: 1) Subject: Software Risk Management. 2) Reference: Standard procedure SP-050-SRM. 3) Purpose: To establish a proactive approach to managing risk by routinely assessing and controlling project, process and product risk. 4) Policy: It is the policy of the organization that risks will be managed by each employee. 5) Scope: New business, proposals, projects, internal research and development. 6) Objectives: Maximize profit and minimize risk. 7) Responsibility: Individuals are accountable for identifying risk to their team leaders. Team leaders are responsible for communicating issues if cannot be resolved at team level to their manager. Managers are responsible for ensuring coordination with affected third parties. The project manager is responsible for ensuring coordination with affected external members. 8) Authority: The project manager delegates authority as required to manage risk. 9) Procedure: The following are five procedures required: 1) Document a risk management plan. 2) Perform a baseline risk assessment early in the project. 3) Report risks at weekly status meetings. 4) Review risks at monthly project reviews. 5) Maintain a risk database and deliver it at project completion.
 * 9.4.2 Outline the Policy Contents**

**9.5 Review Draft Policy**
It is important at this point to have the draft policy reviewed by people where it should promote understanding of the risk management practices expected within the organization and result in incorporating the feedback of the people who will be practicing risk management.

It is a good idea to keep the policy to one page since many people will read it and it would be more easy for people to make suggestions to it. It is important to define on all the terminology where establishing a common vocabulary within the organization is the most important aspect of the policy. If the term shall is too formal or project manager is called principle investigator on internal research and development, than it is wise not to use shall or project manager since this may not be the exact terminology used in the organization.
 * 9.5.1 Promote Understanding**

Incorporating necessary changes are done by personnel assigned to develop the policy and it is their responsibility to ensure all changes are consistent with respect to the intent and vocabulary of the policy. It is appropriate to thank those individuals who contributed in developing the policy and providing an explanation to anyone whose suggestion was not incorporated.
 * 9.5.2 Incorporate the Feedback**

**9.6 Document Policy**
The policy should be documented in a standard format and incorporated in a manual of operating procedures. The personnel department or other specific location should contain a hard copy of the documented policy.

**9.7 Approve Policy**
The policy should be approved at the highest levels to ensure that senior management agrees with the changes from the policy review and the approval should include the signature of a senior manager who represents the management team. This will ensures that the approved policy is critical in managing risk and should be followed appropriately by the organization.

**9.8 Communicate Policy**
The policy should be communicated to the organization in a memo that states when it will take effect and senior management can support the policy by communicating it to their organization.

**9.9 Summary**
Risk management policy is designed which assist in developing a risk ethic within the organization. The risk ethic is important to an organization because it establishes the rules of conduct for managing risk and it has five set of rules which include taking responsibility for risk, not blaming people for risk, communicating risk to the right people, being proactive in managing risk and learning from unexpected outcomes. A risk management policy includes a statement of purpose and responsibility within the organization. Since policy directs the way business is conducted, it sets the expectations for organizational behavior where behavior is controlled by a risk management policy.

**Chapter 10 Define Standard Process**

 * Objective:** A standard process is a minimum set of procedures defined and approved for use by an organization. An organization, whose objective is to ensure product quality, must rely on good process procedures. Having a defined process makes it easier to follow a written script and organize into clearly defined roles. As the defined process is executed, individuals can be identified by their roles and they can work together more efficiently. The focus of this chapter is to establish an action team to define a reusable risk management process for the organization.

**10.1 Establish an Action Team**
The risk management process action team should consist of individuals who represent both large and small projects and various product lines. Organization members who want to participate should be recruited and recognized for their effort. The following three important concepts can be utilized in establishing an effective team: 1) __A team is not a group__ --- Groups communicate and teams collaborate. Communication is the exchange of information or thoughts where collaboration occurs when two more individuals with complementary skills interact to share an understanding that was not previously been possessed. 2) __A team is task oriented__ --- An effective team commits itself to achieving its objectives and team members have a sense of shared purpose. 3) __A team matures over time__ --- Consensus is the decision making process of a mature team and it ensures that everyone live with the decision. Growth and progress are made in spite of obstacles that can occur.

Team members may be assigned one of four roles on rotating basis: 1) __Leader__ --- The leader ensures the team has adequate resources to complete the task; guarantees deadlines and clarify team roles and responsibilities; other responsibilities can be coordinating meeting time, location and team performance. 2) __Facilitator__ --- The facilitator manages the decision making process, encourage participation of all team members and can remind the team of constraints, such as the time remaining in completing the task. The facilitator can also be participants and all team members may act as secondary facilitator. 3) __Recorder__ --- The recorder take notes, assist in decisions and working procedures, document essential facts and distributes meeting minutes. 4) __Participants__ --- They contribute ideas, seek clarification to avoid misunderstanding and they help the facilitator in managing the team progress. A team matures over time in four stages as shown below: 1) __Forming__ --- The team define their mission, task, requirements and determine what behavior is acceptable and how much is expected from them. 2) __Storming__ --- Some team members may ignore goals set by the leader and resist the requirements of the given task which can result in conflict and struggle. 3) __Norming__ --- Members may adopt new roles within the organization as their relationships get closer with each other and the team may be capable of using consensus process which requires team members to share information and openly discuss their views. 4) __Performing__ --- When the team is performing, issues get resolved and goals are achieved.

High-performance teams are created by beginning with an innovative workshop that trains in the basics of teamwork, problem-solving methods, and using data to achieve objectives. The following are several important factors for building a high-performance team: 1) __A shared compelling vision__ --- The members share a common vision and have positive expectations of achieving success. Utilizing brainstorm method for a vision statement can be an efficient way of sharing a vision. 2) __Individual accountability__ --- Each person contributes through clearly defined roles and responsibilities where there is a sense of pride in belonging to the team. 3) __Synergy in collaboration__ --- Team members are open to ideas and they appreciate the diversity that the team is more than the sum of the individual members. Conflict and disagreement can occur easily but it can be handled with mutual trust and respect for each person.
 * 10.1.1 Build a High-Performance Team**

The team can evaluate itself on the following areas to ensure that they are following the steps in building a high-performance team as shown below: 1) A clear elevating goal. 2) Principled leadership. 3) Competent team members. 4) Unified commitment. 5) Results-driven structure. 6) Standards of excellence. 7) Reward and recognition.

The team leader is responsible for creating team notebooks to be given to each team member at the organizational meeting. The following are several advantages for maintaining a notebook: 1) Team member stay on track better when they know the path to completion. 2) Process definition is used though notebook tabs that serve as place holders for the remaining work. 3) If an individual leaves the team, his or her notebook is passed on to the new team member. 4) Notebooks serve later as a good reference that can be checked out from a technical library and it can be used to educate other team members.
 * 10.1.2 Organize the Team for Success**

The team notebook includes the following sections: 1) __Action plan__ --- includes the task requirements, budgets, schedule given to the team leader by the organization, team goals and products to be delivered. 2) __Action team interfaces__ --- is a diagram that illustrates how the action team operates within the context of the organization defined for software process improvement. The following figure 10-1 shows the diagram:



3) __Member directory__ --- contain the names and contact information for all team members. 4) __Activity log__ --- is a one page journal for team members to capture the date, time and activity they performed using the action team charge number. 5) __Meeting rules__ --- includes team operating principles which includes Do and Do not instructions.

The first meeting of a risk management process action team should include the following activities: 1) Introduce team members 2) Clarify team goals 3) Review the action plan 4) Agree on operating principles

After the team is organized, there should be an educational meeting to level the playing field which ensures that everyone begins work with a common understanding of the vocabulary that will be utilized. Skills required include consensus, process definition, and risk management concepts where the team can be trained by using videotapes or a guest lecture.
 * 10.1.3 Level the Playing Field**

**10.2 Develop the Draft Standard Process**
A draft standard process is a brief overview of the final product which is reviewed for understanding and consistency. The following are four steps utilized in developing the draft standard process. 1) Select a process design method. 2) Gather the risk practices data. 3) Scope the effort and products. 4) Define the draft standard process.

The following are three good process design methods: 1) __IDEF0__ --- consist of process elements where they are connected to make a more complex process definition. It is a systematic approach and it can be implemented easily in risk management process. There are extension of IDEF0 also which focus on the tools and information that support the process; for instance, IDEFIX is a parallel information modeling standard which has been used to transform information requirements into physical database schemas. The following figure 10-2 shows the basic building block of the IDEF0 process definition method: 2) __ETVX__ --- is a method which describes a process in terms of entry criteria, task, validation and exit criteria. The following figure 10-3 shows the ETVX process with process elements described: 3) __The 3 R’s__ --- stands for role, responsibility and resources where this approach is based on modeling roles that encapsulate responsibilities and resources associated with the role. Roles decompose hierarchically from general to specific and at each role level, responsibilities and resources are specifically defined. Solutions are developed from recursive design, construction and test of requirements linked to responsibilities. The following figure 10-4 shows 3 R’s process is utilized where customers and suppliers are addressed both internal and external way.
 * 10.2.1 Select a Process Design Method**

The following are several categories of information on risk management practices: 1) __Company practices__ --- consist of information on the current state of practice such as policy, process guidebooks, process assessment findings, and project practices which should be collected and catalogued in the team notebook. 2) __Industry data__ --- provides information on the current state of other organizations which include conferences, training seminars, books etc. 3) __Standards organizations__ --- is the ideal model also called “best practice” such as, IEEE standard for software project management plans which identifies risk management as a managerial process.
 * 10.2.2 Gather the Risk Practices Data**

Understanding the difference between the action team’s objectives and the organization’s current practice will help the team to scope its effort and products. To properly scope its work, the team should consider the risk of defining scope incorrectly. A procedural change has a narrow scope. The degree of change may be an incremental improvement in an existing procedure. A change that is structural or cultural has a broad scope. The degree of change is radical. Sometimes there is a large degree of desired change from “as is” to “should be”. Depending on the constraints of budget and schedule, one can refine the action plan by adjusting the scope of team goals using the purpose hierarchy in order to meet the constraints and ensure success. One needs to decompose the work into task assignments for the team members. There are three intermediate work products that are produced to help bound the task as shown below: 1) __Draft outline__ --- Define the outline like a table of contents that contain the topics in the order that they will be covered and also describing the goal, objective, purpose and context. 2) __Product scope__ --- Define the process as a “black box” with input and outputs only. Define the process entry criteria and prerequisite conditions or products needed to start the process. Also define the process exit criteria and verification criteria needed to end the process. 3) __Process diagram__ --- Draw the process using the standard process notation.
 * 10.2.3 Scope the Effort and Products**

When the task and work products are properly scoped, a draft process is written. There are several common improvement themes to consider in designing the process activities: 1) __Reduce bureaucracy__ --- Remove unnecessary approval cycles and paperwork. 2) __Eliminate duplication__ --- Remove steps that are repeated. 3) __Add value__ --- Assess whether the activity serves the customer’s requirements. 4) __Minimize errors__ --- Make it difficult to introduce an error during the activity. 5) __Standardize__ --- Select a single best way to do the activity. 6) __Automate__ --- Use computers for routine or repetitive tasks.
 * 10.2.4 Define the Draft Standard Process**

The draft process consists of themes and figures that show both content and sequence of activity. The following are three intermediate work products that are produced to generate the draft standard process: 1) __Annotated outline__ --- Develop the structure of the document by writing one or two sentences under each heading that describe what the section will contain. 2) __Process description__ --- Define the activities of each process element at a high level to describe the work flow. 3) __Identify mechanisms__ --- List the methods and tools used by each process element.

**10.3 Review the Draft Standard Process**
When the draft process is defined, the team members review the product and a meeting is scheduled to discuss individual team members’ comments and suggested changes. When the draft standard process is documented, it should be sent out to a large group of reviewers.

The distribution list should be checked to ensure coverage to represent the organization projects and levels of hierarchy. The material should be packaged with a review form which identifies the product under review and the current level of completeness. A checklist should be included in the review package to guide reviewers.
 * 10.3.1 Prepare the Review Package**

The draft process is reviewed for content and focus where important issues that reviewers wish to change or discuss at the interview meeting are recorded on the review form. At the review meeting, reviewers bring their comments and discuss their suggestions. Corrections are submitted to the action team with the completed review form. All major issues are discussed and resolved as action items. The team recorder captures specific decisions reached on key issues and documents action items. For each action item, a determination is made as to whether a review of the altered product is required.
 * 10.3.2 Review the Draft Standard Process**

Reviewers’ corrections will be incorporated into the product and the action team prioritizes the list of action items documented at the review meeting and determines which changes are necessary. When the team incorporates these changes, the action items are closed. Some suggestions will be good improvements, but may need to be incorporated in future versions due to resource constraints.
 * 10.3.3 Incorporate the Recommended Changes**

**10.4 Document the Standard Process**
The standard process is an elaboration of the draft standard process with examples added to increase understanding of the process. To assist process improvement, a change request form is part of the standard process documentation. The purpose of defining a standard process for an organization is to own the process and thus avoid the “not-invented-here” syndrome.

If the draft was properly decomposed, there should not be any new headings in the outline. After team members elaborate the draft standard process to the next level, they may uncover errors of omission in the draft outline. A process diagram will help to illustrate major process elements that can be described in the standard process definition.
 * 10.4.1 Elaborate the Draft Standard Process**

The action team leader and the product review team leader coordinate an evaluation of the standard process. A subset of reviewers three or four are selected based on their experience and position in representing the organization’s business area or product lines. The process is evaluated in the following areas: 1) Implementation of approved organization policy. 2) Compliance to action plan. 3) Compliance to product standards. 4) Closure of action items. 5) Overall quality and usability. As a guideline, if more than 25 percent of the product is change, another evaluation is recommended.
 * 10.4.2 Evaluate the Standard Process**

Review team comments are incorporated into the final product and a brief response for each issue is prepared to indicate what specific changes were made. Action items not addressed are carried forward as open items.
 * 10.4.3 Close the Action Items**

**10.5 Approve the Standard Process**
The action team leader presents the standard process to the organization by sending the final document to those with signature authority. After the process is approved, the team should be recognized by management for their achievement.

**10.6 Distribute the Standard Process**
The action team leader is responsible for ensuring that the approved process is distributed within the organization. New processes should be tested and have a limited distribution until proved on projects. For a new process, associated training material should be given to the training department. For a modified process, updates to existing training material should be provided. Both hard copy and soft copy process definitions should be cross-referenced.

**10.7 Summary**
The activities of the process definition process consist of establishing an action team, developing and reviewing the draft standard process, document, approve and distribute the standard process. A high performance team shares compelling vision, hold individual accountability and provide synergy in collaboration. A process definition notation is important because it is a form of written communication that describes a process in shorthand. A purpose hierarchy is used to describe the action team success criteria. Teams must understand the risk and reward of defining too narrow or too broad a scope for their effort and products.

**Chapter 11 Train Risk Technology**

 * Objective:** Preparation is the key to implementing risk management successfully and it is important to start with risk management concepts where understanding the vocabulary of risk management is essential for communication of risk. The focus of this chapter is to define training metrics that can help in providing instruction in risk management and also how to increase organization’s knowledge of risk management technology through the learning process.

**11.1 Prepare for Training**
Preparation includes all the logistics to obtain approval, determine location, and set the date for training. One of the most important steps to get ready for training is to determine the intended audience. The following are three major considerations: 1) __Need__ --- Ask in advance from the audience what their requirements are for example, quality professionals need risk management to prevent problems. 2) __Level__ --- Consider the level of the audience where executives require more of an overview and practitioners need more details. 3) __Size__ --- Consider the size of the audience.

**11.2 Develop Training Material**
Understanding the building blocks of fundamental knowledge in risk management will enable us to adapt to situations as they change and provide the mental flexibility needed to respond to uncertainty. The following training modules provide risk management instruction: 1) Session 1: Risk management concepts 2) Session 2: Risk assessment methods 3) Session 3: Risk management process 4) Session 4: Risk management measures 5) Session 5: Proactive risk management.

A certain amount of general knowledge is necessary to provide a foundation for reasoning about risk. The vocabulary of risk management can be learned in a series of progressive steps. Using the personal software process, individual become competent in four stages of learning where training material should assist in helping individual through the stages of learning. The following are four stages of individual learning: 1) __Unconscious inability__ --- In this stage, individuals are unaware of risk concepts and they may feel frustrated in a situation that appears beyond their control. 2) __Conscious inability__ --- Individual in this stage may become aware of risk concepts but don not have the ability to manage risk. 3) __Conscious ability__ --- Individuals put risk concepts into practice which increases their capability to manage risk. 4) __Unconscious ability__ --- Our brain can make associations based on new information and past experiences; these associations exist in our mind and can be utilized in resolving risks.

**11.3 Apply Training Metrics**
After determining the correct order of the training material, individuals can allocate time to each topic and they need to fit the training material to time constraints which will require a trade-off between breadth and depth. The training metric shown in figure 11-1 below can assist in refining the training plan by applying rules.



**11.4 Deliver Training**
There are many ways to deliver training, such as video, audio, and direct satellite broadcast. The method that is for a project depends on the budget and the available technology. Regardless of the training method, the pace of the material should be relaxed to facilitate learning. The following are some guidelines for effective communication in risk management training: 1) Have something good to say. 2) Say it well. 3) Read your audience. 4) Use words with emotions. 5) Identify with your audience.

**11.5 Obtain Training Feedback**
It’s a good idea to ask the students what they hope to learn and then set appropriate expectations by reviewing the list and explaining what will and will not be covered. Training evaluations help in improving future training sessions. Utilizing the five point scale where 1 = poor, 2 = fair, 3 = average, 4 = good, 5 = excellent) is an effective way to measure progress over time. Training evaluation should ask the students to rate and comment on their assessment of training in the following areas: 1) Value of training content. 2) Speaker presentation skills. 3) Training facilities. 4) The part of training I liked best. 5) The part of training I liked least. 6) Other comments.

**11.6 Summary**
The steps to train risk management technology include preparing for training, developing training material, applying training metrics, delivering training and obtaining training feedback. People learn in stages and at first they are unaware of risk concepts and lack a vocabulary to reason about risk. They progress to the second stage and become aware of risk concepts. In the third state, individuals put risk concepts into practice and in the fourth stage, our brain makes associations based on new information and past experiences.

**Chapter 12 Verify Compliance**

 * Objective:** The goal of managing risk has an intermediate objective of verifying compliance of project practices to the risk management plan, a way to engineer quality results. The intermediate objective is necessary to overcome the obstacle of a faulty plan or deficient practices. The objective of verifying compliance is to determine improvement potential of the plan and of the practice. The focus of this chapter is to discuss improvement of the plan and the work, but not the process. Quality provides customers with products and services that satisfy their requirements. Quality assurance is the practice of ensuring that quality standards are met through quality control which consists of the methods by which quality is measured and improved.

**12.1 Review the Risk Management Plan**
The first step in verifying compliance to risk management practices is to review the risk management plan in order to understand the activities, agents, and artifacts of the plan to prepare for a compliance audit. Agents are the project roles with responsibility for risk management activities and artifacts are the expected outputs produced by performing risk management. The following steps assist in the planning and satisfying the elements established with the help of QA personnel: 1) __Completeness__ --- Do the contents consider all aspects of risk management? Utilizing an outline of a risk management plan as a checklist should be helpful. 2) __Understandability__ --- Is the plan easy to read and comprehend? Perhaps a glossary is necessary so that new employees or subcontractors can interpret the plan. 3) __Level of detail__ --- Is the level of detail sufficient to execute the plan? A detailed plan specifies what will be done, when, by whom and how much it will cost. 4) __Consistency__ --- Is the plan ambiguous? Check for any contradictions that would confuse the implementation of the plan such as, inconsistent terminology in the plan. 5) __Realistic__ --- Is the perspective of the plan practical? Check for statements that lack common sense.

**12.2 Audit Agents and Artifacts**
Quality assurance can be effective when competent professionals report through an independent chain of command and support the development of product quality. On large projects, managers need assistance in performing the task of quality assurance. Auditing agents and artifacts can also assist in uncovering potential problems. Quality assurance is responsible for auditing the quality actions of agents such as, project personnel and alerting management to any deviations. Quality assurance audits the quality of artifacts to ensure management that the work is performed the way it is supposed to be. The following table 12-1 shows some sample questions for agents, artifacts and activities to investigate risk management practices. The SEI CMM describes the auditing process of software quality assurance at level 2. Software quality assurance involves reviewing and auditing the software product and activities. The verifying implementation common feature in each key process area describes the specified auditing practices to ensure compliance for key process area. The following are goals of software quality assurance are: 1) Software quality assurance activities are planned. 2) Adherence of software products and activities to the applicable standards, procedures, and requirements is verified. 3) Affected groups and individuals are informed of quality assurance activities and results. 4) Noncompliance issues that cannot be resolved within the software project are addressed by senior management.

**12.3 Generate an Audit Report**
An audit report provides visibility into project risk management performance and it is generated to document the review and audit findings. The project audit findings summarize implementation performance and detail any discrepancies against the risk management plan. SEI CMM requires SQA to review and audit activities and work products for defect prevention and reports the results. The CMM’s software Quality Assurance key process area notes that compliance issues are first addressed within the software project and resolved there if possible. For issues not resolved, the SQA group escalates the issue to an appropriate level of management for resolution.

**12.4 Track Action Items**
Quality assurance is responsible for tracking audit action items until closure and the quality system should require a timely response to action items. All quality standards require maintenance of record of each activity to verify compliance. These record needs to be managed so that they can be easily retrieved to provide evidence that the quality management system is being used and that all its requirements are being satisfied.

**12.5 Summary**
The steps to verify compliance to a risk management plan consist of reviewing the risk management plan, auditing agents and artifacts, generating an audit report and tracking action items. The three major goals of quality assurance are ensuring compliance, reporting discrepancies, and monitoring quality. SEI capability Maturity model for software provide guidance for software quality assurance.

**Chapter 13 Improve Practice**

 * Objective:** A system for managing risk requires checks and balances and the flexibility to create new laws to take the place of those that are out dated. We must periodically check our risk management practice for potential improvement. To obtain feedback for improvement, one can survey people’s notion of their own practice. This chapter focuses on utilizing a risk practices survey to measure progress in managing risk quantitatively.

**13.1 Develop an Appraisal Method**
There are various reason for assessing risk management practices and a structured method is used for the following reasons: 1) To report risk management capability. 2) To establish a baseline for improvement. 3) To develop a plan for improvement. 4) To measure progress against an improved plan 5) To select a contractor or subcontractor. 6) To monitor practice performance.

The risk practices survey is an appraisal method to obtain perceptions of risk management practice and quantitative process improvement (QPI) provide objective measures through statistical analysis of subjective perception. The risk practice survey requires survey participants to identify their perceptions of the performance and importance of risk management practices. The time to complete the survey is normally fifteen minutes and the survey itself is inexpensive.
 * 13.1.1 Design a Risk Practices Survey**

The survey is given to all those with responsibilities for risk management. Anyone who has used a project charge number can be considered a candidate survey participant. The survey includes categories for all possible roles, to ensure adequate project representation. Survey results are the collective knowledge and experience of the project team, organization management, and the customer. Responses are categorized by project and organization role, so that they may be compared.
 * 13.1.2 Categorize the Survey Participants**

**13.2 Assess Risk Practices**
There are three steps to assess risk management practices using the risk practices survey and these are covered below.

This is the first step where obtaining permission from the responsible manager is mandatory and the manager can assist in obtaining timely data and can characterize the project in terms of size, structure and application domain.
 * 13.2.1 Administer the Risk Practices Survey**

This is the second step to assess risk management practices which enter responses into a spreadsheet that can be used to graph the survey data. Survey responses from 0 to 4 provide a ranking but this does not measure the distance between the points. To normalize the data, scale the data to fit a normal distribution. Find the scaled value corresponding to each score from 0 to 4 by determining the mean for each slice. By normalizing the data, metric and statistical comparisons are enabled. The table 13-1 shows the scaled values after the data was transformed: This is the third step in which plotting relative importance versus performance and the mean importance and performance provides four quadrants that categorize risk management practices. The quadrant shows relative strengths and weaknesses and may be used to identify areas for improvement. Importance is the key to performance because we prioritize activity based on importance.
 * 13.2.2 Analyze the Risk Practices Survey Results**
 * 13.2.3 Establish a Baseline for Improvement**

**13.3 Develop an improvement Plan**
An improvement plan should define specific areas to be improved and to develop a realistic improvement plan, one must understand the difference between “as is” and “should be”. Improvement plan can be developed based on the difference between “as is” and “should be”.

**13.4 Implement the Improvement Plan**
Management should assign responsibility to implement the improvement plan and to execute the plan, it involves people on projects as required to promote buy-in from the organization. Improvement plans should focus on the evolution of risk management technology that will be utilized to satisfy the project’s risk management needs.

**13.5 Summary**
The three steps to improve risk management practice include developing an appraisal method, assessing risk practices, developing and implementing an improvement plan. Importance is the key to performance because we prioritize activity based on importance and to improve the performance of risk management practices, one must first the value of the practice.