Risk Management in a Dynamic Society: A Modelling Problem
Introduction
The discussion focuses on modeling risk management in a dynamic society, drawing from decades of multi-disciplinary research on industrial risk management at Risø National Laboratory and international collaborations since the Bad Homburg workshop series “New Technology and Work” (Wilpert, 1987).
Initial modeling concepts have evolved through various research disciplines and paradigms.
Research began in systems and control engineering, designing control and safety systems for hazardous industrial process plants.
Evolution of Risk Management Modeling
Attention shifted to human-machine interface problems and human error analysis, incorporating psychological competence.
The focus expanded to the performance of individuals who prepare work conditions for operators, integrating research from management and organizational science to address decision errors at the management level.
Expertise in law and legislation was required to address management's commitment to safety and societal efforts to control management incentives through safety regulation.
Current Challenges in Risk Management
Models created by integrating results from various disciplines are useful for designing work support systems for individual actors and decision-makers.
These models are not effective for analyzing the performance of the total risk management system.
A system model cannot be built by a bottom-up aggregation of models derived from individual disciplines.
A top-down, system-oriented approach based on control theoretic concepts is required.
Attempts to improve system safety using models of local features have been offset by unpredictable human adaptation.
The Problem Space: Risk Management in a Dynamic Society
Loss of control over physical processes can lead to injuries, environmental contamination, and investment losses.
Accidents are influenced by human activities that can trigger or divert event flows.
Safety depends on controlling work processes to prevent accidental side effects that harm people, the environment, or investments.
Safety control involves multiple levels, including politicians, managers, safety officers, and work planners, who use laws, rules, and instructions to control hazardous physical processes.
These levels seek to motivate, educate, guide, or constrain worker and operator behavior to increase safety.
Socio-Technical System in Safety Control
Figure 1 illustrates the socio-technical system involved in safety control, researched bottom-up across several academic disciplines.
Society seeks to control safety through the legal system, balancing safety priorities with employment and trade balance.
Legislation sets boundaries for acceptable human conditions. Political and legal sciences research this level.
Authorities, industrial associations, and workers’ unions interpret and implement legislation into rules to control activities in specific workplaces.
Management scientists and work sociologists operate at this level.
Rules are interpreted and implemented within companies, considering work processes and equipment, involving work psychologists and human-machine interaction researchers.
Engineering disciplines design productive and potentially hazardous processes and equipment, developing standard operating procedures.
Limitations of the Command-and-Control Approach
The classic prescriptive command-and-control approach, deriving rules of conduct top-down, is effective in stable societies where instructions and work tools are based on task analysis.
In the present dynamic situation, this approach is inadequate and requires a different view on system modeling.
The Present Dynamic Society
Compared to the past, the present dynamic society introduces dramatic changes in industrial risk management:
Rapid technological change at the operative level in transport, shipping, manufacturing, and process industry.
Management structures lag behind technological advancements (Savage and Appleton, 1988).
Legislation and regulation lag even further behind.
Different time lags at various levels create significant problems.
Scale of industrial installations is increasing, elevating potential for large-scale accidents. Very low accident probabilities must be demonstrated for societal acceptance.
Models must include rare conditions.
Advances in information and communication technology lead to high system integration and coupling.
Decisions can have dramatic effects propagating rapidly through global society.
It is difficult to model systems in isolation and conduct small-scale experiments.
Companies operate in aggressive, competitive environments, incentivizing short-term financial and survival criteria over long-term welfare, safety, and environmental impact.
Modeling Approaches: Structural Decomposition vs. Functional Abstraction
These trends impact modeling approaches, raising questions about structural decomposition, functional abstraction, cross-disciplinary research, and multi-disciplinary cooperation (Hale et al., 1996; Rasmussen et al., 1994).
Modeling by Structural Decomposition: Tasks, Acts, and Errors
Usual approach decomposes socio-technical systems into elements modeled separately.
Risk management is decomposed according to organizational levels, studied within different disciplines.
Risk management at upper levels is studied with a ‘horizontal’ orientation, across technological hazard sources.
Sociological studies at upper levels rely on industry-wide questionnaires, without detailed consideration of processes at the productive bottom level (Barley, 1988).
Management theories are often independent of the substance matter context of organizations.
Being a manager is viewed as a profession, irrespective of managing a hospital, manufacturing company, or bank.
Commercial companies now narrowly focus on financial operations (Engwall, 1986).
Implications for Societal Control
A marine safety official noted potential naval safety decreases due to ships being operated by banks and investors rather than shipping professionals.
Rees and Rodley (1995) critically reviewed this trend's effects on management behavior in public health care.
Need for Vertical Interaction Studies
More studies are needed on vertical interaction among socio-technical system levels, concerning the nature of technological hazards they control.
Systems are traditionally modeled by decomposing into structural elements, while dynamic behavior is modeled by breaking down the behavioral flow into events like tasks, decisions, acts, and errors.
Work situations provide actors with freedom in choosing means and timing, making task instructions unreliable standards for judging behavior.
Limitations of Task Analysis
Degrees of freedom in task completion require additional performance criteria from instructors, who cannot foresee all local contingencies.
Rules and instructions are often designed separately, but several tasks may be active simultaneously, posing constraints unknown to the instructor.
Rules, laws, and instructions are rarely followed precisely.
Even in highly constrained situations, modifications of instructions are common (Fujita, 1991; Vicente et al., 1995).
Strikes by civil servants often involve “working-according-to-rules”.
Operators' violations of formal rules appear rational given actual work load and timing constraints.
After accidents, individuals who violated a formal rule are often punished, leading to judgments attributing accidents to ‘human error’ (Rasmussen, 1990a,b, 1993a).
Task instructions are unreliable standards for judging behavior in actual work.
Modeling human behavior as a stream of acts is unreliable in dynamic environments.
Task analysis is only useful when behavior is tightly controlled by the control requirements of a technical system.
Decision-Making in Familiar Environments
Traditional decision research views decisions as discrete processes separate from context.
In familiar work environments, actors are immersed in the context and know the normal flow of activities.
Analytical reasoning is replaced by skill- and rule-based choices among familiar action alternatives.
Operational decisions are based on minimal information necessary to distinguish among perceived action altematives.
Separate ‘decisions’ are difficult to identify, requiring simultaneous study of the social context, value system, and dynamic work process.
The skill-, rule-, knowledge-based behavior model of cognitive control was developed (Rasmussen, 1983), leading to paradigms of ‘naturalistic’ decision making (Klein et al., 1994).
Cognitive science has converged the economist’s concept of ‘decision making’, the social concept of ‘management’, and the psychological concept of ‘cognitive control’.
Accident Causation
Deviation from normative work instructions often leads to the conclusion that ‘human error’ is a determining factor in 70-80% of accidents.
Multiple contributing errors and faults are normally found due to planned defenses against accidents.
Commercial success involves operating at the fringes of accepted practice, implying risk of crossing safety limits.
Court reports from accidents like Bhopal, Flixborough, Zeebrugge, and Chernobyl show systematic migration of organizational behavior toward accidents due to cost-effectiveness pressures (Rasmussen, 1993b, 1994b).
Consider the interaction of decisions made by several actors under competitive stress.
Figure 2 shows the Zeebrugge accident causal tree, where decision makers optimized cost-effectiveness, preparing the stage for an accident triggered by a single human act.
Individual decision makers cannot see the complete picture or judge the state of multiple defenses.
Modeling activity in terms of task sequences and errors is ineffective; understanding behavior requires deeper analysis of behavior-shaping mechanisms.
Modeling by Functional Abstraction: Migration Toward the Boundary
Activities naturally migrate toward the boundary of acceptable performance.
Human behavior in any work system is shaped by objectives and constraints.
Actors adaptively search within administrative, functional, and safety-related constraints.
Local work condition changes cause variability in strategies and activities.
Actors identify an ‘effort gradient’ and management supplies a ‘cost gradient’.
This results in systematic migration towards the boundary of functionally acceptable performance, where crossing the boundary may cause errors or accidents.
Well-designed work systems have numerous precautions against occupational risk, using a ‘defence-in-depth’ strategy.
Defense-in-Depth Strategy
Local violation of one defense may have no immediate, visible effect.
The boundary of safe behavior for one actor depends on defense violations by others.
Defenses may degenerate systematically through time due to cost-effectiveness pressures.
Accident investigations often conclude that a particular accident was waiting for its release (Rasmussen, 1993b).
Accidental courses of events are prepared through time by the normal efforts of actors responding to cost-effectiveness requests.
Normal behavior variations can release an accident.
Explaining accidents in terms of events, acts, and errors is not useful for improving systems.
When decision makers managing institutions and companies adapt individually to commercial stresses, resulting interactions may not match overall safety control requirements.
Figure 4 illustrates interaction conflicts between institutions and companies at various levels, identified from super tanker and ro-ro ferry accidents (Shell, 1992; Estonia, 1995; Stenstrom, 1995).
Need for New Representation Approaches
A new approach to representing system behavior is needed, focused on mechanisms generating behavior in the dynamic work context.
Analogy with thermo-dynamic models is useful, considering boundary conditions and gradients.
A higher level of functional abstraction than task analysis is needed.
Representation involves identifying the boundary conditions of the work space and gradients guiding the drift across this space.
This approach requires a detailed study of the means-ends relations of the work system and studies focused on particular system types characterized by their work processes and hazard sources.
Taxonomy of Hazard Sources
A taxonomy of hazard sources and their control characteristics is necessary. A framework for identifying objectives, value structures, and subjective preferences governing behavior within the degrees of freedom faced by decision makers and actors is required (Rasmussen, 1994a; Rasmussen et al., 1994).
This approach relates to Gibsonian concepts of invariants and affordances (Gibson, 1966, 1979), and the ‘space of safe driving’ (Gibson and Crooks, 1938).
For a review of this ecological approach, see Flach et al. (1994).
Control of System Performance
The new approach to modeling accident causation invites a new approach to controlling system performance.
Focus should be on controlling behavior by making boundaries explicit and providing opportunities to develop coping skills at these boundaries, rather than controlling behavior by fighting deviations from a pre-planned path.
Increasing the margin from normal operation to the loss-of-control boundary.
Increasing awareness of the boundary through instruction and motivation campaigns.
Explicit identification of safe operation boundaries, making them visible to actors and providing opportunities to learn to cope with boundaries.
Risk Management as a Control Task
Risk management should be considered a control function focused on maintaining a hazardous, productive process within safe operating boundaries.
A systems approach based on control theoretic concepts should be applied to describe overall system functions.
Management and work planning apply different control strategies based on time horizon, system stability, and disturbance predictability.
Centralized planning based on prognosis from past results is being replaced by customer-controlled, just-in-time production strategies.
These strategies exemplify open-loop and closed-loop approaches requiring different design approaches (Rasmussen, 1994b).
Modeling in Dynamic Markets
Modeling risk management in a dynamic society requires an active, closed-loop feedback perspective.
Use an abstract representation of the entire information network involved in controlling the technical core's hazard.
This control function's stability cannot be studied across systems; analysis is needed for particular system types.
Categorization
Studies of risk management must categorize hazard sources according to their control requirements.
For a particular hazard source:
Identify the control structure.
Identify relevant controllers (actors).
Determine objectives and performance criteria.
Evaluate control capability.
Analyze available information about the system's state concerning production objectives and safety boundaries from a feed-back control perspective.
Identification of Controllers
First step: Identify decision makers (controllers) who may contribute to accident propagation.
One approach maps relationships among decision makers who contributed to accident causation, as shown in the 'AcciMap' (Figs. 5 and 6).
The interaction among decision makers has special features such as:
Decision makers manage their work domains focusing on controlling means and ends of normal productive tasks.
Work Objectives
Proper action targets (productive and safety-related objectives) are critical for controllers, corresponding to their action opportunities.
For normal work activities, business objectives propagate downward through an organization, formulated differently at various levels.
Objectives have many shades, expressed in terms of product specifications, production volume, process optimization criteria, and process constraints (e.g., safety).
Performance criteria are often implicit in company or local work practice.
Objectives and values are generally formulated at higher levels, where degrees of freedom for action multiply as objectives are implemented locally.
Special care should be taken analyzing the influence of different time-lags in responding to change at various levels.
Consider the present trend in legislation and regulation away from prescriptive rules towards performance-centered objectives.
Information on Actual State of Affairs
In a closed-loop, feedback function, observation and measurement of the actual state of affairs and the response to control actions are important.
Control systems perform no better than their measuring channel.
Capability and Competence
The content and form of competence for controllers are critical.
Questions arise when interpreting generic regulation:
Are local decision-makers familiar with hazard control requirements?
Commitment
These aspects determine whether a decision maker can adequately control safety.
Additional questions include:
Are priorities right?
Will decision makers commit to safety?
Are regulatory efforts controlling management priorities?
Are decision makers aware of safety constraints?
Conclusion
Traditional task analysis is inadequate for dynamic workplaces.
Use problem space analysis, formulating constraints and choice options at several levels of a means-ends hierarchy (Rasmussen et al., 1994).
Represent problem space in a means-ends hierarchy (Fig. 6) to identify safe choice boundaries.
Analyze the problem formulation and performance criteria of individual decision makers.
Tight coordination is necessary across all levels (Fig. 1), with deep understanding of the subject matter and competence at each level.
Identification of Constraints and Safe Boundaries
Success depends on explicitly identifying work system constraints and acceptable operation boundaries in a dynamic society.
Predictive risk analysis can identify safe operation preconditions for well-structured and tightly coupled systems.
Risk Management Strategies
Strategies have evolved differently across hazard domains based on the nature of the hazard source (Rasmussen, 1993c, 1994b) (Fig. 7).
A detailed study of control requirements for the dominant hazard sources of a work system is mandatory for risk management.
Typical categories include:
Occupational safety focused on frequent, small-scale accidents.
Protection against medium-size, infrequent accidents.
Protection against very rare and unacceptable accidents.
Classification System for Hazard Sources
Develop a classification system for hazard sources and their different control requirements.
Proactive, ‘no-accident-is-tolerable’ strategy requires analytical risk management strategies.
Classification of hazard sources, control requirements, and effective risk management strategies is needed to select proper risk management policies and information systems.
Each workplace has multiple, potentially hazardous activities requiring a set of management strategies.
Consensus is necessary among decision makers on hazard source characteristics within a company to classify activities and communicate effectively.
Taxonomy for Classification
Preliminary dimensions of a taxonomy for classification include:
Nature of hazard source.
Accident anatomy.
Degree to which defenses can be based on predictive analysis.
Importance of Human Factors
The different features of hazard sources, system configurations, and risk management strategies require coordinated studies involving technical and human sciences.
Models required to plan effective risk management strategies cannot be developed by horizontally oriented research within academic disciplines across different hazard domains.
Vertical studies of the control structure are required for well-bounded categories of hazard sources, characterized by uniform control requirements.
Present Trends in the Paradigms of Human Sciences
The approach is based on the assumption that a dynamic socio-technical system cannot be represented in terms of task sequences and errors referring to ‘correct’ or rational performance.
Developments within academic disciplines of relevance for risk research facilitate a cross-disciplinary approach.
The concept of ‘human error’ and ‘decision bias’ is typically found in a certain phase of evolution, where rational behavior is identified by normative models and actual behavior is described as a deviation.
Acceptance of cognitive models supports modeling actual behavior in terms of behavior-shaping constraints and adaptive mechanisms.
Figure 8 illustrates the parallel evolution of paradigms within decision research and management research, and the concurrent change of paradigms within branches of safety research.
Decision Research
In decision-making research, the shift from normative, prescriptive models, over descriptive models in terms of deviation from rational performance, towards modeling actual behavior is very visible.
Organizational Theory
Normative, rational models take different shapes like Scientific Management focused on economic efficiency, Administrative Management assuming a known master plan, and Bureaucratic Models following similar patterns.
Occupational Safety Research
Efforts to improve safety by counteracting human error sources identified by causal analysis tend to be ineffective.
There tends to be a need for such a research direction is clearly demonstrated by the observation that human adaptation frequently compensates for attempts to improve system safety.
Major Accident Research
The defense-in-depth protection based on multiple barriers was developed systematically for nuclear systems (the minimum critical mass problem).
Perfor-mance was controlled by formal standard operating procedures and effective training (use of simulators).
Conclusion
Industrial risk management, environmental protection and life-cycle engineering for a modern, dynamic, society raise some basic problems
Specific researches include: development of hazard sources and their control and the vertical interaction among decision makers.