Guidelines for Environmental Risk Assessment and Management
[This document refers, in a number of instances, to the then Department of the Environment, Transport and the Regions (DETR). The text of this document has not been updated since the transfer of environmental protection functions to Defra.]
Chapter 6
Quantification and dealing with uncertainty
6.1 Introduction
Risk assessments for complex, high priority risks can be time-consuming and expensive. In Chapter 2, the principle was introduced that the amount of effort put into the risk assessment should be proportional to the severity of the problem. The tiered approach shown in Figure 2.1 is intended to help match effort to severity by providing a series of clear stages, after each of which decisions are taken about whether or not further effort would be justified. If an initial assessment of risk based on a reasonable 'worst-case' scenario indicates little cause for concern then there is little point in moving on to more sophisticated analyses. Alternatively, cause for concern may become apparent at an early stage and there would then be little point delaying the identification of risk management options in order to complete the risk assessment. More detailed data and sophisticated analysis may be required where initial estimates indicate the need for further refinement of the estimation.
Previous and ongoing monitoring programmes are important information sources and modelling and simulation are useful techniques for analysing information. Tools and techniques for risk assessment are being developed all the time. The RiskWorld internet site provides some useful pointers to models for quantifying the probability of release, estimating the consequences and dealing with uncertainty (Section 6.4).
Where information is limited, informed decisions can be based on assumptions or extrapolations. It is important, though, that data gaps or assumptions are acknowledged. Sensitivity analysis offers a useful approach to dealing with such uncertainties. It provides a means to examine the behaviour of a model by measuring the variation in outputs resulting from changes to its inputs.
Estimating the probability of events
In environmental risk assessment, there can be situations in which the probability of an event is 1 (ie it will happen). For example, once the decision to build a dam has been taken, its construction will certainly lead to the loss of habitats, landscape features and structures in the flooded area. In this case, the important parameters to consider are the probability and magnitude of consequences arising from the construction rather than the probability of the event (construction) itself. Another example of such a situation would be the release of planned, routine emissions. In situations outside the system design (ie accidents or malicious releases) the initiating event probability becomes more important. More usually, the event has a probability less than 1, and an estimate of its probability will be required. There are various techniques available to do this, some of which are briefly outlined below.
Actuarial or historical information
This involves looking at the reliability of components or other factors within a system based on past experience or data. To be useful there has to be a statistically significant number of data points. If the event relates to a novel process or is very rare (such as a major industrial accident), then it will not be possible to gather sufficient data for a probability estimate. Other circumstances lend themselves more easily to the use of historical data. For example, the frequency of collisions involving road tankers that can then lead to environmental pollution might be estimated from direct data on past road tanker accidents.
Synthesised analysis
Many processes, industries or sectors do not have sufficient data on which to base such estimates and other techniques involving synthesised analysis are needed. Two of the most widely used and well-known techniques to deal with operation or process failures are fault tree analysis and event tree analysis. These are similar in that logic diagrams are employed to represent the propagation of events or faults through a system.
Fault tree analysis
Fault tree analysis can be used to assess the probability of a system failure in the absence of actual data. The technique requires information on the failure rates of components within a system. Combining such data can provide an estimate of the probability of system failure over time or of failure on demand (eg failure of a safety system to operate). The aim is to take an undesired event (system failure) and describe how it might occur.
Event tree analysis
Event tree analysis operates in the opposite way to fault tree analysis by taking a situation and asking to what system states it might lead. A simple example would be considering how a release of chlorine could affect the local environment and population around a plant. The probabilities would depend on the operation of safety systems, size of release, wind direction, distance from source to receptor, and so on.
Estimating the magnitude of consequences
In some cases there will be a high level of uncertainty in the estimation of the magnitude of consequences, and making some judgement on the possible consequences may be the best option. For example, there is often great uncertainty in ecological risk assessment, and it becomes very difficult to predict the extent to which a target population may decline and the degree of seriousness of the subsequent effects on community and ecosystem function that may result. In such cases cost-effective measures to avoid serious or irreversible harm must be adopted, even in the face of uncertainty.
In most cases, however, it will be possible to quantify the magnitude of the consequences, and possibly even to place a monetary value on them (which will facilitate socio-economic analysis). The significance of the magnitude of a consequence, at least to a certain extent, is a matter of judgement. Where no guidance exists regarding the significance, a rough, ad hoc scale can be developed. An example is presented below ranging from negligible to extremely severe effects. Approaches using coarse scales of this sort have proved useful in risk assessment related to a range of environmental problems, for example assessing suitable clean-up standards for contaminated land.
- Negligible - Sub-lethal effects in individuals that do not cause a change in population structure or size.
- Mild-Moderate - Effects occurring at the population level. Effects on ecosystems that are not regarded as being of high value for whatever reason.
- Severe - Local extinctions (depending on the species) and local dysfunction of communities and ecosystems.
- Very severe - Global extinctions (depending on species) and widespread effects on the functioning of communities and ecosystems.
- Extremely severe - Impacts on the functioning of global ecosystems.
Estimating the probability of consequences is likely to be at best semi-quantitative. There are three primary factors to consider when estimating the probability of consequences (Section 2.2) - whether the event will be initiated; whether exposure to the hazard will occur; and whether harm will result following exposure.
For example, there are well-developed techniques for estimating the probability that a chemical released to the environment will lead to harm to organisms. These are based on comparing a known concentration at which effects occur with a predicted or measured concentration in the environment.
In some cases it might be possible to base exposure predictions on measured levels in environmental compartments. There will be uncertainty in these measurements and, where this uncertainty is unacceptable or data are unavailable, the use of surrogates, models and assumptions will usually be of value. For example, physico-chemical properties of a substance and details of the amounts released into the environment can be used to predict its environmental partitioning and environmental concentrations. Mass-balance models are then used to quantify the amounts of a chemical expected to be present in different compartments within a particular environment.
Where strict quantitative analysis is not possible, expert opinion may be needed. For example, it is often less feasible to carry out a detailed quantification when the risk being considered is from living organisms (genetically modified organisms or alien species, for example). Hence in such cases regulatory decisions are usually based on the opinion of an expert advisory committee.
Sources of uncertainty
Analysing the sources and magnitudes of uncertainty can help determine how much confidence can be placed in the risk assessment as a basis for decision-making. Uncertainties can arise from several sources, including natural or inherent variability over space and time, variability in the accuracy of measurements and data manipulation, and knowledge gaps due to lack of data. They can also arise when models and test systems do not accurately reflect the environment or exposed population of concern.
Analysing uncertainty
Methods for analysing and describing uncertainty may be simple or complex. Where significant knowledge gaps exist a useful approach is to estimate consequences based on alternative scenarios, presented as a series of estimates with different assumptions and descriptions of uncertainty. A common approach to dealing with uncertainty is to adopt a worst-case scenario which assumes that the consequences will definitely occur, or to assign given magnitudes to the consequences. Uncertainty can, in many cases, be reduced by collecting more information (ie increasing the sample size). On the other hand natural variability (eg chemical sensitivities within and between species) cannot usually be reduced by further measurement and must be expressed through the use of statistical descriptions such as probability and frequency distributions. Sensitivity analysis should always be carried out where the degree of uncertainty is critical.
Because many risk estimates will be subject to uncertainty from various sources, 'safety' factors (sometimes called 'protection' or 'uncertainty' factors) are often applied, especially in standard-setting and decision-making. Safety factors are typically applied when extrapolating from animal data to humans, from data derived from a small number of individuals to a population, or from a species to a mixed ecosystem.
The decision process for developing safety factors can involve scientific judgements on a wide range of quantitative and qualitative information to produce a single number expressing those judgements and uncertainties. Safety factors can take account of scientific uncertainties in available data and allow, for example, for the protection of the more susceptible parts of the environment. Determining an appropriate safety factor requires a combination of experience and judgement. Recording the rationale behind such judgements is important.
Key references
Baird DJ, Maltby L, Greig-Smith PW & Douben PET (1996) ECOtoxicology:
Ecological Dimensions, London, UK, Chapman & Hall
An interesting collection of papers addressing the importance of ecological
issues within ecotoxicology, with a particularly relevant contribution
on the evaluation of the importance of indirect effects.
Begon M, Harper JL & Townsend CR (1990) Ecology: Individuals,
Populations and Communities (Second edition), Oxford, UK, Blackwell
Scientific Publications
As above.
Calow P (1997) Controlling Environmental Risks from Chemicals:
Principles and Practice, Chichester, UK, John Wiley & Sons
A concise but informative textbook dealing with the basic principles
of the environmental risk assessment of chemicals, including sections
on European and North American legislation.
Calow P (1998) Handbook of Environmental Risk Assessment and Management,
Oxford, UK, Blackwell Science
A comprehensive treatment of the basic principles of environmental
risk assessment and management.
Department of the Environment/Advisory Committee on Releases to the
Environment (1993) The Regulation and Control of the Deliberate Release
of Genetically Modified Organisms, London, UK, Department of the Environment
Guidance for interpreting the legislation on the release of genetically
modified organisms to the environment.
Department of the Environment/Advisory Committee on Releases to the
Environment (1995) Guidance to the Genetically Modified Organisms (Deliberate
Release) Regulations 1995, London, UK, Department of the Environment
As above.
European and Mediterranean Plant Protection Organisation (1994) Decision-making
scheme for the environmental risk assessment of plant production products;
Terrestrial vertebrates. EPPO Bull, 24, 37-87
An example of the guidelines produced by the European and Mediterranean
Plant Protection Organisation for the ecotoxicological risk assessment
of plant protection products.
Paustenbach DJ (1989) The Risk Assessment of Environmental and
Human Health Hazards: A Textbook of Case Studies, New York, USA, John
Wiley & Sons
A useful collection of case studies concentrating mainly on human health
risk assessment with some ecotoxicological case studies.
Royal Society (1992) Risk: Analysis, Perception and Management
(Second edition), London, UK, The Royal Society
A comprehensive study of risk assessment, management and perception
from a variety of viewpoints.
Schnoor JL (1996) Environmental Modeling: Fate and Transport of
Pollutants in Water, Air and Soil, New York, USA, John Wiley &
Sons
Addresses key questions about fate, transport and long-term effects
of chemical pollutants in the environment.
Walker CH, Hopkin SP, Sibly RM & Peakall DB (1996) Principles
of Ecotoxicology, London, UK, Taylor & Francis
An excellent textbook on the fundamentals of ecotoxicology, including
chemical fate and behaviour, biomarkers, toxicity testing and discussions
on ecotoxicological impacts from the individual through to the ecosystem,
including case studies.
Electronic information sources
RiskWorld internet site - www.riskworld.com/
Key periodicals
Archives of Environmental Contamination and Toxicology
Atmospheric Environment
Chemosphere
Conservation Biology
Environmental Pollution
Environmental Science and Technology
Environmental Toxicology and Chemistry
Ground Water
Journal of Environmental Quality
Journal of Toxicology and Environmental Health
Nature
Toxicology
Defra is not responsible for the contents or reliability of the linked web sites and does not necessarily endorse the views expressed within them. Listing should not be taken as endorsement of any kind. We cannot guarantee that these links will work all of the time and we have no control over the availability of the linked pages.
Page published 2 August
2000;
Page last modified
19 September, 2002