ISSN 1726-5274 MAicrtotrbibioulotginicgalilRlnisekss Ascsaeusssm edenbt y Shiga Gutoidxainnc-pe rfoordFuocoindg Escherichia coli (SGUTIDAENCCE ) to specific foods REPORT 36 MICROBIOLOGICAL RISK ASSESSMENT SERIES 32 MICROBIOLOGICAL RISK ASSESSMENT SERIES 36 MICROBIOLOGICAL RISK ASSESSMENT SERIES Microbiological Risk Assessment Guidance for Food GUIDANCE Food and Agriculture Organization of the United Nations World Health Organization Rome, 2021 i Required citation: FAO and WHO. 2021. Microbiological risk assessment - Guidance for food. Microbiological Risk Assessment Series No. 36. Rome. https://doi.org/10.4060/cb5006en The designations employed and the presentation of material in this information product do not imply the expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United Nations (FAO) or the World Health Organization (WHO) concerning the legal or development status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific companies or products of manufacturers, whether or not these have been patented, does not imply that these have been endorsed or recommended by FAO or WHO in preference to others of a similar nature that are not mentioned. The views expressed in this information product are those of the author(s) and do not necessarily reflect the views or policies of FAO or WHO. ISSN 1726-5274 [Print] ISSN 1728-0605 [Online] FAO ISBN 978-92-5-134518-4 [PDF and print version] WHO ISBN 978-92-4-002489-2 [electronic version] WHO ISBN 978-92-4-002490-8 [print version] © FAO and WHO, 2021 Some rights reserved. This work is available under a CC BY-NC-SA 3.0 IGO licence Some rights reserved. This work is made available under the Creative Commons AttributionNonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/ licenses/by-nc-sa/3.0/igo/legalcode). Under the terms of this licence, this work may be copied, redistributed and adapted for non-commercial purposes, provided that the work is appropriately cited. In any use of this work, there should be no suggestion that FAO or WHO endorses any specific organization, products or services. The use of the FAO or WHO logo is not permitted. If the work is adapted, then it must be licensed under the same or equivalent Creative Commons licence. If a translation of this work is created, it must include the following disclaimer along with the required citation: “This translation was not created by the Food and Agriculture Organization of the United Nations (FAO) or the World Health Organization (WHO). Neither FAO nor WHO is responsible for the content or accuracy of this translation. The original English edition shall be the authoritative edition.” Disputes arising under the licence that cannot be settled amicably will be resolved by mediation and arbitration as described in Article 8 of the licence except as otherwise provided herein. The applicable mediation rules will be the mediation rules of the World Intellectual Property Organization http:// www.wipo.int/amc/en/mediation/rules and any arbitration will be conducted in accordance with the Arbitration Rules of the United Nations Commission on International Trade Law (UNCITRAL). Third-party materials. Users wishing to reuse material from this work that is attributed to a third party, such as tables, figures or images, are responsible for determining whether permission is needed for that reuse and for obtaining permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. Sales, rights and licensing. FAO information products are available on the FAO website (www.fao. org/publications) and can be purchased through publications-sales@fao.org. Requests for commercial use should be submitted via: www.fao.org/contact-us/licence-request. Queries regarding rights and licensing should be submitted to: copyright@fao.org. Cover picture © Dennis Kunkel Microscopy, Inc Contents Preface xii Acknowledgements xiii Contributors xiv Declaration of interests xvi Abbreviations and acronyms xvii 1 Introduction 1 1.1 FAO/WHO series of guidelines on microbiological risk assessment 1 1.2 Scope and purpose of these guidelines 2 1.3 Guiding the reader through this document 3 1.4 How to begin with risk assessment 4 Part 1 General considerations 5 2 Risk assessment in context 7 2.1 Risk analysis framework 7 2.2 Risk management 8 2.3 Risk assessment 10 2.4 Risk communication 12 3 Food microbiological risk assessment (MRA) 14 3.1 Properties and principles of best-practice risk assessments 14 3.2 Purpose and scope of MRA 16 3.2.1 Estimating baseline risk 17 3.2.2 Comparing risk management strategies 18 3.2.3 Research-related study or model 20 3.3 The role of best- and worst-case scenarios 20 3.4 Assessing the results of a risk assessment 21 3.5 Choosing the type of risk assessment to perform 23 3.5.1 Consistency 25 3.5.2 Resources 25 3.5.3 Theory or data limitations 27 3.5.4 Breadth of application 27 3.5.5 Speed 27 3.5.6 Transparency 28 3.5.7 Stage of analysis 28 3.5.8 Responsiveness 28 iii 4 Hazard identification 30 4.1 Objectives of hazard identification 30 4.2 The process of hazard identification 31 4.3 Data sources for hazard identification 33 5 Exposure assessment 34 5.1 The process of exposure assessment 34 5.2 Modelling approaches 36 5.2.1 Introduction 36 5.2.2 Qualitative and semi-quantitative exposure assessment 38 5.2.3 Quantitative exposure assessment 38 5.2.4 Modelling the production-to-consumption pathway 39 5.2.5 Consumption 51 6 Hazard characterization 58 6.1 The process of hazard characterization 58 6.2 Descriptive characterization 59 6.2.1 Information related to the disease process 60 6.2.2 Information related to the hazard 63 6.2.3 Information related to the host 64 6.2.4 Information related to the food matrix 65 6.2.5 Relationship between the dose and the response 65 6.3 Quantifying the dose–response relationship 67 7 Risk characterization 70 7.1 The process of risk characterization 70 7.2 Qualitative risk characterization in risk assessment 71 7.2.1 Introduction 71 7.2.2 Performing a qualitative risk characterization 72 7.3 Semi-quantitative risk characterization 75 7.3.1 Introduction 75 7.3.2 Performing a semi-quantitative risk characterization 76 7.4 Quantitative risk characterization 81 7.4.1 Introduction 81 7.4.2 Quantitative risk measures 81 7.4.3 Integration of hazard characterization and exposure assessment 89 8 Examples 92 8.1 Examples of qualitative and semi-quantitative risk assessments 92 8.1.1 Risk assessment for main determinants of antibiotic resistance in South East Asia 92 iv 8.1.2 Faecal pollution and water quality, WHO 94 8.1.3 Drinking water guidelines, Australian National Health and Medical Research Council 95 8.1.4 BSE/TSE risk assessment of goat milk and milk-derived products, EFSA 96 8.1.5 Geographical BSE cattle risk assessment, EFSA 97 8.1.6 Risk profile of Mycobacterium bovis in milk, New Zealand Food Safety Authority 98 8.1.7 Seafood safety using RiskRanger, Australia 100 8.1.8 Animal and animal product import risk assessment methodology, Biosecurity Australia 102 8.1.9 Multicriteria-based ranking for risk management of foodborne parasites, FAO/WHO 104 8.2 Examples of quantitative risk assessments 105 8.2.1 E. coli O157:H7 in tenderized vs non-tenderized beef, FSIS 105 8.2.2 Listeria monocytogenes in ready-to-eat foods, FAO/WHO 106 8.2.3 Shiga toxin-producing E. coli O157 in steak tartare patties, Netherlands 107 8.2.4 Vibrio vulnificus in raw oysters, FAO/WHO 108 8.2.5 Histamine in fish sauce, Thailand 109 8.2.6 Pathogens in fresh vegetables, Rwanda 110 8.2.7 Campylobacter and Salmonella in chicken meals, Senegal 112 8.2.8 Vibrio parahaemolyticus in bloody clams, Thailand 112 8.2.9 Salmonella in table eggs, EFSA 114 8.2.10 Cryptosporidium in water – a cost–benefit analysis, United States of America 116 Part 2 Detailed considerations 119 9 Qualitative and semi-quantitative risk assessment: further considerations 121 9.1 Qualitative risk assessment 121 9.1.1 The value and uses of qualitative risk assessment 123 9.1.2 Qualitative risk assessment in food safety 124 9.1.3 Characteristics of a qualitative risk assessment 125 9.2 Semi-quantitative risk assessment 127 9.2.1 Uses of semi-quantitative risk assessment 127 9.2.2 Characteristics of a semi-quantitative risk assessment 129 9.2.3 Limitations of semi-quantitative risk assessment 130 v 10 Data 133 10.1 Literature (primary and/or meta-analysis) 137 10.1.1 Analytical epidemiological studies 138 10.1.2 Microbiological studies of prevalence and counts/concentrations 138 10.1.3 Cross-contamination data during food processing 140 10.1.4 Food handling and preparation 140 10.1.5 Human volunteer feeding studies 142 10.1.6 Animal studies 144 10.1.7 In vitro studies 145 10.1.8 Biomarkers 146 10.2 National and international surveillance data 148 10.2.1 Food safety rapid alert systems 148 10.2.2 Outbreak data 150 10.2.3 Foodborne disease surveillance and annual health statistics 153 10.2.4 Systematic food contamination monitoring surveys 155 10.2.5 National food production statistics 157 10.2.6 National consumption databases 157 10.2.7 National population census 160 10.3 Industry data 160 10.3.1 Description of product and supply chain 163 10.4 Unpublished data 164 10.5 Data gaps 164 10.5.1 Model restructuring 166 10.5.2 Surrogate data 166 10.5.3 Expert knowledge elicitation (EKE) 167 10.5.4 Collection of new data 169 10.6 Recommendations on data collection and organization 170 10.6.1 Searching for data 170 10.6.2 Selection of data 171 10.6.3 Formatting of data 171 10.6.4 Level of detail recorded 172 10.6.5 Combining data from different sources 173 10.6.6 Presentation of data 173 11 Quantitative modelling approaches 175 11.1 Deterministic 175 11.2 Stochastic 176 11.3 Monte Carlo simulation 177 11.4 Other model classification schemes 178 vi 12 Predictive microbiology 180 12.1 Modelling microbial growth and inactivation 181 12.1.1 Microbial ecology of foods 181 12.1.2 Predictive microbiology 184 12.1.3 Model types and modelling tools 185 12.2 Application of predictive microbiology within exposure assessment 187 12.2.1 Range of model applicability 187 12.2.2 Spoilage microbiota 188 12.2.3 Sources of variability and uncertainty 189 13 Dose–response 13.1 The infectious disease processes 13.1.1 Infection and illness 13.1.2 Sequelae and mortality 13.2 Modelling concepts 13.2.1 The particulate nature of the inoculum 13.3 Selection of models 13.3.1 Dose-infection models 13.3.2 Dose-illness models 13.3.3 Sequelae and mortality 13.4 Extrapolation 13.4.1 Low dose extrapolation 13.4.2 Extrapolation in the pathogen–host–matrix triangle 13.5 Dose–response model fitting approaches 14 Variability and uncertainty 14.1 Variability 14.2 Uncertainty 14.3 Uncertainty analysis 14.4 Uncertainty and variability together 192 192 194 194 195 195 195 197 197 197 198 198 198 201 203 203 206 208 209 15 Sensitivity analysis 213 15.1 Sensitivity analysis in qualitative risk assessment 214 15.2 Sensitivity analysis in quantitative risk assessment 215 15.2.1 Statistical methods 215 15.2.2 Graphical methods 216 15.2.3 Evaluation of sensitivity analysis methods 216 vii 16 Quality assurance 16.1 Data evaluation 16.1.1 Data collection 16.1.2 Sorting and selecting data sources 16.2 Model quality assurance 16.2.1 Model verification 16.2.2 Model anchoring or calibration 16.2.3 Model validation 16.3 Comparison with epidemiological data 16.4 Extrapolation and robustness 16.5 Credibility of the risk assessment 16.5.1 Risk assessment documentation 16.5.2 Scientific peer review Bibliography Glossary 217 217 219 221 222 222 223 224 227 229 230 230 231 232 264 viii TABLES Table 1. Examples of factors of importance when determining the impact of the postprocessing environment on the level of exposure 46 Table 2. Elements that might be included in characterization of adverse human health effects (Adapted from ILSI, 2000) 62 Table 3. Elements to consider in characterization of the hazard (Adapted from ILSI, 2000) 64 Table 4. Factors related to the host that may affect susceptibility and severity (Adapted from ILSI, 2000) 64 Table 5. Elements to consider in characterization of the effect of the food matrix on the hazard-host relationship 65 Table 6. Elements to consider in describing the dose–response relationship (Adapted from ILSI, 2000) 66 Table 7. Dose–response models and parameter estimates commonly used in QMRA 69 Table 8. A comparison of the process for computing the final risk estimate in risk characterization in quantitative and qualitative risk assessments. (Table adapted from Table 4 in Wooldridge (2008)) 72 Table 9. Example of a possible tabular format for presenting data linked to risk estimates and conclusions 73 Table 10. Example of a possible sectional format for presenting data linked to risk estimates and conclusions 74 Table 11. Example category definitions for the probability of an event occurring and for the frequency of exposure per year 76 Table 12. Example definitions of health effect / severity category labels 76 Table 13. Example of combining category labels 78 Table 14. A hypothetical example of a risk matrix to combine likelihood and severity as could be applicable to risk characterization using probability ratings as presented in Table 15 79 Table 15. Semi-quantitative allocation of categorical labels to probability ranges 79 Table 16. Risk matrix used to combine two consecutive, and dependent events (adapted from Wieland et al., 2011) 94 Table 17. Relative risk potential to human health through exposure to sewage through outfalls (reproduced from WHO, 2003) 94 Table 18. Qualitative measures of likelihood 96 Table 19. Qualitative measures of consequence or impact 96 Table 20. Qualitative risk analysis matrix: level of risk 96 Table 21. Geographical BSE Risk (GBR) in 2003 in seven countries as assessed by EFSA (2004b; Table 3.5). Earlier assessed levels are also shown 98 Table 22. The four generic categories proposed in New Zealand for the incidence (rate) with examples (Appendix 1 in Lake et al., 2005) 99 Table 23. The three generic categories proposed in New Zealand for severity with examples (Appendix 1 in Lake et al., 2005) 99 ix Table 24. Result of using RiskRanger to evaluate hazard–product combinations for various subpopulations in Australia (from Sumner and Ross, 2002) 101 Table 25. Tabulation of risk as a combination of likelihood and consequence (Biosecurity Australia, 2016) 104 Table 26. Average of elicited criteria weights used in the multi-criteria ranking (Table 3 from FAO and WHO, 2014) 105 Table 27. Estimated risk from L. monocytogenes as used in the risk assessment FAO and WHO, 2004) 107 Table 28. Baseline risk model results at the stage of raw steak tartare patties, for different routes of exposure and the means for the Netherlands (NL). (Pos. tartare = STEC O157 contaminated steak tartare patty), where the column headers refer to specific segments of the Dutch industry (article 10 slaughter with ‘industrial’ butcher, article 10 slaughter with ‘traditional’ butcher and article 4 slaughter) 108 Table 29. Mean risk of illness due to V. vulnificus per serving or exposure 109 Table 30. Risk estimates using probabilistic approach (Table 5 in CCFFP, 2011) 110 Table 31. Number of illnesses per year and probability of illness per serving after 100 000 iterations of the baseline model and the what if scenarios (Table 6.4 in Ssemanda, 2018) 111 Table 32. Dates used in the model for the baseline and alternative scenarios (Table 11 in EFSA, 2014b) 114 Table 33. Summary of time and temperature of storage of eggs in the EU, from the ‘on farm’ to the ‘transport to retail’ stages as derived from expert opinion (industry experts) (Table 6 in EFSA, 2014b) 115 Table 34. Example of a Probability-Severity table for individual hazards (indicated by the numbers in the grid) per year (NIL=None, VLO = Very Low; Lo = Low; Med = Medium; Hi = High; VHI = Very High) 128 Table 35. Example risk score calculations for some hazards used in Table 34 129 Table 36. A linear scoring system for probability 130 Table 37. A logarithmic scoring system where the highest score is assigned the lowest probability 131 Table 38. Data required for risk assessment and data collection sources 135 Table 39. Indicative response times for growth and inactivation of vegetative bacterial cells as a function of temperature 182 Table 40. List of most common types of uncertainty affecting risk assessments associated with the inputs and the methodology (EFSA, 2018a) 207 FIGURES Figure 1. Generic risk management framework as presented by FAO/WHO (2006a, Figure 2.1) 8 Figure 2. Components of a risk assessment 11 Figure 3. “With” and “without” intervention scenarios and changes in risk over time (FAO and WHO, 2009a, Box 2.2) 19 x Figure 4. The epidemiology triangle (modified from Coleman and Marks, 1998) 31 Figure 5. An example of an overview of the conceptual model to describe the exposure pathway for a production-to-consumption exposure assessment. To assess exposure, it is necessary to consider both the probability that a unit of food is contaminated with the hazard (denoted P, for ‘prevalence’), and the level, or number, of that hazard in the food (denoted N) at the time of consumption. For microbial hazards, in particular, both prevalence and number can change as the commodity is further processed, and as time elapses before the product is finally consumed (Lammerding and Fazil, 2000). (Reproduced with permission from Elsevier) 36 Figure 6. An example ‘influence diagram’ of a model of a cross-contamination pathway for the preparation of cooked chicken and lettuce salad. (Xcontam = cross-contamination) 49 Figure 7. Process flow diagram for hazard characterization of pathogens 59 Figure 8. Example Salmonella dose–response model, including expected response (solid line), approximate 2.5th and 97.5th uncertainty percentile lines (dashed) and upper and lower uncertainty bounds (dotted) (FAO and WHO, 2002a p. 87) on linear-log scale (left) and on log-log scale (right) 69 Figure 9. Model output (from FSIS, 2002) showing predicted bacteria per serving after cooking (Dose) and corresponding frequency of illness (Dose–response) 106 Figure 10. Schematic representation of the model framework for a production- to-consumption risk assessment of V. parahaemolyticus in Bloody clam (Figure II-6 in FAO and WHO, 2011a) 113 Figure 11. Decision tree for Boil Water Orders for Cryptosporidium showing the probabilities and estimated costs for illness and death outcomes (Ryan et al., 2013a).( Reproduced with permission from John Wiley and Sons) 116 Figure 12. Example of a typical growth curve where A denotes the maximum population density, l denotes the lag, mm denotes the maximum growth rate, and A denotes the maximum population density (Fig 1 of Zwietering et al., 1990) 183 Figure 13. The major steps in the foodborne disease process 193 Figure 14. Linkage between exposure assessment and hazard characterization 205 Figure 15. A matrix of various types of quantitative outputs one can produce from a risk assessment describing variability and uncertainty; variability only is shown in the graphs on the left and uncertainty and variability combined are shown in the graphs on the right 212 Figure 16. The effect of a selected slaughterhouse process on the Campylobacter concentration on chicken carcases. The change in pathogen concentrations (expressed as log CFU per carcase) before and after the process is represented by a line connecting data points originating from the same study (adapted from Christensen et al., 2001) 220 Figure 17. Example of an overview of data from different studies, with 95 percent confidence intervals for a model input parameter 221 xi Preface Risk assessment of microbiological hazards in foods, commonly referred to as Microbiological Risk Assessment (MRA), has previously been identified as one of the priority areas of work by the Codex Alimentarius Commission (CAC). Following the work of the Codex Committee on Food Hygiene (CCFH), CAC adopted Principles and Guidelines for the Conduct of Microbiological Risk Assessment (CXG-30) in 1999. In response to the needs of their member countries and Codex, FAO and WHO launched a programme of work in the early 2000’s with the objective of providing expert advice on risk assessment of microbiological hazards in foods, including technical guidance on microbiological risk assessment. Three technical guidance documents were published in the Microbiological Risk Assessment Series: Hazard characterization for Pathogens in food and water (2003), Exposure assessment of microbiological hazards in food (2008), and Risk characterization of microbiological hazards in food (2009a). Science has evolved over the last decade and there is a need to update and incorporate new developments in the principles and methods for risk assessment of microbiological hazards. To consolidate and update the existing technical guidance documents on microbiological risk assessment, FAO and WHO established a group of experts and convened the Expert Meetings in Rome, Italy on 11-15 March 2019. The discussion and conclusion in this meeting were taken into consideration in finalizing this report. In addition, the document was also subject to peer review and public consultation before finalization. xii Acknowledgements The Food and Agriculture Organization of the United Nations (FAO) and the World Health Organization (WHO) would like to express their appreciation to all those who contributed to the preparation of this report through the provision of their time and expertise, data and other relevant information before, during and after the meeting. Special appreciation is extended to all the members of the Expert Panel for their dedication to this project and to Dr Emma Hartnett for her expert chairing of the Panel, and Dr Andreas Kiermeier for his excellent support as Rapporteur and assistance in preparing the final document. All contributors are listed in the following pages. In addition, the contributors to the FAO/WHO guideline documents for the hazard characterization (FAO and WHO, 2003), exposure assessment (FAO and WHO, 2008), and risk characterization (FAO and WHO, 2009a) steps of risk assessment, which formed the basis for this consolidated and expanded document, are thanked. The preparatory work and expert meeting convened to prepare this report was coordinated by the Secretariat of the Joint FAO/WHO Expert Meetings on Microbiological Risk Assessment (JEMRA). Appreciation is also extended to all those who responded to the public consultation for the draft guidance that were issued by FAO and WHO and provided information and comments. xiii Contributors EXPERTS Aiko Adell Nakashima, Andres Bello University, Chile Ursula Gonzales Barron, Polytechnic Institute of Braganza, Portugal Nourredine Bouchriti, Hassan II Agronomic and Veterinary Institute Rabat, Morocco Emma Hartnett, Risk Assessment of Biological Systems at Risk Sciences International, Canada Iddya Karunasagar, Nitte University, India Andreas Kiermeier, Statistical Process Improvement Consulting and Training Pty Ltd., Australia Kostas Koutsoumanis, Aristotle University of Thessaloniki, Greece Feng-Qin Li, China National Center for Food Safety Risk Assessment, China Tom Ross, University of Tasmania, Australia Donald Schaffner, Rutgers University, the United States of America Marcelo Lisandro Signorini, Estación Experimental Agropecuaria Rafaela, Argentina Bing Wang, University of Nebraska–Lincoln, the United States of America Marcel Zwietering, Wageningen University, the Netherlands PEER REVIEWERS Wayne Anderson, Food Safety Authority of Ireland, Ireland Yuhuan Chen, FDA, the United States of America Eric Ebel, Food Safety and Inspection Service, USDA, the United States of America Leon Gorris, Independent Food Safety Expert, the Netherlands Yukiko Hara Kudo, National Institute of Health Sciences, Japan Ding Tian, Zhejiang University, China REVIEWERS IN RESPONSE TO CALL FOR PUBLIC COMMENTS Susan Petterson, School of Medicine, Griffith University, Australia; Water & Health Pty Ltd, Australia. Brazilian Codex Alimentarius Committee, Brazil xiv Health Canada, Canada; Canadian Food Inspection Agency, Canada; Public Health Agency of Canada, Canada Linda Saucier, Université Laval, Canada Li Bai, China National Center for Food Safety Risk Assessment, China Egyptian Codex, Egypt BIOHAZ Panel and BIOCONTAM Unit, EFSA Risk Assessment Unit, Finnish Food Authority, Finland French Dairy Interbranch Organization, France Oliver Cerf, Professor Emeritus, Alfort Veterinary School, France Moses Gichia, Food Safety Consultant, Kenya Arno Swart, Eelco Franz, Rob de Jonge, Jenny Deng, Lucie Vermeulen, Joke van der Giessen, National Institute for Public Health and the Environment (RIVM), the Netherlands Lawrence Mafulul, National Agency for Food & Drug Administration & Control, Nigeria Microbiological Risk Assessment team, UK Food Standards Agency, the United Kingdom UK Advisory Committee on the Microbiological Safety of Food, Food Standards Agency, the United Kingdom Andy Hart, Independent Expert, the United Kingdom U.S. Codex Office, the United States of America Mark Powell, Office of Risk Assessment and Cost Benefit Analysis, USDA, the United States of America Barakat S. M. Mahmoud, USAID, Office of Policy & Analysis & USDA, Office of International Research Engagement and Cooperation, the United States of America Fernando Sampedro, University of Minnesota, the United States of America SECRETARIAT Jeffrey LeJeune, Food and Agriculture Organization of the United Nations, Italy Kang Zhou, Food and Agriculture Organization of the United Nations, Italy Satoko Murakami, World Health Organization, Switzerland Lisa Scheuermann, World Health Organization, Switzerland Haruka Igarashi, World Health Organization, Switzerland RESOURCES PEOPLE Lingping Zhang, Joint FAO/WHO Food Standards Programme Food and Agriculture Organization of the United Nations, Italy xv Declaration of interests All participants completed a Declaration of Interests form in advance of the meeting. They were not considered by FAO and WHO to present any conflict in light of the objectives of the meeting. All the declarations, together with any updates, were made known and available to all the participants at the beginning of the meeting. All the experts participated in their individual capacities and not as representatives of their countries, governments or organizations. xvi Abbreviations and acronyms ACMSF ALOP ANOVA BSE BWO CAC CCFFP CDF CFU COI DALY EFSA EKE EC EU FAO FAST FSANZ FSIS FSO GATT GBR GEMS HACCP HALY ICMSF ILSI IPCC JECFA Advisory Committee on the Microbiological Safety of Food [of the United Kingdom of Great Britain and Northern Ireland] Appropriate Level of Protection Analysis of Variance Bovine Spongiform Encephalopathy Boil Water Order Codex Alimentarius Commission Codex Committee on Fish and Fishery Products Cumulative Distribution Function Colony Forming Unit(s) Cost of Illness Disability-Adjusted Life-Year European Food Safety Authority Expert Knowledge Elicitation European Commission European Union Food and Agriculture Organization of the United Nations Fourier Amplitude Sensitivity Test Food Standards Australia New Zealand United States of America Department of Agriculture, Food Safety and Inspection Service Food Safety Objective General Agreement on Tariffs and Trade Geographical BSE Risk Global Environment Monitoring System Hazard Analysis and Critical Control Points Health-Adjusted Life Years International Commission on Microbiological Specification for Foods International Life Sciences Institute Intergovernmental Panel on Climate Change Joint FAO/WHO Expert Committee on Food Additives xvii JEMRA MII MPD MPN MPRM MRA MRSA NHMRC NOAEL OECD OIE PCR PRM QALY QMRA RASFF RMF RTE SPS STEC TBT TSE USDA USEPA USFDA USNACMCF WHO WTO YLD YLL YOPI Joint FAO/WHO Expert Meetings on Microbiological Risk Assessment Mutual Information Index Maximum Population Density Most Probable Number Modular Process Risk Model Microbiological Risk Assessment Methicillin Resistant Staphylococcus aureus [Australian] National Health and Medical Research Council No Observed Adverse Effect Level Organisation for Economic Co-operation and Development World Organisation for Animal Health, formerly the International Office of Epizootics Polymerase Chain Reaction Process Risk Model Quality-Adjusted Life-Year Quantitative Microbial/Microbiological Risk Assessment [European] Rapid Alert System for Food and Feed Risk Management Framework Ready-to-Eat Sanitary and Phytosanitary Shiga toxin-producing Escherichia coli Technical Barriers to Trade Transmissible Spongiform Encephalopathy United States [of America] Department of Agriculture United States [of America] Environmental Protection Agency United States [of America] Food and Drug Administration United States [of America] National Advisory Committee on Microbiological Criteria World Health Organization World Trade Organization Years lived with a disability Years of life lost Young, Old, Pregnant, Immunocompromised xviii 1 1. Introduction 1.1 FAO/WHO SERIES OF GUIDELINES ON MICROBIOLOGICAL RISK ASSESSMENT The General Agreement on Tariffs and Trade (GATT), was established under the United Nations in 1947 as a series of international meetings at which nations would work together to reduce tariffs and other barriers to eliminate unfair and discriminatory practices in international commerce. In relation to food, the overarching principle was that export income from agricultural products was the first step in the economic development of many nations. Completion of the eighth, or ‘Uruguay round’, of GATT negotiations, in 1994, led to the creation of the World Trade Organization (WTO). Importantly, the rules and disciplines of the WTO Agreements – the Sanitary and Phytosanitary (SPS) and the Technical Barriers to Trade (TBT) Agreements – are designed to minimize the negative effect on trade of food safety measures that cannot objectively be justified. What this means is that scientific data and arguments and conclusions based on them, i.e. ‘science-based’ arguments, are the only basis for restrictions to international trade in foods. The WTO recommendations specified the need for science-based food safety measures but, when those rules were introduced, there were no established, internationally accepted procedures for science-based assessment of microbiological food safety risk. The development of science-based standards was considered the role of the Codex Alimentarius Commission (CAC). Accordingly, FAO and 1 WHO established the Joint FAO/WHO Expert Meetings on Microbiological Risk Assessment (JEMRA) (FAO, 2021a) – similar to the already well-established Joint FAO/WHO Expert Committee on Food Additives (JECFA) (FAO, 2021b) – to develop the methods and the tools needed to facilitate the WTO ambitions. As part of that process, CAC also developed a set of principles and guidelines for the conduct of microbiological food safety risk assessment (CAC, 1999). FAO and WHO, through JEMRA, launched a programme of work in the early 2000s in response to the needs of their member countries and CAC with the objective of providing expert advice on risk assessment of microbiological hazards in foods. FAO and WHO undertook development of guideline documents for the hazard characterization (FAO and WHO, 2003), exposure assessment (FAO and WHO, 2008), and risk characterization (FAO and WHO, 2009a) steps of risk assessment. The need for such guidelines was highlighted in the work being undertaken by FAO and WHO on risk assessment of specific commodity-hazard combinations and it was recognized that reliable and consistent estimates of risk in the risk characterization step were critical to risk assessment. Over the years, since the guidelines were first developed, much experience has been gained in risk assessment. By 2017, FAO and WHO recognized that a single, updated document on risk assessment was needed, including additional guidance on hazard identification. To this end, this present document is intended to provide practical guidance and a structured framework for carrying out each of the four components of a microbiological risk assessment, whether as part of a full risk assessment, as an accompaniment of other evaluations, or as a stand-alone process. These guidelines are not intended to be prescriptive, nor do they identify preselected compelling options. They provide descriptive guidance on how to conduct a risk assessment, utilizing a variety of tools and techniques. They have been developed in recognition of the fact that reliable estimation of risk, combined with appropriate uncertainty analysis, is critical for transparent and consistent risk management decision making as well as for effective risk communication within the risk analysis framework. 1.2 SCOPE AND PURPOSE OF THESE GUIDELINES This document provides guidance on undertaking risk assessment of all microbial hazards which may adversely affect human health in foods along the food supply chain. Included are microbial toxins that result in acute illness and where the dose of the microbial toxin is stoichiometrically related to the level of contamination 2 of the toxigenic organism in the food, e.g. Staphylococcus aureus. Excluded is the assessment of risks associated with deliberate contamination, i.e. food tampering. This document is also intended to provide practical guidance on a structured framework for carrying out risk assessment of microbiological hazards in foods, focusing on the four components including hazard identification, hazard characterization, exposure assessment and risk characterization. These guidelines therefore represent the best practice at the time of their preparation, and it is hoped that they will help stimulate further developments and disseminate the current knowledge. The overarching objectives of these guidelines are to help the reader to: • identify the key issues and features of a microbiological risk; • recognize the properties of a best-practice risk assessment; • avoid some common pitfalls of risk assessment; and • perform risk assessments that are responsive to the needs of risk managers. 1.3 GUIDING THE READER THROUGH THIS DOCUMENT The primary audience for this Microbiological Risk Assessment (MRA) guidance is the global community of scientists and risk assessors, both experienced and inexperienced in risk assessment, and the risk managers or others responsible for risk decision making and/or communication. Ideally, the reader would begin with the Report of a Joint FAO/WHO Consultation entitled “Principles and guidelines for incorporating microbiological risk assessment in the development of food safety standards, guidelines and related texts” (FAO and WHO, 2002b). That report appropriately establishes the purpose of risk assessment as meeting the needs of risk managers. With that report as background the reader would ideally read the current guidelines for risk assessment next. This document largely reflects the established practice of MRA, based on the Codex principles for Risk Analysis. However, risk assessment is an evolving science, and while some recent developments are incorporated here, the document does not claim to present every new advance to date. However, this should not be considered as invalidating those developments, but rather it is a reflection of the richness of potential approaches available. On some issues, an approach is advocated based on a consensus view of experts to provide guidance on the current science in risk assessment. On other issues, the available options are compared and the decision on the approach appropriate to 3 the situation is left to the analyst. In both of these situations, transparency requires that the approach and the supporting rationale be documented. 1.4 HOW TO BEGIN WITH RISK ASSESSMENT Microbial risk assessment can often seem overwhelming to those faced with the task of developing a risk assessment for the first time. There are several books that can be helpful for the beginner or the advanced beginner. Training courses are also available from recognized experts in the field. Finally, and perhaps of greatest value, is to work with an experienced practitioner over an extended period to develop a risk assessment. The list of texts and training providers below are not meant to be all-inclusive, nor do they imply endorsement, but they represent a good starting place. Books • Haas, C.N., J.B. Rose, and C.P. Gerba. Quantitative Microbial Risk Assessment. 2nd Ed. John Wiley & Sons, 2014. • Schaffner, D.W. (editor). Microbial Risk Analysis of Foods. ASM Press, 2008. • Vose, D. Risk analysis: A Quantitative Guide. John Wiley & Sons, 2008. • Teunis, P. and J.F. Schijven. Generic guidance to quantitative microbial risk assessment for food and water. RIVM [Rijksinstituut voor Volksgezondheid en Milieu], 2019. • WHO/FAO. Food safety risk analysis: A guide for national food safety authorities, 2007. Training • Center for Advancing Microbial Risk Assessment http://camra.msu.edu/ • Epiχ Analytics https://www.epixanalytics.com/ • FAO/WHO/ICD basic awareness course on Microbiological risk assessment available at: http://www.fao.org/waicent/faoinfo/food-safety-quality/mra/ mra_en/index.html • Joint Institute for Food Safety and Applied Nutrition https://jifsan.umd.edu/ training/risk/registration/catalog • Risk Sciences International, Inc. https://www.risksciences.com/course/ quantitative-food-safety-risk-assessment/ • Vose Software https://www.vosesoftware.com/services/training/ 4 Part 1 General considerations Part 1 General considerations 5 2 2. Risk assessment in context 2.1 RISK ANALYSIS FRAMEWORK Risk analysis is defined by CAC as “a process consisting of three components: risk assessment, risk management and risk communication” (CAC, 2019). It should be noted that the Codex definition differs from how risk analysis is defined in other contexts, e.g. animal health protection (OIE, 2018) or water safety management (WHO, 2016). In the current context, the three components of risk analysis are defined as follows: • Risk Assessment: A scientifically based process consisting of the following steps: (i) hazard identification, (ii) hazard characterization, (iii) exposure assessment, and (iv) risk characterization. • Risk Management: The process, distinct from risk assessment, of weighing policy alternatives, in consultation with all interested parties, considering risk assessment and other factors relevant for the health protection of consumers and for the promotion of fair-trade practices, and, if needed, selecting appropriate prevention and control options. • Risk Communication: The interactive exchange of information and opinions throughout the risk analysis process concerning risk, risk-related factors and risk perceptions, among risk assessors, risk managers, consumers, industry, the academic community and other interested parties, including the explanation of risk assessment findings and the basis of risk management decisions. Risk analysis is used to develop an estimate of the risks to human health, to identify and implement appropriate measures to control the risks, and to communicate 7 with stakeholders about the risks and measures applied. It can be used to support and improve the development of standards, to address food safety issues that arise from emerging hazards or from failures in food control systems. It provides risk managers with the information and evidence they need for effective decisionmaking. As a result, risk analysis contributes to better food safety outcomes and improvements in public health. Regardless of the institutional context, the discipline of risk analysis offers a tool that all food safety authorities can use to improve food safety. 2.2 RISK MANAGEMENT A generic process for carrying out risk management is presented in Figure 1. Such an international framework can provide a useful template for countries developing their own risk management systems. In addition, the CAC has developed principles and guidelines for conducting microbiological risk management (CAC, 2008). Preliminary risk management activities • identify food safety issue • develop risk pro le • establish goals of risk management • decide on need for risk assessment • establish risk assessment policy • commission risk assessment, if necessary • consider results of risk assessment • rank risks, if necessary Monitoring and review • monitor outcomes of control(s) • review control(s) where indicated Identi cation and selection of risk management options • identify possible options • evaluate options • select preferred option(s) Implementation of risk management decision • validate control(s) where necessary • implement selected control(s) • verify implementation FIGURE 1. Generic risk management framework as presented by FAO/WHO (2006a, Figure 2.1) 8 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The first phase of the Risk Management Framework (RMF) shown in Figure 1 consists of preliminary risk management activities. After a food safety issue has been identified, available scientific information is aggregated into a risk profile that will guide further action. The second phase of the RMF consists of identifying and evaluating a variety of possible options for managing the risk. These may include controlling, preventing, reducing, eliminating or in some other manner mitigating the risk. The third phase of the RMF refers to the implementation of the selected risk management options by the relevant stakeholders. In many countries, industry has the primary responsibility for implementing regulatory standards or other food safety measures under government or customer oversight. National food safety authorities, or so-called certified third-party auditors, verify implementation of regulatory standards. They also verify the implementation and effectiveness food safety programs, such as Hazard Analysis and Critical Control Points (HACCP) programs. In addition, other risk management options may be adopted to contribute to risk reductions. Examples include quality assurance schemes at the farm level or consumer education packages for food handling in the home. Guidelines on translating microbial food safety risk assessment into risk management actions are presented in “The use of microbiological risk assessment outputs to develop practical risk management strategies: Metrics to improve food safety” (FAO and WHO, 2006c). Once control measures have been implemented, monitoring and review activities should be carried out as part of the fourth phase of the RMF. The goal is to determine whether the measures that were implemented are, in fact, achieving the risk management goals and whether they are resulting in any unintended effects. Both industry and government bodies are likely to be involved in those activities. Both sectors usually monitor levels of hazard control, while government generally carries out surveillance of the level of foodborne illness in the population. If monitoring information indicates a need to review the risk management options, then the risk management process can begin a new cycle, with all relevant stakeholders participating as appropriate. When dealing with a specific food safety issue, the RMF can be entered at any phase and the cyclical process can be repeated as many times as necessary. Further details can be found in the food safety risk analysis guide published by FAO/WHO (2006a). CHAPTER 2 - RISK ASSESSMENT IN CONTEXT 9 2.3 RISK ASSESSMENT Risk assessment is a ‘decision support’ tool. Its purpose is not necessarily to extend scientific knowledge. Its aim is to provide risk managers with a rational and objective picture of what is known about a health risk and its causes at a particular point in time. It is the risk manager’s responsibility to consider the risk alongside other decision criteria. Such factors include nutrition, food security, social and cultural aspects, technical feasibility, cost–benefit, environmental and economic aspects (FAO, 2017). Risk managers need a sound understanding of the scientific approaches and assumptions used by risk assessors. In general, risk assessment is the umbrella term used to describe the complete process of assessing a risk and is often broken down into several stages. The CAC “Principles and guidelines for the conduct of microbiological risk assessment CAC/GL-30” (CAC, 1999) define risk assessment for microbiological hazards in foods as a science-based process comprising the four components described below (Figure 2). These components are systematically addressed in the various chapters of the present guidance document. For all components, the sources and magnitude of variability and uncertainty (see Chapter 14) should be described. The extent to which this can be done will depend on the data available and the risk assessment approach being taken. • Hazard Identification (Chapter 4) is a qualitative process intended to identify microbial hazards of concern in food. Microbial hazards can include infectious agents or toxins produced by microorganisms. For well-documented microbiological hazards this step is straightforward, while more work will be required if the hazard is new or emerging. If a comprehensive risk profile has already been developed, then this step may be very simple. During hazard identification, the associations between microbiological hazards and specific food commodities and certain high-risk groups in the population should be identified. • Exposure Assessment (Chapter 5) is the qualitative and/or quantitative evaluation of the likely intake of a microbial hazard via specific foods. It should provide a qualitative and/or quantitative estimate of the likelihood and level of the hazard in a specified portion of that food or volume of water. The exposure assessment may also identify the frequency and amount of food or water consumed in a given period for a given (sub)population and may combine the information to estimate the population exposure to a microbiological hazard. The exposure assessment should detail the various steps of the farm-to-fork pathway so that the effect of pertinent steps/processes, or changes to them, 10 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD can be assessed. This can be very powerful information for assessing risk management options. • Hazard Characterization (Chapter 6) provides a description of the adverse effects that may result from ingestion of a hazard, whether that is a microorganism or its toxin. This should include a dose–response relationship where possible. Those health effects include, for example, diarrhoeal illnesses, hospitalizations and deaths. In the context of MRA are usually considered to be acute, rather than chronic, health effects. This component may include identification of different adverse effects, including sequalae and their likelihood, for different subpopulations, such as neonates or immunocompromised people. • Risk Characterization (Chapter 7) is the integration of the three previous steps to derive a risk estimate, i.e. an estimate of the likelihood and severity of the adverse effects that occur in a given (sub)population, with associated uncertainties. It is in the risk characterization step that the results of the risk assessment are presented. These results are provided in the form of risk estimates and/or risk descriptions that provide answers to the questions that the risk managers posed to the risk assessors. These answers, in turn, provide the best available science-based evidence to assist risk managers in controlling food safety risks. The World Organisation for Animal Health, formerly the International Office of Epizootics (OIE), has also defined risk assessment (OIE, 2018). However, the components are slightly different as the OIE guidelines focus on risk assessment Hazard Identification Hazard Characterization Exposure Assessment Risk Characterization FIGURE 2. Components of a risk assessment CHAPTER 2 - RISK ASSESSMENT IN CONTEXT 11 from the perspective of import and export of aquatic and terrestrial animals. Similarly, the WHO document “Quantitative Microbial Risk Assessment: Application for Water Safety Management” uses a slightly different framework to deal specifically with water-related hazards (WHO, 2016). 2.4 RISK COMMUNICATION The ultimate objective of risk communication is to inform and enhance risk assessment and risk management strategies. This includes informing people who may be involved in implementing risk management options, and to enable people to be involved in how they protect their own and others’ health from the food safety risk. For this reason, the results should be presented in ways that promote accessibility. Risk communication is an integral and ongoing part of the risk analysis process and, ideally, all stakeholders should be involved from the start. This means that risk communication is a two-way process, which involves understanding and considering all stakeholder feedback, perceptions and willingness to accept risk, and the formulation of the most appropriate risk management strategies. Therefore, a risk communication strategy should be developed early in the risk analysis process, i.e. prior to commissioning a risk assessment (e.g. Ch 7 in FSANZ, 2013). To assist risk managers in communicating food safety risk information more effectively, FAO has developed a handbook on the subject (FAO and WHO, 2016). Communication of relevant scientific information to risk managers by risk assessors can be challenging, especially when there is uncertainty about riskaffecting factors and the ultimate risk to consumers. For this reason, the interaction between risk assessors and risk managers should be ongoing throughout the process. Risk assessors and risk managers should discuss and agree on which stakeholders are consulted throughout the process. While risk managers of the competent authority have the ultimate responsibility for risk management, the risk perception of stakeholders, including industry and consumers, as well as their willingness to operationalize risk management options must be understood. In presenting the results of a risk assessment, the following points should be taken into consideration: • Results should be presented in a transparent, objective manner. They should be in a form that enables people with little mathematical or statistical background to understand the essential aspects of the risk characterization. For example, a technical document, with all modelling details, could be paired with a less technical interpretive summary. Additionally, the use of illustrations, graphs 12 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD and tables for presentation of quantitative information from the model will be more informative than giving just parameter estimates or other statistics as numerical risk outputs. • Numerical estimates should be supported, and communicated, by qualitative information about the nature of the risk and about the weight of evidence that defines and supports it. • All assumptions, and their consequences for the risk estimates, sources of variation and uncertainty should be fully presented and acknowledged. • All the information and data used in the MRA should be explicitly described in the report. • To ensure transparency, all sources of information and data should be given and cited appropriately and unambiguously in the report and detailed in the reference list. A copy of any ephemeral information (e.g. from a website) should be saved and filed for reference. • Any identified needs for additional data should be clearly communicated. CHAPTER 2 - RISK ASSESSMENT IN CONTEXT 13 3 3. Food microbiological risk assessment (MRA) 3.1 PROPERTIES AND PRINCIPLES OF BEST-PRACTICE RISK ASSESSMENTS Codex Guidelines CAC/GL-30 (CAC, 1999) for microbiological risk assessment contain a list of general principles of microbiological risk assessment, including that: • Risk assessment be objective and soundly based on the best available science and presented in a transparent manner; • Constraints that affect the risk assessment, such as cost, resources or time, be identified and their possible consequences described; • MRA should clearly state the purpose, including the form of risk estimate that will be the output; • The dynamics of microbial growth, survival, and death in foods and the complexity of the interaction (including sequelae) between human and agent following consumption (as well as the potential for further spread) be specifically considered; • Data should be such that uncertainty in the risk estimate can be determined; • Data and data collection systems should, as far as possible, be of sufficient quality and precision that uncertainty in the risk estimate is minimized; • The risk estimate should include a description of the uncertainty and where that uncertainty arose; and • MRA should be conducted according to a structured approach that includes hazard identification, hazard characterization, exposure assessment and risk characterization. 14 The scope of the risk assessment in terms of content and timeframe should be appropriate to meet its objectives and fulfil the needs of the risk managers. As such, before embarking on a risk assessment, the purpose and scope should be clearly identified and articulated by those who commission it. Risk assessments should be initiated in response to well-defined risk management questions; where possible these questions should target the evaluation of the specific risk management options under consideration. Discussions with risk managers are needed to define what information is required to support the decisions they have to make and the type of work that needs to be undertaken to provide that information. Depending on the question(s), this may, for example, include provision of surveillance data, or epidemiological data; a qualitative risk assessment; or a quantitative production-to-consumption exposure assessment. Even if a fully quantitative risk assessment is thought to be necessary, it may be useful to commence with a qualitative approach to better define the nature of the work, the feasibility and the time needed to meet the risk manager’s requirements. This approach highlights the likely iterative nature of risk assessments. The risk assessment for microbiological hazards should provide risk managers with a “best estimate” of the risk. The basis of this best estimate, whether the average risk (mean), or the most likely risk (mode), or some other metric, should be clearly communicated and include a description of why that metric is the best measure of risk. The chosen risk estimate should be as free of bias as is possible. Bias describes forms of error that lead to consistent over- or underestimation of the true risk. If bias cannot be eliminated (e.g. the decision to use a worst-case estimate), that bias and the reasons for it should be clearly stated. Risk assessments should represent the real world as closely as possible and reflect the full range of possible outcomes. For example, this may include probabilities and levels of exposure and consequent risk (e.g. through a distribution of risk per serving). A risk manager may also express the need for information on a particular subset of outcomes, such as “most likely” or “worst-case” scenarios, and the MRA should accommodate those. However, deliberately conservative estimates can reduce the usefulness of the estimate for cost–benefit and cost–effectiveness studies and decrease the ability to describe the uncertainty of the risk estimates. However, they may be useful in certain situations, e.g. to better understand the effect of risk mitigations. Specification of uncertainty and variability are critical in terms of correctly understanding and appropriately using the estimate of risk. It is important to CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 15 identify uncertainty and variability to the greatest extent possible. Their implications for the risk estimate(s) should be discussed and a description of uncertainty and variability should be provided as part of the final risk estimate. Uncertainty and variability are discussed in more detail in Chapter 14. Independence and functional separation of the risk assessment from the risk management process are highly desirable. Nevertheless, interaction between managers and assessors is also essential to ensure that the risk assessment provides the best possible support for the decision(s) that the risk managers have to make. In addition, this interaction helps risk managers understand the principles and assumptions underlying the specific risk assessment. The need for transparency of the risk assessment requires full documentation of the process. This includes transparency in the process, including calls for data and information, scientific peer review and public review, etc. The MRA report should include an explanation of the data used, a description of the models used to assess risk, and explanations of any assumptions made, including the likely effect those assumptions have on the risk estimates. 3.2 PURPOSE AND SCOPE OF MRA Risk assessment is commonly undertaken to help risk managers understand which, if any, intervention strategies can best improve food safety outcomes, or if current risk management actions are adequate. Before beginning a risk assessment, the purpose and scope should be clearly defined, either explicitly or implicitly through the risk management questions. This may involve a discussion between all relevant parties, including the risk managers, risk assessment team, risk communication specialist, and, when appropriate, relevant stakeholders. Definition of the purpose and scope usually specifically identifies the population that should be protected (e.g. general population, young children, pregnant women, immunologically compromised), the stages of the food supply chain that are to be included, as well as the metrics of risks best suited for decision-making. The scope may need to be revised during the preparation of the risk assessment if it becomes evident that the original scope cannot be achieved; any change in scope should be discussed and agreed with the risk manager. If the risk assessment aims to find the option resulting in the greatest reduction in risk, then a statement of purpose should be prepared to identify all potential risk management options to be considered. The questions and the statement of purpose 16 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD will, to a great extent, guide the choice of the approach to be taken to characterize the risk. Clearly, this should be done prior to commencing the risk assessment so that the relevant data are gathered, synthesized and analysed in a way that most effectively informs the risk manager. However, if the purpose of the risk assessment is not clear initially, inappropriate data and information may be collected and analysed. While the results may provide insight into some aspects of the risk, they do not provide clear answers to inform the risk manager appropriately. Risk managers initially define the intended use of a risk assessment in their preliminary risk management activities (Figure 1). They may need to iteratively interact with risk assessors to refine the specific questions to be answered, the scope, focus or outputs of the risk assessment, possibly throughout the conduct of the risk assessment. Risk managers are expected to ask risk assessors to answer specific questions about potential risk management options, which when answered, provide the managers with the information and analysis they need to support their food safety decisions (FAO, 2017). One of the more important preliminary risk management activities is the elaboration of a risk profile (CAC, 2008). A risk profile comprises a systematic collection of the information needed to make a risk management decision and whether a full risk assessment is needed. Typically, the risk profile would be a short document, although sometimes it is expanded to a preliminary risk assessment, e.g. the approach used in New Zealand (e.g. Lake and Cressey, 2013) and in the Netherlands (Bogaardt et al., 2004). This may help to determine the structure of the risk assessment, to fine-tune risk management questions, and assess the feasibility of a more comprehensive risk assessment. While the elaboration of a risk profile is the responsibility of the risk manager it may be commissioned out to other parties. The purpose and scope of risk assessment can vary depending on the risk managers’ questions. The following sections contain a discussion of three possible approaches to risk assessment. No correct approach can be recommended or specified; the choice of approach depends on the risk assessment question, the data and resources available, etc. Three approaches, considered as examples, are: • Estimating a baseline risk; • Comparing risk intervention strategies; and • Research-related study or model. 3.2.1 Estimating baseline risk A common and practical starting point for a risk assessment is to estimate the existing level of risk, often termed the baseline risk, i.e. the level of food safety risk posed without any changes to the current system. This risk estimate is most CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 17 frequently used as the baseline against which intervention strategies can be evaluated (Figure 3). Using the current level of risk as a baseline has the advantage that the magnitude of the risk after a change is relative to this baseline. This approach implies that the baseline risk is the starting point of any risk management actions. For some purposes, a baseline other than the existing level of risk might be used as a point of comparison. For example, the baseline risk could be set as that which would exist under some preferred (e.g. least costly) risk management approach, and the risk under an alternative approach compared with that. Estimating a baseline risk may not be for the immediate purpose of managing the risk. It may be to estimate the magnitude of a food safety problem and hence decide whether the risk merits further management. Whilst in theory it may not be necessary to determine a baseline risk to evaluate intervention strategies, it is nonetheless almost always carried out in practice. Baseline risk does not always need a fully detailed farm-to-fork risk assessment and could instead rely mostly on epidemiological data and knowledge of underreporting rates (see also Section 3.2.2). 3.2.2 Comparing risk management strategies Ideally, agencies with responsibility for safety of foods would consider all possible risk management options along the food chain without regard to who has the authority to enact them. This objective has led to the creation of integrated food safety authorities in many nations and regions. For example, Berends et al. (1998) considered the likely effects on exposure (i.e. Salmonella contamination of pork retail cuts) under different intervention strategies, covering various steps from the farm to the retailer. A farm-to-table model may be most appropriate for this purpose, though for some risk questions, analysis of epidemiological data or a model of part of the food chain may be adequate. In practice, however, the scope of the assessment may be limited to those sections of the food chain within the risk manager’s area of authority. Nevertheless, a more comprehensive risk assessment might identify areas where the risk manager needs to work with other stakeholders to achieve effective change in the food chain. Evaluations of potential risk management actions are often based on comparisons of a baseline risk estimate with an estimate that could result from pursuing alternative strategies (FAO and WHO, 2009b; Perrin et al., 2015; USFDA, 2005) as shown in Figure 3. Such alternatives may be evaluated through “what-if ” scenarios. One includes a future with no new intervention, the other a future with a new intervention. Initially, a baseline model is constructed and run to give a baseline 18 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD estimate of risk and what is expected to happen in the future if no intervention is implemented. Then the model or selected model parameters are changed to determine the probable effect of the putative intervention(s). The differences between the two risk estimates offer indications of the public health benefits of the proposed intervention(s) and, if possible, could also indicate the costs required to attain them. Combinations of interventions can be investigated in a similar fashion, to determine their joint effect, in an effort to find the optimal There are many ways to approach an evaluation of risk management options, including gap analysis, before and after comparison, and with and without comparison (as illustrated in this example). The risk estimates, special studies, economic and environmental analyses, opinion surveys, analysis of the legal implications of proposed actions, and the like will vary from case to case. Not all of these elements are within the domain of risk assessment, but a few generic steps in the process can be identified. These include: • Describe the exiting baseline risk condition, i.e. the current state of the risk, given the intervention strategies already in place. • Describe the most likely future condition in the absence of a change in risk man- agement intervention, i.e. the ‘without’ condition. Every option is evaluated against this same ‘without’ condition, labelled ‘Future No Action’ below. This future may exhibit an increasing, decreasing, flat or mixed trend. • Describe the most likely future condition anticipated with a specific risk-management intervention in place, i.e. the ‘with’ condition. Each intervention has its own unique ‘with’ condition: in the example below, it is labelled ‘Future With Intervention A’. • Compare ‘with’ and ‘without’ conditions for each intervention option. • Characterize the effects of this comparison: not all effects are equal in size, some are desirable, others are not. Human health e ects Existing Future no action With and without intervention comparison Existing baseline Future with intervention A Before and a er comparison Target Gap analysis Time FIGURE 3. “With” and “without” intervention scenarios and changes in risk over time (FAO and WHO, 2009a, Box 2.2) CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 19 strategy. However, risk managers should also consider suboptimal strategies in the broader context, i.e. taking into account the multidimensional nature of risk management (FAO, 2017). In some cases, it is possible to estimate the change in risk without producing an estimate of the baseline risk, but caution must be used in these cases. For example, a risk assessment might determine that it is technically feasible to reduce a particular risk one-hundred-fold. However, if this risk was negligible at the start, then reducing it one-hundred-fold may not be a worthwhile course of action. 3.2.3 Research-related study or model Reliable data are needed to do good risk assessment. There are a number of large microbiological risk assessment models that have been initiated as academic exercises (Guo et al., 2015; Pang et al., 2017; Van Abel et al., 2017). These models have helped advance the field of microbiological risk assessment by identifying what techniques are necessary, developing new techniques, and stimulating research that has value within a risk assessment context. In some situations, those models have subsequently been used by risk managers to assist in making risk management decisions. Such models have also made apparent the changes needed in collection and reporting methods for microbiological, epidemiological, production, dietary and other data that would make the data more useful for risk assessment. Risk assessment is also a very useful aid in identifying where gaps in knowledge exist and thus where additional information is needed. A risk assessment may be undertaken specifically or incidentally to identify research needs, to establish research priorities, and to help design commissioned studies. Experience with microbiological risk assessments has proven these assessments to be valuable in aiding understanding of complex systems. The very process of systematically investigating a food chain has contributed to the appreciation and understanding of the complexity of the systems that make up the food chain. 3.3 THE ROLE OF BEST- AND WORST-CASE SCENARIOS It may be useful to evaluate the best- or worst-case scenarios to get a sense of the most optimistic and pessimistic risk estimates. These scenarios may be used as a filtering technique or as part of a risk profile. For example, the worst-case scenario can be used to filter out whether a risk, or an exposure pathway, is worth worrying about. No further analysis is necessary if the most pessimistic estimate shows the risk level to be below some threshold of interest, such as a negligible-risk level or an acceptable level of risk as defined by a competent authority. 20 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Conversely, a best-case scenario can be used as a preliminary filter of possible risk management options. The risk manager can discount any options for which the most optimistic estimate of the benefits does not justify the cost of that option. Best- and worst-case scenarios operate like extreme what-if scenarios. Where there is considerable but quantified uncertainty about a model parameter, a value is used that gives the required extreme. This will usually be an extreme value from the uncertainty distribution of the parameter, e.g. its 1st or 99th percentile. Where there is uncertainty about exposure pathways and risk attribution, the extreme risk estimate is achieved by picking the most pessimistic (or optimistic) pathway, for example, “imagine that all Salmonella came from chicken.” Potential problems with worst-case analyses include focusing the analysis on the consequences of the worst case, without the context of the probability of that scenario occurring – absolute extremes may be limited only by imagination, no matter how unlikely. In addition, there may be difficulty in specifying the conditions that could lead to the worst (or best) case. Conversely, when parameter values or exposure pathways are known with considerable certainty, they should be used to avoid exaggerating the extreme scenario beyond what is likely. Of particular relevance in relation to the use of extreme scenarios is the concept of compounding or compounded conservatism. While a detailed explanation of the concept is beyond the scope of this document, the interested reader is directed towards scientific literature (Bogen, 1994; Burmaster and Harris, 1993; Cullen, 1994), including Cassin et al. (1996) who specifically discuss the dangers of compounding conservatism in quantitative microbial risk assessment. 3.4 ASSESSING THE RESULTS OF A RISK ASSESSMENT When undertaking a risk assessment, the risk assessor needs to consider two basic probability concepts that can affect the outcome. The first is the apparently random nature of the world. The second is the level of uncertainty about how the real world is operating. Together, they limit the ability to predict the future and the consequences of decisions made. Inevitably, a risk assessment will not have included all possible information about a risk issue because of limited data access (for example, time constraints for the collection of data, or unwillingness of data owners to share information) or because the data simply do not exist. Complying with all the requirements of transparency – describing model and parameter uncertainties, and all explicit and implicit assumptions – does not necessarily CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 21 communicate to risk managers the degree of confidence that the risk assessor has in the results or limitations in its application. Thus, risk assessors should clearly explain how confident they are in the risk assessment results. The confidence in the results depends on the extent of variability and uncertainty in the model outcomes. All assumptions should be acknowledged and made explicit in a manner that is meaningful to the risk manager. In particular, it should be explained what the assumption is, why it was made, why it is appropriate, and what the expected effect is if the assumption does not hold. The process of microbiological food safety risk assessment is most affected by uncertainty, such as: • uncertainty about what is happening in the exposure pathways resulting in human illness; • uncertainty about processes that lead from ingestion through to infection and illness; • uncertainty in the factors that dictate the severity of the illness in different people; and • uncertainty about the parameter values that would describe those pathways and processes. In general, risk assessments should be as simple as possible whilst meeting the risk manager’s needs. The MRA should strive to balance greater detail and complexity (e.g. through addressing more questions or alternative scenarios) against having to include more assumptions that this would entail. That is because more assumptions increase the uncertainty in the results. A draft risk assessment, in which the data gaps and assumptions are clearly identified, may elicit new information, if distributed widely to important stakeholders. Sometimes what is known at a particular time is insufficient for a risk manager to be comfortable in selecting a risk management option. If the risk manager’s criteria for making a particular decision are well defined, a risk assessment carried out based on current knowledge can often provide guidance as to what, and how much, information would make a decision clearer. Another benefit of the risk assessment methodology is that it provides a basis for rational discussion and evaluation of data and potential solutions to a problem. Thus, it also helps to identify where additional data are required. The purpose of a risk assessment is to help the risk manager make a more informed choice and to make the rationale behind that choice clear to all stakeholders. Thus, 22 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD in some situations, a very quick and simple risk assessment may be sufficient for a risk manager’s needs. For example, imagine the risk manager is considering some change that has no cost associated with it, and a crude analysis demonstrates that the risk under consideration would be 10 to 90 percent less likely to occur following implementation of the change, with no secondary risks. For the risk manager, this may be sufficient information to authorize making the change, despite the high level of uncertainty and despite not having determined what the baseline risk was. Of course, most problems are far more complicated, and require balancing the benefits (usually human health effect avoided) and costs (usually the commitment of available resources to carry out the change, as well as human health effects from any secondary risks) of different intervention strategies. Thus, depending on the specific question posed, an exposure estimate may be enough to allow comparison between different interventions to be made, allowing the risk manager to make an informed decision. In the process of performing a risk assessment one usually learns which gaps in knowledge are more, and which are less, critical. Some of those uncertainties are readily quantified with statistical techniques where data are available, which gives the risk manager the most objective description of uncertainty. If, however, a risk assessment assumes a particular set of pathways and causal relationships that are incorrect, then the assessment will be flawed. This is clearly different from variability and uncertainty (Chapter 14) and should be avoided as much as possible. 3.5 CHOOSING THE TYPE OF RISK ASSESSMENT TO PERFORM Risk assessments methods span a continuum from qualitative through semiquantitative to fully quantitative. These approaches may vary in their key attributes such as the quality of risk inference, timeliness, complexity, assessor training requirements, and data requirements. Regardless of the approach used, a scientifically sound risk assessment requires collection of suitable information/ data/assumptions which are documented and fully referenced and synthesized in a logical and transparent manner. All are valid approaches to food safety risk assessment, but the appropriateness of a particular method depends on the ability of the risk assessment to address the specific risk question, i.e. that it is fit-forpurpose to support the risk management decision process. A benefit of undertaking a risk assessment, irrespective of the approach, is that solutions to minimize risk often present themselves out of the process of assessing risk. CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 23 • Qualitative risk assessments are descriptive or categorical treatments of information. A qualitative assessment may be undertaken as part of a first evaluation of a food safety issue, to determine if the risk is significant enough to warrant a more detailed analysis. This again highlights that risk assessments tend to be, and frequently are, iterative. Nevertheless, a qualitative exposure assessment alone may, in some circumstances, provide all the decision support needed by the risk manager. If a more detailed analysis is warranted, then a fully quantitative assessment is usually the preferred approach if data, time and resources are available to support it. • Semi-quantitative risk assessments evaluate risks with a score. They provide an intermediary level between the textual evaluation of risk of qualitative risk assessments and the numerical evaluation of quantitative risk assessments. They offer a more consistent and rigorous approach to assessing and comparing risks and risk management strategies than qualitative risk assessment. They also avoid some of the ambiguities that a qualitative risk assessment may produce. Semi-quantitative risk assessments do not require the same mathematical skills of quantitative risk assessments, nor do they require the same amount of data, which means they can be applied where precise data are missing. • Quantitative risk assessments provide numerical estimates of risk and most models use combinations of mathematics and logic statements. Quantitative risk assessments require the development of mathematical models. In these models the relationships between factors affecting exposure can be quantified and explained using logical tests and conditional statements. An exposure estimate may be combined with a mathematical function that quantifies the dose–response relationship to provide an estimate of risk. It should be noted that there is a gradation of model types from qualitative to fully quantitative and while such classifications may be helpful, they are not strictly defined categories. The importance of matching the type of risk assessment to its purpose has been emphasized previously. The United States of America’s National Advisory Committee on Microbiological Criteria for Foods noted (USNACMCF, 2004): Risk assessments can be quantitative or qualitative in nature, but should be adequate to facilitate the selection of risk management options. The decision to undertake a quantitative or qualitative risk assessment requires the consideration of multiple factors such as the availability and quality of data, the degree of consensus of scientific opinion and available resources. 24 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The Australian National Health and Medical Research Council (NHMRC, 2011, p38) cautions that: Realistic expectations for hazard identification and risk assessment are important. Rarely will enough knowledge be available to complete a detailed quantitative risk assessment. ... Staff should have a realistic understanding of the limitations of these predictions, and this should also be conveyed to the public. The decision on the appropriate balance of the continuum of methods from qualitative to quantitative will be based on several factors, including those considered below. 3.5.1 Consistency Risk assessments should limit subjectivity as far as possible and aim for consistency. Qualitative and semi-quantitative risk assessment can be made simple enough to be applied repeatedly across a range of risk issues. In contrast, quantitative risk assessment is more driven by the availability of data and may have to employ quite disparate methods to model different risks. Nevertheless, subjectivity can occur across the spectrum. Qualitative risk assessment is more prone to subjective judgements involved in converting data or experience into categories such as “high”, “intermediate” and “low.” Because it may be difficult to unambiguously define these terms repeatability of an analysis by others is less certain. On the other hand, quantitative risk assessments may involve subjective choices regarding model form and data analysis, e.g. in approaches to the selection and analysis of data. In all cases the basis of these judgements can, and should, be documented in a way that enables others to understand the reasoning and replicate the results. 3.5.2 Resources Some basic capacities are needed to conduct MRA or its components. Risk assessments conducted at the international level (e.g. JEMRA) can assist countries by providing modules or building blocks that can be adapted or modified to suit other exposure or risk assessments. For example, FAO/WHO’s Food Safety Risk Analysis Tools website (FAO and WHO, 2021) contains a risk assessment tool for Cronobacter spp. in powdered infant formula and a risk management tool for the control of Campylobacter and Salmonella spp. in chicken meat. The United States of America Food and Drug Administration’s (USFDA) FDAiRISK® tool (FDA, 2021) allows users to create and share risk assessment models/modules. However, CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 25 it must be remembered that a risk assessment usually requires some country- or region-specific data to be useful. The basic capacities for undertaking an MRA include the following. • Access to expertise. While the assessment may be carried out by one individual or a small team, access to a range of expertise, from multiple disciplines, usually is needed. Depending on the task, this is likely to include trained risk assessors, modellers, mathematicians, statisticians, microbiologists, food technologists, animal and plant health specialists, agriculture technologists, human and veterinary epidemiologists, public health specialists, and other experts as needed. Quantitative risk assessments typically require that at least part of the assessment team have rigorous mathematical training. If this resource is in limited supply, then this may make qualitative risk assessment more practical, provided the risk question is amenable to this approach. Note that, while qualitative risk assessments may not be demanding in terms of pure mathematical ability, they place a considerable burden of judgement on the analyst to combine evidence in an appropriate and logical manner. The technical capability necessary to collate and interpret the current scientific knowledge is almost the same, regardless of the approach used. • Informed risk managers and policymakers who are aware of the need for, use of and limitations of risk assessment. They need to be working in the context of an appropriate risk management framework, whether in government or industry. This framework must facilitate data collection, decision-making and implementation. • Financial and human resources to complete the risk assessment in a timely manner and to an acceptable level that provides useful support for risk management decisions. For very large MRA projects, a dedicated project manager may be desirable. • Communication channels. Good communication is needed between technical experts, risk managers and the risk assessors to facilitate efficient exchange of data and knowledge. • Information technology. Computing facilities, both hardware and software, and access to appropriate information networks are needed to collect, collate and process data, and to provide outputs in a form suitable for communicating results. This should include access to international networks and databases, including access to scientific publications. • Where data on microbiological hazards are not available, the capacity to conduct surveillance for microbiological hazards, including access to microbiologists, epidemiologists, trained field staff and competent laboratories, is needed. 26 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD While the above list is an ideal, benefits can also be obtained from conducting more modest risk assessments, but still according to the principles in these guidelines, even from teams with limited expertise. To assist groups with fewer resources, communication with more established groups should be actively encouraged (e.g. including training, mentoring and technology transfer). With respect to scientific publications, access to subscription-based journals has repeatedly been identified as a substantial limitation in many countries. It is worthwhile to note that Research4Life (www.research4life.org) provides organisations in many low-income countries with free or low-cost access to academic and professional peer-reviewed content online. To assist the risk assessors with their tasks, a range of software tools have been developed, including those listed by Bassett et al. (2012) and those at the Quantitative Microbial Risk Assessment (QMRA) Wiki (QMRA wiki, 2021). These tools are not necessarily specific to food safety risk assessments, although they include a range of food safety specific models and tools. These tools cover areas of risk ranking, predictive microbiology, specific risk assessment and sampling. 3.5.3 Theory or data limitations Quantitative risk assessments tend to be better suited for situations where mathematical models are available to describe phenomena, e.g. dose–response models, and where data are available to estimate the model parameters. If either the theory or data are lacking, then a more qualitative risk assessment is appropriate. 3.5.4 Breadth of application When considering risks across a spectrum of hazards and pathways, there may be problems in applying quantitative risk assessment consistently across a diverse base of theory and evidence, such as comparing microbiological and chemical hazards in food. The methodologies and measurement approaches may not yet be able to provide commensurate risk measurements to support decisionmaking where scope is broad. 3.5.5 Speed Qualitative and semi-quantitative risk assessments generally require much less time to generate conclusions compared with quantitative risk assessment. This is particularly true when the protocols for qualitative and semi-quantitative risk CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 27 assessments have been firmly established with clear guidance in the interpretation of evidence. There may be some exceptions where the process of qualitative risk assessment relies on a process of consultation that requires considerable planning, briefing, and scheduling, e.g. when relying heavily on structured expert elicitation. Quantitative risk assessment may take longer to develop; if it is to be repeated once the model is established, then the speed to generate conclusions is similar to qualitative or semi-quantitative approaches. 3.5.6 Transparency Transparency, in the sense that every piece of evidence and its exact effect on the assessment process is made explicit, is more easily achieved by quantitative risk assessment. However, qualitative or semi-quantitative approaches may be easier to understand by a larger range of stakeholders, who will then be better able to contribute to the risk analysis process. Quantitative microbiological risk assessment often involves specialized knowledge and a considerable time investment. As such, the analysis may only be accessible to specialists or those with the time and resources to engage them. Strict transparency is of limited benefit where interested parties are not able, or find it excessively burdensome, to understand, scrutinize and contribute to the analysis and interpretation. Consequently, errors in quantitative risk assessments may be more difficult to find. 3.5.7 Stage of analysis Qualitative and quantitative risk assessment need not be mutually exclusive. Qualitative risk assessment can be very useful in an initial phase of risk management to provide timely information regarding the approximate level of risk. This allows risk managers to decide on the scope and level of resources to apply to quantitative risk assessment. As an example, qualitative risk assessment may be used to decide which exposure pathways (e.g. air, food, water; or raw versus ready-to-eat foods) will be the subject of a quantitative risk assessment. Where available, comparing the outputs from both approaches, or from different stages of the analysis, may help the detection of errors that may have been made in either assessment. 3.5.8 Responsiveness A major concern often expressed in regulatory situations is the lack of responsiveness of risk assessment conclusions when faced with new evidence. Consider a situation where a risk assessment has been carried out with older data indicating that the prevalence of a pathogen is 10 percent. After the risk assessment is published, it 28 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD is found that the prevalence has been reduced to 1 percent. In most quantitative risk assessments, there would be a clear effect of the reduced prevalence on the risk characterization. In some qualitative risk assessments, this effect may not be sufficiently clear. Qualitative risk assessments, particularly where the link between evidence and conclusion is ambiguous, may contribute to foster or support this lack of responsiveness. This in turn can generate mistrust and concern for the integrity of the risk assessment process. CHAPTER 3 - FOOD MICROBIOLOGICAL RISK ASSESSMENT (MRA) 29 4 4. Hazard identification Hazard identification (HI) is conventionally the first step in MRA. For the purposes of the CAC, hazard identification related to food safety is defined as “the identification of biological, chemical and physical agents capable of causing adverse health effects and which may be present in a particular food or group of foods” (CAC, 1999). In particular, for microbiological agents “the purpose of hazard identification is to identify the microorganisms or the microbial toxins of concern with food” (CAC, 1999). In general, hazard identification is largely a qualitative examination of the foodborne hazard and associated potential adverse health outcomes due to specific foodborne exposure. It is supported by a critical review of knowledge about the hazards and/or food in question. In the context of MRA, the term hazard encompasses any microbiological agent able to cause harm, including bacteria, viruses, parasites, fungi, algae, including their toxins and metabolites, as well as prions. 4.1 OBJECTIVES OF HAZARD IDENTIFICATION The main purpose of hazard identification is to identify the microbiological hazard(s) found in food that is/are the cause of adverse health outcomes. Since a wide range of microbiological hazards can cause foodborne illness, hazard identification should identify whether a potential hazard is realistic for the food product of interest. In some situations, i.e. depending on the risk managers’ questions, the hazard identification may include a list of hazards and therefore, the final product of the hazard identification procedure is a practical list of microbiological hazards related to the specific food product (e.g. FAO and WHO, 2006b, 2007). 30 4.2 THE PROCESS OF HAZARD IDENTIFICATION Hazard identification serves to establish the hazard as likely or real in the food product and to document the important information known about the relationships and interactions between the hazard, the food and host, as well as their relationship to human illness (Figure 4). With respect to the food these factors including intrinsic characteristics, environmental factors and production conditions. Matrix Disease Pathogen Host FIGURE 4. The epidemiology triangle (modified from Coleman and Marks, 1998) There is some overlap between the information collated as part of the hazard identification step and the exposure assessment and hazard characterization steps – the hazard identification may provide only a general overview, while the latter steps document more detailed information, e.g. extent of exposure to the hazard and dose–response relationship. The information documented as part of microbiological hazard identification includes the following. • What is/are the hazard(s) of concern associated with specific food in question? • Is the hazard of concern to public health and what is the likelihood of the hazard causing an adverse health effect? • What is the population at risk? • What is the epidemiological evidence, including outbreaks and sporadic illness, that this hazard poses a potential risk in the food product? • What adverse health effects could be associated with the exposure to the hazard and through what mechanisms? • What host factors and life stages could affect the type and severity of adverse health outcomes among the population at risk? • How do common exposure pathways link the adverse health effects with the hazard? CHAPTER 4 - HAZARD IDENTIFICATION 31 • How often does the hazard occur in the food product of interest? • How do environmental conditions affect the hazard’s transfer and fate along the exposure pathway? A wide range of microbiological hazards are associated with foodborne illness. To identify the most significant hazards in the food of concern, characteristics of a range of hazards can be collectively evaluated. These include inherent properties of hazards such as invasiveness, virulence, pathogenicity, natural reservoir, transmissibility, and resistance to environmental factors and interventions in the food supply chain. In addition, hazard identification highlights issues such as sensitive populations, acuteness of the illness (acute versus chronic disease) and other complications such as long-term sequelae. These may be considered consideration in more detail the hazard characterization (Chapter 6). Sensitivity to infection depends on the integrity of the hosts’ immune system, the virulence/potency of the hazard and level of exposure to the hazard. The integrity of a host’s immune system can be affected by life stage and health conditions. For example, due to their immature or compromised immune systems, young children and the elderly may be more sensitive to microbiological infection compared to young healthy adults. In turn this can lead to more serious and longer-lasting health outcomes. The exposure level of and ability of a hazard to elicit an adverse health effect at the time of consumption can be cumulatively affected by a series of environmental conditions throughout the food chain. The physical and chemical properties of the food matrix may affect the hazard’s survival and persistence in the food. Together with growth, inactivation and survival characteristics of the hazard, the properties of the food can be elaborated in the exposure assessment (Chapter 5). For example, the presence of high levels of fat in food can protect Salmonella against thermal inactivation (Gurman et al., 2016; Krapf and Gantenbein-Demarchi, 2010). The transmission and fate of a hazard may be affected by the complex interaction between the hazard and various intermediate pathways. For example, bacterial pathogens from food-producing animals may reach the human population directly through the consumption of contaminated animal products or indirectly through the consumption of crop products contaminated with animal faeces. Sometimes evidence clearly identifies the significance of foodborne transmission for specific microbiological hazards and which foods are implicated before a microbiological risk assessment is conducted. In this situation, less effort can be expended in the investigation of the causal relationship between the occurrence of adverse health outcomes and the exposure to the foodborne hazard. Conversely, 32 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD emerging hazards are continually being identified through the mechanism of acquiring new traits. Through vertical or horizontal transfer of genetic traits among microorganisms, newer pathogenic or opportunistic strains can be produced. Consequently, this transfer could result in new microbiological hazards with higher virulence and/or persistence to various environmental conditions. In this situation, when a particular food is suspected, more thorough investigation is needed to indicate whether the hazard is likely associated with the food product of interest. 4.3 DATA SOURCES FOR HAZARD IDENTIFICATION A large amount of relevant evidence-based information needs to be collected, appraised and interpreted in hazard identification. The main types of data sources providing useful information to the hazard identification process are as discussed in Chapter 10. Epidemiologic data from disease monitoring programs, or investigations of foodborne outbreaks are often the first well documented indication of a food safety problem associated with a hazard. Food contamination surveillance data, together with product/process evaluations can aid the identification of hazard– food combinations. Evidence from these sources is usually quantitative, i.e. includes information about the concentration or number of units of the hazard in the food. These data may provide useful information for exposure assessment and/ or establishing a dose–response relationship. Whole genome sequencing is being used increasingly for foodborne pathogen surveillance, outbreak investigation and contamination source tracking throughout food supply chains (Rantsiou et al., 2018; WHO, 2018). Clinical research usually provides qualitative data, highlighting the mode of action with which the hazard affects the host, such as through the action of toxins, either in the food or, alternatively, through infectious mechanisms. Inferences from microbiological and clinical studies can be used to support the epidemiological and observational evidence. More details regarding the strength and limitation of different data sources can be found in Chapter 10. CHAPTER 4 - HAZARD IDENTIFICATION 33 5 5. Exposure assessment 5.1 THE PROCESS OF EXPOSURE ASSESSMENT Codex defines exposure assessment as “the qualitative and/or quantitative evaluation of the likely intake of biological, chemical, and physical agents via food as well as exposures from other sources if relevant.” (CAC, 1999). Consequently, exposure assessments are often specific to the production, processing and consumption patterns within a country or region. Exposure assessment may be undertaken as part of a risk assessment, or it can be a stand-alone process, such as when there is not enough information available to undertake a dose–response assessment (i.e. a Hazard Characterization) or when the risk management question only involves quantifying or seeking ways to minimize exposure. The process of exposure assessment can be, and usually is, iterative. Discussions between risk managers and risk assessors may lead to a refinement of the initial question to be addressed. Similarly, consultation with other parties may result in new information, that can in turn lead to revision of assumptions or to further analysis. Also, nongovernmental bodies, such as food manufacturers or the food industry, may use exposure assessment, as a stand-alone process or as part of an MRA, to assess the safety of their food products. This may be particularly useful as part of food innovation research and before putting products on the market (van Gerwen and Gorris, 2004; Membré and Boué, 2018; Pujol et al., 2013). The goal of an exposure assessment may be to provide an estimate of the level of exposure to a hazard in a given population. The risk manager may also wish to limit the scope to specific regions, or populations, periods of time or parts of the supply 34 chain. This again reinforces the need for the risk managers to clearly articulate their needs to the assessors, including the level of detail required in the exposure assessment, and any constraints that would limit the range of management options. For example, when potential mitigations are to be compared, the risk managers should provide an indication of which measures they not consider acceptable under any circumstances. Once there is a clear understanding of the requirements of the exposure assessment in relation to risk management, the next step is to consider the factors that have a direct effect on consumer exposure to the hazard. These including frequency of consumption of the product or commodity; frequency and levels of contamination with the hazard; and factors that affect the exposure. These factors may include potential for microbial growth, inactivation during cooking (or other processes), meal size, seasonal and regional effects, etc. In addition, the exposure assessment should describe the relevant pathways of exposure. For example, if the purpose of the risk assessment is to identify and compare different mitigation strategies to be used from production to consumption, then the entire production-to-consumption pathway has to be addressed (Figure 5). In other cases, only the pathways from retail to consumers may be relevant. Thus, if the purpose of the assessment is to reach a decision on the maximum tolerable level of a pathogen in a ready-to-eat (RTE) product at the point of sale, then the assessment would be used to determine the potential for further changes in exposure due to consumer handling (such as time and temperature of storage, effect of cooking or other food preparation steps, potential for cross-contamination in the home, etc.). The level of detail required in the different pathways reflects the question asked and the information needed by the risk managers and may be modified based on the information available. For example, if it has been shown that the prevalence and/ or numbers of a hazard differs between carcases according to the type of abattoir, then such information might influence the level of detail required and the selection of pathways in the exposure assessment. Food supply pathways can be multiple and complex, for example, ‘ready-to-eat’ meals are a synthesis of food components (e.g. meat, vegetable and dressing) that arise from different pathways. Risk managers may have specific questions concerning specific processes, such as organic farming, logistic slaughtering (the order in which animals are slaughtered (e.g. Nauta et al., 2009)), or imported foods (e.g. Skjerve, 1999) that they want to be addressed. Accordingly, these specific interests would need to be taken into account in selecting the pathways to consider and the types of data to include. CHAPTER 5 - EXPOSURE ASSESSMENT 35 Prevalence (P) Pf Pp Pr Farm Processing Retail Home Nf Np Nr Number (N per unit measure) Probability and level of exposure FIGURE 5. An example of an overview of the conceptual model to describe the exposure pathway for a production-to-consumption exposure assessment. To assess exposure, it is necessary to consider both the probability that a unit of food is contaminated with the hazard (denoted P, for ‘prevalence’), and the level, or number, of that hazard in the food (denoted N) at the time of consumption. For microbial hazards, in particular, both prevalence and number can change as the commodity is further processed, and as time elapses before the product is finally consumed (Lammerding and Fazil, 2000). (Reproduced with permission from Elsevier) 5.2 MODELLING APPROACHES 5.2.1 Introduction The goal of exposure assessment is to deduce, from the available information, the probability and magnitude of exposure to the hazard. Detailed exposure data, characterizing the extent of microbiological hazards present in foods at the time of consumption, are usually not available. Thus, exposure assessment will often rely on a model, encompassing knowledge of the factors and their interactions that affect the number and distribution of the hazard in foods, to estimate exposure at consumption. This chapter is primarily concerned with development and application of models used as part of the exposure assessment. General data needs and sources are considered in greater detail in Chapter 10. A model can be defined as “the description of a system, theory, or phenomenon that accounts for its known or inferred properties and may be used for further study of its characteristics” (McMeekin et al., 2008). The model is a simplified description of some more complex system or phenomenon. Models are also used to communicate an understanding, or hypothesis, concerning some aspect of reality that may, or 36 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD may not, be able to be directly observed. Thus, another description is that a model is “a hypothesis or system of beliefs about how a system works or responds to changes in its inputs” (Cullen and Frey, 1999). That hypothesis or description can be expressed in words or as “a system of postulates, data, and inferences presented as a mathematical description of that entity or state of affairs” (Merriam-Webster, 2021).When developing a model – whether it is a full risk assessment or any part thereof – it is important to ensure that the model is fit-for-purpose. As a result, a model should be as simple as possible, but as complex as necessary.1 Among the benefits of a model is that it can be used to predict the outcome of events that have not occurred, or have not been observed, e.g. the probability of infection from low doses. However, a fundamental rule of modelling is that no possibility should be modelled that could not actually occur (Vose, 2008). In the context of exposure assessment, the models synthesize data and knowledge from other observations about the pathways of exposure, the behaviour of microbial hazards in foods, patterns of consumption, and so on, to infer what could happen in other circumstances of interest. Models can be used to interpolate among discrete values of observed data and, in some circumstances, to extrapolate beyond the range of observations. In either case, the validity of the interpolation or extrapolation depends on validation of the model (see Sections 16.2). There is a spectrum of approaches available for exposure assessment, ranging from qualitative to fully quantitative in nature. Quantitative exposure assessments may, in turn, be deterministic or stochastic, with the latter encompassing and representing variability and uncertainty in the data and knowledge as fully as possible and necessary (see Chapter 14). Although qualitative exposure assessments lack numerical precision, they are still valuable and may, in some circumstances, provide all the decision support needed by the risk manager. Also, as an example, a qualitative assessment may be undertaken as part of a Risk Profile, to determine if the risk is significant enough to warrant a more detailed analysis. This again highlights that risk assessments tend to be, and frequently are, iterative. If a more detailed analysis is needed to answer the risk question and to provide the needed decision support for the risk manager, then a fully quantitative assessment is usually the preferred approach if data, time and resources are available to support it. 1 A rephrasing of Einstein’s principle “A scientific theory should be as simple as possible, but no simpler.” CHAPTER 5 - EXPOSURE ASSESSMENT 37 5.2.2 Qualitative and semi-quantitative exposure assessment A qualitative assessment may be developed by assigning descriptive ratings of probability, such as ‘negligible’, ‘low’, ‘medium’ or ‘high’, to the factors considered in the assessment (ACMSF, 2012; Fazil, 2005). As noted in Section 3.5, semi-quantitative exposure assessment provides an intermediary level between qualitative and quantitative exposure assessment. It does not require the same mathematical complexity as quantitative exposure assessment, nor does it require the same amount of data, which means it can be applied to exposure and exposure minimization strategies where precise data are missing. See also Sections 9.1 and 9.2 for more detailed discussion of these qualitative and semi-quantitative risk assessment approaches. Examples of semiquantitative risk assessment approaches, including exposure assessment, being used to make risk management decisions (Cardoen et al., 2009; Hald et al., 2006; Omurtag et al., 2013; Sumner and Ross, 2002). 5.2.3 Quantitative exposure assessment As noted above, quantitative exposure assessments provide numerical estimates of exposure. They require models to be developed, in which all relationships between factors affecting exposure are described mathematically. Consequently, quantitative exposure assessments generally require more data than qualitative or semi-quantitative exposures assessments. Quantitative models can be divided into two categories (Bassett et al., 2012): 1. Deterministic, sometimes also referred to as fixed-value or point-estimate and which in some situations can be solved analytically, and 2. Stochastic, sometimes also referred to as probabilistic. In some limited circumstance, these models may be able to be evaluated analytically, though most need to be evaluated using Monte Carlo simulation. These are discussed in more detail in Chapter 11. In a mathematical model, input variables are those that determine the type and magnitude of the response, or output, variables. The output variables in exposure assessment are the frequency and magnitude of exposure of consumers to the microbiological hazard in the food of interest. Depending on how much of the food supply chain is included in the exposure assessment, input variables could include factors such as time, temperature, production volume and dilution during processing (see data sources in Chapter 10). If a modular process framework is utilized for the exposure assessment (e.g. Figure 5), then outputs from one module are the inputs for the next module. Parameters quantify the distribution of input variables; they can be 38 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD fixed values or described by distributions. For example, while bacterial growth rate is often related to temperature, a mathematical model is needed to quantify that relationship (see Chapter 12). The parameters of that model could be fixed for a specific strain of a hazard but will differ between species and perhaps even for different strains of the same species. In the latter situation the between-strain variability in growth rates, which are a function of temperature, could be described by a distribution. Stepwise approach to quantitative exposure assessment As described above, exposure assessments often involve description of very complex systems, where each process step may not contribute equally to exposure and where not all the desired data may be available. In the context of MRA, van Gerwen et al. (2000) suggested that, under such conditions, it could be beneficial to conduct an exposure assessment in a series of stages of increasing complexity/ sophistication. Similar approaches have been suggested by Cullen and Frey (1999), the USEPA (2006) and WHO (2016) and may be particularly useful when there is an urgent need for an estimate of exposure or risk. A rough estimate is first made of the order of magnitude that individual factors or parameters may contribute to exposure or consequent risk. This could be considered as part of a risk profile. For those factors that contribute most significantly, a more detailed assessment is performed, or more data are gathered and combined in, for instance, a deterministic approach. Where relevant, an even higher level of detail can be achieved using stochastic modelling. Van Gerwen et al. (2000) propose that, when using a stepwise approach, both efforts and resources are focused where they add most to reducing uncertainty in the exposure estimate. 5.2.4 Modelling the production-to-consumption pathway Introduction The methods by which exposure is estimated depends on the combination of risk management questions being addressed and the amount of data and other resources available, such as expertise and time. An exposure assessment that considers the events from agricultural production through to consumption will demand the most time and resources. Such an exhaustive approach may be appropriate if: • the risk management questions require consideration of all stages, e.g. the effectiveness or feasibility of mitigation at the farm to estimates of exposure in consumed product, and • there are sufficient data, knowledge, time and expertise to allow each stage to be considered. A generic full production-to-consumption pathway is outlined in Figure 5, and various approaches for modelling of this pathway are outlined below. It is important CHAPTER 5 - EXPOSURE ASSESSMENT 39 to emphasize that the final approach utilized depends on the available data and the risk management questions being addressed and is therefore assessment specific. Thus, the following should be viewed as guidance, or examples, rather than as being prescriptive. Model development Conceptual model is a term used to describe the understanding of the routes by which the population of interest is exposed to the hazard of concern, including all the factors and their interactions that affect the probability and level of exposure. The conceptual model may be expressed in text, diagrams, as a mathematical model or a combination of these. There is no preferred method to develop and describe the conceptual model. Rather, whatever form the conceptual model takes, it should adhere to the principles and guidelines for the conduct of microbiological risk assessment (CAC, 1999). For the purposes of communication of the conceptual model to nonmathematicians, a diagrammatic representation may be useful and more readily understood than a text-only description, or the mathematical model, alone. Different approaches can be used to develop the conceptual model. The Event Tree approach describes a scenario from a contamination event to a defined endpoint of the assessment, e.g. consumption (Roberts, Ahl and McDowell, 1995). This approach serves to describe or identify the most likely pathways that lead to contamination and subsequent disease and may identify variables in need of further data or modelling. Conversely, the Fault Tree approach begins with the occurrence of a hazard and from there describes the events that must have occurred for the hazard to be present (Roberts, Ahl and McDowell, 1995). This approach can provide a framework to analyse the likelihood of an event by determining the complete set of underlying conditions or events that would allow the given event to occur (Jaykus, 1996). Additional approaches to modelling used in assessments of microbial food hazards include the Dynamic Flow Tree model (Marks et al., 1998) and the Process Risk Model (PRM) (Cassin, Paoli and Lammerding, 1998). The Dynamic Flow Tree model emphasizes the dynamic nature of bacterial growth and incorporates predictive microbiology using statistical analysis of data. In contrast, the PRM focuses on the integration of predictive microbiology and scenario analysis to provide an assessment of the hygienic characteristics of a manufacturing process. A general framework is the Modular Process Risk Model (MPRM) (Nauta, 2001, 2008; Nauta et al., 2001), which can be thought of as an extension of the PRM approach. The fundamental assumption of the MPRM approach is that at each of 40 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD the steps in the various intermediary stages from production to consumption, at least one of several processes can be assigned. These processes can be divided into microbial and product handling processes. The microbial processes include growth and inactivation, and the food and product handling processes include mixing of units, partitioning of units, removal of parts of units and cross-contamination of organisms among units. The transmission of infection among live animals during primary production could be viewed as an additional biological process, which provides the starting estimates of prevalence in a full production-to-consumption model. When developing mathematical models, the model structure can facilitate or hinder stochastic modelling and sensitivity analysis (Chapter 15). It is recommended that the models should be formulated such that independent variables affecting exposure are clearly specified. In addition, data for each iteration of the model should be stored for all inputs and outputs for which sensitivity analysis is required. Depending on the modelling approach selected, a one-to-one relationship between the input and output may not be possible when partitioning or combining of units is included (e.g. Kiermeier, Jenson and Sumner, 2015). The definition of unit is crucial when modelling the processes from production to consumption. A unit is defined as a physically separated quantity of product in the process, e.g. an animal, a (part of a) carcase, or a package of ground beef. It may be that one unit from primary production is also the consumer package, such as an egg or whole chicken. However, most examples are more complex, e.g. beef carcase transformed to ground beef burger or milk made into cheeses. In this case, units have to be redefined at each partitioning or mixing stage and thus both the number of organisms (N) in a unit and the prevalence (P) across units (see Figure 5) can be treated as uncertain and variable throughout the model. This makes it possible to assess the uncertainty and variability in the final exposure, and thus the uncertainty in the final risk estimate. It should also be noted that prevalence and concentration are related. If the (mean) concentration of the pathogen in a batch of food is low (e.g. 1 cell per 5 kg), then the prevalence of contamination will depend on the size of the unit of food. For example, if the unit size is 100 g, then it is expected that one in 50 units contains the pathogen, i.e. the prevalence is 2 percent. But if the unit size is 500 g, then it is expected that one unit in 10, on average, contains the pathogen, i.e. the prevalence equals 10 percent. Similarly, if the unit size is 5 kg, then it is expected that the prevalence is 100 percent. In practice, however, the cells are not expected to be perfectly uniformly distributed, and hence the prevalence will be less than 100 CHAPTER 5 - EXPOSURE ASSESSMENT 41 percent, because some units contain more than one cell and, consequently, some others contain none. It is possible to estimate the concentration in a batch from the prevalence and the number of positive samples, provided that not all samples of that size are positive. This approach is based on the same statistical principles as the Most Probable Number technique used in microbiology (Cochran, 1950). For a good exploration of the distribution of microbes in food see Bassett et al. (2010). Approaches to mathematical modelling of microbial growth and inactivation and their application are outlined in Sections 12.1 and 12.2. It is difficult to suggest a general model framework for cross-contamination but useful discussion of this topic can be found in Schaffner (2003, 2004), Pérez-Rodríguez et al. (2008) and later in this section. As noted above, different modelling approaches have been proposed and used. The approach used therefore depends on the perspective of the assessor and on the problem being modelled, as indicated by the risk question. Discussion of modelling strategies for the stages from production to consumption is presented below; which stages to include will depend on the scope and purpose of the risk assessment. Primary production (farm) The main focus of the primary production or “farm” stage of the exposure assessment is to estimate the prevalence and concentration of the microbiological hazard in the animal population, crop or product of interest; the same approach applies for wild capture/harvest situations. For example, this might be prevalence and contamination levels per live cow, per bird, per homestead, per kg of lettuce leaves, per apple or per vat of raw milk. For animal products, it is important to differentiate infection and colonization from contamination of skin surfaces. These may depend on each other, such as where excretion by infected or colonized animals can result in contamination of that animal’s external surface as well as that of other animals in the group. Recognizing and incorporating dependencies between variables is an important aspect of constructing robust and logical models. This is particularly important when constructing stochastic models in which the variables are described by distributions. Thus, if dependencies are not included in the model structure, then impossible outcomes could occur during the model simulation. For example, consider two variables that are highly positively correlated; ignoring their correlation could result in combinations of values that are not practically feasible. Such model iterations can greatly distort the results of stochastic models, unless 42 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD the dependencies between variables are explicitly recognized and included in the modelling. These issues are further exemplified in Section 12.2. As always, the level of detail required in the farm model depends on the risk questions being addressed and specifically if on-farm control is relevant. This detail will relate to whether or not transmission of infection or contamination is included. For example, the model of Hartnett et al. (2001) considers transmission on farm, while the models of Cassin et al. (1998) and FSIS (2001) do not. Similarly, FAO/ WHO (2009c, 2009d) included on-farm modelling of Campylobacter infection in broiler chickens and transmission from fomites, contaminated water, other birds, etc. Conversely, FAO/WHO (2002a) were unable to usefully model pathways of transmission of Salmonella on farms. It must be remembered that animals and plants harvested for food may become infected/ contaminated from many sources including drinking/irrigation water, contaminated feed, vermin and feral animals, bird faeces, etc., or in the case of fish and especially shellfish from the water itself. Transport to processing plants Transport from primary production to processing can also be included in the exposure assessment. During transport, cross-contamination of primary production units can occur, or infection can spread between units in close proximity, and thus the overall microbial load can increase. In particular, stress of animals during transport can lead to increased faecal shedding and dissemination of pathogens to uninfected animals. Microbial loads on produce can also increase due to microbial growth during transport (Arthur et al., 2007; FSIS, 2001). Processing The stages in processing need to be defined before a model can be constructed to describe the changes in prevalence and in the number of organisms (see also comments above about the interplay between prevalence, unit size at a given contamination level). There can be many stages in food processing, though not all will necessarily have a strong effect on the ultimate risk to human health. Cassin et al. (1998), for example, identified 36 distinct processing steps during the slaughter of beef cattle. It is unlikely that all these stages will be followed by all processors, and an added difficulty is elaborating processing scenarios that are both representative of the majority of processors yet take into account differences between processors. Flow diagrams developed for HACCP systems can be useful sources of information on process steps and conditions. CHAPTER 5 - EXPOSURE ASSESSMENT 43 Modelling of processing involves considering how: • unit size changes from stage to stage and how this affects prevalence and concentration of organisms; • prevalence and concentration change as a result of cross-contamination, or recontamination after the application of a critical control point, without unit size changing; and • concentration changes due to microbial growth or inactivation. Much effort is expended during food processing operations to minimize microbial growth and/or to maximize microbial inactivation, and to prevent cross-contamination from other materials or the processing environment through cleaning and sanitation. For example, Dogan et al. (2019) evaluated the effectiveness of various interventions in processing plants to protect the safety of chicken consumers, through the development of quantitative exposure assessment models. Similarly, Smith et al. (2013) evaluated the relative effects of preharvest and processing interventions on public health risk for the consumption of ground beef and beef cuts contaminated with Escherichia coli O157:H7 in Canada. In contrast, mechanisms of recontamination of products in factory environments were discussed by den Aantrekker et al. (2003) and Tenenhaus-Aziza et al. (2014). When the process is not relevant to the decision, then detailed modelling is not needed. The reduction (or increase) in numbers is, thus, sometimes modelled using a black box approach whereby the changes are modelled without attempting to describe any of the underlying microbial processes. Important factors controlling the extent of growth and inactivation are the duration of conditions and severity of treatment (particularly temperature) prevailing during the process. Similarly, where changes are due to growth or inactivation, the effects of process duration and conditions on microbial numbers can be estimated using well-established predictive models (e.g. Tenenhaus-Aziza and Ellouze, 2015; Zwietering and Hasting, 1997a, 1997b); predictive microbiological models are discussed in detail in Chapter 12. Finally, the MPRM methodologies for mixing, partitioning and removal can be used to model the effects of changes to unit size (Bassett et al., 2010; Nauta, 2008). Studies of the effects of processing steps on the levels of microbiological hazards often report on the results of analysis of “before and after” samples, for example, the number of organisms contaminating a broiler carcase before and after defeathering. The results are often reported in terms of log10 concentrations. Caution is needed, however, when modelling contamination events and when the initial contamination levels are reported as log10 populations. For example, if a contamination event adds 1 000 organisms per unit (i.e. 3 log10) to a unit containing 100 organisms (i.e. 2 44 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD log10) it is incorrect to conclude 2 log10 + 3 log10 = 5 log10 (or 100 000 organisms per unit). The correct calculation involves converting the log counts to their arithmetic value and then adding the numbers, i.e. 100 + 1 000 = 1 100, which means the final contamination is 3.04 log10 organisms per unit, from the original 2 log10. This is because contamination is an additive process. In contrast, microbial growth is a multiplicative process because growth is exponential, i.e. where the increase is based on the initial number of organisms in the product and numbers change exponentially over time. In those cases, the log values can be added, e.g. 2 log10 initial plus 3 log10 growth = 5 log10 at the end of growth, because every cell initially present increased in number by 1000-fold. These are examples where errors would result in causal relationships that are incorrect and thus resulting in a flawed assessment; such errors should clearly be avoided. The variation and uncertainty associated with modelling the change in numbers should also be given careful consideration. When choosing the approach, thought should be given to what the data represent (variation, uncertainty or both) and how representative they are. For example, a problem with modelling the results of carcase samples is ensuring that the sampled portion is representative of the entire carcase. A potential remedy to this challenge is to estimate the magnitude of the bias (due to different carcase parts) in a separate study and include this in the model. A practical corollary of this is that if contamination on the carcases is unevenly distributed, then when the carcase is broken down into smaller pieces, not all will carry the same level of contamination. This is a good example of the consequence of partitioning and where contamination on each smaller unit may vary. Consequently, the prevalence and distribution of contamination levels on subunits would need to be described. During processing, formulation of products can be altered, and such alterations may change the potential for microbial growth. Examples include adding growth inhibiting compounds, such as salt or organic acids, to processed food; drying/water removal leading to reduction of water activity; acidification during fermentation; addition of water; etc. Similarly, packaging can affect the potential for microbial growth, inactivation, and cross-contamination. Thus, changes in the condition of the product over time must be modelled as part of exposure assessment. Processing often involves steps designed to reduce or eliminate microbial loads so that not only the expected magnitude of the reductions due to these steps, but also their uncertainty/variability, will need to be modelled. Also, if the initial contamination levels are low, and a typical unit size is small, then not all units will contain the hazard so that increased risk, in the absence of cross-contamination, can only come from growth in the units that do contain the hazard. CHAPTER 5 - EXPOSURE ASSESSMENT 45 Postprocessing The postprocessing environment includes storage and transport/distribution, retail display and handling, food service operations and home kitchens. These steps can allow microbial growth, crosscontamination, but also hazard reduction through cooking, physical removal of contamination, etc. Table 1 lists some of the factors of the postprocessing environment that could affect hazard frequency and level of exposure. While some of these environments may differ in some respects, there are often important similarities and some data collected in one environment may be suitable surrogates for assessing changes in exposure in other environments, e.g. cross-contamination from cutting boards. TABLE 1. Examples of factors of importance when determining the impact of the postprocessing environment on the level of exposure Factor Example Temperature Static (though variable) Refrigerated storage temperature Dynamic Cooling times and temperatures for cooked food Product formulation pH and water activity of the food, preservative compounds (sorbate, lactate, nitrite, nisin, etc.) Biotic factors in food Relative level of spoilage or other microorganisms on the (inter-species competition) product compared to pathogens, e.g. fermented food, lactic acid bacteria in vacuum-packed foods. Time Time on a salad bar, time between cleaning the blade of a processed meat slicer Cross-contamination Foods Transfer from chicken Surfaces Food contact surface Transfer from food to cutting board Hand contact surface Transfer from refrigerator door or utensils Cleaning (sponge, cloth) Survival on a sponge Hands Transfer from hands Bodily orifices From diarrhoea via hands, fomites Survival on surfaces Survival on stainless steel Cleaning Washing Effect of washing hands with soap and water for 20 seconds Sanitizing Effect of 200 ppm chlorine Discards Decision to use lunch meat beyond its use-by date 46 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Transport and storage postprocessing can include: • Transport from the processor to a food service establishment or retail outlet, possibly via a distribution centre, and subsequent storage; • Warehousing; • Retail storage and retail display; • Storage and handing in food service; and • Transport from retail to the home by the consumer and subsequent home storage. This type of transport and storage is likely to be less well controlled as most consumers do not have a refrigerated vehicle and frequent access to upright, domestic refrigerators means frequent loss of temperature control. Transport and storage conditions may also be less-well controlled in different regions. For example, in countries where street food vending is common, street vendors often lack the facilities for proper temperature, insect and vermin control. Similarly, farmers markets may pose challenges in terms of temperature control during transport, storage and retail (Young et al., 2017b). In general, relatively little information is available in the published literature on transport temperature and durations. With respect to transport between processor and retailers (or further processing), information on durations is likely known by the processors, indicating the need for good risk communication and involvement of stakeholders early in the risk assessment process. However, less is known about the temperature profile during transport, although the increasing availability of relatively cheap data loggers, possibly location enabled, are helping to remedy this situation (e.g. Sumner, 2016). Similarly, not many published research articles exist about retail or food service storage. An example of temperature data collection at retail is provided by Ecosure (2008), who collected data on cold temperature storage of products in various areas of retail stores (available in spreadsheet format). The FRISBEE project collected similar cold-chain data in Europe for various food products (FRISBEE project, 2021). Less is known about the treatment of food during transport to the home, likely related to the logistical difficulties of obtaining such data. Ecosure (2008), however, also collected data from consumer volunteers on transport to the home. The volunteers also reported how product was transported, the temperature in the part of the vehicle where product was located, the outside temperature, and time between purchase and placing each product into the refrigerator/freezer at home. Similarly, Kim et al. (2013) reported on temperature profiles of various food products during transport to the home. CHAPTER 5 - EXPOSURE ASSESSMENT 47 Using information about duration and temperature at each stage during post processing, predictive microbiology models may be used to predict the growth and inactivation of the hazard. Depending on the hazard and the durations involved, the effects of shelf life (limit on total duration between production and consumption) and competing and spoilage bacteria may need to be considered (see also Section 12.2). Cross-contamination Cross-contamination has been recognized as an important factor directly related to outbreaks of foodborne diseases and food spoilage and therefore may need to be included in the exposure assessments (Possas et al., 2017). Postprocessing environments can be more complex than processing environments because of: • the variety of foods involved – restaurant menus, for example, may have dozens of items, and a cafeteria may have hundreds; • the complexity of food preparation operations – nonlinear compared with food processing operations; • differences in preparation setting – home versus food service; • differences in the physical layout between operations – one kitchen versus another; • and level of training – new worker or home cook versus a highly experienced worker. The need to evaluate how microorganisms are transmitted along the food chain has motivated the study of other phenomena besides growth and death. The potential complexity involved in modelling cross-contamination during food preparation is shown in Figure 6 for the act of preparing a cooked chicken product and a lettuce salad. Despite its complexity, a number of simplifying assumptions are made in Figure 6. • The lettuce and the person preparing the food do not contribute any microbiological hazard to the exposure, except for cross-contamination originally arising from the chicken. • Hands and cutting board are the only cross-contamination vehicles, and other kitchen surfaces and utensils (knives, plates, sponges, towels, aprons, countertops, etc.) do not contribute to exposure. • No changes in microbial numbers occur, due to growth or inactivation, during any step except storage and cooking (e.g. bacterial populations on cutting board do not change). • The frequency at which each event occurs is not specified, and multiple contamination events may occur in any food preparation procedure. 48 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Initial Storage effect Raw chicken Cooked effect Cooked chicken Xcontam rate Xcontam rate Xcontam rate Hand Xcontam rate Xcontam rate Xcontam rate Cutting board Xcontam rate Lettuce Exposure FIGURE 6. An example ‘influence diagram’ of a model of a cross-contamination pathway for the preparation of cooked chicken and lettuce salad. (Xcontam = crosscontamination) Some of the simplifying assumptions listed above can be shown to be false in many situations, e.g. that no change in microbial numbers occur during any step except storage and cooking. Growth on contact surfaces does occur and may be important. Surfaces that become contaminated with films of nutrient-rich liquids from raw product may contain bacterial pathogens which could grow in the film. This surface is then replenished with new material from each subsequent unit and can promote cross-contamination to other units. Consider that a work–shift may be 4 to 8 hours in duration and that the working environment is maintained at 10-15 °C (such temperatures are maintained in some food processing operations because at lower temperatures workers became less dextrous and are more likely to have accidents and injuries). Based on estimates from published predictive models, pathogens could increase by 10- to 1000-fold in some products, e.g. Vibrio parahaemolyticus on fish and shellfish (100- to 1000-fold), Listeria monocytogenes on smoked fish (10-fold) and E. coli on raw meat (10-fold). The rate of potential growth on contact surfaces can be used to determine the maximum time interval between successive cleanings of equipment in contact with raw product. However, predicted increases may be quite different under processing settings where food products are moved on and off the preparation surface throughout the shift, each potentially depositing and/or removing some of the contamination. CHAPTER 5 - EXPOSURE ASSESSMENT 49 Another difficulty in populating the diagram in Figure 6 with real numbers and mathematical relationships is a lack of published data on many consumer storage and handling practices and on cross-contamination rates. The large uncertainty and variability associated with preparation and cooking practices has been recognized in national and international reports of exposure assessments. For example, the FAO/WHO exposure assessment models for Salmonella spp. and Campylobacter spp. in broilers suggest that cross-contamination during preparation and cooking can affect exposure (FAO, 2001; FAO and WHO, 2002a; WHO, 2001). However, despite the large number of studies reviewed by Pérez-Rodríguez et al. (2008) the authors concluded: The main objective and challenge when modelling bacterial transfer is to develop reliable mathematical models … However, with today’s knowledge, such models are a Utopia, since information is imprecise and scarce, and data show major experimental errors. The available cross-contamination modelling approaches in foods as well as the available evaluation methods for model robustness are provided by Possas et al. (2017). Given the limited amount of suitable data available for quantifying the effects of crosscontamination, most exposure assessments have considered this event in a simplistic manner. For example, such simplifications can be achieved by including a limited number of pathways, and by estimating both the probability of transfer and the numbers of organisms transferred (e.g. Hartnett, 2002). Other approaches have also been adopted, such as the Health Canada Campylobacter risk assessment, where the transfer of organisms in the drip fluid was also considered (Fazil et al., 1999). Schaffner (2004) modelled the cross-contamination of Listeria species using a quantitative mathematical model using Monte Carlo simulation techniques. Chen et al. (2001) quantified the probability of bacterial transfer associated with various steps in the food preparation process and provided a scientific basis to include cross-contamination in the exposure assessment with the aim to support risk management strategies to reduce or prevent cross-contamination in the kitchen. Zilelidou et al. (2015) evaluated the cross-contamination phenomena that might take place between cutting equipment and leafy vegetables in common households or in food preparation environments and provided quantitative data regarding the transfer rate of E. coli O157:H7 and L. monocytogenes from contaminated lettuce to kitchen knives and subsequent transmission to fresh lettuce. Other studies have 50 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD evaluated the cross-contamination rates of L. monocytogenes (Gallagher et al., 2016), Salmonella (Smid et al., 2013), Campylobacter (Hayama et al., 2011; Moore, Sheldon and Jaykus, 2003; Mylius, Nauta and Havelaar, 2007), and E. coli O157:H7 (Jensen et al., 2015; Pérez-Rodríguez et al., 2011). In summary, postprocess food preparation is a highly complex, and poorly characterized, part of the food chain. Limited data are available, and numerous data gaps have been identified. Given the complexity of this part of the food chain, research to better understand and describe these processes is ongoing. Publication of the results of that research will contribute to improved exposure assessment where cross-contamination may be an important route of exposure. However, cross-contamination is initially a redistribution process and, unless that redistribution alters the fate of the hazard, that is, either due to growth or reduction the benefits of cross-contamination modelling should be carefully considered. 5.2.5 Consumption To characterize the risk from exposure to microbiological hazards in food, it is necessary to know the amount of food consumed per meal, how often it is consumed, the form in which it is consumed (raw or cooked), and by whom because susceptibility is variable and some groups (e.g. very old, very young) are more likely to develop illness from foodborne hazards. The specific characterization of food consumption patterns used in the MRA depends on the question to be answered, and the food consumption data that are available to the risk assessor (see also Chapter 10). The data collated and published by WHO through the Global Environment Monitoring System (GEMS) cluster diets may be useful when no other data are available (WHO, 2021). However, care needs to be taken, as for any consumption data, to ensure correct interpretation (see below). Modelling the amount of food consumed When modelling food consumption, it is important for risk assessors to understand the specifics of how the food consumption data were collected and analysed. It is also necessary to clearly describe how these data are used in the model, including any assumptions used in arriving at the estimates. CHAPTER 5 - EXPOSURE ASSESSMENT 51 The important aspects of calculating the amount of food consumed, particularly when using results from food consumption surveys, include: • the population divisor, that is, whether the total consumption amount is divided by the total population (amount per capita) or only those who consumed the food (amount per consumer); • the frequency of consumption (per day/week/month/year); and • the amount consumed per consumption event. These are discussed below. Amount per capita vs per consumer The per capita amount is calculated by dividing the total amount of a food by the total number of people in the population. The per consumer amount is calculated by dividing the total amount of food only by the number of people who consumed the food. For foods that are consumed regularly by most of the population (e.g. bread), the per capita and per consumer amounts will be nearly equal. For foods that are consumed by fewer individuals (e.g. raw oysters), the per capita and per consumer amounts will be quite different. For example, consider that 10 million kg of a food are consumed by 10 percent of the population, which consists of 10 million people. The average consumption per capita equals 1 kg, while the average consumption per consumer equals 10 kg. Amount per year, per day or per eating occasion Consumption may be calculated as the amount per time period (e.g. year, month, week or day) or per eating occasion. Definition of the consumption period is particularly important in MRAs because acute, rather than chronic, exposure is of concern. In contrast, chronic exposure may be relevant for some microbial toxins that are released into foods before consumption, e.g. mycotoxins, and in such situations chemical risk assessment approaches are appropriate (e.g. see FAO and WHO, 2009e). For microbial toxins that cause acute illness, such as Staphylococcus aureus enterotoxin, the dose of microbial toxins is stoichiometrically related to the level of contamination of the food by the toxigenic organism and microbial risk assessment approaches are generally appropriate. National food production statistics (e.g. FAOSTAT) (FAO, 2021c) generally report an amount of food produced per year, and care is needed to fully understand the 52 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD values. For example, if the amount of fish caught is reported, does the amount relate to whole fish landed, or does it relate to the amount after gilling and gutting? Clearly, amounts ultimately consumed need to be adjusted to remove inedible parts of the food and any losses incurred during processing. Similarly, food wastage in the supply chain due to spoilage or other reasons needs to be accounted for if possible. For highly perishable products (meat, fish, fruits, salad vegetables, etc.), this may be as high as 20 to 25 percent of production (Gustavsson, Cederberg and Sonesson, 2011). A consumption amount may be estimated by dividing the total annual amount (per capita or per consumer) by the average number of eating occasions. Returning to the example above, if the food product is thought to be consumed daily then the average amount would equal 10 kg divided by 365 days, or about 27.4 g per day. This amount may be too small to be realistic and hence the data and assumptions for the calculations may need be reassessed and adjusted, if necessary. For example, it may be that a typical amount consumed in a meal is closer to 100 g and hence this would imply that the food is consumed about 100 times per year, or approximately once every 3-4 days. Meal size and consumption data may be available from surveys for some countries. Food consumption surveys of individuals allow much more flexibility in estimating the consumption amount. Survey results are frequently summarized and reported on the basis of daily consumption. If the raw data from the survey are available, then it may also be possible to calculate the amount of food consumed per eating occasion (depends on coding system and questions in the questionnaire) and the frequency of consumption. The basis for consumption is particularly important when considering foods that may be consumed more than once in a single day. For example, if a person drinks a 250-ml glass of milk at each of three meals, the amount per meal would be 250 ml, whereas the amount per day would be 750 ml. When calculating daily food consumption from food consumption survey data, it is also important to note whether the amount was calculated as an average over all days of the survey or for only the days on which a food was consumed. As an example, consider a study where five days of dietary records were collected for individuals participating in the survey. From those data, consumption could be calculated as consumption on the days the food was actually consumed or as the average, or total, over five days for which each person participated in the survey. Of course, a portion size can vary from meal to meal and different people will consume different amounts per meal (on average), e.g. young children or the elderly might have smaller portion sizes than young adults. In this case, serving size can be modelled as a distribution, if the data are available. In general, all other CHAPTER 5 - EXPOSURE ASSESSMENT 53 things being equal, larger serving sizes would be correlated with slightly higher risk of illness. If there is a correlation between serving size and particular consumer characteristics, these correlations can also be modelled to reflect the differential risk to different consumers. In addition, when the total exposure from several foods is assessed it may be necessary to consider correlations between the respective serving size. Examples might include consumption of apples in the form of raw apples, apple juice and apple pies, or consumption of cheeses in combination with deli meats. However, accessing data that allows quantification of such correlations will most likely be challenging. Importance of characterizing the distribution of contamination The risk to an individual depends on the dose ingested which, in turn, depends on the serving size. The importance of modelling the physical distribution of the number of organisms in a food, i.e. the dose, will depend on the dose–response relationship for that organism. If a high level of growth occurs in a single unit of food prior to consumption, only one person is likely to be affected because that single unit of food will be consumed by one person. Assuming that there are more than enough cells of the hazard present to cause infection in most individuals, if that same dose were spread equally over 100 servings, then the same dose might be enough to infect many of the 100 consumers, assuming a pathogen with a high probability of infection per infectious particle, e.g. norovirus (Teunis et al., 2008). Conversely, for a pathogen with a very low probability of infection per cell, e.g. L. monocytogenes (see Table 7), the predicted risk to the entire population from the exposure is largely independent of the distribution of doses among units of food and is effectively estimated from the average dose. This is because there is, effectively, a direct proportionality between the dose and probability of infection for all realistic doses (see Chapter 6) and for those realistic doses the probability of infection is much less than one. In this situation, there is less need to characterize the distribution of the pathogen among different servings. Nauta (2000) provides advice on modelling distribution among individual servings. This is relevant for the physical distribution of doses, but also if we consider the statistical variability of the doses. To realistically determine the population risk, variability in doses should be included since the risk is often determined by the right tail of the distribution. These extremes are affected by the distribution in the initial concentration, the effects of processing and variability of all other factors in the exposure assessment. Consumption frequency The frequency of consumption refers to how often an individual consumes a food in a specific period. In MRAs (e.g. FAO and WHO, 2002a; FSIS, 2001; USFDA, 54 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 2005; USFDA/FSIS, 2003), frequency of consumption has been expressed in a variety of ways: • Number of days per year on which the food is consumed. • Number of eating occasions over a year: ■ annual number of meals, ■ number of times the food is consumed per year, or ■ number of 100-g portions consumed in a year. For a food consumption survey, the number of days of consumption during the survey period can be determined directly from the survey results; from that, an annual number of days of consumption may be extrapolated. The number of meals, eating occasions or individual food items may also be calculated directly from the survey results, if the survey covers more than one day per individual. Alternatively, data from single 24-hour recall surveys can be combined with information from food frequency surveys on the proportion of the population who usually consume a food in a given period to estimate the annual number of consumption days. It may be possible to refine or verify the estimated frequency of consumption by combining food consumption data with other industry information, such as annual sales volume or market share information (Chapter 10). For example, if the food consumption data report the frequency of consumption of a broad category such as cheese, market share or loyalty card data may be used to predict the frequency of consuming a particular type of cheese (e.g. Camembert). Note that it might be reasonable to assume that the amount of cheese consumed is similar across types of cheese although the frequency differs by cheese type. As noted above, consideration should be given to the proportion of production that is never consumed due to spoilage, not sold by specified use-by or best-before date, or due to other forms of wastage. A useful reality check is to combine food consumption amounts with frequency of consumption, and number of consumers to calculate approximate production volumes, taking into account wastage, imports and exports, etc. These estimates should be comparable to actual production volumes and big discrepancies may indicate that some of the estimates or assumptions are not valid. Considerations and challenges in modelling food consumption There are a number of aspects of food consumption data that should be considered when developing the food consumption model. Extrapolating data from results of food consumption surveys Food consumption surveys generally collect information from a subset of the CHAPTER 5 - EXPOSURE ASSESSMENT 55 population (e.g. van Rossum et al., 2011). If the sample is representative of the total population and statistical weights developed for the survey are used in the data analyses, survey results may be used to predict food consumption patterns for the population as a whole. For MRAs, it may be important to estimate the consumption for sensitive population groups, such as the elderly or the immunocompromised. In the absence of specific data for these groups, it is often assumed, if appropriate, that their consumption patterns are the same as the normal, healthy population of the same age and gender. Infrequently consumed foods Estimates of consumption based on a small number of observations, i.e. small number of food consumption records, will be more uncertain than estimates based on larger samples. For this reason, care should be taken when interpreting and extrapolating survey results for infrequently consumed foods, even if the overall survey size was large and survey weights are used in the data analysis. If the survey data are used to model consumption for an infrequently consumed food, it is important that the consumption amount be calculated from the day or eating occasion on which the food was consumed, rather than as the average over all survey days. Food consumed as discrete items vs components of mixed dishes Some foods may be consumed both as discrete items and as components of combination foods or food mixtures. For example, milk may be consumed as a beverage, but also as an ingredient (often in small amounts) in many food items. The normal usage of those foods can also affect hazard levels, e.g. milk consumed in meals may be heated which could reduce pathogen numbers compared to milk consumed as part of a cold milk drink. When modelling food consumption, it is important to know whether the consumption estimate includes all sources of the food or only the amount of food consumed as a discrete item. If the consumption estimate includes consumption of the food from all sources, it may be necessary to consider the recipes for foods containing that ingredient. This will not only allow estimation of the total consumption from all sources, but also the form in which the food is eaten, including the effects on the hazard (if any) due to food preparation. Similarly, it may be necessary to estimate the proportion of the total consumption in which the hazard could be present, such as unpasteurized juice or milk, or hot dogs eaten without reheating. As another example of the effect of mixing and partitioning, while consumption data for shell eggs may indicate that a person eats 60 g of shell egg per day, in some situations the serving may have been made from 56 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD many eggs combined, such as scrambled eggs in an institutional setting. In such a case, many consumers might be exposed to a single contaminated egg compared to another situation where a single consumer eats the entire contaminated egg. Aggregation or grouping of foods If the risk assessment is focused on food groups rather than individual foods, then consider the way in which foods are aggregated for estimating consumption. The average consumption amount for a food category is affected by the number of foods it represents and how similar the foods are in terms of the usual amount and frequency of consumption. If the foods are too dissimilar, the average amount and frequency of consumption may be misrepresented. For example, if fluid milk and cheese are grouped together as ‘dairy products’, the consumption amounts may be quite different, and the average consumption will likely underestimate consumption of milk and overestimate consumption of cheese. This is because fluid milk is generally consumed more frequently and in greater amounts than cheese. Again, if a food category includes seasonal items as well as foods that are available year-round, the frequency of consumption may be under- or overestimated for the seasonal foods. Some consumption surveys do, however, identify seasonal effects, e.g. by sampling individuals at many times throughout the year. CHAPTER 5 - EXPOSURE ASSESSMENT 57 6 6. Hazard characterization 6.1 THE PROCESS OF HAZARD CHARACTERIZATION Codex defines hazard characterization as “the qualitative and/or quantitative evaluation of the nature of the adverse health effects associated with biological, chemical and physical agents, which may be present in food” (CAC, 1999). Hence, the hazard characterization provides a description of the adverse effects that may result from ingestion of a hazard, whether that is a microorganism or its toxin. Where possible the hazard characterization should include an indication, for the population of interest, of the probability to cause an adverse health effect as a function of dose. This would ideally take the form a dose–response relationship, if available, or using the Median Dose or Infectious Dose 50 (ID50), the dose at which 50 percent of consumers become infected, or ill (see Section 6.3 for details). The hazard characterization may also include identification of different adverse effects for different subpopulations, such as neonates or immunocompromised people. Hazard characterization can be conducted as stand-alone process or as component of risk assessment. A hazard characterization for a particular hazard may serve as a common module or building block for risk assessments conducted for a variety of purposes and in an assortment of commodities. A hazard characterization developed in one country may serve the needs of risk managers in another country when combined with an exposure assessment specific to that country, unless there are countryspecific population effects. A hazard characterization developed for one specific food product may be adapted to another food product by taking into consideration 58 the food matrix effects, where possible. In general, hazard characterizations are adaptable between risk assessments for the same hazard. This is because the human responses to a specific hazard are not considered to be based on geography or culture. Instead, they are about the interaction between the hazard and the host only, recognizing that some hosts will be more susceptible than others. Similar to other parts of risk assessment, hazard characterization can be iterative. For well-established hazards, such as Campylobacter or L. monocytogenes, the hazard characterizations tend to be well developed and may not require much revision unless considerable new information is available. However, for emerging hazards the hazard characterization may be less certain due to lack of data and information, and thus may require more frequent updating to reflect the increasing knowledge about the hazard. Characterization of hazards in food and water follow a structured, step-wise approach, as outlined in Figure 7 and described below. Data collection and evaluation Descriptive characterization Dose-response modelling Review FIGURE 7. Process flow diagram for hazard characterization of pathogens 6.2 DESCRIPTIVE CHARACTERIZATION Descriptive hazard characterization serves to structure and present the available information on the spectrum of human illness associated with a particular hazard, and how this is affected by the characteristics of the host, the hazard and the matrix, as indicated in Chapter 4. This is based on a qualitative or semi-quantitative analysis of the available evidence and will take the different illness mechanisms into account. CHAPTER 6 - HAZARD CHARACTERIZATION 59 6.2.1 Information related to the disease process When a hazard characterization is being undertaken, one of the initial activities will be to evaluate the weight of evidence for adverse health effects in humans to determine, or confirm, the ability of the hazard to cause disease. The weight of evidence is assessed based on causality inferences appropriately drawn from all available data. This entails examination of the quantity, quality and nature of the results available from clinical, experimental and epidemiological studies; analyses of hazard characteristics; and information on the biological mechanisms involved. When extrapolating from animal or in vitro studies, the biological mechanisms involved need to be considered to ensure they are relevant to humans. When undertaking hazard characterization for waterborne and foodborne microbial hazards, the biological aspects of the disease process should be considered. Each of these steps is composed of many biological events. Careful attention should be given to the following general points. • The process as a whole, as well as each of the component steps, will vary by the nature of the hazard. • Hazards may be grouped in regard to one or more component steps, but this should be done cautiously and transparently. • The probability of an event at each step may, or may not, depend on other steps. • The sequence and timing of events are important. For (toxico-)infectious hazards, it is recommended that the factors related to infection, and those related to illness as a consequence of infection (discussed in Section 13.1), are considered separately. While doing so, the following points should be considered when evaluating the available evidence. • The definition of infection may differ between studies, i.e. is not universally accepted. • Infection can be measured dichotomously (yes or no), but some aspects can be measured quantitatively. • Detecting/measuring infection depends on the sensitivity of diagnostic assay. • Target cells or tissue may be specific (one cell type) or nonspecific (many cell types), and local (noninvasive) or invasive or systemic, or a combination. • The sequence of events and the time required for each may be important and may vary according to the hazard. 60 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The information related to the disease should provide detailed insights into the disease process, either qualitatively and/or quantitatively. In most cases, this would be based on the available clinical and epidemiological studies. Narrative statements are helpful to summarize the nature of and confidence in the evidence, based on limitations and strengths of the data. Each source of information has its advantages and limitations, but collectively they permit characterization of potential adverse health effects. The analysis should include evaluations of the statistical characteristics of the studies, and appropriate control of possible bias, while identifying uncertainties, their sources, and their effects. Characterization of the adverse human health effects should consider the whole spectrum of possible effects in response to the hazard, including asymptomatic infections and clinical manifestations, whether acute, subacute or chronic (e.g. long-term sequelae (Carbone, Luftig and Buckley, 2005)), or intermittent (see Table 2). Where clinical manifestations are concerned, the description would include consideration of the diverse clinical forms, together with their severity, which may be variable among strains and among hosts infected with the same strain. Severity may be defined as the degree or extent of clinical disease produced by a hazard, and may be expressed in a variety of ways, most of which include consideration of possible outcomes. For mild gastrointestinal symptoms, severity may be expressed as duration of the illness, or as the proportion of the population affected (morbidity). Where the symptoms require medical care or result in longterm illness, or both, severity may be expressed in terms of the costs to society, such as the proportion of workdays lost or cost of treatment. Some hazards and the related clinical forms may be associated with a certain degree of mortality and therefore severity may be expressed as mortality rate (e.g. Vibrio vulnificus infections and L. monocytogenes infections). Some hazards cause chronic illness, that is, the disease leaves long-term sequelae (Carbone, Luftig and Buckley, 2005), e.g. foodborne trematode infections. For these it may be desirable to consider and include the effects on quality of life as a result of the disease. Quality of life may be expressed in a variety of ways, depending on the nature of the illness. For instance, human life expectancy may decrease, chronic debilitation may occur, or quality of life may be affected by episodic bouts of disease. Increasingly, concepts such as Quality-Adjusted Life Year (QALY) or Disability-Adjusted Life Year (DALY), discussed further in Section 7.4.2, are being used to integrate and quantify the effects of different disease endpoints on the health of individuals or populations (Batz, Hoffmann and Morris, 2014; e.g. Havelaar et al., 2000; WHO, 2000, 2015). CHAPTER 6 - HAZARD CHARACTERIZATION 61 TABLE 2. Elements that might be included in characterization of adverse human health effects (Adapted from ILSI, 2000) Clinical forms Duration of illness Severity (morbidity, mortality, sequelae) Pathophysiology Epidemiological pattern Secondary transmission Quality of life In addition to a description of the human adverse health effects, information on the disease should include consideration of the epidemiological pattern and indicate whether the disease may be sporadic, endemic or epidemic. The frequency or incidence of the disease or its clinical forms, or both, should be addressed, together with their evolution with time and possible seasonal variations. The description should include consideration of the repartitioning of clinical forms according to specific groups at risk. Finally, the potential for, extent of or amount of transmission, including asymptomatic carriers, and secondary transmission, should also be characterized. Information collected on these aspects is important to guide the risk characterization. In all cases, and with particular regard to further modelling, it is important that the characterization includes a definition of possible endpoints to be considered. Thought needs to be given to the appropriate criteria when defining infection of the host by the hazard, and the criteria of what constitutes a clinical case. In addition, a definition of the severity scale should be provided, specifying the indicator chosen (e.g. disease endpoint or consequences) and how it can be measured. The description should also include information on uncertainties and their sources. To the extent possible, the characterization should incorporate information on the pathophysiology of the disease, i.e. on the biological mechanisms involved. Depending on the information available, this would include consideration of elements such as: • the entrance route(s) of a hazard into a host; • the effect of growth conditions on expression of virulence by and survival mechanisms of the hazard; • the effect of the conditions of ingestion, including matrix effects; • the effect of gastrointestinal status; 62 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD • the mechanisms involved in the penetration of the hazard into tissues and cells; • the status of the hazard relative to nonspecific (innate) and cell-mediated immunity; • the status of the hazard relative to humoral defences; • the effect of intercurrent illnesses and treatments, such as immunosuppressive or antimicrobial therapy; • the potential for natural elimination; and • the behaviour of the hazard in a host and its cells. The natural history of the disease needs to be completed by specific consideration of factors related to the hazard, the host and the food matrix, insofar as they may affect development of health effects, their frequency and severity. 6.2.2 Information related to the hazard Basically, this information is analysed with a view to determining the characteristics of the hazard that affect its ability to cause disease in the host via transmission in food. The analysis needs to consider the biological nature of the hazard as well as the relevant mechanisms that cause illness (infectious, toxico-infectious, toxigenic, invasive or not, immune-mediated illness, etc.). In principle, the descriptive hazard characterization is applicable to all types of hazards and all associated illnesses. In practice, by nature of the data collected, the focus will be on acute effects, associated with single exposures rather than long-term effects associated with chronic exposure. Note that the possible interaction between repeated exposures, e.g. the development of acquired immunity, is an integral part of the descriptive characterization. The ability of a hazard to cause disease is affected by many factors (Table 3). Some of these factors relate to the intrinsic properties of the hazard, such as phenotypic and genetic characteristics that affect virulence and pathogenicity, and host specificity. The characteristics of the hazard that determine its ability to survive and multiply in food and water, based on its resistance to processing conditions, are critical components of both exposure assessment and hazard characterization. Ecology, strain variation, infection mechanisms and potential for secondary transmission may also be considered, depending on the biology of the microorganism and on the context of the hazard characterization, such as the scenario that has been delineated during the problem formulation stage of a full risk assessment. CHAPTER 6 - HAZARD CHARACTERIZATION 63 TABLE 3. Elements to consider in characterization of the hazard (Adapted from ILSI, 2000) Intrinsic properties of the hazard (phenotypic and genetic characteristics) Virulence and pathogenicity mechanisms Pathological characteristics and disease caused Host specificity Infection mechanisms and portals of entry Potential for secondary spread Strain variability Antimicrobial resistance and its effect on severity of disease If not already included, then specific consideration should be given to the intrinsic properties of the hazard that affect infectivity, virulence and pathogenicity; their variability; and the factors that may alter the infectivity, virulence or pathogenicity. As a minimum, elements to addressed, as best as possible, are summarized in Table 3. 6.2.3 Information related to the host Host-related factors are the characteristics of the potentially exposed population that may affect susceptibility to the particular hazard. These should take into account host intrinsic and acquired traits that modify the likelihood of infection or, most importantly, the probability of illness and its severity. There are many pre-existing (innate) host barriers, though they are not all equally effective against hazards. Each barrier component may have a range of effects depending on the hazard, and many factors may affect susceptibility and severity. These are identified in Table 4. TABLE 4. Factors related to the host that may affect susceptibility and severity (Adapted from ILSI, 2000) Age General health status, stress Immune status Underlying conditions, concurrent or recent infections Genetic background Use of medications Pertinent surgical procedures Pregnancy Breakdown of physiological barriers Nutritional status, bodyweight Demographic, social, and behavioural traits 64 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Not all of the factors listed in Table 4 are relevant, or important, for all hazards. In all cases, however, an important issue in hazard characterization is to provide information on whom is at risk and on the stratification of the exposed population for relevant factors that affect susceptibility and severity. 6.2.4 Information related to the food matrix The factors related to the food matrix are principally those that may affect the survival of the hazard through the hostile environment of the stomach. Such effects may be induced by protection of the hazard against physiological challenges, such as gastric acid or bile salts. These are related to the composition and structure of the food matrix, e.g. highly buffered foods and entrapment of bacteria in lipid droplets. Alternatively, the conditions in the food matrix may phenotypically affect the ability of the hazard to survive the host barriers. Examples include increased acid tolerance of bacteria following pre-exposure to moderately acidic conditions, or induction of stress–response by starvation in the environment. Stress conditions encountered during the processing or distribution of food and water may alter a hazard’s inherent virulence and its ability to resist the body’s defence mechanisms. These potential matrix effects can be important elements in hazard characterization. The conditions of ingestion may also affect survival by altering the contact time between hazard and barriers, e.g. initial rapid transit of liquids in an empty stomach. These factors are summarized in Table 5. TABLE 5. Elements to consider in characterization of the effect of the food matrix on the hazard-host relationship Protection of the hazard against physiological barriers, e.g. fatty foods, ingestion of pathogen in, or after, ingesting a large volume of fluid Induction of stress response Effects on transport of hazard through the gastrointestinal tract 6.2.5 Relationship between the dose and the response The final, and essential, element in the descriptive hazard characterization is the relationship, if any, between the ingested dose, infection and the manifestation and magnitude of health effects in exposed individuals. Specific modelling aspects are covered in Sections 6.3 and Chapter 11. Description of the dose–response relationship involves consideration of the elements or factors related to the hazard, the host and the matrix, insofar as they may modulate the response to exposure. Where appropriate information is CHAPTER 6 - HAZARD CHARACTERIZATION 65 available, it also involves a discussion about the biological mechanisms involved, in particular whether synergistic action of hazards, may be a plausible mechanism for any harmful effect, or whether a single hazard may cause adverse effects under certain circumstances. Elements to consider are listed in Table 6. TABLE 6. Elements to consider in describing the dose–response relationship (Adapted from ILSI, 2000) Organism type and strain Route of exposure Level of exposure (the dose) Adverse effect considered (the response) Characteristics of the exposed population Duration – multiplicity of exposure Where clinical or epidemiological data are available, discussion of the dose– response relationship will generally be based on such data. However, the quality and quantity of data available will affect the characterization. The strengths and limitations of the different types of data are addressed in Chapter 10. A specific difficulty is obtaining data to characterize infection, the translation of infection into illness, and illness into different outcomes. In many cases, the analysis may only be able to describe a relationship between a dose and clinical illness. Other difficulties arise from several sources of variability, including variation in virulence and pathogenicity of the microorganisms; variation in attack rates; variation in host susceptibility; and type of matrix, which modulates the ability of hazards to affect the host. Therefore, it is essential that the dose–response analysis clearly identify what information has been utilized and how the information was obtained. In addition, the variability should be clearly acknowledged and the uncertainties and their sources, such as insufficient experimental data, should be thoroughly described. In cases where a dose–response model cannot be ascertained or is not needed, such as for a qualitative MRA, an indication of the likely dose required to cause a certain probability of infection/illness should still be given. In particular, the dose that results in infection/illness in 50 percent of exposed consumers – often referred to the ID50 or median dose – may be a simple, yet practical, indicator. However, such a dose should not be interpreted as a threshold or minimal infective dose (see box below). For example, some hazards are highly infective and only a very small dose is required, such as for norovirus, for which it has been estimated that the ID50 66 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD may be as low as 18 viruses (Teunis et al., 2008). For other hazards a larger dose is required to cause 50 percent illness, as is the likely case with L. monocytogenes in the general population (FAO and WHO, 2004; Buchanan et al., 2017). It should be recognized that for many organisms a very low dose may cause illness, even though the probability of this happening may be very low. However, often the exposure distributions, i.e. distribution of doses, is highly right-skewed and so most exposures occur at (very) low doses. As a result, these low doses, together with a small probability of illness may still represent a large number of illnesses in a population; such exposures are consistent with the concept of sporadic illness. 6.3 QUANTIFYING THE DOSE–RESPONSE RELATIONSHIP Illness can be the result of intoxication, toxico-infection or infection. In the first case the illness is the result of ingestion of toxins which are preformed in the food. The health risks of certain toxins, e.g. cyanobacterial toxins in water or aflatoxins in foods, usually relate to repeated exposures and hence tend to be chronic; these require another approach, which resembles hazard characterization of chemicals. Other toxins have more acute effects like botulinum toxin, S. aureus enterotoxin or Bacillus cereus cereulide. In toxico-infection organisms produce toxins in the intestines that either produce adverse effects there, or are transported and create effects in other places in the human body. For infections the organisms invade human cells, either in the intestine or elsewhere in the human body. To determine the probability of adverse effects, a dose–response relation is needed to translate the doses resulting from exposure assessment. For this, a mathematical model is needed, as well as the value(s) of its parameter(s), including variability and uncertainty where possible. Attention should be paid to the following aspects. • The dose ingested is characterized by the multiplication of the concentration and the amount of food (or water) ingested (that are both variable). • The definition of the response(s), e.g. infection, disease, sequelae. • The specific model used, e.g. exponential, Beta–Poisson. • The set of parameters, including their variability and uncertainty, potentially relevant for a specific population group and/or food commodity and/or organism subgroup. CHAPTER 6 - HAZARD CHARACTERIZATION 67 BOX 1 The Minimal Infective Dose model posits that there is a dose below which there is no infection and hence no risk, and above which infection always occurs. However, such models are now considered invalid. Microbial dose–response models today are based on the single-hit assumption, i.e. each individual cell has a discrete, nonzero probability of establishing infection. Models based on this assumption can be found in numerous peer-reviewed papers and are also recommended in the WHO/FAO Guidelines for Hazard Characterization of Pathogens in Water and Food (FAO and WHO, 2003). Therefore, the Minimal Infective Dose concept, the phrases “minimal infective dose,” “infectious dose,” or statements like “the dose–response is between 104 and 105 cells” should not be used. It is appropriate to use an infectious dose for a certain (quantitative) response like ID50 or ID10, representing the dose at which 50 or 10 percent, respectively, of those exposed get infected. This concept holds true for toxico-infectious and infectious organisms. Sometimes the ID50 is used or interpreted as a threshold of infection; however, such an interpretation is incorrect and should be avoided. A minimal toxic dose might exist for illness cause by food containing preformed toxins (e.g. staphylococcal enterotoxins), where there is a level below which there is no observable response. Plots of empirical datasets relating the response of a group of exposed individuals to the dose (often expressed on a logarithmic scale) frequently show a sigmoid shape (Figure 8, left) and a large number of mathematical functions can be used to model such dose–response relationship (Haas, Rose and Gerba, 2014; Teunis, 1997). It is important to also investigate this curve on the log-log scales, since the “low exposure” (X-axis) and “low probability” (Y-axis) part of the relationship (Figure 8, right) is often particularly relevant (Williams, Ebel and Vose, 2011a) as explained at the end of Section 6.2.5. It should be noted that the uncertainty bounds appear different in width when viewed on the log-log scale compared with the linear scale, though this is simply the result of the mathematical transformation. When extrapolating outside the region of observed data, different models may predict widely differing results (Coleman and Marks, 1998; Holcomb et al., 1999). It is therefore necessary to select between the many possible dose–response functions and justify the decision. In setting out to generate a dose–response model, the biological aspects of the hazard–host–matrix interaction should be considered carefully (Teunis, 1997). For some dose–response models, some of the well-established models and parameter values may be appropriate (see Table 7). In those cases, relevant assumptions need to be evaluated. It could also be decided to extend the dose–response relation with additional data or derive a fully new dose–response model. Guidance is provided in Chapter 13 for deriving new or updating existing dose–response models. 68 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Probability of illness log10 (Probability of illness) 1 0 0.9 0.8 -0.5 0.7 -1 0.6 0.5 -1.5 0.4 0.3 -2 0.2 -2.5 0.1 0 -3 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 log10(Dose) log10(Dose) FIGURE 8. Example Salmonella dose–response model, including expected response (solid line), approximate 2.5th and 97.5th uncertainty percentile lines (dashed) and upper and lower uncertainty bounds (dotted) (FAO and WHO, 2002a p. 87) on linear-log scale (left) and on log-log scale (right) TABLE 7. Dose–response models and parameter estimates commonly used in QMRA Organism Reference Model Salmonella spp. FAO/WHO Beta– (2002a) Poisson Parameters α=0.1324 β=51.43 Lower bound Upper bound (Percentile) (Percentile) 0.0940 (2.5th) 0.1817 (97.5th) 43.75 (2.5th) 56.39 (97.5th) Listeria monocytogenes a FAO/WHO (2004) Exponential (susceptible) Exponential (healthy) r=1.06×10-12 r=2.37×10-14 2.47×10-13 9.32×10-12 (95th) (5th) 2.70×10-13 (95th) 3.55×10-15 (5th) Campylobacter spp.b Shigella dysenteriae/ E. coli O157 FAO/WHO Beta– (2009d) Poisson Cassin et Beta– al. (1998) binomial α=0.21 β=59.95 α=0.267 β=Lognormal (5.435, 2.472) Vibrio vulnificus FAO/WHO (2005) α=9.3×10-6 β=110 000 a For L. monocytogenes, newer animal model data (Roulo et al., 2014; Smith et al., 2003, 2008; Williams et al., 2007, 2009) and outbreak data (Pouillot et al., 2016) suggest much higher r-values and hence lower ID50 values than predicted by this model which was based on the method of Buchanan et al. (1997) of matching expected loads of L. monocytogenes across the food supply to the total annual cases in a community, and which relies on many untested assumptions. b The dose–response relation is for infection. The conditional probability of disease following infection was 33 percent (29/89) and can be described by a beta(30,61) distribution. CHAPTER 6 - HAZARD CHARACTERIZATION 69 7 7. Risk characterization 7.1 THE PROCESS OF RISK CHARACTERIZATION Codex defines risk characterization as … the process of determining the qualitative and/or quantitative estimation, including attendant uncertainties, of the probability of occurrence and severity of known or potential adverse health effects in a given population based on hazard identification, hazard characterization and exposure assessment (CAC, 1999). Hence, the risk characterization integrates the findings from those three components (see Figure 2) to estimate levels of risk, which can subsequently be used to make appropriate risk management decisions. Risk characterization is the final step in the risk assessment process (Figure 2), which is initiated by risk managers who pose specific questions to be answered by the risk assessment. As noted previously, the questions posed by risk managers are usually revised and refined in an iterative process of discovery, discernment and negotiation with risk assessors. Once answered, the risk managers have the best available science-based information they need to support their decision-making process. Risk characterization is the risk assessment step in which the risk managers’ questions are directly addressed. While risk characterization is the process, the 70 result of the process is the risk estimate. The risk characterization can often include one or more estimates of risk, risk descriptions, and evaluations of risk management options. Those estimates may include economic and other evaluations in addition to estimates of risk attributable to the management options. Although the Codex risk assessment framework is a common context for undertaking risk characterization, it is by no means the only context. In actual practice, an assessment of the risk may include some or all of these steps. The scientific analyses comprising any one of these steps may be sufficient on their own for decision-making. Risk assessments can follow a bottomup or topdown approach. A bottom-up approach links knowledge about the prevalence and concentration of a hazard in a food source with knowledge about the causal pathways, transmission routes and dose–response relations. Alternatively, topdown approaches use observational epidemiological information to assess risk, typically making use of statistical regression models (Williams, Ebel and Vose, 2011b). Also, models exist that use elements from both approaches, e.g. for source attribution. These approaches have different starting points, use different types of data and serve different purposes. For example, in Denmark (Hald et al., 2004) and the United States of America (Guo et al., 2011), the number of human cases of salmonellosis attributed to different animal sources was estimated without a precise exposure assessment and without using a dose–response model. A further example is provided by De Knegt et al. (2015). Bottomup and topdown MRA approaches have been published on aiding risk managers in the use of risk metrics, such as the Appropriate Level of Protection (ALOP) and Food Safety Objective (FSO) with case studies using L. monocytogenes in deli meats (Gkogka et al., 2013a) and Salmonella spp. in raw chicken meat (Gkogka et al., 2013b). 7.2 QUALITATIVE RISK CHARACTERIZATION IN RISK ASSESSMENT 7.2.1 Introduction The risk characterization generated as part of a qualitative risk assessment will ideally be based in numerical data for exposure assessment and hazard characterization. Nevertheless, it will generally be of a descriptive or categorical nature that is not directly tied to a more precisely quantified measure of risk (e.g. CFIA, 2019 Section 3.4.1). Qualitative risk assessments are commonly used for screening risks to determine whether they merit further investigation and can be useful in the preliminary risk management activities described in (FAO and WHO, 2002b), but may also provide the needed information and analysis to answer CHAPTER 7 - RISK CHARACTERIZATION 71 specific risk management questions. The major difference between qualitative and quantitative risk characterization approaches is in the way the information is synthesized and the communication of the conclusions. 7.2.2 Performing a qualitative risk characterization Qualitative risk characterization requires an overall textual estimate of the risk that will be based on a combination of the various prior steps. This is a complex process as it should still obey basic principles of probability theory when combining probabilities but there are no clear rules to the outcome of the combination of (possibly subjective) textual descriptions of probability. As a hypothetical example, Table 8 illustrates a comparison between the process for computing risk estimates in quantitative versus qualitative risk assessments. When combining the equivalent qualitative statements, the only inference that can be made is that the final risk is either of equal magnitude or lower than the probability at Stage 1. This qualitative process can lead to errors in probability logic and may be impossible if there is uncertainty to address or multiple pathways to combine (Wooldridge, 2008). Alternatively, Wooldridge (2008) proposes the risk characterization process consist of a summary of the individual conclusions for each of the steps of the risk assessment (including descriptions of uncertainty). TABLE 8. A comparison of the process for computing the final risk estimate in risk characterization in quantitative and qualitative risk assessments. (Table adapted from Table 4 in Wooldridge (2008)) Stage 1 2 Quantitative risk assessment Probability Computation 0.1 0.001 P(Stage 2) = P(Stage 1) x 0.001 = 0.0001 3 0.5 P(Stage 3) = P(Stage 2) x 0.5 = 0.00005 4 Risk estimate 0.9 P(Stage 4) = P(Stage 3) x 0.9 = 0.000045 0.000045 Qualitative risk assessment Probability Computation Low Very Low P(Stage 1) x “Very Low” → Very Low (or lower) Medium P(Stage 2) x “Medium” → further reduction from very low High P(Stage 3) x “High” → further (small) reduction Very low (or lower) 72 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Despite its name, a qualitative risk assessment still relies on as much numerical data as possible to provide suitable inputs. The search for information, and thus for numerical data, should be equally as thorough as for a quantitative risk assessment. Also, where there are crucial numerical data deficiencies, expert opinion must be utilized. The major difference between qualitative and quantitative risk assessment approaches lies in how the data and expert opinion are treated and combined. Transparency in reaching conclusions A qualitative risk characterization should show clearly how the risk estimate is reached. The precise way of doing this will vary depending in part on the complexity of the risk assessment and in part on the risk assessor(s) preferences. Methods used include: • a tabular format, with data presented in the left-hand column, and the conclusions on risk in the right column; or • a sectional format with a summary or conclusion at the end of each data section. Examples of these formats that illustrate good practice (i.e. documentation of evidence and logic) are presented in Table 9 and Table 10. The examples are based on particular steps in an overall risk assessment for which the question is “What is the probability of human illness due to microbe ‘M’, in country ‘C’, due to the consumption of meat from livestock species ‘S’ infected with microbe M?” TABLE 9. Example of a possible tabular format for presenting data linked to risk estimates and conclusions Step being estimated: What is the probability of a randomly selected example of species S in country C being infected with microbe M? Data available Risk estimate and conclusions The prevalence of microbe M in species S in Country C was reported as 35 percent (Smith & Jones, 1999*). The prevalence of microbe M in region R, a district within country C, was reported as 86 percent (Brown, 2001*). There are no particular geographical or demographic (with respect to S) differences in region R, compared with the rest of C (Atlas of World Geography, 1995*). The diagnostic test for microbe M, used in the livestock surveillance programme in country C is reported to have a sensitivity of 92 percent and a specificity of 99 percent (Potter & Porter, 1982*). *Fictional references for illustrative purposes only The studies suggest that the probability of a randomly selected example of species S in country C being infected with microbe M is medium to high. However, the two studies indicate that considerable variability by region is likely. With only two studies available, there is also considerable uncertainty of the actual range of prevalence by region, as well as the probability of infection in a randomly selected example of S. In addition, the timing of these surveys may suggest an increasing prevalence of M in C. The reported parameters for the diagnostic test used do not alter these conclusions. CHAPTER 7 - RISK CHARACTERIZATION 73 TABLE 10. Example of a possible sectional format for presenting data linked to risk estimates and conclusions SECTION X. What is the probability of human ill health, given infection with microbe M? Data available • No specific dose–response data have been found for microbe M. • Health authorities for country C provide the following data (Zhou and Kopko, 1999*). > Incidence over the period was reported as 22 cases per million of the population per year (22 per million is 0.000022 percent of the population per year). • Clinical incidence recording and reporting systems in Country C are considered to be of high quality (Bloggs, pers. comm.*). • Experts’ opinions indicate that once clinical symptoms appear, cases are likely to consult a medical practitioner (Lopez et al., 1992*). • Cases tend to be seen in the very young or the very old (Lopez et al., 1992*). • A surveillance study undertaken by practice-based serological testing indicated that 35 percent of the population of C had been exposed to microbe M and had sero-converted (Hunt, Hunt and Seek, 2001*). This was a countrywide, statistically representational study. *Fictional references for illustrative purposes only Conclusions Data suggest a high level of exposure to microbe M in country C, but a very low incidence of clinical disease. Expert opinion indicates underreporting of clinical disease due to lack of medical practitioner involvement is unlikely to account for this. Overall, therefore, the probability of human ill health, given infection with microbe M, is likely to be low. The level of uncertainty in the data specific to country C appears to be low, making this conclusion reasonably certain. However, data also indicate that there are specific groups at higher risk of clinical illness, specifically the very old and very young. From the data currently available it is not possible to indicate how much higher this risk is likely to be. Limitations of qualitative risk characterization It may be difficult to conceive of a fully qualitative risk assessment that will provide useful advice to risk managers, except in a few special cases. In those special cases, the number of factors that could affect the risk may be very low or every factor that affects the risk may change the risk in the same direction. Since risk managers may make decisions on the basis of economics, qualitative descriptions may be difficult to translate directly to financial benefits and/or costs. In other cases, it may be virtually impossible to assess the combined effect of multiple stages because the relative contributions of factors, expressed in qualitative terms, cannot be logically combined to determine their overall effect on risk. In some cases, a qualitative best-effort may still be needed, and any assumptions and uncertainties need to be clearly explained. Thus, while a fully qualitative risk assessment can identify pathways or scenarios that lead to extremes of risk, the relative risk from all other scenarios cannot be logically differentiated. Logical qualitative reasoning can provide conclusions like “the risk of X is logically less (or greater) than that of Y” 74 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD where Y is another, more precisely quantified, risk that has previously been deemed acceptable (or unacceptable). One can also argue that both of these approaches are forms of best- and worst-case quantitative risk assessment. Cox, Babayev and Huber (2005) discuss these limitations in greater detail and provide examples. Qualitative analyses often suffer from the inability to determine what pieces of evidence were influential, how they were combined, and ambiguity concerning the meaning of any assigned risk characterization labels. Without explicit criteria identifying what is meant by descriptions such as high, moderate, and low risk, there is little to distinguish the conclusions from arbitrary and possibly value-laden judgements about the level of risk. These shortcomings tend to make qualitative risk characterization unacceptable in many decision-support situations. Another limitation of qualitative risk assessment may be to blur the lines between risk assessment and risk management. For example, a risk assessment that concludes the level of the risk under consideration to be “low”, may be perceived to be making a management evaluation of the risk, and therefore confusing the roles of assessor and manager. It is possible to present an unstructured analysis as a more structured analysis by including standard documentation headings such as exposure assessment, hazard characterization and risk characterization. Examples that illustrate qualitative approaches that do link evidence and conclusion are presented in Section 8.1. If the risk assessment will be read by a broader audience, assessors should be mindful that interpretation of words or terms used as descriptors might vary between languages or regions. Even when there is a consensus between assessors and managers over the interpretation of the terms used, some limitations of qualitative risk assessment can be identified. 7.3 SEMI-QUANTITATIVE RISK CHARACTERIZATION 7.3.1 Introduction Semi-quantitative approaches to risk characterization involve assigning numbers to qualitative estimates in the form of probability ranges, weights or scores. These are combined by addition, multiplication, or other mathematical operation with the objective of achieving a greater level of objectivity compared to a qualitative approach. It is the role of risk characterization to provide an unbiased estimate of the level of the risk being considered. Semi-quantitative approaches avoid this problem by using a specific, quantitative meaning rather than terms like “Low probability.” CHAPTER 7 - RISK CHARACTERIZATION 75 Table 11 and Table 12 provide some example definitions for probability, exposure and severity categories where probability ranges have been assigned to qualitative descriptions. TABLE 11. Example category definitions for the probability of an event occurring and for the frequency of exposure per year Category Negligible Very Low Low Medium High Very High Certain Probability range (Probability of event per year) Indistinguishable from 0 < 10-4, (except 0) 10-4 to 10-3 10-3 to 10-2 10-2 to 10-1 > 10-1 (except 1) 1 Category Exposures per year Negligible Very Low Low Medium High Very High Indistinguishable from 0 1-2 3-10 11-20 21-50 >50 TABLE 12. Example definitions of health effect / severity category labels Category None Very low Low Medium High Very high Severity description No effect Feel ill for few days without diarrhoea Diarrhoeal illness Hospitalization Chronic sequelae Death 7.3.2 Performing a semi-quantitative risk characterization Semi-quantitative methods require the development of decision rules guiding how the categorical risk levels are combined and which are logical, align with general principles of probability, and are transparent in terms of the operations performed. The options to conduct the risk characterization using semi-quantitative methods spans the continuum between qualitative and quantitative approaches, with no single approach endorsed as the single “best” approach in all circumstances. Approaches include, but are not limited to, the combination of labels or scores in algebraic form with a fixed equation (e.g. specifying multiplication or addition of scores); using specified probability ranges/bounds in place of quantitative point estimates of risk; or using a combinatorial risk matrix. The level of complexity 76 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD of the approach varies widely as the exact set of rules to combine the categorical risk levels are often designed specifically for the risk assessment being conducted. Examples of the types of approach that may be used include the following: Using an algebraic approach: Components of the risk characterization (and overall risk assessment) are assigned numerical values to represent categorical levels and an equation is specified that defines how these values are combined. An example using an algebraic approach is RiskRanger by Ross & Sumner (2002). The probabilities involved in exposure and severity are converted to scores from 0 to 1, which are combined (usually by multiplication but including additions, e.g. for recontamination) and subject to logical tests in the software (e.g. to prevent unfeasible risk estimates). The result is a “comparative risk” and, in conjunction with the number of consumers, a predicted number of cases of illness is obtained. An example of its use is presented in Section 8.1.7. Using probability bounds: The categorical labels are assigned probability ranges which are then combined. Often, in the course of carrying out a qualitative risk assessment, one can roughly estimate the probability of exposure, etc., from comparison with other, previously quantified risks or from good data pertaining to the problem. If time or the available data are insufficient to carry out a complete quantitative risk assessment, one can use these categorical labels to express the risk level in a more structured way than a simple description of the evidence one has acquired. However, when terms like “low risk” or “very low risk” are used, it is very important to consider the number itself, but even more so to examine the context to see what the number means. For example, consider where the probability of botulinum toxin in one tin of food from a single supplier is 0.0001. This number itself (0.0001) seems very low. However, since this number refers to only a single tin in a potentially very large population of tins, e.g. 10 million, the resulting number of “toxic tins” equals 0.0001×10 000 000 = 1 000, which would be considered a very large number of toxic tins, given the nature of the illness. On the other hand, if the probability of one tin per year in the entire world containing botulinum toxin is 0.01, then this value is 100 times larger than the value above (0.0001), but the actual risk is much lower (i.e. one “toxic tin” in 100 years) – this risk is actually quite low, considering that the yearly worldwide tin use is in the trillions rather than millions. Therefore, the denominator of the probability needs to be clearly defined (per serving, per person per year, over the whole population, etc.) and the probabilities need to be considered in this context (risk per serving, for a person per year or for the whole population), to classify them as “high” or “low”. In addition, the severity needs to be considered when moving from probability to risk. CHAPTER 7 - RISK CHARACTERIZATION 77 As an example, consider that a qualitative risk assessment has determined that: • the probability a serving could be contaminated is “Very High”, • the number of servings a random person consumes is “Medium” and • the probability of illness given consumption of the contaminated product is “Low”. Using the example definitions from Table 11 and Table 12, one can conclude the composite probability to be between “Low” and “Medium” by multiplying the corresponding bounds from each of the probability ranges, as shown in Table 13. TABLE 13. Example of combining category labels Component Probability that serving is contaminated Number of servings in a year Probability of illness from a contaminated serving Probability of illness in a year Category Very High Medium Low Low to Medium Numerical range 10-1–1 10–20 10-4–10-3 10-4–2×10-2 This approach enables people to make more consistent, logical conclusions: a “Low” exposure probability per serving and a “High” probability of illness given exposure cannot, for example, be categorized as a “Very High” probability of illness per serving. It is possible to use categorical labels to perform some rudimentary type of probability manipulation. For example, by carefully defining the ranges assigned to each term, it is possible to combine a ”Low” exposure with a “High” probability of subsequent health effect to determine the appropriate categorization for the total risk. It is only possible to maintain consistency and transparency in combining categorical labelling of elements of a risk assessment if numerical ranges have been defined for each label. Nonetheless, combining categorical labels should be approached with considerable caution (see Chapter 9). Using a risk matrix: A risk matrix uses combination rules to combine categorical labels, and an example of such a matrix is show in Table 14 This approach has been adopted for many years in other areas of risk assessment but has also received criticism because of the difficulties of defining a robust, defensible treatment of risk characterization (and risk assessment in general). See Levine (2012) and Cox Jr. (2008) for a discussion of these issues and suggestions for improvement. 78 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD TABLE 14. A hypothetical example of a risk matrix to combine likelihood and severity as could be applicable to risk characterization using probability ratings as presented in Table 15 Likelihood Very likely Likely Negligible Minor Low Medium Medium Low Low Medium Severity moderate Medium High Medium Possible Low Low Medium Medium Unlikely Very unlikely Low Low Medium Low Medium Low Low Low Medium Significant Severe High High Medium High Medium High Medium Medium High Medium High Medium High Medium TABLE 15. Semi-quantitative allocation of categorical labels to probability ranges Probability Risk Rating >70% Very likely 40% to 70% Likely 10% to 40% Possible 1% to 10% Unlikely <1% Very Unlikely Limitations of semi-quantitative risk characterization Any semi-quantitative risk characterization has limitations which can result in inaccuracies in risk estimates. These are discussed in more detail in Section 9.2.3, and include: • Number of categories: There is no rule regarding the number of categories that should be used, e.g. 5 or 25 categories of severity. • Granularity of scale: Consider a risk whose probability of occurrence falls just above the boundary between two categories. If a risk management strategy reduces that probability by a small amount, then it could be dropped down one category. However, this change is indistinguishable from a change that reduces the probability by a factor of 10, and thus also reduces the category by one level. • Difficulty combining probability scores: It is difficult to create a rule with scores that replicates the probability rules. CHAPTER 7 - RISK CHARACTERIZATION 79 Data requirements The basic principle of risk assessment is to collect as much data as possible, providing that the inclusion of more data may affect the decision being made. The data collected for a qualitative risk assessment are often sufficient for semiquantitative risk assessment needs. The difference between the two is that semiquantitative risk assessment has a greater focus on attempting to evaluate the components of the risk to within defined quantitative bounds. Thus, at times, one may do a statistical analysis on a data set to attempt to more precisely estimate a probability providing it will give the assessor more confidence about how to categorize the risk. Semi-quantitative risk assessment is usually used as a means to compare several risks or risk management strategies. At times there may be sufficient data to be able to perform a full quantitative risk assessment for a select number of risks, i.e. food–pathogen combinations. A quantitative model can provide more information about specific strategies to apply to that particular risk issue, but the quantitative results can also be used to place these more precisely evaluated risks into context with others of concern in a semi-quantitative environment. Transparency in reaching conclusions Semi-quantitative risk assessment is a system for sorting out risks, focusing on the big issues, and managing the entire risk portfolio better. The scoring system is inherently imperfect, but so is any other risk evaluation system. If the scoring system being used can be shown to produce important errors in decision logic, then one can use potentially more precise quantitative risk assessment arguments or change the scoring system to something more suitable. Semi-quantitative risk assessment may offer some advantages in achieving transparency. No sophisticated mathematical model is necessary, for example, which is appealing to the lay person. However, the use of mathematical models as an obstacle to transparency may be overemphasized. Most food safety risk assessments require understanding of complex microbiological information and a reasonable understanding of human medicine and of epidemiological principles, which tend to be postgraduate topics. In contrast, quantitative risk assessment uses mathematics that are generally covered at an undergraduate level. The main obstacle to transparency of quantitative models is that there are only a few people who have specialized in the field. The key transparency issue with semi-quantitative risk assessment arises from the granularity of the scales used in scoring. The usually rather broad categories mean 80 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD that any distinction between risks, that can be considerably different in probability and/or severity magnitude, is lost. This means, for example, that one food industry could be unfairly penalized because its product lies just above a category bound, or that industry or regulator only have the incentive to push a risk just over, or below, a category boundary. 7.4 QUANTITATIVE RISK CHARACTERIZATION 7.4.1 Introduction As described in Section 5.2.3, quantitative assessment can be either deterministic or stochastic. Examples of deterministic quantitative risk assessments can be found most readily in the food additive safety assessment (also known as chemical risk assessment) literature. However, most of the literature, guidance and the bestknown examples of quantitative microbiological risk assessments are stochastic. This approach offers many advantages over deterministic risk assessment, and these advantages are described in Chapter 11. FAO and WHO have produced numerous examples of stochastic QMRAs, through the Microbiological Risk Assessment Series, as have many food safety authorities around the world; some examples are provided in Section 8.2 Quantitative risk characterization addresses risk management questions at a finer level of detail than a qualitative or semi-quantitative risk characterization and facilitates a more precise comparison between risks and between risk management options. This extra level of detail can be at the expense of a far greater time to completion, a reduction in scope and a greater difficulty in understanding the model. Probabilistic techniques are more complex and therefore introduce a greater likelihood of error or misunderstanding. Quantitative risk assessments may also rely on subjective quantitative assumptions (WHO and OECD, 2003), and the mathematical precision of these quantitative results can inadvertently give a false impression of the degree of accuracy in characterizing risk. This has been recognized for a long time in the risk analysis community, e.g. Whittemore (1983) noted, “Quantitative risk analyses produce numbers that, out of context, take on lives of their own, free of qualifiers, caveats and assumptions that created them.” 7.4.2 Quantitative risk measures Quantitative measures of risk combine the two quantitative components of risk: (a) a measure of the probability/amount of the hazard consumed (i.e. exposure) and (b) the subsequent severity of the health effect (Kaplan and Garrick, 1981). CHAPTER 7 - RISK CHARACTERIZATION 81 Measure of exposure The probability of exposure in microbiological food safety risk assessment must relate to a specified level of exposure, which is the result of the exposure assessment component (Chapter 5). The subsequent probability measures of risk are expressed generally as risk of an outcome (e.g. probability of illness per serving) or as population risk (e.g. probability of the population experiencing more than 10 illnesses per year). There are advantages and disadvantages in selecting each probability measure. The first option underlines the probabilistic content of the risk measure, while the second can be misread to make one believe that the risk event will occur deterministically with the specified frequency. However, explicit identification of the distribution of the risk measure, or associated probability intervals, helps to counter that perception. The probability measure needs to be chosen carefully and in collaboration with the risk managers. This allows any explanation of the risk assessment results to be made as clear as possible to the intended audience. Measure of health effect There are different ways of expressing risk (EFSA, 2012a). Codex Alimentarius defines risk as “A function of the probability of an adverse health effect and the severity of that effect, consequential to a hazard(s) in food” (CAC, 2019). There are different metrics that have been developed to characterize and compare risk including the number of an adverse outcome, the QALY, the DALY, as well as metrics for monetary valuation of public health (EFSA, 2012b). Each of these metrics has some advantages and disadvantages, and there is no universally preferred choice. Each individual metric provides a different perspective on the public health risk of foodborne pathogens and the choice should be based on the purpose and scope of the risk assessment. The selected measure(s) of health effect will reflect what the risk manager cares about. There are many potential adverse health effects that a risk manager might be interested in, in addition to those about which the affected individual is directly concerned. This, in turn, means that there are many possible ways to measure and express the magnitude of the risk, sometimes called the risk metric, that might be selected as the required output from a risk assessment. The selection of a particular measure of risk is therefore not necessarily straightforward, and must be discussed between the risk manager, the risk assessor, and other interested stakeholders. In addition, for quantitative modelling, the unit(s) required must be defined whilst 82 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD considering the practical aspects of modelling so that the outputs can be produced and reported in those units. A) Number of adverse outcomes: The number of adverse outcomes (e.g. illnesses, hospitalizations, deaths) is the simplest metric that can be used in risk assessment. This number (or the probability) of adverse outcomes can be estimated as “per serving” or “per annum” and standardized for population size (e.g. per 100 000 per year). In general, the per annum relative risks inherently have a greater degree of uncertainty than the corresponding per serving relative risk because of the additional uncertainty associated with the number of annual servings (EFSA, 2012b). Another factor that affects relative risk on a per annum basis is the size of the susceptible subpopulations, relative to the total population, which are substantially different, e.g. young, old, pregnant, immunocompromised (YOPI). Note that not all subpopulations may be equally susceptible to all hazards, e.g. susceptibility to infection may differ between subpopulations. • Risk of some outcome per serving requires that a serving be defined (e.g. 100 g of cooked chicken, 150 ml of orange juice, or use of a serving size probability distribution). The risk of some outcome per serving measure provides an easy comparison of the risk from direct consumption of different food products. It can also be helpful in establishing cost–benefit type arguments where, for example, one is looking for the lowest risk for a given nutrition requirement. • Individual risk can be specified for a random individual within the population of concern, or for a random consumer of the product. If a random consumer of the product is assumed this presupposes that there are no significant secondary infections or cross-contamination effects. Random individuals can be assumed to be part of various subpopulations if one wishes to explore the risk to different subpopulations. Examples of different individual risk estimates include: 1. The probability per year that a random individual will suffer illness X from exposure to bacteria Y in food Z. 2. The probability per year that a random individual will suffer any deterioration in health X from exposure to bacteria Y in food type Z. 3. The probability that a person will suffer some adverse health effect in their lifetime from exposure to bacteria Y in foods. 4. The expected number of foodborne-related adverse health events for a random individual from consuming food type Z in a year. 5. The distribution of the number of foodborne-related adverse health events for a random individual from consuming food type Z in a year. 6. The per capita expected incidence of health impact X from food type Z. CHAPTER 7 - RISK CHARACTERIZATION 83 7. The expected incidence of health impact X per kg consumed of food type Z by the nation. This risk per person is usually a very low number (e.g. 0.000 013 expected illnesses per person per year), making it difficult to understand and compare. These values can be made more understandable by considering the risk over a large number of people (e.g. 1.3 expected illnesses per 100 000 people per year). • Population–level risk considers the risk distributed over the population or subpopulation of interest. It may or may not distinguish between subgroups within that population, such as by region, ethnicity, age or health status. The following are some examples of population–level risk estimates: 1. Total expected number of cases of foodborne illness within the population in a year. 2. Expected number of hospital bed–days taken up per year as a result of a particular foodborne pathogen. 3. Probability that there will be at least one outbreak (or one death, one illness, etc.) in the population in a year. 4. Probability that there will be more than 10 000 illnesses in the population in a year. These estimates can be produced for separate subpopulations if required and aggregated to a single measure for the whole population. B) Health adjusted life years (Burden of disease): Summary measures of public health can characterize and compare the health effect of diverse risks and health outcomes. These are particularly useful when a risk assessment is considering different pathogens. For example, deciding between risk management options that pertain to two different pathogens requires a method that accounts for the differences in severity between those pathogens. In contrast, if a risk assessment is concerned with a particular product–hazard pairing, and the severity of outcomes is independent of exposure pathway, then these summary metrics are less critical. For example, deciding between risk management options that pertain to controlling illnesses for a particular product–hazard pair is less dependent on the differences in severity between the options (because this is the same). Different methods have been developed that provide a common metric for more fully valuing and comparing health risks. Health-Adjusted Life Years (HALYs) are nonmonetary health indices and are summary measures of population health 84 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD permitting morbidity and mortality to be simultaneously described within a single number (Gold, Stevenson and Fryback, 2002). HALYs are used in economic cost– effectiveness analyses, also sometimes referred in the literature as cost–utility analysis or weighted cost–effectiveness analysis (Mangen et al., 2010). The two most prominent HALYs are QALYs and DALYs. The DALY is based on the amount of life quality lost multiplied by the duration of that health state (Van der Fels-Klerx et al., 2018). They are useful for overall estimates of burden of disease, comparisons of the relative effect of specific illnesses and conditions on communities, and in economic analyses. The DALY method presumes perfect health for the entire life span, and therefore measures the loss due to ill health. The QALY concept is analogous, but measures the increase in quality of life, and its duration, as a result of an actual or putative intervention. The DALY approach allows one health state to be compared with another, and with mortality itself. Integrated health measures provide information to put diverse risks into context. DALYs lost is the summation of two quantities: 1. YLL: Years of life lost (the difference between the age at death and the life expectancy) 2. YLD: Years lived with a disability (multiplied by the extent of the disability) Given values of these disability rates, and data on time course (distribution) of severity of outcomes, the DALYs in units of total years of impact in the population under consideration can be computed (Ssemanda et al., 2018). This formulation recognizes that different illnesses will have different patterns of severity and longevity of disability (Haas, Rose and Gerba, 2014). The DALY methodology has been widely used in both national (Lake et al., 2010; Monge et al., 2019; Scallan et al., 2015; Ssemanda et al., 2018) and global (Mangen et al., 2010) disease burden estimations or to compare the burden of disease estimates attributed to different cooking practices (Berjia, Poulsen and Nauta, 2014). The DALY approach has also been used by WHO to quantify the global burden of foodborne disease as it incorporates life years lost through specific types of disability, pain or other reduced quality of life, including premature mortality. The WHO Initiative to Estimate the Global Burden of Foodborne Diseases (WHO, 2015) provides estimates of global foodborne disease incidence, mortality, and disease burden in terms of DALYs for 31 foodborne hazards (including 11 diarrhoeal disease agents, 7 invasive infectious disease agents, 10 helminths and 3 chemicals). A related approach to integrate the spectrum of health outcomes is the QALY approach. QALYs differ from DALYs primarily by the nature of the weights used. CHAPTER 7 - RISK CHARACTERIZATION 85 Rather than using expert-derived “disability weights,” the QALY concept uses “quality weights” which are based on survey or preference data to assess the relative perceived quality of life under certain health impairments. Such an approach allows for the differentiation among subpopulations, socioeconomic conditions, and differences in underlying society (Haas, Rose and Gerba, 2014). The DALY method is considered by some to be preferable to the QALY method for making societal resource allocation decisions. The QALY method was intended to evaluate the benefit in quality of life improvement through a medical intervention, i.e. compared to the cost, while DALY mostly seeks to quantify the burden of disease due to a particular hazard in a particular context. A strong point of the overarching HALY approach is that utility and disability weights are not income constrained. However, HALYs do not capture effects that are not health related and HALY effects cannot be compared to other nonhealth projects, as would be the case if all effects would be expressed in monetary values (Mangen et al., 2010). HALYs are based on the assumption that a life–year is the appropriate metric for measuring health; as a result, the valuation of permanent disability and mortality is linearly valued by age of patients. DALYs and QALYs are semi-quantitative estimates based on disability scoring, and their accuracy depends greatly on the quality of input data and risk assessment models used for estimating the incidences of relevant health outcomes (Van der Fels-Klerx et al., 2018). C) Monetary risk metrics: The public health effect of foodborne disease can also be characterized using monetary metrics (Mangen et al., 2010). However, health economics is a branch of economics with additional complexities (Arrow, 1963). Factors that distinguish health economics from other areas include extensive government interventions, uncertainty in several dimensions, asymmetric information (the physicians know more than the patients), barriers to entry, externalities (communicable diseases, fear of catching disease) and the presence of a third-party agent (professional health care provider). Several different approaches have been developed for the monetary valuation of risk (Mangen et al., 2010). The three general approaches are: • the human capital approach, measuring a person´s production in the marketplace; • cost of illness (COI) methods; and • revealed or stated preferences which also include immeasurable factors such as suffering and pain. 86 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD With the human capital approach, the benefits of a health program or costs of disease is measured by the effect on a person’s productive input. The human capital approach is generally restricted to the effects on labour productivity (e.g. foregone income) and makes no attempt to include intangible costs. It is therefore not considered a measure of individual or social welfare. Opportunity costs of time or a replacement cost approach are two methods usually used to value the time for nonmarket activities (e.g. home-keeping). A second approach to measuring the public health impact of disease is the COI method. The COI approach does not measure intangible costs but traces the economic flow associated with an adverse health outcome through the quantification of measurable monetary costs. COI measures include (Mangen et al., 2015): 1. the costs related to the resources used within the healthcare sector; 2. the resources used by patients and their families; and 3. productivity losses and other, not healthcare related, resources used that are indirectly related to illness (e.g. special education). The COI method estimates the money spent on medical expenditures and the value of the productivity of the patient foregone as a result of foodborne illnesses, complications and deaths. It can be applied wherever there are quantitative data relating to the effect of disease and sufficient cost data for calculating resultant treatment costs and loss of income. Subject to data availability, it is possible to compare large numbers of food risks using COI (Van der Fels-Klerx et al., 2018). COI can be applied for comparing diseases (Mangen et al., 2015), for food–disease combinations (Thomsen et al., 2019), for supply chain analysis of a single food– disease combination (Duncan, 2014; McLinden et al., 2014; Monge et al., 2019), and for comparing the cost–effectiveness of different interventions to reduce the foodborne risk (Lawson et al., 2009). A third approach uses stated preference studies that are based on the presentation of hypothetical scenarios on which to evaluate how much a person would pay for reductions in the risk of death or other adverse health states. Stated preventative studies can be designed for a specific health state, but are based on a hypothetical construction and, therefore, describe the intention of individuals to adopt particular decisions (Mangen et al., 2010). Matching dose–response endpoints to the risk measure Exposure to microbiological agents can result in a continuum of responses ranging from asymptomatic carriage to death. Risk characterization needs to consider the CHAPTER 7 - RISK CHARACTERIZATION 87 reported health outcome used in developing the dose–response relationship and may require estimating the desired risk assessment endpoint(s) from a more or less severe measurement endpoint. A fraction of exposed individuals will become infected. Infection may be measured as the multiplication of organisms within the host, followed by excretion, or a rise in serum antibodies. The morbidity ratio is the fraction of those infected who will exhibit symptomatic illness, as measured by clinical observation or reported by patients or consumer responses (Haas, Rose and Gerba, 2014). A fraction of those becoming ill will suffer severe symptoms (e.g. bloody diarrhoea), require medical care or hospitalization, or will die. In the case of death, this fraction is known as the case–fatality rate or mortality ratio (Haas, Rose and Gerba, 2014). It should also be noted that DALY and QALY are not typically dose–response endpoints; rather, the endpoints are infections, illness, death. A template (e.g. DALY/case) must be used to translate the risk estimate (e.g. cases) from a quantitative microbial risk assessment to DALYs, etc. In addition, care must be taken to ensure that the implications of the case definition used in a clinical trial or epidemiological investigation are understood. For clinical trials, typical measurement endpoints include infection (e.g. as indicated by a faecal positive) or illness (e.g. as indicated by diarrhoea). Epidemiological surveys may provide information on morbidity and mortality ratios. These ratios might be dose-dependent, but epidemiological data may not indicate this relationship. In some cases, clinical trials have used a continuous dose–response measurement endpoint (e.g. volume of diarrhoea excreted) that might provide some insight about the dose-dependency of outcome severity (Coleman et al., 2004). Accounting for subpopulations Subpopulations may vary with respect to susceptibility, exposure, or both. If the risk characterization seeks to distinguish risk by subpopulation (e.g. by age class), then the exposure assessment outputs should be kept separate for each subpopulation to reflect variation in exposure among them (e.g. the frequency, size and preparation of servings consumed by members of each age class). Even where separate dose– response relationship by subpopulation cannot be specified, it may be informative to characterize risk by subpopulation. The subpopulations of interest to the risk managers (e.g. susceptible consumers) may not correspond directly to easily identified categories (e.g. age classes). There should be a reasoned basis for classifying consumers as members of different subpopulations, and that subpopulation definitions are consistent between the exposure and dose–response analyses. 88 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 7.4.3 Integration of hazard characterization and exposure assessment Codex guidelines describe the need to assess exposure to a hazard and assess the level of risk that the exposure represents. Most quantitative risk assessments will implement the exposure and dose–response models separately, and risk characterization will connect these to estimate the risk. This need for connection should be included in the planning stage of the modelling whenever possible, to avoid having to adjust the output of exposure or the input of the dose–response to achieve consistency. When there is a logical separation between variability and uncertainty in either the exposure assessment or hazard characterization, this distinction should be propagated through the integration to determine both the variability and uncertainty in the relevant risk measures. Failure to maintain separation between variability and uncertainty can profoundly affect the risk characterization (Nauta, 2000). Additionally, assumptions implicit to specific dose–response models or potential biases in the dose–response estimates can limit how exposure and dose– response can be combined. In the following sections the dose concepts formulated previously are briefly reviewed and suggestions are offered to address issues of maintaining consistency of units, dose–response model rationales and reducing biases when integrating potentially inconsistent exposure and hazard characterizations. Units of dose in exposure assessment According to Codex (CAC, 1999) the output of the exposure assessment is defined as an estimate, with associated uncertainty, of the likelihood and level of a pathogen in a specified consumer portion of food. This exposure estimate is commonly represented by a distribution of the probability that a randomly selected portion of food is contaminated with the hazard, combined with a probability distribution representing the numbers (or concentration) of hazard in contaminated portions of food (i.e. contain one or more cells of the pathogen). Whether the level of contamination is expressed as a number, i.e. colony forming units (CFU), or concentration (CFU/g or CFU/ml) is important when linking this exposure output to a dose–response model. Numbers of CFU potentially ingested are necessarily positive integers, so a discrete distribution may be the most natural choice for the estimated exposure. The use of a continuous distribution for modelling of individual exposures would be most appropriate when pathogen concentrations are relatively high but these can always be converted back to a CHAPTER 7 - RISK CHARACTERIZATION 89 discrete distribution with some rounding function. Continuous distributions are often used for bacterial counts because they are more flexible and easier to manipulate than discrete distributions. If a concentration is used to express the level of exposure, the concentration has to be multiplied by the amount of food ingested to determine the individual exposure. If the concentration being modelled is in the form of a probabilistic mean, then one needs to use dose–response functions for which inputs are probabilistic (usually, Poisson) mean doses rather than those whose input is an actual dose (Haas, 2002; Pouillot, Chen and Hoelzer, 2015). Units of dose in dose–response assessment Dose–response models in microbiological risk assessment typically apply the concepts of “no threshold” mechanisms, independent action and the particulate nature of the inoculum (Chapter 11). This results in the application of single-hit models like the exponential model, the Beta-Poisson model approximation, the Weibull-Gamma model and the hypergeometric model (Haas, 1983; Teunis and Havelaar, 2000). These models assume each ingested cell acts independently, and all cells have the same probability of causing infection. The “no threshold” assumption implies the existence of some level of risk for any dose greater than zero. A review of dose–response models is provided Chapter 13. The two principle types of data used for dose–response modelling are clinical feeding trials with human volunteers, and epidemiological outbreak data or data on disease incidence associated with foodborne exposure. These different types of human data have varying strengths and weaknesses, as discussed in Chapter 10. Combining exposure and dose–response assessments Consistency is important when combining exposure and dose–response assessments. The exposure assessment and hazard characterization should be applicable to the same hazard and the same population. For example, one might mistakenly apply a dose–response relationship, estimated using data from young healthy volunteers, to a less homogenous population that includes susceptible individuals. Such extrapolations should be avoided, if possible, by looking at alternative modelling approaches. However, if extrapolation is done, then it should be clearly explained, and the potential biases and uncertainties of such extrapolation should be incorporated in the assessment. The output of the exposure assessment should be in units of ingested organisms (CFU, cells, virus particles, etc.) per individual and usually on a per-exposure event basis. In contrast, the input of the dose–response may not be on a per-individual level. For example, the exposure may be expressed as a mean or other summary 90 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD of a distribution of exposures over a group of individuals (e.g. Teunis et al., 2010), though this should be avoided. Differences between individual– and group– level exposure summaries in a hazard characterization may create problems of consistency when combining them. Exposure assessment and hazard characterization can be combined in a Monte Carlo simulation by calculating a probability of infection (or illness) associated with each sample from the exposure distribution. For a given sample containing a known number of cells from the exposure distribution, the probability of infection from the specified dose, would then be calculated based on the dose–response relationship. Exposure and risk predictions will generally be uncertain due to the uncertainty associated with alternative models of the exposure distribution and the risk of illness at any specified dose level. These uncertainties extend to predictions of risk when the exposure and dose–response are combined and should be properly represented in the output of the assessment. Limitations of quantitative risk characterization Just as with qualitative and semi-quantitative risk characterization, there are limitations of quantitative risk characterization. These primarily stem from its advantages and are related to the potential need for large quantities of data, as well as the use of complex models. Because of the data and modelling needs, some multidisciplinary teams tasked with performing quantitative risk characterization can be quite large, and thus costly and time-consuming. Deciding on how to incorporate uncertainty in the risk assessment and explaining uncertainty can be challenging. The complex nature of the models often makes the review of such models limited to select experts, as well as time consuming. This complexity can also provide a challenge to transparency as complex models may not be easily interpreted by nonexperts. CHAPTER 7 - RISK CHARACTERIZATION 91 8 8. Examples The examples below are provided to give a perspective on the breadth and depth of published risk assessments, to make the concept of risk assessment more tangible for Codex, FAO and WHO member countries, and to provide example guidance for their own development of risk assessment activities. These examples cover a range of situations, including country and regional level; from the early days of risk assessment to more recently; completed by government employees or in partnership with academic experts; a focus on a particular food product or covering large food categories; for one or more pathogens; for a specific part or the whole food chain. Most of the risk assessment focus on infectious pathogens, but one focuses on the toxin histamine produced by microbial action. Examples of qualitative and semi-quantitative risk assessments are provided in Section 8.1, while quantitative risk assessments can be found in Section 8.2. 8.1 EXAMPLES OF QUALITATIVE AND SEMIQUANTITATIVE RISK ASSESSMENTS 8.1.1 Risk assessment for main determinants of antibiotic resistance in South East Asia The emergence of antibiotic resistant bacteria and genes has been observed. A qualitative risk assessment was conducted to evaluate the relative effects of the main determinants of antibiotic resistance, and to estimate the risk of the emergence and spread of antibiotic resistance among humans in the WHO South East Asia region (Chereau et al., 2017). Factors were examined at the policy 92 level (e.g. scope of policies and guidelines), system level (e.g. implementation of healthcare, wastewater, or agriculture and livestock management options), and at individual level (e.g. human behaviour). The region considered includes the 11 countries Bangladesh, Bhutan, Democratic People’s Republic of Korea, India, Indonesia, Maldives, Myanmar, Nepal, Sri Lanka, Thailand, and Timor-Leste. Seven bacteria with high levels of antibiotic resistance were considered as part of the hazard identification. The study focused on those causing infections with high mortality, namely extended spectrum β-lactamase and carbapenemase producing Enterobacteriaceae and methicillin resistant S. aureus (MRSA). The processes leading to the acquisition, selection, and spread of the resistant bacteria and genes in humans was described as part of the exposure assessment. This included the reservoirs, transmission routes, and biological determinants of the emergence and transmission of resistance. Exposure routes considered included the release from human and animal waste, aquaculture, and pharmaceutical industry, ingestion of contaminated food and water, direct contact with reservoirs (animals, soil, water), and human-to-human transmission (including health case workers). A context assessment was also conducted to look at the environment in which the event is taking place, considering socioeconomic, ecological, other factors that may affect the exposure and/or risk. The likelihood of occurrence of each event was rated using a qualitative approach using the following categories: • Negligible: the event occurs under exceptional circumstances • Low: the event occurs some of the time • Moderate: the event occurs regularly • High: the event occurs in most circumstances. The events in the chain were chronologically integrated leading to transmission of antibiotic resistance in the human population using a matrix to calculate the risks from two consecutive, and dependent events. When multiple independent events contributed to the estimation of risk, the highest risk was used. The risk matrix used was from Wieland et al. (2011), which is designed to combine two risk estimates based on the assumption that the second event is fully conditional on the previous event (see Table 16). The risk assessment concluded that South–East Asia is at high risk of the emergence and spread of antibiotic resistance in humans. The assessment provides an overall CHAPTER 8 - EXAMPLES 93 picture of the factors affecting the emergence of antibiotic resistance emergence in humans in the region, and highlights the limited benefit of interventions that are sector specific as opposed to an overall holistic “One Health” approach. TABLE 16. Risk matrix used to combine two consecutive, and dependent events (adapted from Wieland et al., 2011) Event 2 Negligible Event 1 Negligible Negligible Low Negligible Moderate Low High Low Low Negligible Low Low Moderate Moderate Negligible Low Moderate Moderate High Negligible Low Moderate High 8.1.2 Faecal pollution and water quality, WHO The Annapolis Protocol (WHO, 1999) was developed in response to concerns regarding the adequacy and effectiveness of approaches to monitoring and managing faecal-polluted recreational waters. One of the most important changes recommended in the Annapolis Protocol was a move away from sole reliance on guideline values of faecal indicator bacteria to the use of a qualitative ranking of faecal loading in recreational-water environments. The protocol was tested in several countries, and an expert consultation was convened by WHO. A revised Chapter 4 in Volume 1 of the guidelines was produced from the expert consultation, which described a suitable approach to risk assessment and risk management (WHO, 2003). Tables were produced for water bodies affected by three different sources of human faecal contamination: sewage outfalls, riverine discharges and bather shedding. The tables were based on qualitative assessment of risk of exposure under “normal” conditions of sewage operation, water levels, etc., and classified the potential human risk. Table 17 reproduces the classification for sewage outfalls. TABLE 17. Relative risk potential to human health through exposure to sewage through outfalls (reproduced from WHO, 2003) Treatment Nonec Preliminary Primary (including septic tank) Secondary Directly on beach Very High Very High Very High Discharge type short outfalla High High High Effective outfallb NAd Low Low High High Low (cont.) 94 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Treatment Directly on beach Discharge type short outfalla Effective outfallb Secondary plus disinfectione Moderate Moderate Very Low Tertiary Moderate Moderate Very Low Tertiary plus disinfection Very Low Very Low Very Low Lagoons High High Low a) The relative risk is modified by population size. Relative risk is increased for discharges from large populations and decreased for discharges from small populations. b) This assumes that the design capacity has not been exceeded and that climatic and oceanic extreme conditions are considered in the design objective (i.e. no sewage on the beach zone). c) Includes combined sewer overflows. d) NA = not applicable. e) Additional investigations recommended to account for the likely lack of prediction with faecal index organisms. 8.1.3 Drinking water guidelines, Australian National Health and Medical Research Council As part of Australia’s National Water Quality Management Strategy the Australian National Health and Medical Research Council produced the Australian Drinking Water Guidelines (NHMRC, 2011). The guidelines are not mandatory standards but are designed to provide an authoritative reference document and framework for good management of drinking water supplies to assure safety at point of use by consumers in all parts of Australia. The guidelines consider that the greatest risks to consumers of drinking water are pathogenic microorganisms, and as such cover similar issues for water that microbiological food safety risk assessment covers for food. However, it should be noted that microbiological growth and inactivation are likely to play a much larger role in microbiological food safety risk assessment because of the greater potential for microbial growth in foods, and the application of strong inactivation processes that do not occur in water in nature. The extensive guidelines document includes a qualitative method for assessing human health risks and recommends that risks should be assessed at two levels so that priorities for risk management and application of preventive measures can be established: • Maximum risk in the absence of preventive measures; and • Residual risk after consideration of existing preventive measures. It is stated in the document that “the aim should be to distinguish between very high and low risks” (NHMRC, 2011). The level of risk of each hazard (pathogen or hazardous event) can be qualitatively assessed by combining the likelihood of the hazard occurring and the subsequent severity of the consequences, according to the example categories listed in Table 18, Table 19 and Table 20 (Tables 3.1, 3.2 and 3.3 in the original document), though these can be modified as needed. The guidelines document also includes qualitative hazard identification characterizations for a wide range of waterborne hazards that can be used to assist in the application of the tables. CHAPTER 8 - EXAMPLES 95 TABLE 18. Qualitative measures of likelihood Level A B C D E Descriptor Almost certain Likely Possible Unlikely Rare Example description Is expected to occur in most circumstances Will probably occur in most circumstances Might occur or should occur at some time Could occur at some time May occur only in exceptional circumstances TABLE 19. Qualitative measures of consequence or impact Level Descriptor Example description 1 Insignificant Insignificant impact; little disruption to normal operation; low increase in normal operation costs 2 Minor Minor impact for small population; some manageable operation disruption; some increase in operating costs 3 Moderate Minor impact for large population; significant modification to normal operation but manageable; operation costs increased; increased monitoring 4 Major Major impact for small population; systems significantly compromised and abnormal operation, if at all; high level of monitoring required 5 Catastrophic Major impact for large population; complete failure of systems TABLE 20. Qualitative risk analysis matrix: level of risk Likelihood A (almost certain) B (likely) C (possible) D (unlikely) E (rare) 1 Insignificant Moderate Consequences 2 Minor 3 Moderate High Very high 4 Major Very high 5 Catastrophic Very high Moderate Low Low Low High High Moderate High Low Moderate Low Moderate Very high Very high Very high Very high High Very high High High 8.1.4 BSE/TSE risk assessment of goat milk and milk-derived products, EFSA A research group in France found a suspected case of Bovine Spongiform Encephalopathy (BSE) infection in a slaughtered goat in 2002. As a result, the European Commission (EC) requested advice from the European Food Safety 96 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Authority (EFSA) on the safety of milk and meat in relation to Transmissible Spongiform Encephalopathy (TSE) in goats and sheep. EFSA (2004a) published the following preliminary statement: From the limited data available today it is concluded that in the light of current scientific knowledge and irrespective of their geographical origin, milk and milk derivatives (e.g. lactoferrin, lactose) from small ruminants are unlikely to present any risk of TSE contamination provided that milk is sourced from clinically healthy animals. Exclusion of animals with mastitis is considered to reduce the potential risk. Further assurance of healthy milk could include milk tests for total somatic cell counts indicative of inflammation. EFSA also commented in a press release (EFSA, 2021): A comprehensive and quantitative assessment of the risks involved in the consumption of goat meat, milk and dairy products will only be possible if more scientific research data on the occurrence of TSE in small ruminants can be obtained. Such a quantitative risk assessment, if feasible, will take considerably more time. It is extremely difficult to assess the risk of BSE-contaminated product because there is no means to measure the number of prions present in a food product. In addition, no human dose–response relationship for prion levels currently exists. EFSA nonetheless needed to provide comment on the level of the above risk and relied on an expert panel to review the available data. 8.1.5 Geographical BSE cattle risk assessment, EFSA In 2003, EFSA was requested by the EC to reassess geographical BSE risk (GBR) and concluded the following (EFSA, 2004b): The Geographical BSE-Risk (GBR) is a qualitative indicator of the likelihood of the presence of one or more cattle being infected with BSE, pre-clinically as well as clinically, at a given point in time, in a country. Where its presence is confirmed, the GBR gives an indication of the level of infection. The GBR assessments are based on information submitted by countries concerned in response to a European Commission recommendation in 1998 setting out the information requirements for such an assessment. CHAPTER 8 - EXAMPLES 97 The information concerns in particular imports of bovines and meat and bone meal (MBM) from the United Kingdom and other BSErisk countries, rendering standards for animal by-products, use of so called Specified Risk Materials (SRMs), feeding of MBM to ruminants, etcetera. Table 3.5 [Table 21] shows the current GBR levels of the seven countries assessed by EFSA so far, as well as their former classification where available. TABLE 21. Geographical BSE Risk (GBR) in 2003 in seven countries as assessed by EFSA (2004b; Table 3.5). Earlier assessed levels are also shown GBR level Presence of one or more cattle clinically or preclinically infected with the BSE agent in a geographical region or country I Highly unlikely II Unlikely but not excluded III Likely but not confirmed or confirmed at a lower level IV Confirmed at a higher level Note: N/A = not applicable, i.e. not assessed before GBR of the country or region Current status (status before) Australia (I) Norway (I), Sweden (II) Canada (II), Mexico (N/A), South Africa (N/A), USA (II) none 8.1.6 Risk profile of Mycobacterium bovis in milk, New Zealand Food Safety Authority The New Zealand Food Safety Authority2 commissioned the New Zealand Institute of Environmental Science & Research Ltd to provide a risk profile of Mycobacterium bovis in milk (Lake et al., 2009). The risk profile is used in the New Zealand food safety system to rank food safety issues for risk management. It forms part of the preliminary risk management activities (Figure 1). The pathogen was selected for assessment because Although it is likely to have minimal public health significance, demonstration of the safety of New Zealand produced food with respect to this pathogen may have trade implications. The food most commonly associated with transmission to humans is cow’s milk. 2 The former New Zealand Food Safety Authority is now part of the New Zealand Ministry of Primary Industries. 98 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The system for assigning a category for a food–hazard combination uses two criteria: incidence (rate) and severity. A four-category scoring system was proposed for the rate (see Table 22), based on foodborne disease rates experienced in New Zealand (Lake et al., 2005). Note that this generic scoring system was adapted to M. bovis in milk. Similarly, a three-category scoring system was proposed for the severity (see Table 23), based on a comparison of the proportion of New Zealand foodborne cases that result in severe outcomes, namely long-term illness or death (Lake et al., 2005). This generic scoring system was also adapted to M. bovis in milk. TABLE 22. The four generic categories proposed in New Zealand for the incidence (rate) with examples (Appendix 1 in Lake et al., 2005) Rate Category 1 2 3 4 Rate range (per 100 000 per year) Examples of food hazard combinations >100 Significant contributor to foodborne campylobacteriosis 10–100 Major contributor to foodborne salmonellosis Significant contributor to foodborne noroviruses 1–10 Major contributor to foodborne yersiniosis, shigellosis <1 Major contributor to foodborne listeriosis TABLE 23. The three generic categories proposed in New Zealand for severity with examples (Appendix 1 in Lake et al., 2005) Severity Category 1 2 3 Fraction of cases that experience severe outcomes 5% 0.5-5% <0.5% Examples Listeriosis; Shiga toxin-producing Escherichia coli (STEC); hepatitis A; typhoid Salmonellosis; shigellosis Campylobacteriosis; yersiniosis; noroviruses; toxins Analysis for M. bovis in milk was hampered by a complete lack of prevalence information, so it was considered impossible to make even qualitative statements of exposure. The only available dose–response data were from animal experiments from 1934 and earlier, making it meaningless to consider a usual food safety risk assessment of exposure and hazard characterization. Therefore, the risk profile is based solely on epidemiological data in an attempt to inform decision–makers of CHAPTER 8 - EXAMPLES 99 how important the issue is among other food safety issues that need to be managed. The analysis discussed the available evidence and gave the following scores: • Severity: 1 (>5% serious outcomes) • Incidence: 4 (<1 per 100 000 people per year) • Trade importance: High Note that the risk assessment title described this as a “qualitative” risk assessment. However, the numerical definitions of the broad category bands would place it within the range of semiquantitative risk assessments, as used in this document. 8.1.7 Seafood safety using RiskRanger, Australia Sumner et al. (2004) discuss the continuum between qualitative and quantitative risk assessment for seafood, and introduce a semi-quantitative risk assessment method that had been coded into a freely-available software tool called RiskRanger (Ross and Sumner, 2002; Sumner and Ross, 2002; CB Premium, 2021). The tool requires answers to 11 questions, which describe the factors throughout the food chain that affect the food safety risk. The questions can be answered in either qualitative (with predetermined categories) or quantitative terms. Qualitative answers are converted to quantitative values according to lookup tables. The model is intended to be population specific, so key inputs like total and/ or regional population size are required. A score is calculated from the inputs, allowing the ranking of various food–hazard combinations. The scoring system is designed to have a scale of 0 to 100, where 100 represents the worst imaginable scenario, i.e. that every member of the population consumes a lethal dose every day. A score of 0 was arbitrarily set to equate to one mild diarrhoeal case per 100 billion people per hundred years, the logic being that the Earth’s population is significantly less than 100 billion, so one would not expect to see an occurrence of the risk anywhere within a person’s lifetime. The chosen range extends over 17.6 orders of magnitude, which equates to 100/17.6 ≈ 6 “risk ranking units” for each factor of 10 between risks. The method has been designed to screen risks and to screen major categories of risk management options. The interface allows a risk manager to instantaneously consider what-if scenarios that can stimulate discussion of possible risk management strategies. The simplicity and generic nature of the model means that its results remain fairly crude. It also means that the questions that are posed are very general. The authors go into considerable detail to warn the reader of these limitations. There is, for example, no incorporation of uncertainty and variability in the model. 100 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The tool was used to evaluate ten Australian seafood hazard–product combinations, considering different consuming subpopulations in Australia. The results are shown in Table 24 (from Sumner and Ross, 2002). The authors compared the ranked risks against observations in Australia. There had been no documented cases in Australia for risks with a score <32. All risks with scores between 32 and 48 (a range of three orders of magnitude) had caused several outbreaks of foodborne illness in Australia, with the exception of Vibrio cholera. Risks with scores >48 had all caused outbreaks of large numbers, some in specific regions. TABLE 24. Result of using RiskRanger to evaluate hazard–product combinations for various subpopulations in Australia (from Sumner and Ross, 2002) Hazard–product pairing Selected population Ciguatera in reef fish General Australian population Ciguatera in reef fish Recreational fishers, Queensland Scombrotoxicosis General Australian population Algal biotoxin in shellfish – controlled waters General Australian population Algal biotoxin — during an algal bloom Recreational gatherers Mercury in predaceous fish General Australian population Viruses in oysters — contaminated waters General Australian population Viruses in oysters — uncontaminated waters General Australian population Vibrio parahaemolyticus in cooked prawns General Australian population Vibrio cholerae in cooked prawns General Australian population Vibrio vulnificus in oysters General Australian population Listeria monocytogenes in cold-smoked seafoods General Australian population Listeria monocytogenes in cold-smoked seafoods Susceptible (aged, pregnant, etc.) Listeria monocytogenes in cold-smoked seafoods Extremely susceptible (AIDS, cancer) Clostridium botulinum in canned fish General Australian population Clostridium botulinum in vacuum packed General Australian population smoked fish Parasites in sushi or sashimi General Australian population Enteric bacteria in imported cooked shrimp General Australian population Enteric bacteria in imported cooked shrimp Susceptible (aged, pregnant, etc.) Risk ranking 45 60 40 31 72 24 67 31 37 37 41 39 45 47 25 28 31 31 48 CHAPTER 8 - EXAMPLES 101 As an important caution the authors cite that they have not been able to systematically and objectively evaluate the model’s performance because there are few data sets describing exposure and foodborne disease incidence. That caution, however, is evidence that full quantitative models would also not have been possible. The authors also found that the model was a powerful tool for teaching the principles of risk analysis. 8.1.8 Animal and animal product import risk assessment methodology, Biosecurity Australia In 1998, a trade dispute between Canada and Australia over Australia’s 24-year ban of uncooked salmon went to the WTO court (WTO, 1998). The former Australia Quarantine Inspection Service had produced a qualitative risk assessment analysing the disease threat in 1995, and another in 1996 – the former assessed the risk to be acceptably low while the latter reached the opposite conclusion. The difference in conclusion was due to a different qualitative risk assessment approach being used, rather than through the emergence of new information. The WTO Appellate Body found in favour of Canada because, inter alia, it considered that Australia had not implemented a proper risk assessment of salmon imports. This highlighted to the risk analysis community the potential problems of relying on a purely qualitative risk assessment methodology. Australia’s regulatory body assessing import risk was restructured, and it now falls under the responsibility of Biosecurity Australia. They have developed a semiquantitative approach to assessing import risk (Biosecurity Australia, 2016). The risk evaluation is based on placing the estimated risk in a risk matrix (Table 25). The band of cells marked “very low risk” represents Australia’s ALOP, or tolerance for loss. The guidelines describe qualitative (e.g. low, medium, high), semi-quantitative (e.g. 0 → 0.0001; 0.0001 → 0.001; 0.001 → 0.01; 0.01 → 1) and quantitative (exact probability calculation) evaluation of likelihood of entry of an exotic disease into Australia. This has the potential advantage of using one environment to incorporate risk assessments along the qualitative to quantitative continuum. Qualitative evaluations of steps in a sequence that results in exotic disease entry are allowed through a matrix rule for combining such qualitative probabilities. The consequence assessment component of the risk estimate for an exotic disease import risk is generally considered far more difficult than evaluating the probability of disease entry. This is because imports are regulated and fairly simple to model, 102 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD and their probabilities are well understood, whereas there are no data on the spread of disease in the naïve country, and disease spread is extremely complex to model. Biosecurity Australia aimed to evaluate the probability and magnitude of a variety of effects should the disease enter the country. They devised a series of rules that allowed the incorporation of the geographical extent of the consequence (local, district, regional, national), and the level to which the consequence would be felt at that scale. Other rules combined the qualitative or semi-quantitative estimates of likelihood of these consequences (given the disease has entered Australia) to allow a placement of the unrestricted risk estimate (i.e. the risk from a product where no specific controls are in place to protect against the hazard in question) in Table 25. If the unrestricted risk estimate fell into an acceptable region, the import would be allowed without any restrictions. If not, restrictions (testing, heat treatment, etc.) would be evaluated to determine the least trade-restrictive option that would allow the import product to meet Australia’s ALOP. Whichever approach (or combination of approaches) is chosen, the guidelines state that the approach should provide for the following: • an assessment based on sound science; • an assessment that is structured and transparent; • an assessment that is internally consistent, and that can be repeated (with the same or a similar outcome) by another operator using the same framework and data; • an outcome that will support the estimation of risk (a combination of likelihood and consequences); • an outcome that will enable risk to be evaluated against the importing country’s ALOP; and • a framework within which the efficacy of risk management and the acceptability of a mitigated risk can be evaluated. CHAPTER 8 - EXAMPLES 103 TABLE 25. Tabulation of risk as a combination of likelihood and consequence (Biosecurity Australia, 2016) Likelihood of pest entry, establishment and spread High Moderate Low Very Low Extremely Low Negligible Consequence of pest entry, establishment and spread Negligible Very Low Low Moderate High Extreme Negligible Very low risk risk Low risk Moderate High risk Extreme risk risk Negligible Very low risk risk Low risk Moderate High risk Extreme risk risk Negligible Negligible Very low Low risk Moderate High risk risk risk risk risk Negligible Negligible Negligible Very low risk risk risk risk Low risk Moderate risk Negligible Negligible Negligible Negligible Very low Low risk risk risk risk risk risk Negligible Negligible Negligible Negligible Negligible Very low risk risk risk risk risk risk 8.1.9 Multicriteria-based ranking for risk management of foodborne parasites, FAO/WHO FAO and WHO were asked to review the current status of knowledge on parasites in food and their public health and trade impact (FAO and WHO, 2014). This was done to provide the Codex Committee on Food Hygiene with advice and guidance on the parasite–commodity combinations of concern, issues that need to be addressed by risk managers, and the options available to them. As part of this charge some work was undertaken to develop a quantitative ranking tool using expert opinion. The experts defined global criteria for evaluating the 24 foodborne parasites and rated each parasite along these criteria. The criteria were: (1) number of global illnesses; (2) global distribution; (3) acute morbidity; (4) chronic morbidity; (5) percentage chronic; (6) mortality; (7) increasing illness potential; (8) trade relevance; and (9) socioeconomic effect. Each criterion was then weighted by the experts for importance and averaged. The three criteria for disease severity (3, 4 and 5) were combined into one criterion, giving a total of 7 criteria weights, reflecting the relative importance of each criterion to the overall score. The average of the elicited criteria weights used in the multi-criteria ranking are shown in Table 26. The overall score for each parasite is calculated as follows. Score = C1*W1+C2*W2+{C3*(1-C5)+C4*C5}*W345+C6*W6+C7*W7+C8*W8 +C9*W9 The resulting tool was able to give a global ranking of foodborne parasites by importance and their primary food vehicle. 104 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD TABLE 26. Average of elicited criteria weights used in the multi-criteria ranking (Table 3 from FAO and WHO, 2014) Scoring Criterion Criterion Weight W1 Number of global foodborne illnesses 0.22 W2 Global distribution 0.14 W345 Morbidity severity 0.22 W6 Case-fatality ratio 0.15 W7 Increased illness potential 0.07 W8 Trade relevance 0.10 W9 Impacts on economically vulnerable communities 0.10 8.2 Examples of quantitative risk assessments 8.2.1 E. coli O157:H7 in tenderized vs non-tenderized beef, FSIS Mechanical tenderization, performed using stainless steel blades or needles, moves pathogens from the surface of intact beef cuts to beneath the surface, thereby potentially shielding those pathogens from the lethal effects of heat during cooking. The United States of America Department of Agriculture’s Food Safety and Inspection Service (FSIS) aimed to estimate whether blade-tenderized steak posed a significantly greater risk than its nontenderized equivalent (FSIS, 2002). They created a quantitative simulation model that predicted the change in survival of bacteria due to the extra protection that was afforded by being embedded in the meat through the tenderizing process. They estimated the bacterial load on steaks after cooking and used this concentration as input into a dose–response model to estimate risk. FSIS concluded that the probability of E. coli O157:H7 surviving typical cooking practices in either tenderized or non-tenderized steaks is minuscule and that differences in bacterial dose after cooking attributable to either type of steak were minimal. They predicted seven additional illnesses due to tenderization for every billion steak servings. This can be seen from Figure 9 below, where the dotted and solid lines for tenderized and non-tenderized steaks are virtually indistinguishable. This was a comparative risk assessment, so the model contained only the elements that were necessary to make the comparison. Thus, the model began with the distribution of bacteria on steak prior to tenderizing, and then looked at the difference in human health risk posed by the same steak under different processing. Consequently, there was no need to consider any factors involved in the rearing and slaughtering of the animal. CHAPTER 8 - EXAMPLES 105 Fraction of servings exceeding dose Fraction ill 1 0.1 0.01 0.001 0.0001 0.00001 0.000001 0.0000001 0.00000001 1 10 Not tenderized 100 1 000 BPSAC (i.e., Dose) Tenderized 10 000 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 100 000 Dose response FIGURE 9. Model output (from FSIS, 2002) showing predicted bacteria per serving after cooking (Dose) and corresponding frequency of illness (Dose–response) 8.2.2 Listeria monocytogenes in ready-to-eat foods, FAO/WHO FAO/WHO convened a drafting group to address three questions relating to L. monocytogenes that were posed by the Codex Committee on Food Hygiene (CAC, 2000). Those questions were to (i) Estimate the risk of serious illness from L. monocytogenes in food when the number of organisms ranges from absence in 25 g to 1 000 CFU/g (or CFU/ml) or does not exceed specified levels at the point of consumption; (ii) Estimate the risk of serious illness for consumers in different susceptible population groups (elderly, infants, pregnant women and immunocompromised patients) relative to the general population; and (iii) Estimate the risk of serious illness from L. monocytogenes in foods that support its growth and foods that do not support its growth at specific storage and shelf life conditions. The risk assessment did not need to complete a full farm-to-fork model to answer these questions (FAO and WHO, 2004). The questions are also not specific to a particular country or product, which would require defining the scope of the model. The team decided to focus on the level of L. monocytogenes at retail; model the growth and inactivation from retail to consumption; and use a dose–response function to estimate the subsequent risk. The team selected four ready-to-eat foods to be reasonably representative of the many different foods available. The quantitative analysis produced the results shown in Table 27. 106 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD TABLE 27. Estimated risk from L. monocytogenes as used in the risk assessment (FAO and WHO, 2004) Food Milk Cases of listeriosis per 109 people per year 910 Cases of listeriosis per 109 servings 0.5 Ice cream 1.2 0.0014 Smoked fish 46 2.1 Fermented meats 0.066 0.00025 The risk assessment report provides a detailed explanation of the important limitations of the quantitative analysis. In particular, these included the need to rely on mostly European quantitative contamination data and on multiple sources for the prevalence estimates. Consumption data were mainly from North America, and the dose–response relationship was derived from epidemiological data from the United States of America. The summary response to the three Codex questions recognizes the caution that should be applied in interpreting the quantitative figures, by providing qualitative context. The report notes that the risk assessment demonstrates that most cases of listeriosis result from the consumption of high numbers of Listeria. Those cases arise from foods where the L. monocytogenes level exceeds the criteria (either 0.04 or 100 CFU/g). The model predicts that consumption of low numbers of L. monocytogenes has a low probability of illness. Eliminating higher levels of L. monocytogenes at the time of consumption has a large impact on the predicted number of illnesses. (FAO and WHO, 2004) 8.2.3 Shiga toxin-producing E. coli O157 in steak tartare patties, Netherlands Nauta et al. (2001) simulated the exposure of the population in the Netherlands to Shiga toxinproducing E. coli O157 in steak tartare, using a farm-to-fork Monte Carlo model. This risk assessment provided an example of integration of exposure assessment and hazard characterization with a low-level dose and an individual– level dose–response relation. The baseline model predicted 0.29 percent contaminated tartare patties and a mean dose of 190 CFU per contaminated patty, as shown in Table 28. CHAPTER 8 - EXAMPLES 107 TABLE 28. Baseline risk model results at the stage of raw steak tartare patties, for different routes of exposure and the means for the Netherlands (NL). (Pos. tartare = STEC O157 contaminated steak tartare patty), where the column headers refer to specific segments of the Dutch industry (article 10 slaughter with ‘industrial’ butcher, article 10 slaughter with ‘traditional’ butcher and article 4 slaughter) Prevalence Mean cfu/pos. tartare Pos. tartare with one cfu STEC O157 Art 10, ind. 0.29% 3.4 72% Art 10, trad. 0.30% 670 38% Art 4 0.21% 1700 36% NL 0.29% 190 64% The dose–response model developed for the hazard characterization was based on a well-documented outbreak in a primary school in Japan (Shinagawa, Hu and Yoshida, 1997). An exponential model, fitted separately to the data for children and adults, resulted in point estimates for the probability of infection by a single cell of r = 0.0093 for children and r = 0.0051 for adults. The exposure distribution was combined with the dose–response model in a Monte Carlo simulation by applying the single-hit model in the form 1-(1-r)n, with n a random sample from the exposure distribution. The risk characterization predicted an attack rate of 0.0015 percent infections per person per year in the Netherlands; or 2 335 infections per 15.6 million people per year. This example incorporated variability but not uncertainty. 8.2.4 Vibrio vulnificus in raw oysters, FAO/WHO An FAO/WHO assessment of the risk of illness due to V. vulnificus in raw oysters adapted a risk model previously developed in the United States of America for V. parahaemolyticus (FAO and WHO, 2005). A principle objective was to investigate potential effectiveness of mitigations after development of a baseline model. This risk assessment provides an example of integration of exposure assessment and hazard characterization, with different assumptions used in estimating the dose– response. A dose–response relationship for V. vulnificus was obtained by fitting a Beta-Poisson model to estimated arithmetic mean risk for the population versus arithmetic mean dose (grouped by month and year). The magnitude of the difference between risk predictions obtained under two alternative interpretations of the dose– response is shown in Table 29. Assuming that the fitted population–level risk 108 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD versus dose relationship applied at the individual level resulted in predictions of risk that were consistently lower (by up to 75 percent) than the epidemiological estimates of mean risks. The predictions of risk obtained based on an aggregatelevel interpretation of the dose–response were more consistent, on average, with the epidemiological estimates of mean risks used to obtain the dose–response fit, so this latter interpretation was used for risk characterization. TABLE 29. Mean risk of illness due to V. vulnificus per serving or exposure Season Winter Spring Summer Autumn Estimated data based on case reports and consumption statistics 1.40E-06 2.80E-05 4.90E-05 1.90E-05 Fitted as individual-level risk versus dose Risk Ratio to Estimated Data 5.10E-07 1.70E-05 2.80E-05 5.10E-06 0.36 0.61 0.57 0.27 Fitted as mean risk versus mean dose Risk Ratio to Estimated Data 1.10E-06 3.40E-05 3.90E-05 2.30E-05 0.79 1.21 0.80 1.21 8.2.5 Histamine in fish sauce, Thailand Fish sauce is a fundamental ingredient used in many South-East Asian dishes and is also used as a dipping condiment. Due to the nature of raw materials and the production methods for traditional fish sauce, high levels of histamine are found in many samples. A risk assessment of histamine in Thai fish sauce was undertaken to respond to the request of the Codex Committee on Fish and Fishery Products (CCFFP) for sound scientific advice as a basis for the development of guidelines for the control of histamine in fish sauce (CCFFP, 2011). Previous human trials and outbreak data were used to build a histamine dose– response model. Subsequently, the risk of developing histamine poisoning from fish sauce among Thai consumers was estimated. Consumption of fish sauce alone yielded a very small histamine intake for consumers. Different scenarios reflecting the effect of different histamine standards were also evaluated and are shown in the Table 30 below. As the analysis shows, the risk from fish sauce alone is essentially zero, and clearly less than the risk of histamine poisoning from fish alone. When the risk of histamine poisoning from fish and fish sauce was evaluated under two different standards, the risk increased slightly. CHAPTER 8 - EXAMPLES 109 TABLE 30. Risk estimates using probabilistic approach (Table 5 in CCFFP, 2011) Scenario 1. Fish sauce alone (a FS daily dose was consumed in one meal) 2. Fish aloneb (a fish daily dose was consumed in a meal) Mean risk per meal (SD)a 0.00 (0.00) 8.12×10-6 (0.4×10-5) 3. Fish + fish sauce (a FS daily dose was consumed in one meal) • 200ppm FS standard • 400ppm FS standard 8.39×10-6 (0.46×10-5) 8.47×10-6 (0.52×10-5) a Risk per meal refers to the predicted risk of an individual becoming ill of histamine poisoning when he or she consumes a daily dose of fish sauce (FS) or a daily dose of a scombroid fish or a scombroid fish with fish sauce. The risk was estimated as a probability of the histamine intake to exceed the No Observed Adverse Effect Level (NOAEL) limit of 50mg using Monte-Carlo simulations. b Assumption: a fresh scombroid fish had a lognormal distribution with an average of histamine concentration of 5ppm and standard deviation of 10ppm. 8.2.6 Pathogens in fresh vegetables, Rwanda This study analysed the farm-to-fork microbial risk for the fresh vegetable supply chain in Rwanda (Ssemanda, 2018). One of the major data gaps identified by the authors was that they could not attribute the estimates of food related illnesses to any food vehicle based on the available data. Despite these limitations, the authors were able to evaluate several scenarios related to the distribution chain. 1. Moving all vegetables from farms to food service establishments without going through markets. 2. Moving all vegetables from farms via supermarkets (with specialized refrigeration systems) to food service establishments. 3. Holding all vegetables under refrigeration (2 °C and 8 °C) from farm to fork and the introduction of a die-off model. 4. All vegetables are effectively washed and sanitized, accomplished by increasing the modelled logreduction by washing. 5. Assuming no contamination and crosscontamination occurs between vegetables and other surfaces throughout the chain. 6. Assuming that preventive measures and interventions implemented at farm level reduce prevalence and levels of pathogenic E. coli by 90 percent. 7. Assuming that the last three scenarios (4 to 6) are combined. Simulation of the 7 scenarios resulted in varying fold-changes in the predicted microbial risk as shown in Table 31. Improvement in washing and sanitization at food service establishments resulted in less than a twofold change in the predicted microbial risk. About a two-fold reduction in risk was observed for the scenario 110 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD of channelling all vegetables through supermarkets instead of traditional markets (Route 3 in Table 31). Farm interventions reduced the predicted prevalence and levels of pathogenic E. coli in the baseline model by 90 percent, introducing a cold chain and skipping the market step resulted in a tenfold reduction in predicted microbial risk. The scenario of avoiding contamination and cross contamination along the supply chain led to a more than 4000-fold reduction in the predicted microbial risk. Lastly, combining the final three farm-to-fork measures resulted in an estimated reduction in risk of 1 million-fold. TABLE 31. Number of illnesses per year and probability of illness per serving after 100 000 iterations of the baseline model and the what if scenarios (Table 6.4 in Ssemanda, 2018) No. of illnesses per year (in millions) Probability of illness per serving What if scenarios b Mode 5th, 95th Percentile Mode 5th, 95th Percentile Fold change# Baseline/ Route 1a 12.1 6.96, 32.6 0.100 0.0572, 0.169 – Improving washing and sanitization at FSEs 10.63 2.13, 27.8 0.1039 0.0151, 0.156 1.14 Route 3 6.26 0.828, 17.3 0.0535 0.0395, 0.0057 1.93 Farm 1.13 Interventions 0.517, 3.101 0.01029 0.00395, 0.0165 10.71 Introduction of cold chain 0.288 0.218, 15.1 0.00042 0.0015, 0.1016 42.01 Route 2 (market step skipped) 0.139 0.195, 10.87 0.000455 0.0013, 0.0728 87.1 No contamination and cross contamination along the supply chain 0.00272 0.00339, 9.4 0.0000183 0.0002, 0.0564 4 449 Farm to fork measures and interventions 0.00001108 0.0000144,0.694 7.33×10-8 0.000, 0.00494 1.1×106 a Baseline model or Route 1 represents a simulation of the supply chain through which about 90 percent of the vegetables are channelled from farms via traditional markets to food service establishments (FSEs) – not applicable # Fold change were calculated by dividing the mode for the numbers of illness per year in the baseline model with the mode for the numbers of illness per year in the what if scenarios. b What if scenarios arranged in descending order of the number of illnesses per year and probability of illness per serving. CHAPTER 8 - EXAMPLES 111 8.2.7 Campylobacter and Salmonella in Chicken Meals, Senegal The authors used a QMRA model to describe the risk of Campylobacter and Salmonella infection linked to chicken meals prepared in households in Dakar, Senegal (Pouillot et al., 2012). The authors note that prevalence and concentration of pathogens in foods available in many countries are well-known data gaps for risk assessment. They also suggest that more information on home cooking practices, cooking processes, and the length and temperature of food storage before and after preparation are needed. They used data collected specifically for purposes of QMRA, including prevalence and level of bacteria on chickens from local markets, time–temperature profiles of chickens from purchase to consumption, observational data from meal preparation in kitchens, and data on pathogens prevalence on utensils, equipment and cooks’ hands. Their model was developed in R software using the mc2d package for second-order Monte Carlo simulations. The simulation used 10 001 iterations in the variability dimension and 1 001 iterations in the uncertainty dimension. The model predicted that crosscontamination led to a high expected frequency of pathogen ingestion, and that significant Salmonella growth was predicted during food storage at ambient temperature before and after meal preparation. The model also predicted a significant decrease in risk could be achieved through reducing prevalence of chicken contamination at slaughter, and by using simple hygienic measures in the kitchen. The model indicated that most effective modification to home cooking practices include the use of a new board, knife, and dish when manipulating the cooked chicken, assuming that these objects are bacteria-free. 8.2.8 Vibrio parahaemolyticus in bloody clams, Thailand A microbiological risk assessment of V. parahaemolyticus risk from Anadara granosa (Bloody clam) was conducted by researchers from Thailand and Japan. They developed two risk assessments (a farm-to-fork model and a fractional change model) based on new data collected primarily from Hat Yai City in southern Thailand, where seafood consumption is popular. The QMRAs were published as part of the FAO/WHO Microbiological Risk Assessment Series in a book entitled “Risk assessment of V. parahaemolyticus in seafood” (FAO and WHO, 2011a). The purpose of the risk assessment was to estimate the risk of V. parahaemolyticus infection associated with consumption of one type of seafood in a defined setting and during a limited period. The work documents an example of a case study where scientists were able to conduct a series of clinical and microbiological studies to generate locally relevant data and elaborate a risk assessment model for a shellfish species other than oyster. 112 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The authors report that the study estimated that only a few people per 10 000 people per year acquire V. parahaemolyticus infection as a result of consuming the boiled Bloody clam food. The risk estimate does not support the common perception that Bloody clam is a major cause of diarrhoeal illness, including V. parahaemolyticus illnesses. At the same time, the investigators caution that this study may also underestimate the risk of Bloody clam-associated V. parahaemolyticus illness due to several critical data gaps. The authors recommended that a case–control study be conducted using patients in Hat Yai City with microbiologically confirmed V. parahaemolyticus infections, as this could provide data on various food and environmental exposure paths. These investigations might also provide more realistic evidence of behaviour that reduces or increases the risk of V. parahaemolyticus illness. The investigators also suggested that more bacterial data on Bloody clam throughout the food chain should be collected, focusing on detection of virulent strains. Finally, the authors encouraged the collection of more detailed data on behaviour regarding harvesting, storage, cooking and consumption patterns need to be collected. A representation of the QMRA model for V. parahaemolyticus in Bloody clam from production to consumption is shown in Figure 10. Temperature Prevalence Concentration Harvest Temperature Prevalence Concentration Transportation Time Temperature Retail Transportation Time Temperature Temperature Prevalence Concentration Cooking Quantity Frequency Consumption Experimation simulation FIGURE 10. Schematic representation of the model framework for a production-toconsumption risk assessment of V. parahaemolyticus in Bloody clam (Figure II-6 in FAO and WHO, 2011a) CHAPTER 8 - EXAMPLES 113 8.2.9 Salmonella in table eggs, EFSA This risk assessment, originally developed by Thomas et al. (2006), was adapted by EFSA to answer a EC’s, question about the risk of Salmonella in eggs (EFSA, 2014b). The EC asked EFSA to assess the public health risk posed by Salmonella from table eggs and to quantify the relevance of the period of time between laying and consumption and the storage conditions of eggs. The period of time between laying and consumption is related to the “Sellby date” and the “Bestbefore date”. The Sellby date applicable to eggs is fixed at 21 days by the European Union (EU) Hygiene Regulation (EC) No 853/2004. This means that table eggs must be delivered to the consumers within of 21 days after laying. The Bestbefore date applicable to eggs is fixed in Regulation (EC) No 589/2008 at 28 days from laying. EFSA applied a quantitative risk assessment model for S. Enteritidis in eggs to answer the question. The quantitative model excluded all stages before laying. A baseline scenario was defined according to the current Sellby and Bestbefore dates in the EU. Changes to time and temperature of storage at retail and in the household, were used to assess the effect of different storage practice scenarios (Table 32). TABLE 32. Dates used in the model for the baseline and alternative scenarios (Table 11 in EFSA, 2014b) Days post Sell-by date (retail) lay Scenarios 21 28 35 Baseline ● Alternative 1 ● Alternative 2 ● Alternative 3 Alternative 4 ◇ Alternative 5 ◇ Alternative 6 Worst-case scenario Best-before date (household/catering) 42 28 35 42 56 63 70 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ◇ ● ● ● ● ● ● ● Scenarios with egg storage at retail under current conditions ◇Scenarios with egg storage under refrigeration conditions in all retail establishments 114 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Storage temperature and time were modelled using distributions based on expert opinion. The remaining distributions were adapted from the model using expert opinion distribution or based on scientific literature. Table 33 below shows a summary of time and temperature of storage of eggs in the EU, from farm to retail as derived from industry expert opinion. TABLE 33. Summary of time and temperature of storage of eggs in the EU, from the ‘on farm’ to the ‘transport to retail’ stages as derived from expert opinion (industry experts) (Table 6 in EFSA, 2014b) Stage Time (hours) Most Min. likely Max. Temperature (°C) Min. Most likely Max. On farm 0 45 168 4 15 30 Transport to grading 0 6 48 4 15 30 Grading 0 18 168 5 15 30 Transport to wholesale 0 5 48 0.1 14 30 Wholesale/ distribution centre 0 23 336 0.1 13 28 Transport to retail 0 7.5 36 0 14 30 In the household setting, extending the storage time for table eggs resulted in an increase in the number of illnesses per million servings, except when eggs are wellcooked. Extending the Sellby date by one week (from 21 to 28 days), but leaving Bestbefore date unchanged, was estimated to result in a relative risk of illness of 1.4 and 1.5 for uncooked and lightly cooked egg meals, respectively, compared with the baseline. If the Bestbefore date was also extended by one week (from 28 to 35 days), the relative risk was 1.6 and 1.7. In the worstcase scenario considered in this assessment (Sellby date of 42 days, Bestbefore date of 70 days), the relative risks of illness were 2.9 and 3.5. EFSA found that the implementation of refrigeration of all eggs during the retail stage (i.e. with temperatures assumed to range from 0 °C to 12 °C) limited this increase in risk in the household setting to some extent. Compared with the baseline scenario, the risk was reduced with an extension of up to three weeks in the Sellby date, and one or two weeks of the Bestbefore date for a sell-by date of 35 and 28 days, respectively, if refrigeration was applied in all retail establishments. If the Sellby date or the Bestbefore date were prolonged beyond three weeks, CHAPTER 8 - EXAMPLES 115 then the risk estimates were greater compared with the baseline scenario, even if refrigeration at retail was applied, assuming that the proportion of consumers who do not store their eggs under refrigeration remained unchanged. 8.2.10 Cryptosporidium in water – a cost–benefit analysis, United States of America The authors developed a simple decision tree (Figure 11) for Boil Water Order (BWO), including the effectiveness of the BWO and illness and death as possible outcomes (Ryan et al., 2013a, 2013b). For each branch in the decision tree the authors assigned the relevant probabilities, including the probability of illness, probability of death, and probability of the boiling process being ineffective. The boiling process may not be effective due to the boiling time being too short or the boiled water being transferred to a nondisinfected container, or other factors. Estimates for these probabilities, for costs of implementation and for the various outcomes were based on published literature. The uncertainty in these estimates were evaluated using a Monte-Carlo sensitivity analysis. No Boil water order Yes Yes, I Illness No Yes, I x Q Illness No Yes, p = 2.13 x 10-4 Death No, 0.999787 Nocost Yes, p = 2.13 x 10-4 Death No, 0.999787 $69.25 $7.87 million $1 000.20 $7.87 million $1 069.45 I–the probability of illness, Q–the probability that the boil water order is ineffective, p–the probability of death from illness, 0.999787–the probability of no death from illness FIGURE 11. Decision tree for Boil Water Orders for Cryptosporidium showing the probabilities and estimated costs for illness and death outcomes (Ryan et al., 2013a). (Reproduced with permission from John Wiley and Sons) The authors used the decision tree to calculate a threshold value for the oocyst concentration in treated water using an exponential dose–response model. This 116 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD was done by equating the BWO and No BWO branches and solving for the daily dose and associated concentration. The authors concluded that this threshold concentration was equal to 0.046 oocysts/L in treated water or 46 oocysts/L in raw water, which was considered to be more practical to assess using water sampling. These concentrations were estimated to result in 9 illnesses per 10 000 people exposed, given the assumed 3-log10 reduction during water treatment. However, the authors also noted that: […] many water supplies that exceed this concentration may already be applying additional treatment, given that a concentration of 46 oocysts/L would require treatment beyond the 3-log removal required by the Long Term Enhanced Surface Water Treatment Rule. (Ryan et al., 2013a) CHAPTER 8 - EXAMPLES 117 Part 1 Part 1 General considerations Detailed considerations Part 2 Detailed considerations 119 9 9. Qualitative and semiquantitative risk assessment: further considerations 9.1 QUALITATIVE RISK ASSESSMENT There are several examples of published qualitative risk assessments (e.g. Lake et al., 2009; King, Lake and Cressey, 2011), although they tend to elicit less scientific attention than quantitative risk assessments. It should be emphasized that the attributes of good risk assessment, as described in Chapter 3, apply equally to qualitative as they do to quantitative risk assessment. Appropriate data must be collected, documented and fully referenced and synthesized in a logical and transparent manner, whichever method is employed. Despite a number of large and well-publicized quantitative microbiological food safety risk assessment projects that have been completed, it is probable that the majority of risk assessments utilized by risk managers and policymakers in the fields of food safety, health and microbiology are not fully quantitative in the sense described in Chapter 3. There may be a variety of reasons for this. Quantitative microbiological risk assessment is a specialized field and methods are still being developed, and the expertise and resources to complete them are not widely available. Equally, as noted earlier, the results of such assessments are not always “accessible” to risk managers and other stakeholders. Thus, where a formal risk assessment (i.e. a body of work presented in a way that conforms to a set of risk assessment guidelines and specifically designed to estimate the magnitude of a risk) is commissioned by a 121 risk manager, a qualitative risk assessment may be specified for reasons including: • a perception that a qualitative risk assessment is much quicker and simpler to complete; • a perception that a qualitative risk assessment will be more accessible and easier for the risk manager or policymaker to understand and to explain to third parties; • an actual or perceived lack of data, to the extent that the risk manager believes that a quantitative assessment will be impossible; or • a lack of mathematical or computational skills and facilities for risk assessment, coupled with a lack of resources or desire to involve an alternative or additional source of expertise. Whatever the reasons, many of them involve perceptions about the process of defensible qualitative risk assessment that, for reasons also mentioned above, are frequently not valid. Data and knowledge are required for any type of risk assessment, irrespective of whether qualitative, semi-quantitative or quantitative approaches are used. Numerical data are preferred, and a lack of appropriate crucial data will affect all approaches adversely. As data collection and documentation is usually the most time-consuming part of any risk assessment, and defensible logic is required to synthesize the data, a qualitative risk assessment will not necessarily be quicker or simpler to complete. In many cases, however, qualitative and semi-quantitative risk assessments are quicker to complete, and, whilst they require an equal degree of logic and considerable numeracy, they require fewer specialized mathematical and computational resources. A qualitative risk assessment has descriptions of the probability of an unwanted outcome in terms that are, by their very nature, subjective. It means that it is not necessarily easier either for the risk manager to understand the conclusions obtained from the risk assessment, or to explain them to a third party. Crucial to any formal risk assessment method is transparency, whether to describe how a numerical or qualitative description of risk was achieved, because this enables users to understand the basis of the assessment, to understand its strengths and limitations, to critique the assessment, or provide additional information to improve the assessment. Additionally, all approaches require specialized medical, microbiological, biological, veterinary, epidemiological and other expertise. As a result, the inclusion of information and concepts from such a wide variety of areas of knowledge can make the risk assessment less accessible. Section 16.5 contains information about ways in which the results of risk assessment can be communicated. 122 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 9.1.1 The value and uses of qualitative risk assessment Risk assessment, at its simplest, is any method that evaluates, or attempts to evaluate, a risk. Qualitative risk assessment is not, however, simply a literature review or description of all of the available information about a risk issue: it must also arrive at some conclusion about the probabilities of outcomes for a baseline risk and/or any reduction strategies that have been proposed. Both CAC (1999) and OIE (2018) state that qualitative and quantitative risk assessments have equal validity, though they did not specifically consider semi-quantitative risk assessment. However, neither organization explains the conditions under which qualitative and quantitative risk assessments are equally valid, and there is debate among risk experts about methods and approaches to be applied for qualitative risk assessment, and criteria for their validity. The WTO Committee on SPS Measures notes some advantages of quantitative expressions of risk (WTO, 2000): ... quantitative terms, where feasible, to describe the appropriate level of protection can facilitate the identification of arbitrary or unjustified distinctions in levels deemed appropriate in different situations ... use of quantitative terms and/or common units can facilitate comparisons. However, when developing risk assessments, numerical results should be explained and put in context with a discussion of the limitations of the data and analysis, the important assumptions made, and the qualitative aspects of the risk not illuminated by quantitative analysis. The same requirement applies whether the assessment is quantitative or qualitative. It is sometimes the case that a qualitative risk assessment is undertaken initially, with the intention of following up with a quantitative risk assessment if it is subsequently thought to be needed. It may be the case that a qualitative assessment provides the risk manager or policymaker with all the information they require. For example, perhaps the information gathered includes some piece of evidence that shows that the risk is effectively indistinguishable from zero, and no more currently needs to be done. Conversely, perhaps evidence shows that the risk is obviously unacceptably large, or that one or more consequences are so unacceptable that safeguards are needed whatever the magnitude. Analogously, qualitative assessments can be used as a first step to quickly explore protective measures where there is expert consensus that such measures would be immediately effective and useful. As such, if there are obvious sources of risk that can be eliminated, one does not need to wait for CHAPTER 9 - QUALITATIVE AND SEMI-QUANTITATIVE RISK ASSESSMENT: FURTHER CONSIDERATIONS 123 the results of a full quantitative risk assessment to implement risk management actions. A qualitative risk assessment may also provide the necessary insights into the pathway(s) associated with the risk of concern, but not previously identified, which also allows the risk manager to make decisions or apply safeguards without further quantification. For example, FAO/WHO (2004) noted: Qualitative risk assessments may be undertaken, for example, using the process of ‘expert elicitation’. Synthesizing the knowledge of experts and describing some uncertainties permits at least a ranking of relative risks, or separation into risk categories. … As assessors understand how qualitative risk assessments are done, they may become effective tools for risk managers. 9.1.2 Qualitative risk assessment in food safety Qualitative risk assessments have been extensively used in import risk assessments of animals and of animal products (OIE, 2018). Many animal products are also food intended for human consumption; therefore, many of these import risk assessments have also involved such food products. However, the focus of such import risk assessments has historically been to assess the risk of a particular exotic pathogen entering a potential importing country, carried within the food in question. The intention is generally to assess whether the risk of importing the pathogen in the product is too high to be acceptable to the importing country, and whether safeguards should therefore be applied (such as cooking, freezing, testing or total ban). Frequently, further consequences, in particular any potential consequences to human health, have not been the focus of those risk assessments, even when the pathogen might be a zoonotic organism. Human health and safety risk assessments of food products, in general, not only set out to assess the probability of the presence of a pathogen, but also the amount of pathogen present, so that the human response to the probable dose can be assessed. The latter aspect is sometimes perceived to make qualitative risk assessments less useful in food safety applications, despite the fact that many quantitative dose– response data are very subjective in their estimation methods. However, not all steps in the risk assessment process (see Figure 2) are necessary in all cases to assist food safety risk managers to deduce appropriate risk management actions. Even in the absence of dose–response data, actions to reduce exposure would in many cases be appropriate risk management steps and could be determined from an incomplete risk assessment (i.e. without Hazard Characterization), whether qualitative or quantitative. 124 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 9.1.3 Characteristics of a qualitative risk assessment The complementary nature of qualitative and quantitative risk assessments The main principles of a risk assessment apply equally anywhere along the qualitative to quantitative risk assessment continuum. These include identification of the hazard; defining the risk question; outlining the steps of the risk pathway; gathering data and information, including information on uncertainty and variability; combining the information in a logical manner; and ensuring all is fully referenced and transparently documented. It follows from this that many of the activities are the same, up to and including the gathering of the data. Therefore, it is sometimes the case that a qualitative (or semi-quantitative) risk assessment is included in a risk profile, with the intention of following up with a quantitative risk assessment if it is needed. The detailed investigative nature of a qualitative risk assessment may provide the risk manager with all the information they require. A qualitative risk assessment may also provide the necessary insights into previously unidentified pathway(s) associated with the risk of concern, which allows the risk manager to make decisions or apply safeguards without further quantification. In these circumstances additional quantitative assessments will probably be deemed unnecessary by the risk manager. A qualitative risk assessment can be informative even if a quantitative assessment is being planned. It can be used to identify the data currently available, the uncertainties surrounding that data, and uncertainties about exposure pathways, to decide if quantification is both feasible and likely to add anything to the current state of knowledge. It can identify areas of data deficiency for targeting future studies necessary prior to quantification. It can examine the probable magnitude of the risks associated with multiple pathways, such as exposure pathways, prioritizing them for the application of quantification. Whatever the initial intention, when a qualitative risk assessment has already been undertaken, much of the work for a quantitative risk assessment has been done. For the same risk question, quantification will be able to build on the information and data already collected, to provide a numerical assessment of the risk. Subjective nature of textual conclusions in qualitative risk assessments Assessing the probability of any step in the risk pathway, or the overall risk, in terms of “High”, “Medium”, etc., is subjective, as the risk assessor(s) will apply their own meaning to these terms. These meanings may (and probably will) differ CHAPTER 9 - QUALITATIVE AND SEMI-QUANTITATIVE RISK ASSESSMENT: FURTHER CONSIDERATIONS 125 from person to person. This is one of the major criticisms levelled at qualitative risk assessments. However, these final risk estimates should never be viewed in isolation, just as numerical outputs from quantitative risk assessments should not. This reinforces the need for transparent documentation of the data and logic that lead to the assessor’s estimate of the risk. For a qualitative description of a risk to be useful to a risk manager, the assessor and manager must have similar perceptions of the meaning of subjective terms such as “Low”, “Negligible”, etc., or other descriptors (see also Section 7.2). A final risk characterization label, e.g. “Low”, is largely meaningless to a risk manager without some sort of indication of what constitutes “Low” in the eyes of the risk assessor. For example, the Canadian Food Inspection Agency defined five levels of health risk (including “Unable to assess”) as part of their risk classification (CFIA, 2019). Also, a label of “Low” gives little indication of which particular pieces of evidence would change the assigned label to something other than “Low”. Thus, if evidence were to be presented that 25 percent of the product was not stored frozen, would the risk increase to “Moderate”? Judgements will be used within any risk assessment. These may be the risk assessor’s judgements, or expert opinion, or both, and these will always be subjective. This will apply when defining the scope of the problem, selecting (and rejecting) data, delineating the risk pathways, applying weightings to data or model pathways, selecting the distributions in a stochastic model, etc., as well as selecting a description of “High”, “Low”, etc., in a qualitative assessment. Therefore, any risk manager, policymaker or other stakeholder who needs to use a given risk assessment, irrespective of where on the qualitative to quantitative spectrum the risk assessment lies, should not simply look at the final result. They should have some knowledge of how that result was arrived at. A definition of “Negligible risk” used in qualitative risk assessment is that, for all practical purposes, the magnitude of a negligible risk cannot, qualitatively, be differentiated from zero; for example, see the use of the term in OIE (2018). The term “zero risk” is not used because in microbiological food safety there is generally no such thing as absolutely no risk. Note that, since “Negligible” may be understood as “may be neglected”, it can be argued to be a risk management term because it involves a judgement. It must be emphasized, that qualitative risk assessment relies on as much numerical data as possible to provide model inputs despite their textual nature, and the process of data gathering should be equally as thorough as for a quantitative risk assessment. 126 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 9.2 SEMI-QUANTITATIVE RISK ASSESSMENT Semi-quantitative methods involve assigning labels to qualitative estimates in the form of probability ranges, weights or scores, and combining them by addition, multiplication, or other mathematical operation. The objective is to achieve a greater level of objectivity compared with qualitative approaches. There must be clear rules that dictate how the labels (scores, weights, ranges etc.) are combined. These rules should follow the basic probability principles and should be fully described and transparent regarding operation and result generation. This approach provides an intermediary level between the textual evaluation of qualitative risk assessment and the numerical evaluation of quantitative risk assessment. It offers a more consistent and rigorous approach to assessing and comparing risks than does qualitative risk assessment. This also avoids some of the greater ambiguities that a qualitative risk assessment may produce and it does not require the same mathematical skills as quantitative risk assessment. Semi-quantitative risk assessment may be an attractive option when data are limited, but as noted before, all forms of risk assessment require the greatest possible collection and evaluation of data available. Hence, all the data collection and analysis activities for qualitative risk assessment described in the previous section are also required for semi-quantitative risk assessment. As noted in the previous section, Codex and OIE consider just two categories of risk assessment: qualitative and quantitative. Semi-quantitative risk assessment, as described here, has often been grouped together with qualitative risk assessment, but this understates the important differences between them in their structure and their relative levels of objectivity, transparency and repeatability. Nevertheless, OIE consider that semi-quantitative methods do not offer any advantages over wellresearched, transparent, peer-reviewed qualitative approaches (OIE, 2018). 9.2.1 Uses of semi-quantitative risk assessment Semi-quantitative risk assessment is most useful in providing a structured way to rank risks and for ranking risk reduction actions for their effectiveness. This is achieved through a predefined scoring system that allows one to map a perceived risk into a category, where there is a logical and explicit hierarchy between the categories. Comparing hazards One example of the utility of the semi-quantitative risk matrix approach is a probability-severity table. This table is similar to the risk matrix shown in Table 14 in Section 7.3, with the difference that several hazards are listed in the cells of the table according to their likelihood and severity. This approach offers a quick way CHAPTER 9 - QUALITATIVE AND SEMI-QUANTITATIVE RISK ASSESSMENT: FURTHER CONSIDERATIONS 127 to visualize the relative “riskiness” (a term sometimes used for the combination of probability and severity) of several identified hazards within the domain of analysis. Table 34 illustrates a hypothetical example, where all hazards (e.g. the list of pathogens that might appear in a particular food type) are recorded in one table. This approach allows for the easy identification of the most threatening hazards (those closer to the upper right corner) and provides a general picture of the overall risk associated with the food type. The numbers in the table are indices for identified hazards. Hazards 2 and 13, for example, have the highest risk; hazards 3 and 7 have very low risk. Hazards with zero probability (hazards 11 and 14) or no severity (hazards 8, 9 and 10) do not pose a risk, but may be useful to document as having been identified and subsequently determined to have negligible risk. TABLE 34. Example of a Probability-Severity table for individual hazards (indicated by the numbers in the grid) per year (NIL=None, VLO = Very Low; Lo = Low; Med = Medium; Hi = High; VHI = Very High) VHI 6 13,2 HI 14 15 12 Severity MED 5 4 1 LO VLO 11 7 3 NIL 8,9 10 Zero VLO LO MED HI VHI Probability Risk scores can then be used to rank the identified risks. A scaling factor, or score, is assigned to each label used to describe each type of severity. If a log scale is used to define each categorical scale, as in the example provided in Table 11 for probability, then the probability and severity scores can be designed such that the risk score equals their sum, or some other simple mathematical equation. Table 35 provides an example of the type of scaling factors that could be associated with each probability and severity combination. 128 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD TABLE 35. Example risk score calculations for some hazards used in Table 34 Risk Index 13 1 5 Probability VHI HI VLO Probability Score 5 4 1 Severity VHI MED MED Severity Score 6 3 3 Risk Score 5+6=11 4+3=7 1+3=4 Comparing risks and risk management strategies Semi-quantitative risk assessment is also frequently used where one is attempting to optimize the allocation of available resources to minimize the effect of a group of risks. It helps achieve this in two ways: first the risks can be placed onto a type of “map” so that the most important risks can be separated from the less important; second, by comparing the total score for all risks, before and after any proposed risk reduction strategy, one can get a feel for how relatively effective the strategies are and whether they merit their costs. In particular, risk managers have to consider many factors in addition to risk, and multi-criteria decision making methods can be useful in these situations (e.g. FAO, 2017). 9.2.2 Characteristics of a semi-quantitative risk assessment Categorical labelling is the basis for semi-quantitative risk assessment. It uses nontechnical descriptions of a risk’s probability, severity, and risk (the combination of probability and severity), for example: “Very low”, “Low”, etc., or a scaling like A–F. For this type of labelling to be unambiguous and useful, risk managers must provide a list of the nonoverlapping, exhaustive categorical terms that are to be used, together with clear definitions of each term. For example, a “Low” probability might be defined as an event having between 10-3 and 104 probability of occurring in a year, and a “High” severity might be defined as an individual suffering longterm sequelae that materially affect their quality of life. This step is crucial, as a number of studies have shown that even professionals, who are well-versed in probability ideas and who regularly make decision based on risk assessments, have no consistent interpretations of probability phrases, such as “Likely”, “Almost certain”, etc. This lack of consistent interpretation could lead to inconsistent assessment of risk and inadvertent lack of transparency. Without numerical definitions of probability, subjective descriptions such as “Low” can be affected by the severity: for example, a 5 percent probability of diarrhoeal illness from some exposure might be considered “Low”, but a 5 percent probability of death from an CHAPTER 9 - QUALITATIVE AND SEMI-QUANTITATIVE RISK ASSESSMENT: FURTHER CONSIDERATIONS 129 exposure might be considered “High”. The number of categories used to express probability and severity should be chosen so that one can be sufficiently specific without wasting time arguing about details that will not ultimately affect the risk management decision. A five-point scale has been the most commonly used in the risk community. Often, while carrying out a qualitative risk assessment, one can roughly estimate the probability of exposure, etc., from comparison with other, previously quantified risks or from good data pertaining to the problem in hand. If time or the available data are insufficient to carry out a complete quantitative risk assessment, one can use these categorical labels to express the risk level in a more structured way than a simple, qualitative description of the evidence one has acquired. An example is presented in Section 7.3.2. 9.2.3 Limitations of semi-quantitative risk assessment A semi-quantitative risk assessment has its limitations and can cause errors in conclusions, see for example Cox Jr. (2008), Levine (2012), and Vatanpour, Hrudey and Dinu (2015) for a discussion of the issues with an emphasis on risk matrices. These issues arise from several difficulties in defining how categorical labels should be interpreted and manipulated. The risks are placed into usually quite broad sets of categories, and as noted before it is common to use five or so for probability and for severity, not including zero, which gives 25 possible combinations. It is therefore imperative that the categories are carefully constructed. For example, one could break up the probability range into five categories, as in Table 36. TABLE 36. A linear scoring system for probability Score Probability range 1 0 – 0.2 2 0.2 – 0.4 3 0.4 – 0.6 4 0.6 – 0.8 5 0.8 – 1 However, under this scheme, a risk with a probability of 0.1 would sit in the same category (Score 1) as a risk with probability 0.000 001, despite being 100 000 times more likely. This is one reason why a log scale is often chosen for probabilities. The nature of food safety risk means that probabilities often span several orders of 130 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD magnitude, which also makes the use of a log scale more appealing and informative. It is not easy to combine probability scores for components of a risk pathway to get a probability score for the overall risk. For example, food safety risk estimation is often split into two parts: the probability of exposure; and the probability of illness given exposure. Using the scheme above, if the exposure had a 0.3 probability (score = 2) of occurring within a certain period for a random individual, and the probability of illness from that exposure was 0.7 (score = 4), then the combined probability is 0.21 (0.3×0.7=0.21), which receives a score 2. It is not easy to create a rule with scores that replicates the probability rules, and this limitation is well recognised (see references above). Taking the minimum of the two scores is one partial solution, but this generally overestimates the result. For example, changing the probability of illness given exposure to anything from 0.2 to 1.0 would give the same combined probability score of 2 using this approach. The use of a log scale for probability relieves the problem, to some extent, if the probability score order described so far is reversed, i.e. to assign the highest probability to the lowest score, as shown in Table 37. TABLE 37. A logarithmic scoring system where the highest score is assigned the lowest probability Category Negligible Very Low Low Medium High Very High Certain Probability range Indistinguishable from 0 < 10-4, (except 0) 10-4 to 10-3 10-3 to 10-2 10-2 to 10-1 > 10-1 (except 1) 1 Score NA 5 4 3 2 1 0 Using this scheme, the scoring system equivalent of multiplying probabilities is to add scores. For example, if the exposure has a 0.2 probability (score = 1) of occurring within a certain period for a random individual, and the probability of illness from that exposure is 0.004 (score = 3), the combined probability is 0.0008 (score 4). However, it does not always work out so neatly. An exposure with probability 0.5 (score = 1) and a probability of illness from that exposure of 0.003 CHAPTER 9 - QUALITATIVE AND SEMI-QUANTITATIVE RISK ASSESSMENT: FURTHER CONSIDERATIONS 131 (score = 3) gives a combined probability of 0.0015 (score = 3), yet the individual scores sum to 4. Adding scores in a log system like the one in Table 37 will often overestimate the probability by one category. This is one reason for having an amber region in the traffic light system (Table 14), because risks may be overestimated, and risks falling into an amber region may in fact turn out to be acceptable. In addition, there is the problem of the granularity of the scale, as discussed in 7.3.2. However, there is nothing to stop the risk assessor from using score fractions if it seems appropriate. The integer system is designed for convenience and simplicity and could be changed to include fractions if this better represents the available knowledge. Using the semi-quantitative risk assessment scoring system as a surrogate for probability calculations is also likely to cause more severe inaccuracies when one assesses a longer sequence of events. This is because the “errors” are being compounded; see for example the “Probabilities Are Inconsistent with Qualitative Aggregation Rules” (Cox Jr., Babayev and Huber, 2005) 132 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 10 10. Data Risk assessment studies are developed by compiling information from a variety of data sources. Each of these data sources contributes in varying degrees to an understanding of the interaction between the pathogen, host, and matrix (Figure 4) that affect the potential public health risks attributable to a disease agent. An appreciation of the strengths and limitations of the various data sources is critical to selecting appropriate data for use, and to establishing the uncertainty associated with different data sets and test protocols. Active data collection is often required, because reliance on passive data submission or data in published form often does not provide enough information in sufficient detail needed, especially for QMRA models. Relevant data come preferably from peer-reviewed journals. In case of insufficient data from published sources, it is also advisable to evaluate the availability of unpublished, high-quality data sources. Risk assessors should communicate with experimenters, epidemiologists, food or water safety regulators, and others who may have useful data that could contribute to the analysis. An example is the outbreak information collected by the Japanese Ministry of Health (Kasuga et al., 2004) and which was used for dose– response modelling of Salmonella, along with other data (FAO and WHO, 2002a). When such data are used, the criteria and results of evaluation must be carefully documented. If using material published on the Internet, then care should be taken to establish the provenance, validity and reliability of the data, and the original source, if possible. 133 Understanding the characteristics of data sources is important to the selection and interpretation of data. Risk assessors often use data for a purpose other than that for which it was originally intended. Risk assessors and modelers need to know how the data they use were collected, and the purpose of their collection. Two categories of data are necessary for the development of a risk assessment model: first, data that, in text format, describe the biological and physical processes as well as the human factors involved, and second, numerical data that allow quantitative estimates to be calculated. The extent to which numerical data are required will vary from one risk assessment to another, depending on the defined purpose, scope, modelling approach and details chosen. An overview of the types of data required for conducting a risk assessment and their sources is presented in Table 38, and these are described in detail in the following sections. Data should be collected to represent reality as closely as possible. This principle applies irrespective of the risk assessment, that is, it applies to fisheries as it does to primary production, or to food service (catering) or home preparation as the point of consumption. Note that the specific scope and purpose of a risk assessment can be much narrower in practice and these will determine the type and detail of data required. Since data are not available in all instances, alternative (surrogate) data may need to be used. It is important to clearly describe the rationale and suitability for selecting the alternative data and evaluate the effect of using such data on the final risk estimates (Chapters 14 and 15). This chapter presents a summary of the types of data typically required for constructing a risk assessment, capturing in brief the strengths and limitations of each of the data sources. 134 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD TABLE 38. Data required for risk assessment and data collection sources Type of Data Description Hazard Identification Association between The evidence that can be utilized to pair the food and exposure and adverse health microbiological hazard and link the exposure to hazard in outcome specific food to human illnesses Microbiological hazard characteristics General characteristics of food and conditions of supply chain Adverse health outcomes in exposed population Prevalence and concentration Characteristics of the organisms and mechanism with which the organism affects the host are described, while detailed dose– response analysis is done in hazard characterization Intrinsic characteristics of the food (e.g. pH, water activity) and process evaluation (e.g. time, temperature) Disease and sequelae in population and subpopulations by demographic and/or social-economic factors, sensitive population Exposure Assessment Data on prevalence and concentration of the hazard in the food at the starting point of the risk assessment and other points of the food chain Processing conditions Effect of processing stages and/or interventions Product characteristics Data describing the conditions of food processing which may affect prevalence and concentration of the hazard, i.e. time– temperature of thermal processing, fermentation, partitioning, etc. Data on the effect of a processing stage/intervention on prevalence and concentration of the hazard Data on food characteristics (pH, aw, concentration of antimicrobials, packaging atmosphere, use-by-date, etc.) that may affect the behaviour of the hazard during storage Collection Source • Outbreak data • Foodborne disease surveillance and annual health statistics • Food safety rapid alert systems • Literature: Analytical epidemiological studies • Systematic food contamination monitoring surveys • Literature: Microbiological studies • Industry data and literature: Description of product and food supply chain • Scientific and medical literature • Systematic food contamination monitoring surveys • Literature: Prevalence and concentration surveys • Expert Knowledge Elicitation (EKE) • Literature and Industry data: Description of product and supply chain • EKE • Literature: Intervention studies • EKE • Literature and Industry data: Description of product and supply chain • EKE (cont.) CHAPTER 10 -DATA 135 136 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Type of Data Distribution and storage conditions Conditions of food handling and preparation Kinetics of hazard’s behaviour Consumption Population segments Description Time–temperature data for distribution and storage of the food at retail and domestic level Data describing the conditions of food handling and preparation which may affect prevalence and concentration of the hazard, i.e. time and temperature of cooking, partitioning, etc. Data on the kinetics of hazard’s growth/survival/inactivation during food processing, distribution, storage, handling and cooking. Data on serving size, frequency of consumption, and number of annual servings for different population groups (normal, susceptible, etc.). Data on population size by segments Collection Source • Literature and Industry data: Description of product and supply chain • EKE • Literature: Cross-contamination, food handling and preparation • EKE • Literature: Predictive microbiology models • Online modelling tools • National consumption databases • Total diet studies • EKE • National population census Annual production of the food commodity Dose–response data Parameters of dose– response models Annual cases of the foodborne illness and prevalence of the pathogen in a food commodity Annual cases of foodborne illness Data on amount of food produced in a country and information of imports and exports, if necessary Hazard Characterization Dose–response data that can be used in fitting a dose–response model Parameters estimated by fitting dose–response models to data • National food production statistics • Outbreak data • Volunteer feeding studies • Animal studies Literature: fitted dose–response models Data on reported cases of illness and prevalence of the causative • Foodborne disease surveillance and annual hazard in the food commodity to validate a dose-response health statistics relationship • Systematic monitoring surveys Risk Characterization Data used for anchoring and/or validating a risk assessment model • Foodborne disease surveillance and annual health statistics 10.1 LITERATURE (PRIMARY AND/OR META-ANALYSIS) Data required for risk assessments may come from academia and other organizations and a wide variety of published sources. This can be in the form of documents that have been peer-reviewed within the scientific community or via non-peerreviewed written communications (conference proceedings, books, internet sites). Data from different sources may be helpful in confirming the degree of scientific agreement or uncertainty on a particular point. In particular, knowledge syntheses or systematic reviews may provide a good starting point when evaluating the literature (e.g. via Health Evidence) (Health Evidence, 2021), and they can also be developed as a stand-alone document as part of a risk assessment. In most cases, data need to be extracted from sources that are not intended for that specific purpose. Consequently, data may not be readily available in the exact form or detail required for the risk assessment. At this point, meta-analysis can be considered as a useful tool for combining or pooling data from different sources in a structured way. In building the risk assessment model, separate meta-analyses can be carried out to estimate the overall effect of a certain processing stage or intervention on likelihood/concentrations of a hazard at any particular point in the food chain (Gonzales-Barron et al., 2017). Multilevel meta-analysis models that account for the effect of selected moderators can also be used. For example, in Prado-Silva et al. (2015), such models were developed to summarize the effects of sanitizing treatments on Salmonella spp., E coli O157:H7 and L. monocytogenes in fresh produce, as affected by type of sanitizer and washing time and temperature. Risk assessors familiar with meta-analysis techniques may conduct meta-analysis on summary statistics or on raw data. Original data may need to be requested from authors when the data are critical for a risk assessment. Human resources and availability of sufficient primary research sources will constrain the use of metaanalysis. Scientific publications often give a good level of detail about the subject matter being investigated. The conditions under which the data were obtained, and the methods used are often well documented. If a number of individual studies addressing the same research question have been found, meta-analysis can be conducted to obtain a more representative overall estimate. However, a drawback of published research is that, in many cases, aggregate data rather than raw data are published and that raw data may be difficult to access. CHAPTER 10 -DATA 137 Some journals are encouraging authors to make their raw data and supplementary materials available, e.g. International Journal of Food Microbiology. The diversity in languages used for publications can pose a barrier to general access and use. Uncertainty and variability in the data are generally not described, and authors might need to be consulted to obtain information on those aspects. Some research may be published but hard to locate due to a lack of readily accessible computer listings for items like fact sheets, conference proceedings, theses, dissertations, etc. Another potential downside of published research is the potential for publication bias. This type of bias occurs because publishers prefer to publish novel research findings, rather than confirmatory research. As a result, the reported effects, e.g. for the efficacy of an intervention, may be larger than what might be expected in general and this type of “error” has been referred to as a Magnitude Error (M-Error) by Gelman and Carlin (2014). Publication bias directly affects meta-analysis, although there are procedures to adjust the meta-analytical estimates when publication bias is likely to be present (Rothstein, Sutton and Borenstein, 2006). 10.1.1 Analytical epidemiological studies Epidemiological surveys concern studies that have been commissioned to specifically investigate the causal relationship between the occurrence of foodborne illness and exposure to certain microbiological hazards through food consumption. They are most commonly undertaken as part of outbreak investigations, e.g. case– control or cohort study. These studies can be useful for hazard identification and characterization. Strength Epidemiological studies are very specific and provide a large amount of detailed information on the hazard and the consumer group investigated. Limitations Data are often generated for a relatively small number of consumers, and thus are not necessarily representative of larger consumer groups. 10.1.2 Microbiological studies of prevalence and counts/ concentrations The microbiological studies discussed here refer to studies reporting the prevalence and count/concentration of target microorganisms at various stages in the food chain. Included are those studies reporting the change in hazard prevalence/ concentration, such as the efficacy of a processing intervention. These studies may 138 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD report findings throughout the production and processing chain, including in the final food product. They are especially useful for the exposure assessment but may also inform the hazard identification. Strength Results from those studies provide useful information as the initial input or data needed for connecting the parts of the exposure model (e.g. Figure 5). Microbiological surveys undertaken at retail can provide valuable data to verify that the exposure model prediction (up to the retail stage) are comparable with what is observed at retail, i.e. a reality check. For studies related to interventions (or growth or survival) the processing conditions such as durations, temperatures, etc. are often reported and these provide useful inputs into predictive microbiological models. Limitations These studies often present results in an aggregate form, e.g. mean and standard deviation. Where possible, the raw data, without identifying information, should be requested from the authors as this will allow more detailed interrogation of the data than may be presented in a scientific publication. This also allows statistical distributions of the data to be better ascertained and summary statistics (including variances) to be calculated and assessed for different components of the study. Such intricacies may not be included in the scientific publication, possibly because the data were not specifically collected for use in a risk assessment. It often happens that different laboratories use different microbial testing methods that are not measuring the same feature. Therefore, when reviewing published articles investigating the same research question, comparability of results should be appraised to see if these sources are in effect measuring the same thing or not, and if so, whether the same level of uncertainty exists. Differences in testing method comparability are probably the most difficult to resolve when attempting to compare final estimates. Ensuring that internationally validated microbiological methods are used can facilitate this comparison. For example, in some tests, different laboratories may use methods with different detection limits. Nevertheless, there have been advances to take into account the analytical test performance when analysing data, without having to resort to biased “substitution” methods (Busschaert et al., 2011; Evers et al., 2010; Gonzales-Barron et al., 2010; Lorimer and Kiermeier, 2007; Shorten, Pleasants and Soboleva, 2006; Williams and Ebel, 2014). CHAPTER 10 -DATA 139 Studies that utilise only molecular methods to detect microorganisms, without culturing the organism of interest, are unable to determine whether the organism is viable, i.e. is infective. Molecular methods rely on detecting specific fragments of DNA, and these fragments may originate from damaged and unviable units. This is particularly relevant for organisms that cannot readily be cultured, such as some viruses, and as a result the prevalence and/or concentration estimates will be overestimated. 10.1.3 Cross-contamination data during food processing The potential for microbial cross-contamination within the food processing environment is well recognized. Data and models that give insight into the extent to which this occurs, e.g. transfer rate, are therefore required. Important areas will include, for instance, the level of contact between live and slaughtered animals or between raw and processed vegetable material, worker hygiene, operating equipment, plant design, sanitation protocols, and methods of packaging (Gallagher et al., 2016; Pouillot et al., 2015a; e.g. Zoellner et al., 2019). Strength These studies can provide quantitative information on the frequency, extent and type of cross-contamination events that occur in a food processing environment. This allows better modelling of the cross-contamination as part of exposure assessment. Limitations Due to the amount of time involved in observing a reasonable number of crosscontamination events, and the variability between observation times (e.g. days or shifts), these types of studies likely involve only one or a few different food processing environments. Consequently, the results may be specific to the environment that has been observed and may not be representative of the whole industry. 10.1.4 Food handling and preparation Storage and preparation practices, both in the home and in the catering environment, can affect the level of exposure. In particular, hazard growth or reduction may occur during storage prior to preparation if the temperature favours either of these processes; reduction in hazard contamination may occur as a result of cooking; and hazard concentration in cooked products may increase due to cross-contamination. To address these issues, data should be accompanied by descriptions of relevant details such as: times and temperatures of storage; typical handling practices and the potential cross-contamination events that could occur 140 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD during preparation; the extent to which these events occur and the likely numbers of organisms transferred to different locations within the kitchen; the extent to which consumers are exposed to the organisms that are transferred; and typical cooking times and temperatures. Predictive microbiology models will be needed for these stages as well to assess potential changes in levels of hazards and the resultant effect on risk. Research has been undertaken on consumer practices, although the work tends to be product and situation specific (e.g. DeDonder et al., 2009; Kosa et al., 2015). As a result, still relatively little information exists on food handling practices in the home that affect the safety of foods, although this situation is gradually changing (Chardon and Swart, 2016; Murray et al., 2017; Young et al., 2017a). Food handling practices vary by geographical region or even within the same country, based, for example, on ethnicity, gender and education. Consumer storage times, extent of cross-contamination, cooking times and temperatures (such as reported by EcoSure, 2008), hot holding temperatures and times, and other data are not generally available. Likewise, relatively little information is available about food handling practices by restaurant and food service operations, including street food, which accounts for an increasingly greater proportion of meals in many countries. This data gap is also gradually being addressed (Pichler et al., 2014; Samapundo et al., 2015; Tessema, Gelaye and Chercos, 2014). Some research is now being undertaken using human volunteers who are asked to prepare specific foods in custom kitchen that allow observation and video recording of the study participants, so that food handling practices can be quantified and objectively evaluated against prespecified criteria. Strength Directly observing food handling practices and measuring food storage, cooking or associated metrics (e.g. temperature) result in less uncertainty than information obtained through an interview. That is, observation allows recording of what people do, rather than what they say they do. Recording video footage of food preparation is also a good way to reduce researcher bias. The actual food handling practices can be assessed “blindly” through an independent third-party. However, care must be taken that the specific practices that are assessed have been well described and documented to ensure consistency. Limitations It is difficult to observe food handling practices directly as they are practiced in homes and food service operations, especially when researchers want to capture CHAPTER 10 -DATA 141 video footage. The best alternative is to use purpose-built food preparation kitchens that allow observation. However, these are costly to establish and to maintain (including the qualified staff to undertake studies). These types of studies can pose ethical problems and they cannot be undertaken in a “blind” way. That is, volunteers know that they are being observed and because of this they may change the way they handle the food. Where measurements are involved (e.g. EcoSure, 2008) care must be taken that equipment is properly calibrated and that raw data are critically checked for recording errors. 10.1.5 Human volunteer feeding studies The most obvious means for acquiring information on dose–response relations for foodborne and waterborne pathogenic microorganisms is to expose humans to the hazard under controlled conditions. There have been a limited number of hazards for which feeding studies using volunteers have been carried out. Most have been in conjunction with vaccine trials. Examples of the use of volunteer studies to develop dose–response models for a range of enteric pathogens are provide by Teunis et al. (1996), which includes references to the original experimental studies. These studies are generally conducted only with primarily healthy individuals between the ages of 18 and 50, and thus do not examine the segments of the human population typically most at risk. Hazards that are life threatening or that cause disease only in high-risk subpopulations are not amenable to volunteer studies. Typically, the studies investigate a limited number of doses with a limited number of volunteers per dose. The dose ranges are generally high to ensure a response in a significant portion of the subjects. However, the doses are generally much higher than those of interest to risk assessors. The process of (self-)selection of volunteers may induce bias that can affect interpretation of findings. Feeding studies are not a practical means to address strain virulence variation. The choice of strain is therefore a critical variable in such studies. Most feeding studies have used only rudimentary immunological testing prior to exposure. More extensive testing could be useful in developing susceptibility biomarkers. Usually, feeding studies involve only a few strains, which are often laboratory domesticated or collection strains and may not, or no longer, represent strains 142 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD that occur in food. In addition, the conditions of preparation immediately before administration are not usually standardized or reported, though these may affect tolerance to acid, heat or drying, as well as altering virulence. For example, passage of Vibrio cholerae through the gastrointestinal tract induces a hyperinfectious state, which is perpetuated even after purging into natural aquatic reservoirs. This phenotype is expressed transiently, and lost after growth in vitro (Merrell et al., 2002). In many trials with enteric organisms, they are administered orally with a buffering substance, specifically used to neutralize the effect of gastric acidity, which does not directly translate into what the dose–response would be if the hazards is ingested in food or water. Strengths Using human volunteers is the most direct means of acquiring data that relates an exposure to a microbial hazard with an adverse response in human populations. If planned effectively, such studies can be conducted in conjunction with other clinical trials, such as the testing of vaccines. The results of the trials provide a direct means of observing the effects of the challenge dose on the integrated host defence response. The delivery matrix and the pathogen strain can be varied to evaluate food matrix and pathogen virulence effects. These studies can provide information on both infection, e.g. by testing faecal matter for the hazard of interest, and illness, e.g. by observing symptoms in the volunteers. Limitations There are severe ethical and economic limitations associated with the use of human volunteers. Especially because of the ethical implications these studies are no longer undertaken. However, for the purpose of better interpretation and utilization of the data reported in the literature, the aspects that are commonly considered in the development and assessment of an experimental design are listed below. • What isolate, species, serotype and/or genotype, strain, etc. of the hazard was used? • How is dose measured (both units of measurement and the process used to measure a dose)? • How do the units in which a dose is measured compare with the units of measurement for the hazard in an environmental sample? • Total units measured in a dose may not all be viable units or infectious units. CHAPTER 10 -DATA 143 • Volunteers given repeat doses may not all receive the same amount of inoculum. • How is the inoculum administered? Does the protocol involve simultaneous addition of agents that alter gastric acidity or promote the passage of microorganisms through the stomach without exposure to gastric acid? • How is it known that the volunteers are naïve? Serum antibodies may have dropped to undetectable levels or the volunteer may have been previously infected with a similar pathogen that may not be detected by the serological test. • How is infection defined? • What is the sensitivity and specificity of the assay used to determine infection? • How is illness defined? 10.1.6 Animal studies Animal studies are used to overcome some of the logistical and ethical limitations that are associated with human volunteer feeding studies. There are a large variety of different animal models that are used extensively to understand the hazard, host and matrix factors that affect characteristics of foodborne and waterborne disease, including the establishment of dose–response relations. Strengths The use of surrogate animals to characterize microbial hazards and establish dose– response relations provides a means for eliminating a number of the limitations of human volunteer studies while still maintaining the use of intact animals to examine disease processes. Animal models can be relatively inexpensive, thus increasing the potential for testing a variety of strains and increasing the number of replicates and doses. The animals are generally maintained under much more controlled conditions than human subjects. Immunodeficient animal strains and techniques for suppressing the immune system and other host defences are available and provide a means for characterizing the response in special subpopulations. Testing can be conducted directly on animal subpopulations such as neonates, aged or pregnant populations. Different food vehicles can be investigated readily. Limitations A major limitation is that the response in the animal model has to be extrapolated to that in humans. There is seldom a direct relationship between the response in animals and in humans. Often, differences between the anatomy and physiology of humans and animal species lead to substantial differences in dose–response relations and the animal’s response to disease. For a number of food pathogens, 144 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD it can be challenging to select an appropriate animal model, as the successful extrapolation from the animal to the human population depends on several factors, such as the similarity of pathogenic mechanisms, the physiological and immune responses between animals and humans (Buchanan, Smith and Long, 2000). Several highly effective models (e.g. primates or pigs) can be expensive and may be limited in the number of animals that can be used per dose group. Some animals used as surrogates are highly inbred and consequently lack genetic diversity. Likewise, they are healthy and usually of a specific age and weight range. As such, they generally do not reflect the general population of animals of that species, let alone the human population. Ethical concerns over animal experimentation need to be carefully considered and, in many countries, limit the range of biological endpoints that can be studied. When data derived from humans are absent, the validation of dose–response models built on animal studies is challenging. However, there are some general considerations regarding animal models to narrow the difference between animal models and human target. When surrogate pathogens or surrogate animal models are used, the biological basis for the use of the surrogate must be clear. Using data obtained with animal models to predict health effects in humans could take advantage of the use of appropriate biomarkers. It is important to use pathogen strains that are identical or closely related to the strain of concern for humans, because, even within the same species and subspecies, different strains of pathogens may have different characteristics that cause variation in their abilities to enter and infect the host and cause illness. 10.1.7 In vitro studies In vitro studies involve the use of cell, tissue or organ cultures and related biological samples to characterize the effect of the hazard on the host. They are of most use for qualitative investigations of pathogen virulence but may also be used to evaluate in detail the effects of defined factors on the disease process. For example, the effect of food processing and preservation conditions on a pathogen’s virulence and toxin production can be evaluated by in vitro studies (Greppi and Rantsiou, 2016; Haddad et al., 2018). Strengths In vitro techniques can readily relate the characteristics of a biological response with specific virulence factors (genetic markers, surface characteristics and growth potential) under controlled conditions. This includes the use of different host cells or tissue cultures to represent different population groups to characterize differences CHAPTER 10 -DATA 145 in dose–response relations between them. Furthermore, in vitro techniques allow the environment under which the host cells or tissues are exposed to the hazard to be modified. In vitro techniques can be used to investigate the relations between matrix effects and the expression of virulence markers. Large numbers of replicates and doses can be studied under highly controlled conditions. These techniques can be used to readily compare multiple species and cell types to validate relationships between humans and surrogate animals (Section 10.1.6). They are particularly useful as a means of providing information concerning the mechanistic basis for dose–response relations. Limitations The primary limitation is the indirect nature of information concerning the dose– response relationship. One cannot directly relate the effects observed with isolated cells and tissues to disease conditions that are observed within intact humans, such as the effect of integrated host defences. To compare with humans, a way to relate the quantitative relations observed in the in vitro system to those observed in the host is needed. For many organisms, the specific virulence mechanisms and markers involved are unknown, and may vary between strains of the same species. Similar to some other data types, such as public health surveillance, these types of studies are usually limited to providing details of factors affecting dose–response relations and to augmenting the hazard characterization. However, they are unlikely to be a direct means of establishing dose–response models useful for risk assessments. 10.1.8 Biomarkers Biomarkers are measurements of host characteristics that indicate exposure of a population to a hazard or the extent of adverse effect caused by the hazard. Examples include serological assays, counts of subsets of white blood cells and production of gaseous oxides of nitrogen. Biomarkers generally involve minimally invasive techniques that have been developed to assess the status of the host. Also “omics” (transcriptomics, metabolomics) type biomarkers can be used (Haddad et al., 2018). The United States of America’s National Academy of Science has classified biomarkers into three classes (National Research Council, 1989; Slikker Jr., 2018), as follows: • Biomarker of exposure: An exogenous substance or its metabolite, or the product of an interaction between a xenobiotic agent and some target molecule or cell, that is measured in a compartment within an organism. 146 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD • Biomarker of effect: A measurable biochemical, physiological or other alteration within an organism that, depending on magnitude, can be recognized as an established or potential health impairment or disease. • Biomarker of susceptibility: An indicator of an inherent or acquired limitation of an organism’s ability to respond to the challenge of exposure to a specific xenobiotic substance. Even though this classification was developed against the background of risk assessment of toxic chemicals, these principles can be useful in interpreting data on microbial hazards. In future, the gut microbiome might be related to disease susceptibility. Strengths These techniques provide a means of acquiring biologically meaningful data while minimizing some of the limitations associated with various techniques involving human studies. Typically, biomarkers are measures that can be acquired with minimum invasiveness while simultaneously providing a quantitative measure of a response that has been linked to the disease state. As such, they have the potential to increase the number of replicates or doses that can be considered, or to provide a means by which objectivity can be improved, and increased precision and reproducibility of epidemiological or clinical data can be achieved. Biomarkers may also provide a means for understanding the underlying factors used in hazard characterization. A biomarker response may be observed after exposure to doses that do not necessarily cause illness (or infection), e.g. Lefkowitz et al. (1992) noted antibodies to V. vulnificus in shellfish industry workers. Biomarkers can be used either to identify susceptible populations or to evaluate the differential response in different population subgroups, for example, Egorov et al. (2018) noted the application of salivary immunoassay in a prospective community study of waterborne infections. It should also be noted that the most useful biomarkers are linked to illness by a defined mechanism, that is, the biological response has a relationship to the disease process or clinical symptom. If a biomarker is known to correlate with illness or exposure, then this information may be useful in measuring dose–response relationships, even if the subjects do not develop clinical symptoms. Biomarkers such as these can be used to link animal studies with human studies for the purposes of dose–response modelling. This is potentially useful because animal models may not produce clinical symptoms similar to humans. In which case, a biomarker may serve as a surrogate endpoint in the animal. CHAPTER 10 -DATA 147 Limitations Biomarkers are often indicators of infection, illness, severity, duration, etc. As such, there is a need to establish a correlation between the amplitude of the biomarker response and illness conditions. Biomarkers primarily provide information on the host status, unless protocols are specifically designed to assess the effects of different pathogen isolates or matrices. The only currently available biomarkers for foodborne and waterborne pathogens are serological and salivary assays. The main limitation for such assays is that, in general, the humoral immune response to bacterial and parasitic infections is limited, transient and nonspecific. For example, efforts to develop an immunological assay for E. coli O157 infections have shown that a distinctive serological response to the O antigen is seen typically in the most severe cases, such as those with bloody diarrhoea, but can be absent in less severe cases, such as cases with bloodless diarrhoea. In contrast, serological assays are often quite good for viruses. Another limitation is that some biomarkers, such as serological assays, can result in false positives. For serological assays, the presence of antibodies that crossreact with microbial antigens used in the assay or interfering substances that interact with assay components can also lead to falsepositive results. Thus, positive Immunoglobulin M (IgM) assay results require cautious interpretation, that is, consideration of clinical course compatibility and epidemiological factors and/or confirmation by other serological or molecular testing methods (Woods, 2013). Other biomarkers, such as counts of subsets of white blood cells or production of gaseous oxides of nitrogen are possible but have not been tested extensively in human populations. 10.2 NATIONAL AND INTERNATIONAL SURVEILLANCE DATA 10.2.1 Food safety rapid alert systems A food safety rapid alert system allows national food control authorities to share information about measures taken in response to serious risks detected in relation to food, and as such can provide useful information for hazard identification. This exchange of information helps countries to act more rapidly and in a coordinated manner in response to health threats caused by food. One example of such a system is the European Rapid Alert System for Food and Feed (RASFF) (European 148 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Commission, 2021). Through the RASFF consumers’ portal, the latest information on food recalls, public health warnings and border rejections in all EU countries can be accessed. The functioning principle of the RASFF is simple: if a member of the network has any information relating to the existence of a direct or indirect risk to human health deriving from food or feed, that information must be immediately notified to the EC and, where the EU member states are involved, to EFSA. The EC disseminates this information immediately to all members of the network. Strengths Food safety alert systems enable data sharing between geographically linked parties in an efficient manner and provide a real-time service to ensure that urgent notifications are sent, received and responded. The data should be representative of the food within a diverse but geographically linked region. Limitations These systems are only as good as the least active member. If one country does not have the resources or expertise to easily contribute data, then the resulting dataset is limited or skewed toward the other countries in the system. Similarly, the system will likely have good information about common and well recognized hazards, which tend to be part of national surveillance activities or outbreak investigations (Sections 10.2.1 and 10.2.3). Emerging hazards, those that are not actively surveyed or those that do not require reporting under a national health system may be less likely to be captured in a rapid alert system, unless a large enough outbreak has been identified and reported. While rapid alert systems can be excellent sources of information for when a hazard has been identified in a food, they usually do not provide useful information about prevalence of the hazard occurring. This is because the denominator is not generally captured, i.e. information about the food units in which a hazard has not been detected are not reported. In addition, if a hazard has not been reported for a particular food product in a rapid alert system, then this does not imply that the hazard does not occur in that food – it simply means that the food–hazard combination has not be reported in the system, either because the food has not been tested for the hazard, or because the hazard has not (yet) been detected in the food. CHAPTER 10 -DATA 149 10.2.2 Outbreak data When there is a common-source outbreak of foodborne or waterborne disease of sufficient magnitude, an epidemiological investigation is generally undertaken. This investigation aims to identify the cause of the problem, to limit its further spread, and to provide recommendations on how the problem can be prevented in the future. Such information can be particularly valuable for hazard identification and characterization. An outbreak of confirmed aetiology that affects a clearly defined group can provide very good information about the range of illness that a hazard can cause, particular host characteristics that may increase or decrease the risk, and – if there is clinical follow up – the risk of sequelae. When the outbreak is traced to a food or water source that can be quantitatively cultured, the subsequent data may contribute to the dose–response relationship to be estimated. Even when that is not possible, dose–effect relations can often be observed that show variation in clinical response to changes in relative dose and this is part of the classic approach to an outbreak investigation. Outbreak investigators may look for higher attack rates among persons who consumed more of the implicated food but may also include variation in symptom prevalence and complications. There are good public health reasons for gathering information on the amount of the implicated food or water consumed. An outbreak that is characterized by a low attack rate in a very large population may be an opportunity to define the host–response to very low doses of a hazard, if the actual level of contamination in the food can be measured. In addition, data from outbreaks are the ultimate “anchor” for dose–response models and are an important way to validate risk assessments (see also Section 16.2.2). In general, information on several outbreaks – including the dose and the attack rate – is needed to establish a dose–response model, as each outbreak essentially contributes one data point to which the dose–response model is fitted. Examples include the dose–response models for Salmonella (FAO and WHO, 2002a) and E. coli O157:H7 (Strachan et al., 2005). Strengths An outbreak investigation can capture the diversity of host responses to a single pathogenic strain, down to the DNA level, e.g. using whole genome sequencing (e.g. Smith et al., 2019). This can include the definition of the full clinical spectrum of illness and infection, if a cohort of exposed individuals can be examined and tested for evidence of infection, e.g. using a case–control study. This may be undertaken independent of whether they were ill enough to seek medical care 150 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD or diagnose themselves. It also includes definitions of subgroups at higher risk, and host factors that may increase or decrease that risk, given a specific exposure. Collecting information on underlying illness or pre-existing treatments is routine in many outbreak investigations. Obtaining highly specific details of the food source and its preparation in the outbreak setting is often possible, because of the focus on a single food or meal. The investigation may suggest specific correlates of risk that cannot be determined in the routine evaluation of a single case. Often, the observations made in outbreaks suggest further specific applied research to determine the behaviour of the hazard in that specific matrix, handled in a specific way. For example, after a large outbreak of shigellosis was traced to chopped parsley, it was determined that Shigella sonnei grows abundantly on parsley left at room temperature if the parsley is chopped, but does not multiply if the parsley is intact (Wu et al., 2000). Such observations are obviously important to someone modelling the significance of low-level contamination of parsley. Where samples of the implicated food or water can be quantitatively assayed for the hazard, in circumstances that allow estimation of the original dose, an outbreak investigation has been a useful way to determine the symptoms associated with a defined dose in the general population. Follow-up investigations of a (large) cohort of cases may allow identification and quantification of the frequency of sequelae, and the association of sequelae with specific strains or subtypes of a pathogen. If preparations have been made in advance, then the outbreak may offer a setting for the evaluation of methods to diagnose infection, assess exposure or treat the infection. Limitations The primary limitation is that the purpose and focus of outbreak investigations is to identify the source of the illness and to prevent additional cases. The case definitions and methods of the investigation are chosen for efficiency and often do not include data that would be useful in a hazard characterization and may vary between different investigations. Because the primary goal of the investigation is to quickly identify the specific source(s) of illness, key information that would allow data collected in an investigation to be useful for risk assessments is therefore often missing or incomplete. Estimates of dose or exposure in outbreaks may be inaccurate because of several reasons: CHAPTER 10 -DATA 151 • It was not possible to obtain representative samples of the contaminated food or water. • If samples were obtained, then they may have been held or handled in such a way, after exposure occurred, that make the results of testing meaningless. For example, microbial growth may have occurred if food is held at room temperature for extended periods. • Laboratories involved in outbreak testing are mainly concerned with presence/ absence, and they may not be conducting enumeration testing. • It may be difficult to detect and quantify viable organisms in contaminated food or water, e.g. viable Cryptosporidium oocysts in water or norovirus in oysters. • Estimates of the amount of food consumed by infected (and not infected) individuals, and of the variability therein, are poor. • There is inadequate knowledge concerning the health status of the exposed population, and the number of individuals who consumed food but were not identified to have become ill. This may be involve the various reasons that affect underreporting of foodborne illness, including consumers of the food not becoming infected; consumers having asymptomatic infection; consumers not seeking medical attention; a diagnostic sample not being collected and the laboratory test not being able to identify the etiological agent. • The size of the total exposed population is uncertain. In such instances, using outbreak data to develop dose–response models generally requires assumptions concerning the missing information. Fairly elaborate models may be necessary to reconstruct exposure under the conditions of the outbreak. If microbiological risk assessors and epidemiologists work together to develop more comprehensive outbreak investigation protocols, then this should promote the collection of more pertinent information. This might also help to identify detailed information that was obtained during the outbreak investigation but was not reported. Even when all needed information is available, the use of such data may bias the hazard characterization if there are differences in hazard strains associated with outbreaks versus sporadic cases, see for example Frank et al. (2014). The potential for such bias may be evaluated by more detailed microbiological studies on the distribution of growth, survival and virulence characteristics in outbreak and endemic strains. Attack rates may be overestimated when they are based on signs and symptoms rather than laboratory confirmation. Alternatively, in a case–control study 152 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD conducted to identify a specific food or water exposure, the attack rate may be difficult to estimate, and may be underestimated, depending on the thoroughness of case finding. The reported findings depend strongly on the definition of a case. Case definitions may be based on proximity in time and geography, clinical symptoms, on laboratory data or a combination thereof. The most efficient approach could be to choose a clinical case definition and validate it with a sample of cases that are confirmed by laboratory tests. This may include some nonspecific illnesses among the cases. In investigations that are limited to culture-confirmed cases, or cases infected with a specific subtype of the pathogen, investigators may miss many of the milder or undiagnosed illness occurrences, and thus underestimate the risk. The purpose of the outbreak investigation may lead the investigators to choices that are not necessarily the best for hazard characterization. While outbreaks can be a valuable source of information for hazard identification and characterization, an outbreak ultimately only provides one data point – a combination of dose and estimated proportion of infected or ill persons. Consequently, numerous outbreaks involving the hazard (though possibly different strains and different food matrices) are required to allow the fitting of a dose– response model. 10.2.3 Foodborne disease surveillance and annual health statistics Countries and several international organizations compile health statistics for infectious/zoonotic diseases, including those that are transmitted by food and water. The data included in many cases are very specific, with detailed descriptions of the food (e.g. type, amount, composition), hazard (reliably identified, often subtyped) and consumer (e.g. age, gender, health condition) being collected. This is often done in pursuit of identifying and investigating outbreaks (see also Section 10.2.1). Enhanced surveillance networks have in recent years improved the accumulation of data generated in foodborne disease investigations. These include Foodnet (CDC, 2021a), Pulsenet (CDC, 2021b) and Pulsenet International (PulseNet International, 2021). Such data are critical to adequately identify and characterize microbial hazards in specific food products. In cases where no surveillance data or health statistics are available, it may be possible to use surrogate sources. For example, for infections involving Taenia CHAPTER 10 -DATA 153 saginata sales data of taenicidal drugs have been used as an indication of the public health burden (Dorny and Praet, 2007). Strengths Active public health surveillance for foodborne illness can provide useful information about different disease endpoints and their proportional likelihood. Depending on the amount of information available different estimates may be obtained for various subpopulations of interest. However, care must be taken to account for the effect of underreporting, which, depending on the hazard, may be substantial (e.g. Hall et al., 2008; Scallan et al., 2011). Annual health statistics provide one means of both anchoring and validating dose–response models (see Sections 16.2.2 and 16.2.3). The effectiveness of dose– response models is typically assessed by combining them with exposure estimates and determining if they approximate the annual disease statistics for the hazard; this process is sometimes referred to as a “reality check.” In addition, surveillance statistics may provide useful information about different morbidity ratios, i.e. rates with which different severities are observed. For example, Scallan et al. (2011) provide information on hospitalization and mortality rates. These rates can differ between hazards or between different countries/regions (WHO, 2015). Similarly, surveillance information together with microbiological and genomic analyses can support the understanding of the severity of a hazard, e.g. some STEC strains have greater potential to cause more severe illness (from diarrhoea, bloody diarrhoea to haemolytic uremic syndrome) than others (FAO and WHO, 2018a). Finally, annual disease statistics data have been used in conjunction with food survey data to rapidly estimate a simple dose–response relationship. It must be noted that, usually, analysis of such aggregated data requires many assumptions to be made, which increases the uncertainty in the results. This approach is highly cost–effective since the data are generated and compiled for other purposes. Available data often have sufficient detail to allow consideration of subpopulations. Limitations The primary limitations of these data are that they are highly dependent on the adequacy and sophistication of the surveillance system used to collect the information. Data only concern a limited range of microbiological hazards and do not necessarily reflect sporadic cases. Typically, public health surveillance for foodborne diseases depends on laboratory diagnosis. Thus, it only captures those 154 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD who were ill enough to seek care (and were able to pay for it) and who provided samples for laboratory analysis. This can lead to a bias in hazard characterizations toward health consequences associated with developed nations that have an extensive disease surveillance infrastructure. Within developed countries, the bias may be towards diseases with relatively high severity, that more frequently lead to medical diagnoses than mild, self-limiting diseases. Comparisons with other countries are difficult because a set of defined criteria for reporting is lacking at an international level. Another major limitation in the use of surveillance data is that they seldom include accurate information on the attribution of disease to different food products, at the levels of hazard in food and the number of individuals exposed. Use of such data to develop dose–response relations is also dependent on the adequacy of the exposure assessment, the identification of the portions of the population actually consuming the food or water, and the estimate of the segment of the population at increased risk. Nevertheless, these national surveillance data have been used in combination with national production or consumption data to provide crude comparisons across commodities (Hsi et al., 2015). 10.2.4 Systematic food contamination monitoring surveys Frequently, governments set up proactive programmes to sample food and water for the occurrence of microbiological hazards of concern. The results can be reported as the percentage of contaminated samples (the prevalence) and/or the number of microorganisms, e.g. CFU/g of food. In addition, some government agencies carry out routine surveillance monitoring. Such data can be useful for hazard identification and also for exposure assessment. Most pathogen testing is presence/absence testing, because of the low expected contamination, and usually involves sample enrichment to allow the target organism to grow enough to improve detection. Thus, these tests are not enumerative, unless multiple samples are tested in which case the proportion of samples that become positive can be used to estimate the concentration, similar to the Most Probable Number (MPN) method (e.g. Kiermeier et al., 2011). There are some hazards for which tests do not yet exist, so even prevalence data may not be easily obtained. For example, until relatively recently, no reliable diagnostic tests were available for norovirus. This situation has now been addressed using molecular methods, though it still is not yet possible to differentiate between infective and noninfective virus particles (e.g. DNA fragments and damaged capsid). Finally, it should be noted that the efficacy of testing frequently depends on the size of the analytical unit tested, e.g. 1 g versus 25 g (Funk, Davies and Nichols, 2000; Vimont et al., 2005). CHAPTER 10 -DATA 155 In many of the exposure assessments published to date, the lack of specific data on primary production has often been identified as a weakness. Occasionally, governments or other stakeholders specifically survey establishments involved in primary production. However, such programmes are often run for other purposes, e.g. to better understand pathogen ecology and production hygiene with the aim of improving or refining control measures when necessary. Such studies may be small and specific, i.e. they typically concern one hazard and one commodity (e.g. Salmonella in broiler chickens). Nevertheless, they may be enough for a specific risk assessment. If national data on foodborne pathogens are not systematically collected in a country or region, it may be possible to utilize data from another country. In that case, the rationale for the choice of country and information on the possible limitations of the data need to be clearly documented. Strengths National surveillance activities generate substantial amounts of data, both in the form of prevalence or contamination level information. The potential for the use of such data in exposure assessments should be good, especially for systematic monitoring that covers a wide range of products in a certain category and a significant area (a country or region). To allow optimal evaluating of data on prevalence and level of contamination, proper descriptions of the details (i.e. year, season, geographical location, country, etc.) should be provided. Limitations Surveillance data collected by different government agencies are rarely pooled and the raw data may not be readily available. Also, a detailed description of the product or hazard may not be provided. Additionally, a major drawback is that these data may not be random or fully representative. They are generated as part of official control systems that often take account of resource limitations by targeting foods that are known to be problematic. Alternatively, they are generated to support food inspection processes where samples are only taken if there appears to be something wrong with the hygiene of the premises or process, and hence these data are often biased. In many cases, the lower (and upper) limit of detection (LoD) and analytical unit size are not reported, and neither are the sensitivity and selectivity (or specificity) of the detection method(s) utilized. Surveillance data collected at both primary production and processing/retail have a clear limit in terms of geography and time. 156 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 10.2.5 National food production statistics Food production statistics provide an estimate of the amount of food commodities available to the population, and as such can be useful for exposure assessment. Examples of this type of data include the FAO Food Balance Sheets (FAOSTAT) (FAO, 2021c) and other national statistics on total food production, imports, exports, wastage or utilization. Because these data are available for most countries and are compiled and reported fairly consistently across countries, they can be useful in conducting exposure assessments at the international level. Strength These reports contain detailed information and provide a good overview of a country’s production of food commodities, including imports and exports. Limitations Figures reported may be outdated and for some food commodities, production statistics may not be available. It is important to note that production statistics are not necessarily specific to how much of the product is destined for the food supply as compared to other uses, e.g. biofuels. In addition, total amounts of a commodity may need to be adjusted to account for spoilage and other losses to arrive at the total amount that is consumed as food. A reality check relating food consumption to food production should be undertaken where possible. That is, if food consumption statistics are available and they are aggregated over the whole population, does the total amount of the food consumed approximately equal the total production for the food, considering imports and exports, (likely) losses during processing and preparation and general wastage? If not, then some of the assumptions underlying the calculations may need to be critically assessed and revised. 10.2.6 National consumption databases Two types of food consumption data are frequently used for characterizing food consumption patterns for MRAs: food production statistics and food consumption/ nutrition surveys. These data can be very useful in exposure assessments. Other sources of information such as retail food sales or purchase data may be useful in filling data gaps in either food production or food consumption survey data. When using such data, allowance should also be made for the effects of food wastage and food spoilage. CHAPTER 10 -DATA 157 Some countries have carried out “Food Basket” studies to describe the amounts and frequency of foods consumed. In countries where household food surveys have been carried out, useful information for exposure assessments might be available. In addition, the use of “Participatory Epidemiology” methods (Mariner and Paskin, 2000) could be of value in data collection as well, being based on participatory techniques for gathering information based on community observations and traditional oral history (Bergold and Thomas, 2012). Another data source of potential use is WHO’s GEMS Food consumption database (WHO, 2021). This database provides information for a total of about 500 items at up to three levels of statistical food categorization on a country/cluster basis. These data may provide a useful starting point, though care needs to be taken with respect to interpreting the results. Where possible data should be checked against other sources. Food consumption patterns will probably differ based on population demographics (age, gender, ethnicity, health status, socioeconomic group) and seasonal and regional (both national and international) differences in food availability. Consideration of food consumption patterns for sensitive subpopulations (e.g. young children, pregnant women, the elderly and the immunocompromised) and high-risk consumer behaviour (e.g. consuming unpasteurized dairy products or undercooked or raw meat products) are particularly important. Information that enables estimation of variability in serving size will also be important. Strengths Food consumption surveys can provide detailed information regarding the types and amounts of foods consumed by individuals or households and sometimes also the frequency with which the foods are consumed (van Rossum et al., 2011). These surveys usually include a representative sample of individuals or households, from which consumption for the total population or specific subpopulation can be extrapolated. When surveys are repeated over time then changes in consumption patterns may be observed. Since serving size directly affects the amount of a hazard consumed (i.e. dose), these surveys may provide a method to determine a distribution of amounts consumed. Although the surveys are usually short in duration (one or two days to a week for each survey participant or household), they provide detailed information about the types of food consumed, as well as when and where foods are consumed (van 158 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Rossum et al., 2011). Limitations Food consumption patterns may vary widely within a country and the consumption estimates derived from national food balance sheets will not reflect this variability. For example, in sub-Saharan Africa the majority of the population live on the land and eat what they produce, though there may be considerable differences in consumption from the population that lives along the coastal areas. National food consumption surveys would be of great value here, but they are conducted in relatively few countries worldwide. Not all national survey data sets contain information by time of day and place of consumption as well as a total amount of each food consumed. Even if they do, then it is often difficult to extract this type of information and analyse it, e.g. the time of day needs to be clearly defined at the time of the survey, as well as when data are subdivided for analysis, etc. It also requires fairly sophisticated software to be able to analyse individual dietary data at this level of detail, as opposed to deriving mean or median population statistics. This is particularly true if all sources of a food are required to be aggregated at an individual person level (e.g. apples from raw apples, apple juice and apple pies). In terms of microbiological risk assessment, this addition of food consumed from different sources also has additional problems as each food source is likely to have a different level of contamination of the hazard due to different food processing and preparation routes. Food consumption surveys generally do not record descriptive information about the foods that may relate to food safety. For example, they may not report whether milk was raw or pasteurized, whether a soft cheese was made from raw milk, whether cooked shrimp were domestically produced or imported, or whether a food was packaged by the processor or at retail. For this information, food sales data from industry, trade associations, retail stores and other sources can be combined with results of food consumption surveys to estimate the frequency with which very specific food products might be consumed. Whenever possible these data should be compared with information from epidemiological studies (case– control, cohort or outbreak investigations) to verify or calibrate that food survey data capture the actual risk factors. Finally, food consumption surveys are resource intensive and for this reason may be one-off surveys or may be undertaken infrequently, e.g. every five years. As a result, rapid trends and changes in food consumption may not be available. CHAPTER 10 -DATA 159 10.2.7 National population census Governments regularly publish reports on population size by region, gender, age, etc. strata. These figures may be useful when characterizing the risk at population level and/or by type of population. Strengths These reports contain detailed information and provide a good overview of the country’s population demographics, including age (possibly grouped, e.g. 20-25 years, etc.), gender, socioeconomic status, etc. Limitations Reports on population census may be outdated as censuses are very resource intensive and are therefore undertaken relatively infrequently, e.g. every seven or ten years. Care should be taken to ascertain how the census was administered and what subgroups of the population may not have been captured, e.g. homeless people, and what approaches, if any, have been used to adjust for these. Population statistics generally do not capture “at risk” groups, unless they are specifically related to demographic characteristics, e.g. age. A specific problem for international exposure assessments is that information and data may not be accessible due to language barriers. Finding relevant data and correctly interpreting their context may be a problem. 10.3 INDUSTRY DATA Both textual and numerical data can be obtained from industry stakeholders, including occurrence of microbiological hazards, production stages and processing conditions, description of the final product and product pathways. Data on product sales and market share may also be available from private marketing agencies, trade associations and industry. These data are very relevant for the exposure assessment. Industry can provide information on whether the product is fresh or frozen, whether it is sold cooked or uncooked, whether or not it is further processed and the extent to which ingredients are mixed. A complete description of the food, including salt levels, pH, packaging and other relevant information may also be obtained. Such data may also refer to other factors that may affect the prevalence and/or concentration of hazard in the food, e.g. the extent to which the product 160 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD and subproducts are domestically produced or imported; the different ingredients added; or other products typically consumed with the product. The food chain consists of all stages from primary production to the consumption (including home, restaurant, foodservice, and/or institutional locations), and thus data relating to each of these stages are required as part of an exposure assessment. Using meat processing and distribution as an example, the various stages will include the farm; transport to and holding at a slaughterhouse or processing plant; slaughter; processing; packaging; storage; distribution and retail; transport to the home; handling and home storage; food preparation; and consumption. Some of these stages and processes may vary between producers, retailers and consumers and thus it is important to obtain information to describe and account for this variation – this is particularly pertinent for exposure assessments where formal and informal supply chains exist. Certain stages or processes may be regulated, for example, with respect to the use of chemicals or additives; such regulation and information on the extent to which they are actually followed may give relevant data to be collected. Considering growth and survival of a microbial hazard, the times, temperatures, and other ecological factors, such as pH, at the various stages are important. Particular examples of requirements include the duration of, and temperature during, storage and transport; freezing temperatures; pasteurization times and temperatures; cooking times and temperatures; and the addition of ingredients that may alter pH. Data that enable description of the variation in these parameters, for example from producer to producer or day to day, are also important. Often, individual stages in the food chain are considered to be static for a specified period. However, certain conditions, such as temperature, are more likely vary over time and the data should reflect this. While data may be readily available on thermal inactivation, data on other processing steps that affect microbial growth and survival may not be as readily available. It is also important to gather information relating to the stages of mixing and partitioning. For example, the meat from an individual beef carcase can be partitioned and then perhaps mixed with meat from other beef carcases to produce a ground beef burger patty. Partitioning and mixing will affect the microbial status of the product, in terms of both likelihood of contamination and number of organisms, and thus data that describe and quantify these processes should be collected. Typical requirements will include the extent to which these events occur, the numbers of units that contribute to a mixed product, and characteristics of CHAPTER 10 -DATA 161 products obtained through partitioning (including distributions in quantity and size). Retail surveys also represent another source of industry data, including information on geographical area, season, and the degree to which the data represent all manufacturers, distributors or retailers. Strengths Industry collects vast amounts of product/commodity specific data, which it stores in a wide array of private systems. Gaining access to such data and information about product pathways, can provide important information about the realities of the food production, that might otherwise not be known with confidence. Where sampling, testing and monitoring programs are in place, information will be available over time and at various stages during the production, from supplied raw ingredient through to finished product. Such data will be useful for application of predictive microbiological models when the fate of microorganisms is to be predicted. In some businesses, sampling and testing of the raw material and end product is extensive and frequent, as it is the primary means of “ensuring” food safety. Other businesses employ a preventative approach to food safety such as the implementation of a food safety management system based on the principles of HACCP. In these businesses microbiological testing may be less frequent and solely for the purpose of verifying the effective working of the HACCP system. Furthermore, the food production environment is sampled due to considerations of cross-contamination Limitations Major limitations to the inclusion of industry data are the facts that they may not be hazard specific and are difficult to combine when generated in various industrial settings. Because sampling and testing is usually done for verification purposes or to satisfy regulatory requirements, the data often concern the presence/absence of a microbiological hazards rather than the levels/concentration. When testing is done for indicator organisms the levels of contamination are usually recorded, but often these relate poorly to the presence/absence and levels of the hazard of interest. Accessing and retrieving data is often a problem in practice. In this regard, there is also a need to address the issue of confidentiality, which may be a stumbling block 162 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD in relation to access. How to keep proprietary information and data confidential needs to be discussed and agreed prior to their provision, as is done by FAO/WHO (2018b). In addition, potential biases need to be considered, especially relating to the difference in processing and food safety programs related to business size. Large manufacturers process on an industrial scale with better and more automated equipment than small food producers. As a result, they can supply more geographically diverse retailers and supermarkets and access a different consumer segment than small food producers who sell their products at informal markets. Similarly, large enterprises are more likely to have food safety programs in place, including spending (more) money on microbiological testing programs. This contrasts with small or very small enterprises, which are less likely to undertake much, or any, microbiological testing. If a food is tested for microbial contamination anywhere in the food supply chain, then the industry stakeholders should provide sufficient information on the food, microbiological methods, sampling design and frequency of sampling, etc. However, such information may not readily be available. An important implication of collecting retail data by any group (e.g. trade association, academia, consultant) is that the identification of a contaminated food might trigger a recall (e.g. L. monocytogenes in an RTE food or E. coli O157 in ground beef). This may make such surveys of limited value because any kind of recall may change the foods in distribution and impede future industry cooperation. Alternatively, when such studies are commissioned by industry, there may be a limitation placed on the type of microbial data that is collected, e.g. hygiene indicators and/or presence/absence of specific genes rather that direct isolation of the pathogen. 10.3.1 Description of product and supply chain Throughout the food chain, many control options are available to reduce the risk of microbiological contamination of the final food product. These may be incorporated in HACCP plans that are specific for each product and manufacturing site, and thus may vary substantially between manufacturers. Data should be collected that describe both the methods of control and the extent to which these vary. Examples include cleaning and disinfection methods and the extent and frequency with which these are undertaken; inactivation methods and their critical limits; any testing of raw materials and intermediate or final products, with estimates for test sensitivity and specificity; and handling practices. CHAPTER 10 -DATA 163 10.4 UNPUBLISHED DATA Potentially vast amounts of data generated throughout the world are never published in a form that can be used by others, and this can be due to many different reasons. For example, the subject is not attractive to publishers or the scientific community, i.e. results in publication bias; there are barriers in communication (resources, language); or researchers suffer from time and/or resource constraints. This is an unfortunate situation as such data could give new insights, reduce uncertainty and avoid unnecessary duplication of studies. However, like other data sources, the quality of unpublished data needs to be ascertained carefully before they are used in a risk assessment. Some steps can be taken towards improving access to such data. Building networks is very important in this regard, as these can be used to inform a wider audience of the data needs for risk assessment and also provide a means of gaining information about, and even access to, unpublished studies. Building a relationship with potential data providers is essential in establishing trust and instilling confidence that the data will be used properly and remain confidential, if necessary. There is a need for networking, especially with others who might be working in areas where data are required. Another avenue for gaining access to unpublished data is through public calls for data. This approach is usually used for international risk assessments by FAO/ WHO, as well as national competent authorities. Such calls for data also form an important part of risk communication and helps to involve different stakeholders. 10.5 DATA GAPS All risk assessments require data and knowledge, irrespective of whether they are qualitative or quantitative. Data and knowledge gaps affect the assessor’s confidence in the risk characterization and the robustness of the estimates. The form of a risk assessment is determined primarily by looking at what decision questions need to be answered, taking into account the decision criteria described in Section 3.5. Then a search is done to see what data and knowledge are available that would help construct the risk assessment to answer these questions. A balance is generally needed: taking a particular risk assessment approach may not be able to answer all questions but may provide a better-quality answer. Data may not be available to answer the question at all. Thus, defining the form of a risk assessment may require considerable dialogue between assessor and manager. 164 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Both numerical and textual data are required to model all stages of the exposure pathway. Often, data are limited or do not exist. However, a lack of knowledge about a process should not necessarily inhibit the conduct of a risk assessment. When deficiencies in the data exist, they must be clearly communicated to the risk managers and documented in the risk assessment. Such communication will ensure that additional data requirements are identified. Even in situations where appropriate and representative data are known to exist, problems can still occur. For example, there may be institution or company confidentiality to consider, the data may be politically sensitive or there may be a charge for using the data. The iterative nature of risk assessment allows for the continuous upgrading of data as new information becomes available. This process will often lead to a better understanding of the value of other information that is not currently available. One can ask what else could be done if some specific data could be found. Depending on the circumstances, the risk manager may consider it worth waiting, or expending the resources to acquire additional data in anticipation of being able to make more informed decisions as a result. It is tempting to plan the structure of a risk assessment that will answer all the risk managers’ questions, and then attempt to find the data required to “populate” the risk assessment. However, in the food safety area this may not be a practical approach. Food safety management is beset by a lack of data, so writing a wish list of all the data one would like will inevitably lead to disappointment. Other approaches, such as building simplified models to describe the system before considering the data availability (Ebel et al., 2012), have been proposed as preliminary activities to aid in determining the form of the risk assessment. A brief list of reasons for data gaps includes: • it has not previously been seen to be important to collect these data; • data are too expensive to obtain; • data are impossible to obtain given current technology; • past data are no longer relevant; • data from other regions are not considered relevant; or • the data have been collected or reported, or both, in a fashion that does not match the risk assessment needs. Data that have not previously been seen to be important often arises in contamination studies with infrequent detection data. Such data are not usually valuable for CHAPTER 10 -DATA 165 scientific journals; therefore, researchers have less interest in conducting such studies. However, data on nondetections are important for risk assessment, e.g. to estimate prevalence. Using the risk assessment framework, it may be possible to determine which gaps have the biggest effect on being able to address the risk management questions. This identification process can be used to set priorities for future data collection and experimental research. There are a number of approaches that can be used to help overcome limitations in data. These include model design, surrogate data, expert opinion and the collection of new data. 10.5.1 Model restructuring Ideally, all stages in the exposure pathway that affect the hazard are included in the model structure. However, in many situations, data for specific stages may be limited or even not exist. Also, the statement of purpose for the risk assessment may not require detailed analysis of all processing stages, i.e. a farm-to-fork exposure assessment may not always be required. When this is the case, it may be possible to restructure the model to exclude the stage for which data are not available or in such a way that alternative available data can be used. For example, it may be possible to begin the exposure assessment after the processing stage and obtain prevalence and concentration using monitoring data. Clearly any changes in the scope must be discussed and agreed with the risk managers. In addition, simplification of the model may have the benefit of reducing the compounding of uncertainties. There are limitations with this technique, as important factors that have an effect on the risk may be overlooked and lead to errors. Cullen and Frey (1999) provide a useful discussion of trade-offs regarding various levels of model complexity. 10.5.2 Surrogate data In one sense, nearly all data are surrogate data unless specifically collected for an exposure assessment. Pilot plant data, for example, are a surrogate for production facilities; thermal death time values obtained via capillary tubes are surrogates for inactivation in the plate pasteurizers used in food processing. Classically, certain benign species or strains of microorganisms are used as surrogates for pathogenic strains. In such cases, the relevant characteristics of the surrogate organisms should be the same as the organism of interest, or the differences documented and taken into account. Surrogate organisms are more appropriate for quantifying or 166 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD predicting treatment efficacy than for predicting or quantifying health effects such as actual dose–response relationships. The appropriateness of the surrogate data must be judged when assigning uncertainty to the data. For transparency, use of surrogate data must be described and justified. Indicator microorganisms for particular microbiological hazards have been used in some exposure assessments where data on the hazard is not available or cannot be collected. An example is the cross-contamination rate of E. coli O157:H7 from faeces to animal carcases. Because of the low prevalence of E. coli O157:H7 in faeces, a direct measure of contamination cannot readily be obtained. The easily measured generic E. coli is therefore used as an indicator of faecal transfer to the carcase, which can then be related back to E. coli O157:H7. When using surrogate data, care should be taken to clearly identify where it was used and any underlying assumptions, such as proportionality between the pathogen and surrogate, should be made explicit. Regarding food consumption data, if there is insufficient detail to provide estimates for “at risk” populations (pregnant women, immunocompromised, elderly, etc.), data for comparable age and gender groups in the normal population may be used. Data from other countries or regional data may also be used for food consumption if it is known that food consumption patterns are similar. Sensitivity analysis (Chapter 15) of the final model can be used to determine if the parameter, for which surrogate data were used, has a significant effect on the final risk. If the parameter is important in estimating the risk, then an additional study may need to be undertaken to try to collect more relevant data. 10.5.3 Expert knowledge elicitation (EKE) Expert knowledge elicitation is a formal approach to the acquisition and use of expert opinions, in the absence of or to augment available data. It will inevitably be necessary to elicit expert estimates for parameter values in the model where there is a critical lack of data, and where it is essential to assess that risk in the relatively near future. Problems here include, for example, decisions on identification and selection of experts; the number of experts required; techniques for eliciting information; overcoming bias, and methods are still being developed in this area (e.g. Jenkinson, 2005; Hemming et al., 2018; Dias, Morton and Quigley, 2018). Where possible, expert opinion should be elicited using formalized and documented methods that avoid bias and can be used to formulate appropriate probability distributions (Gallagher et al., 2002; Nauta et al., 2001; Vose, 2008). In CHAPTER 10 -DATA 167 situations where experts’ opinions differ markedly, weighting methods can be used to best integrate information. Experts should strive to transparently document the rationale supporting their opinion to the greatest extent possible. When expert opinion is required, the problems and methods of selection, overcoming bias, etc., are likely to be similar irrespective of the level of quantification used for the risk assessment. It is accepted that ideally a “sufficient number” of experts should be utilized. Techniques like the Delphi method (Linstone and Turoff, 2002), and modifications such as the IDEA3 protocol (Burgman, 2015; Hemming et al., 2018), which aim to achieve consensus among a panel of experts, can help produce more reliable estimates. However, there are situations when there truly are very few experts on the specific topic worldwide. Sometimes there are no true experts. This leads to the inputs with very large uncertainty, whatever the risk assessment type – this is far from ideal but may be the only option in the short term. In a quantitative risk assessment, it is necessary to convert expert opinion into a numerical input, and once again various methods exist (e.g. Gallagher et al., 2002; Burgman, 2015; Dias, Morton and Quigley, 2018; EFSA, 2014a). Even in a qualitative risk assessment, these methods may be used to convert expert opinion into numerical values for specific model steps and this is the preferred method. An alternative and less sophisticated way of using expert opinion in qualitative risk assessments, however, may be to ask directly for an opinion on the probability of a specific step in narrative terms, such as “High”, “Low”, etc. The meanings of these words will have the same subjectivity problems as those discussed for qualitative risk assessments in general (see Section 7.2). In principle, such a method should be only a temporary measure until better data are available. The estimation of dose–response model parameters is unlikely to be based on expert elicitation and instead based on model fitting. However, the choice of dose– response function, that is, the mathematical form, is often based on the modellers’ expertise, and thus forms a type of expert opinion. When no dose–response model exists the likely dose needed to result in a specific human health effect, e.g. ID50, may require expert elicitation. This is particularly true for emerging hazards that have not been studied extensively. Readers with interest in the use of expert opinion should consult Morgan and Henrion (1992), who present a sequence of chapters summarizing the heuristic biases in expert elicitation, a typical formal expert elicitation protocol intended to 3 IDEA = Investigate, Discuss, Estimate and Aggregate 168 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD overcome such biases, and examples. Informal EKE can be performed with fewer experts and without the presence of an experienced facilitator in the sense that a small group of scientists wish to quantify their own knowledge about an uncertain quantity, for the purposes of some scientific endeavour. In any case, however, the scientists’ judgements should be made as carefully and objectively as possible and documented fully, according to the principles of the formal EKE. Additionally, EFSA Guidance on Expert Knowledge Elicitation in Food and Feed Safety Risk Assessment (EFSA, 2014a) and the Intergovernmental Panel on Climate Change (IPCC, 2001) discuss the process of expert elicitation in scientific assessments. Strengths When there is a lack of the specific data needed but there are suitable scientific experts, EKE provides a potential means of acquiring and using pertinent information to. This can involve the development of a distribution for a parameter in a model. Expert elicitation is generally not expensive, particularly in relation to short-term needs. Limitations The results obtained depend on the methodology used and are inherently subjective and thus open to debate. The results also depend on the experts selected and may have limited applicability for issues involving an emerging hazard. 10.5.4 Collection of new data At times, there is a need to collect new data, e.g. prevalence and concentration data for a foodborne hazard at a specific point of the food chain. The process of obtaining an estimate of the prevalence or the enumeration of microbiological hazards usually involves the following steps: • Define the research question • Identify the reference population and study population and obtain an appropriate sampling frame • Design a sampling scheme and identify the sample population • Collect and analyse appropriate samples • Conduct statistical analysis of the data Those contemplating the collection of new information for use in risk assessments should consult a statistician or someone trained and experienced in data collection, especially someone who is familiar with the underlying research domain, e.g. microbiology, consumer behaviour, etc. CHAPTER 10 -DATA 169 10.6 RECOMMENDATIONS ON DATA COLLECTION AND ORGANIZATION The characteristics of the data that might be needed at a particular stage are likely to vary from assessment to assessment. Whilst certain characteristics may be considered ideal, in practice it is often necessary to use whatever relevant data are available. This brings into focus the iterative nature of a risk assessment, which is concerned with the fact that initial attempts to model a process are likely to utilize data with a high degree of uncertainty. This process can be used to identify where the greatest uncertainty lies, allowing targeted data collection for subsequent model updating. Gradually, with further iterations of the modelling process, the uncertainty is reduced. Thus, the first iteration of the assessment might be undertaken specifically to identify data needs and/or data gaps. The second iteration may assess the likely exposure, but with wider uncertainty limits; and the third iteration, using ‘new’ data, may allow an estimate of the exposure with a narrower uncertainty band and higher predictive ability. There may be considerable time delays between these stages. The level of uncertainty should be included in the data description. 10.6.1 Searching for data Search protocols using computer-searchable literature databases and data repositories should be devised that are comprehensive and reproducible but are also appropriately selective. Example of such databases include the following (in alphabetical order). • ComBase: http://www.combase.cc • EBSCO Food Science Source: https://www.ebsco.com/products/research- databases/food-science-source • EFSA Knowledge Junction: https://zenodo.org/communities/efsa-kj/ • FAOSTAT: http://www.fao.org/faostat/en/ • FoodRisk.org: http://foodrisk.org/ • Food Science and Technology Abstracts: https://www.ifis.org/fsta • OVID Current Contents: http://www.ovid.com/site/catalog/databases/862.jsp • Promed: http://www.promedmail.org/ • Pubmed: https://www.ncbi.nlm.nih.gov/pubmed/ • Scopus: https://www.scopus.com/ • Web of Science: https://clarivate.com/products/web-of-science/ • WHO/GEMS: https://www.who.int/nutrition/landscape_analysis/nlis_gem_ food/en/ 170 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Systematic plans for obtaining literature that predates these databases, or that is not indexed in them, need to be devised using citations in more recent publications, reviews and book chapters. Criteria for search protocols and data selection should be transparent, with appropriate explanation recorded in the documentation. The Research4Life resources noted in Section 3.5.2 may be helpful in gaining access to scientifically published articles and associated data. 10.6.2 Selection of data It is frequently stated that “all data are biased.” Nevertheless, data should be as representative as possible of the food, hazard and process being assessed and the population consuming the food. Preferred data generally come from peer-reviewed publications, followed in importance by nonpeerreviewed or unpublished data (government documents, theses, proceedings, etc.). Some data are not available in the peer-reviewed literature (e.g. consumption data), and it should be remembered that even peer-reviewed data are, in most instances, not collected for the purpose of being used in risk assessments. Thus they may not comply fully with all data requirements or be fully representative for the case at hand. Any biases or limitations in the degree to which data represent any particular point of view should be identified and documented (e.g. funding source). When no or too few data are found, expert opinion will need to be used (see Section 10.5.3). Generally, the data should be as close as possible to, or specific to, the requirements of the risk assessment. For example, if the assessment were to calculate the exposure in a particular country, the preferred data would come from that country. The next choice would be data from that region or a comparable country. The final choice would be from somewhere else in the world (keeping in mind the purpose of the risk assessment). Selection criteria should include consideration of factors such as geography, time, microbial strain, methodology, equipment type and design, and population demographics. Food consumption data should provide sufficient detail to allow estimates of consumption of the food(s) of interest per meal or per day. The data should be representative of the total population, and ideally will provide information about subgroups within the population. 10.6.3 Formatting of data The ideal format of the data will vary with the particular type of data required; there is no one ideal format for all data. In particular, data that are descriptive of the biological and manufacturing processes will generally be textual, whereas parameter and model input data would be numerical. CHAPTER 10 -DATA 171 However, there are some underlying principles that should be considered when formatting data: • Data should be fully referenced as to source (within the confines of commercial sensitivity). • Units should be given where appropriate. • Raw data, rather than average or other summary statistics, should be used wherever possible. • When raw data are not available, a description of the distribution, the level of uncertainty and the amount of variability should be included to the greatest extent possible. 10.6.4 Level of detail recorded When collecting data for use in an exposure assessment, it is useful to record and report to the greatest level of detail possible. This should be done in an appropriate way which does not interfere with the flow of the report to the extent that it hampers clear communication. The additional information that describes the data set is often referred to as metadata, and there are a number of metadata standards available, though these are not specifically for microbiological data. Examples of some details that might best be routinely recorded and reported are: • Information on data source or provenance. This should include: the full reference to the source; the name of the provider if a personal communication or unpublished data; the date of the collection of the data; affiliation and funding source of the data provider. • Information on the study itself. This should indicate whether it was a laboratory- or field-based study. • Details of sample, including: livestock species (giving scientific name where appropriate) or product definition; source (country, region, category of producer, chain of retailer, etc.); selection method (in particular for livestock, whether samples are clinical cases or random selection); population size; season of collection, if appropriate; portion description or size, if appropriate; and method of collection of samples. • Information on microbiological methods. This should include: sampling method, microbial species, subspecies, strain, in as much detail as is available (and for prespecified exposure assessment, the required detail should be specified and collected); tests used, including any variation from published methods; test performance characteristics; units used; and precision of measurement. • Information on the results obtained. This should be recorded as the raw data, and include: number tested, together with results (including units) given for all samples tested. 172 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 10.6.5 Combining data from different sources Representative data are often limited, and it is usually preferable to use all of it. However, decisions need to be made when different data sets have different degrees of applicability and relevance to the parameter being modelled. Techniques such as meta-analysis can be used for the purpose of combining data sets (e.g. Petitti, 2000). Gonzales-Barron et al. (2016) provide an overview on how to integrate prevalence data of pathogens from different sources. More generally, Bayesian approaches may be useful when considering existing knowledge in the light of recently collected information (Gelman et al., 2013; Kruschke, 2014). In certain situations, using Bayesian techniques allows a better estimate of the parameter to be obtained than if the most recent data are used in isolation (Ranta et al., 2015). When a data set is biased, the data may be adjusted before being combined with other data or used in the risk assessment. An example would be when recent studies demonstrated that data collected by one method consistently underestimated the true parameter value by a known amount. Weighting is often employed so that data sets considered more relevant have more influence on the estimated parameter value. Weighting by the number of samples is frequently used, so that larger studies have more influence. Weights may also be used to reflect an expert’s belief in the quality and appropriateness of the data. Older data or data from another geographical area might be used in estimating the parameter value but be given less weight. The selection of the numerical weighting factors is highly subjective and should be explained for full transparency. Composite data sets may be obtained by averaging, method of moments (Hansen, 1982), or maximum likelihood estimates. Careful examination of the different data sets may facilitate estimates of variation (e.g. different microbial strains used in different studies) or uncertainty (residual errors in statistical analyses). Meta-analysis and mixed-effect models can also be used to evaluate data variation. To avoid inserting the risk assessor’s biases into the parameter values, data should not generally be ignored or deleted. However, certain data sets may clearly be inconsistent with the greater collection of data and knowledge. Comparing the size of the remaining distribution with the divergence of the particular data set may suggest that a particular data set should be excluded. This should be done with caution, as the outlier may indicate another source of variation that is otherwise being overlooked (e.g. see Figure 17). 10.6.6 Presentation of data The format of the data will affect the method of presentation. The underlying principle is that the presentation should be clear and easy to follow. Again, the CHAPTER 10 -DATA 173 data may be textual or numerical. When presenting a large amount of data for a particular exposure assessment, a contents table or list is desirable. An introduction or overview of the assessment puts the data to be presented in context. The data should then be presented in a logical order. In general, with an exposure assessment, there are one or more pathways by which the consumer may be exposed to the microbiological hazard. The first part of the data to be presented is generally the textual data that describes these pathways. For complex pathways, a high-level overview of the process may be required, followed by a more detailed description for each step in the pathway. Also, graphical presentation of the pathways, such as in the form of a flow chart, is generally helpful. When presenting numerical data, this should also follow a logical order, and this is again likely to follow the order of the steps in a particular pathway. A tabular format is frequently useful, particularly for raw data. However, enough text should be provided to fully describe the relevance of the data, and how they are utilized in the assessment. Summary data are often also best tabulated. Graphs, such as histograms, may be used to visualize data but should not be used without explanation. Titles of tables or graphs should allow them to be fully identified and should be unambiguous. References should be clear within the text, diagram or table, and a comprehensive reference list given. Any web pages or similar are probably best attached as appendixes. 174 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 11 11. Quantitative modelling approaches As introduced in Section 5.2.3 there are different categories of quantitative models. The following categories increase in model complexity and thus also increase in the potential richness of the model outputs. This is achieved by incorporating variability and uncertainty into the model and this allows their effects, and those of the model inputs, on the outputs to be evaluated (see also Chapters 14 and 15). 11.1 DETERMINISTIC Deterministic models assume that inputs to a model are known and fixed values with no variability or uncertainty. Although they are simple models, they generally require more data than for a qualitative assessment. A single value, e.g. average, mode, 95th percentile, etc., is chosen to characterize each input variable in the model. The individual point estimates are combined using mathematical equations to generate a point estimate of exposure, and, through a dose–response model, the consequent risk. An example of a deterministic model, implemented in a generic framework, is RiskRanger (Ross and Sumner, 2002; Sumner and Ross, 2002). The effects of changes to model variables can then be investigated by what-if scenarios to generate outputs. For example, the initial scenario may be based on the average for each input variable. Subsequently, the 95th percentile value might be used and the risk estimate compared with that from using the average. When conducting deterministic exposure assessments, selecting a conservative value for each variable has often been used to develop deliberately conservative, or 175 worst case, estimates. Propagating such conservatism through the model, however, can result in an unrealistic overestimate of exposure because the exposure estimate can be based on a highly improbable scenario. Thus, a drawback of the deterministic approach is that the likelihood or probability of the estimated output occurring is unknown. Some values are more likely to occur than others, and without knowledge of the likelihood of each outcome, the risk manager may inappropriately allocate valuable resources to reduce an event that occurs very rarely. Stochastic models can overcome this problem. 11.2 STOCHASTIC The stochastic, or probabilistic, assessment represents all the information available for each input variable, which is described by a probability distribution. Most parameters such as hazard prevalence in primary production, hazard concentration and growth, storage times and temperatures, and serving size can vary. These variables are better described as distributions which represent a realistic range and frequency of values. In stochastic models, scientific data are used to obtain probability distributions for each input variable. They are then combined to determine the probability distribution of an adverse outcome (Ruzante et al., 2013). Consequently, the outcome of a stochastic exposure assessment is a statistical distribution that describes both the range of doses of the hazard that might be experienced by an individual or population, and the likelihood of each dose occurring. For example, consider a hazard in a food product. • The concentration of the hazard in the food prior to heating is lognormally distributed with mean and standard deviation of 1.0 and 0.8 log10 cfu/g. • The effective reduction from heating the food is also lognormally distributed with mean and standard deviation of 2.5 and 0.7 log10 cfu/g. • The reduction is independent of the concentration of the hazard in the food prior to heating. Analytically it can be determined that the concentration of the food after heating is also lognormally distributed, with mean 1+(-2.5) = -1.5 log10 cfu/g and standard deviation of √(0.82+0.72) = 1.06 log10 cfu/g. However, stochastic assessments often involve many inputs. Hence, finding analytical solutions is usually not possible, particularly if the distributions are not normal distributions. For this reason, Monte Carlo simulation is usually used to perform the assessment (see below). 176 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The distribution used to describe a data set depends on the number and pattern of data points available, and on the knowledge about the nature of the process being modelled. Detailed reviews of important probability distributions are available in the literature (Cullen and Frey, 1999; Haas, Rose and Gerba, 2014; Morgan, Henrion and Small, 1992; Vose, 2008). Uncertainty in parameter values can also be expressed by probability distributions, as discussed in Chapter 14. The transition from qualitative assessment to deterministic assessment to stochastic assessment usually represents an increase in both information and time required. However, due to the availability of simulation modelling software, the time involved for a stochastic assessment may not be much greater than for a deterministic analysis. Despite its increased computational complexity over the deterministic approach, much of that complexity is dealt with by the software and the stochastic method is favoured among most risk assessors because it generates more information to support decisions. For example, it allows the risk assessor to identify the range of possible exposure levels from all possible exposure routes from which the most likely level of exposure, or any specified percentile value, can be determined. This output provides much greater information than a single point estimate. In addition, stochastic modelling allows for explicit identification, modelling and separation of variability and uncertainty (Chapter 12). However, with the increased complexity also comes the increased risk of introducing errors into the assessment, and the output is more difficult to understand, review, interpret and use for decisions. 11.3 MONTE CARLO SIMULATION As noted above, stochastic models are generally complex in nature, and as a result are usually difficult, or impossible, to solve analytically. To overcome this problem, the model can be evaluated on a computer, using Monte Carlo (MC) simulation, which consists of repeated random sampling from the distributions that characterise the input variables of a simulation model; these input realisations are used to evaluate the model, which results in distributions of the output variable(s). A variety of specialized computer software packages are available to support this approach and are discussed in various texts (e.g. Cullen and Frey, 1999); a good summary is provided in Table 1 of Basset et al. (2012). Commonly used programs are spreadsheet add-ons, such as @RISK© and Crystal Ball©. Microbial risk assessors have also used the stand-alone package called Analytica® or the USFDA’s web-based, and free to use, FDA-iRISK® system. Other mathematical (e.g. Matlab) or statistical packages (e.g. SAS, R) can also be used for simulation modelling, including various free add-ons, such as the mc2d package for R (Pouillot and CHAPTER 11 -QUANTITATIVE MODELLING APPROACHES 177 Delignette-Muller, 2010) to assist with separating variability and uncertainty. Models can also be constructed using general-purpose programming languages, including FORTRAN, Python, Visual BASIC or C. Commercial software packages may be less flexible to use compared with programs developed in programming languages, although both require specialist expertise to model the processes appropriately. Exchange of models may be hampered if the chosen software is not widely available, and open-source software packages may help to improve the ability for the risk assessment to be audited by others. Simulation models that can be placed and run on the internet may also be desirable to further facilitate model evaluation and reuse (e.g. FDA-iRISK®). To undertake a Monte Carlo simulation, a mathematical model is constructed, including all variables that affect the exposure and their probability distributions. Collectively, the result of the combined equations is an expression of consumer exposure. The software then evaluates the model by generating, at random, a value for each variable from its corresponding probability distribution. The generated values are then combined according to the mathematical equations that comprise the exposure assessment model, and the exposure is calculated. Subsequently, the dose–response model is used to simulate whether the consumption event results in infection or illness. A single realization of this generation and calculation process is called an iteration of the model and represents the exposure from one possible combination of circumstances. There are many such sets of circumstances, however, some more or less likely than others and leading to greater or lesser exposure. To estimate the full range of possible exposures and the likelihood of each, the simulation software repeats the calculations many times: hundreds of thousands or millions of iterations are commonly performed. The result of each iteration is recorded and the distribution of exposures and probability of each is generated, as is the subsequent risk output. Intermediate results may also be recorded to provide insights into the model. 11.4 OTHER MODEL CLASSIFICATION SCHEMES In addition to the classification of models used in quantitative risk assessment as deterministic or stochastic, other not mutually exclusive classification schemes might be encountered. That is, the use of one description does not necessarily preclude an additional description from another classification scheme. Some common schemes are mentioned below though others exist also. Models can also be categorized as empirical or mechanistic. Empirical models simply describe data or relationships in a convenient mathematical form, without 178 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD necessarily having an understanding of the underlying biological mechanisms. For example, a smoothing spline (de Boor, 2001) may describe a set of data points adequately, even though there is no biological basis for it. Mechanistic models have theoretical bases formed from the understanding of the behaviour of a system’s components, e.g. binary fission bacterial growth. If correctly formulated, then a mechanistic model should provide a good fit to experimental data and thus allow the interpretation of the response in terms of known phenomena and processes. In practice, risk assessment models will probably contain both mechanistic and empirical elements. Estimates of exposure and risk can also be viewed from a temporal perspective: they can be defined as static or dynamic. Static estimates relate to a particular point in time, e.g. the probability and level of exposure associated with a random serving of the food product, or the number of contaminated servings consumed per year. In contrast, a dynamic approach would consider the way in which exposure changes over time, for example, reflecting seasonality of exposure (Anderson and May, 1992; Bailey, 1975) or the increasing contamination of a processing line as time from last clean-up increases (Nauta, Van der Fels-Klerx and Havelaar, 2005; Zwietering and Hasting, 1997a, 1997b). CHAPTER 11 -QUANTITATIVE MODELLING APPROACHES 179 12 12. Predictive microbiology Predictive microbiology can play an important role in exposure assessment and is used to fill in data gaps that would otherwise require more extensive data collection programmes. Predictive microbiology, in conjunction with mathematical models and data describing various environmental factors, e.g. including storage time and temperature, pH, water activity, etc., can be used to estimate the final level/concentration of pathogens or spoilage organisms in the food. For example, while data on the number of pathogenic bacteria in food at retail may be available, the number in the food immediately prior to consumption is not. It may, however, be possible to model the number of pathogenic bacteria in the food immediately prior to consumption, considering the storage, preparation and cooking conditions. Predictive microbiology also has limitations. Not all hazards that are of interest have been characterized. Therefore, not all microbial kinetic parameters are available, uncertainties surrounding predictions are not always given, and predicted values may not truly represent the real world if models have not been validated. In spite of the limitations, predictive models remain valuable tools for exposure assessment of pathogenic microorganisms in foods. Detailed descriptions of the application of predictive microbiology in MRA can be found in Ross and McMeekin (2003) and Ross (2008). 180 12.1 MODELLING MICROBIAL GROWTH AND INACTIVATION 12.1.1 Microbial ecology of foods The possible responses of most microorganisms in foods include stasis, growth or death. In general, viruses, protozoa and parasites are inert in foods, requiring a living host to be able to reproduce. While they cannot grow, they can be inactivated by various treatments and processing steps. Similarly, prions are not infectious organisms but are proteins. While they also cannot grow in foods they may be inactivated by some treatments, although they are very resistant to denaturation. Populations of microorganisms in foods may display stasis, growth or death, depending on the formulation of the food (intrinsic factors) and the processing, distribution or storage conditions (extrinsic factors). They may even display different responses at different times in a single unit of food because conditions can change during processing, transport, storage and preparation. While each organism may have a qualitatively similar response to changes in temperature, pH, preservatives, etc., the magnitude and type of response (i.e. growth, death, stasis) to different levels of these factors is specific to the hazard. While pH, water activity and temperature are the most frequently cited properties and typically have the greatest effect on microbial behaviour, many foods will have additional properties with important consequences. These include the levels of fat, oxygen, phosphates, certain spices, organic acid anions (especially acetate, lactate, sorbate and benzoate), nitrite, ionic and non-ionic humectants (sugars, salts, etc.), and antimicrobials, such as benzoate or sorbates. Food structure has also been shown to play an important role in affecting microbial behaviour in some foods (e.g. Wilson et al., 2002). To estimate exposure at the time of consumption, it will be necessary to model the cumulative effect over time of the food’s composition (which may change over time) and processing or storage conditions on the microbiological hazard. In some cases, changes in microbial numbers during processing may occur as a result of cross-contamination, rather than growth or inactivation. Note that the same considerations may apply to microorganisms in water, whether used for recreation, irrigation, drinking or food preparation. It is important to understand under what circumstances growth, inactivation or cross-contamination may need to be considered. In Table 39 are provided indicative CHAPTER 12 -PREDICTIVE MICROBIOLOGY 181 values, based on expert opinion, for the effect of temperature on the rates of growth or inactivation of many vegetative bacteria; inactivation of endospores requires considerably longer time and/or higher temperature. Growth rates for fungi will be lower, but inactivation rates are generally in the same range. TABLE 39. Indicative response times for growth and inactivation of vegetative bacterial cells as a function of temperature Temperature (°C) -80 -20 0 5 10 20 30 35 50 60 70 80 Time for 10-fold increase in numbers (hours) 15-75 10-30 5-20 3-10 2-3 1-2 growth not possible for most Time for 10-fold decrease in numbers (for vegetative cells) years to decades months days to weeks hours seconds to minutes fractions of seconds to seconds Each type of microorganism has a finite range of temperatures over which it can grow, some preferring lower and others higher temperatures. Note also that the effect of temperature depends on the temperature range considered. At low temperature, survival is enhanced, while at intermediate temperatures, growth rate increases with increased temperature. At temperatures above the limit for growth, however, death results at a rapidly increasing rate with increasing temperature. Each organism also has a finite range for growth as a function of pH, water activity, organic acid level, preservatives, etc., so that there are upper and lower limits for each factor, as well as an optimal level at which the growth rate is fastest. In general, the inhibitory effects of suboptimal factors interact in two way. The first is to reduce the range of each factor over which growth is possible, when one or more factors are suboptimal, and the second is to reduce the overall growth rate. At conditions beyond those that allow growth, stasis – or more probably death – will result. The rate at which death/inactivation occurs depends on the conditions but is most strongly affected by temperature (McQuestin, Shadbolt and Ross, 2009; Zhang et al., 2010). 182 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The growth of microorganisms in a unit of food follows the pattern of a batch culture, often with a period of adjustment (lag), involving no growth. This period is followed by exponential growth until some maximum population density (MPD) is reached and population growth ceases (see Figure 12). For many organisms and many foods, the MPD is in the range 109-1010 cells per g, ml or cm2 of food, though the actual MPD depends on the hazard, food, inhibitory conditions (such as salt concentration) and other microbes in the food (see also the discussion in Section 12.1.2). In(N/No) A μm Time λ FIGURE 12. Example of a typical growth curve where A denotes the maximum population density, λ denotes the lag, µm denotes the maximum growth rate, and A denotes the maximum population density (Fig 1 of Zwietering et al., 1990) Similarly, the death or inactivation of microorganisms in a food is characterised by an initial period of no decrease in the microbial population (shoulder), followed by an exponential death phase until the tail is reached and population decline ceases. Although the ecology of microbiological hazards in food can be complex, predictive microbiology models can be used to estimate changes in microbial levels in foods as the product moves through the food chain. Ross (2008) provides a detailed discussion of the microbial ecology of foods in the context of the exposure assessment part of risk assessment. CHAPTER 12 -PREDICTIVE MICROBIOLOGY 183 12.1.2 Predictive microbiology In recent years, significant advances have been made in the field of predictive microbiology. Some models are based on data obtained from liquid microbiological media and have been developed to predict the microbial behaviour when the physicochemical characteristics of the food (e.g. pH, water activity, organic acids concentrations) and the storage temperature are known. Some of these models can fail to accurately describe the microbial behaviour in foods, although the more robust models of this type have been validated in foods. Other models have been developed to predict the behaviour of microorganisms in particular foods, irrespective of what their storage conditions might be. The food-based models can effectively describe the effect of storage conditions on a specific food but their ability to describe the effect of the variability of physicochemical characteristics of the food or to make predictions in other foods is questionable. Some intermediate approaches have also been developed trying to overcome the limitations of these two major approaches. For certain products, it has been shown that proliferation (rate or extent, or both) of the spoilage microflora of a product affects the behaviour of the pathogen concerned, e.g. L. monocytogenes on cheese and coldsmoked salmon (Giménez and Dalgaard, 2004; Mellefont, McMeekin and Ross, 2008; Cadavez et al., 2019). For many bacterial pathogens, responses to environmental conditions have been described and summarized in mathematical models that can be used to predict their behaviour in foods. These include models for growth rate, lag time, death rate, probability of growth occurring, and probability of toxin production within the storage life of the product. Models relating the number of a microbial organism and time, assuming that all other factors are constant, are known as primary models (Buchanan, 1993). The physiological and physical state of the microorganism in the food remains a relatively unexplored area. Stress, injury and recovery also affect the initiation of growth, leading to a distribution of lag/germination times. Many studies use stationary phase cells that were grown in a nutrient-rich broth at favourable temperatures, and the predicted lag phase duration represents those conditions. Cells that contaminate a food may be in a different physiological state. The extent to which the organisms are clustered or aggregated may also affect growth, survival and crosscontamination. In predictive microbiology, foods are characterized in terms of their properties that most affect microbial growth and survival, such as temperature, pH, organic acid levels, salt levels and preservative levels. Microbial responses to analogous 184 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD conditions are systematically studied and quantified, usually in a simplified laboratory broth model system under static and axenic conditions. The data are collated and summarized as predictive mathematical models. In particular, models that relate these properties to growth or inactivation rates are known as secondary models (Buchanan, 1993). Tertiary models are usually considered models that combine primary and secondary models, often in software that enables predictions (Buchanan, 1993). However, it has been argued that the term “tertiary models” should be used for “patterns in the parameters of the secondary models as a function of the organism and the nutrient source” (Baranyi, Buss da Silva and Ellouze, 2017). Conditions that foods and microbes are exposed to are often unstable and dynamic. The effects of these changing conditions on rates of growth or inactivation have to be mathematically integrated over time for each of those distinct processes or stages. Thus, measurements of processing and handling conditions, and the duration for which these are experienced, are integrated and used to predict changes in hazard levels (i.e. population size or concentration). Some predictive microbiology models, however, recreate the growth curve, i.e. the number of cells present, assuming a defined starting level, as a function of incubation time. Outputs from such models would normally have to be converted to rates of growth before their application in exposure assessment models. A potential weakness of many predictive models is that they are usually developed based on microbial growth in broth culture media under laboratory conditions. Under such conditions interactions with other microbes in the food or effects due to the physical structure of the food itself are not representative of what occurs in the food matrix. For example, lactic acid bacteria may suppress pathogen growth in vacuum-packed or modified-atmosphere packed foods, and matrix effects may be important in water-in-oil emulsions (e.g. butter). 12.1.3 Model types and modelling tools Models are available that describe: • Rate of growth as a function of multiple environmental factors. • Rate of inactivation, mainly as a function of a single lethal factor. One should be aware, however, that microbial inactivation is usually considered a stochastic process, i.e. the probability of survival of cells decreases (more or less) exponentially per unit of time. Thus, although the number of viable cells in an individual unit of food may be predicted to be less than one, one might CHAPTER 12 -PREDICTIVE MICROBIOLOGY 185 still find survivors if a larger unit of the product (e.g. the total volume of a batch), or many units of the product, were examined or considered. • Limits to growth as a function of multiple environmental factors, so-called growth/no growth or interface models. Absolute limits to growth of many pathogens due to individual environmental variables have been documented (ICMSF, 1996). • Probability of growth or toxigenesis within a defined period as a function of multiple environmental factors. In addition to numerous small-scale research projects to model microbial responses in foods, two large-scale predictive microbiology research programmes were undertaken in the early 1990s. They were funded by the governments of the United States of America and of the United Kingdom of Great Britain and Northern Ireland. These programmes resulted in the development of a suite of models for responses of populations of foodborne microbial pathogens and some spoilage organisms. The outcomes of those programmes, and subsequent developments, are now available without cost through the Predictive Microbiology Information Portal (USDA, 2021) which hosts the Pathogen Modelling Program and links to ComBase (ComBase, 2021a). These software packages include growth models for many pathogens and some spoilage organisms, and inactivation models for some pathogens. In particular, ComBase is a database of observations for many published and unpublished sources on microbial growth and inactivation rates, and at the time of writing contains close to 60 000 records. The database is derived from the government-funded research programmes referred to above, from data extracted from the published literature and from data (both published and unpublished) donated by researchers and research organizations around the world. Additional models for a range of pathogens and spoilage organisms are also available and a comprehensive lists of predictive microbiology modelling tools are available on the ComBase (ComBase, 2021b) and the OpenML for Predictive Modelling in Food websites (SourceForge, 2021), as well as Tenenhaus-Aziza and Ellouze (2015) and Koutsoumanis et al. (2016). These offer a variety of tools for the most important foodborne pathogens including databases, fitting tools, predictions for growth, growth/no growth and inactivation, probabilistic models, and risk assessment modules. This allows for a wide range of applications including exposure assessment. The most important benefit for the users, however, is that software can assist decision-making in a short timeframe and allow action to be taken almost in real time. Additionally, there are many modelling programmes and studies that have not resulted in the release of software but that are published (often including the data 186 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD on which the model is based) in the scientific literature. These can be found readily by undertaking a literature search. The integration of models for microbial growth, growth limits or inactivation into unified models that can predict both increases and decreases in microbial populations over time will also improve the utility of predictive models for exposure assessment. Several unified models have been proposed, but none have been widely used or endorsed. Many reviews of predictive microbiology, including discussions of potential pitfalls, have been published. McMeekin et al. (1993) and Ross et al. (2014) provide a good introduction to the concept and its practical application, and the texts edited by McKellar and Lu (2003), Brul et al. (2007) and Pérez-Rodríguez and Valero (2013) provide more contemporary reviews of the state of the art. 12.2 APPLICATION OF PREDICTIVE MICROBIOLOGY WITHIN EXPOSURE ASSESSMENT In practice, two features of a predictive microbiology model are critical to its utility. One is the ability to accurately predict microbial responses under all conditions to which the model applies. Evaluation of this ability is loosely termed model validation (see Section 16.2.3). The second is the range of independent variables and variable combinations to which the model applies. If the model does not include terms for all factors relevant to the microbial ecology of the hazard in the food, then that model is incomplete. While predictive microbiology has matured considerably as a scientific discipline over the last two decades many currently available models are still incomplete or unvalidated, or both. Thus, exposure modelling should include consideration of the validity and reliability of any predictive microbiology models used. 12.2.1 Range of model applicability No predictive models currently in use are fully mechanistic (i.e. derived entirely from fundamental theoretical bases). Therefore, microbial growth or death cannot be reliably predicted in a food in which the conditions are beyond the range of any individual factor included in the data used to develop the model (i.e. predictions should be made by interpolation only). Different models have different interpolation regions depending on the experimental design used to develop the model. The determination of the true CHAPTER 12 -PREDICTIVE MICROBIOLOGY 187 interpolation region and the consequences of extrapolation were discussed by Baranyi et al. (1996). Those authors concluded that models that were over-fitted using a large number of parameters were more prone to poor predictions resulting from inadvertent extrapolation, because the predictions of the model often changed dramatically near the limits of the interpolation region. Inadvertent extrapolation can also occur when using stochastic modelling techniques to describe effects of fluctuating variables. This problem may occur for any factor, but temperature is the factor most likely to fluctuate in most real-world situations. Consideration should be given to truncating the tails of the temperature (and other) distributions used to predict microbial growth or death to match the interpolation range of the predictive microbiology model used. This should be done by utilizing a suitable truncated distribution so that the mean, variance and other properties of the chosen distribution are not changed in unintended ways (Johnson, Kemp and Kotz, 2008). The growth limits for the pathogen of concern, and potential for inactivation (if conditions are beyond those limits) should be considered and included in exposure modelling. Growth/no growth models may assist in this regard and have been included in some exposure assessment models. 12.2.2 Spoilage microbiota The effect of spoilage bacteria on the shelf life of the product should also be considered. Conditions that lead to rapid growth of pathogens may also lead to rapid microbial spoilage. Contaminated products that are obviously spoiled are less likely to be consumed, and consequently less likely to lead to foodborne disease, despite the fact that they contain a microbiological hazard. Thus, it may be necessary to consider the effect of storage conditions on the shelf life of the product so that unrealistically long times at high temperatures are not simulated. This can be implemented by correlating model variables that affect growth, e.g. storage time and temperature. Stochastic modelling texts offer advice on how such correlations can be included in models and examples include Ross et al. (2009), Smith et al. (2013) and Kiermeier et al. (2015). Other microorganisms growing in the food can also affect the growth of pathogens. Exposure assessments that rely on empirical data derived from pure culture broth systems are likely to overestimate potential growth of pathogens in food matrices due to the coexistence of numerous competing bacterial populations (Coleman, Sandberg and Anderson, 2003). Pathogen growth rates and maximum densities are thought to be a function of the total microbial community composition and density in the food, due to competition for nutrients, the production of inhibitory 188 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD substances, and overall density (Powell, Schlosser and Ebel, 2004). The final cell density of a pathogenic bacterium can be suppressed when the total concentration of all bacteria in the food reaches stationary phase, a phenomenon that has been termed the Jameson Effect (Jameson, 1962; Stephens et al., 1997) and reported by many authors (e.g. Ross, Dalgaard and Tienungoon, 2000; Le Marc, Valík and Medveďová, 2009; Al-Zeyara, Jarvis and Mackey, 2011). 12.2.3 Sources of variability and uncertainty In stochastic modelling, it is important to characterize the magnitude of the variability and its distribution about the mean. Traditionally, the approach to fit predictive microbiological models was through a two-step fitting approach. For example, first primary models were fitted separately for each temperature and the parameter estimates were extracted (esp. max growth rate). Subsequently these estimates were used as the response for the secondary model, i.e. a model was fitted to relate growth rate to temperature; the implicit assumption is that the parameter estimates are known values rather than estimates. This approach was likely due to the nonlinear nature of problem (i.e. when primary and secondary models are combined) and the result of limited computing power in the early days of the discipline. However, fitting nonlinear models is no longer a problem, though the actual fitting process can still be challenging; good starting estimates and suitable parameter transformations can help in this regard. In addition, it has been shown that the one-step model fitting process, i.e. where the primary and secondary models are combined and estimated in a single model, is more efficient than the two-step process (Jewell, 2012; Dolan and Mishra, 2013; Huang, 2017). Distribution of response times Using the limited amount of replicated published data concerning growth rate estimates under varying environmental conditions, Ratkowsky et al. (1991) concluded that growth rates became increasingly variable at lower growth rates. Microbial response times, as a function of environmental conditions, are often not normally distributed. Distributions describing growth rate and/or response time variability relative to temperature have been described by various researchers (Ratkowsky et al., 1991, 1996; Alber and Schaffner, 1992; Dalgaard et al., 1994; Zwietering et al., 1994). Ratkowsky (1992) presented a general relationship between the variance in growth response times and the mean of those responses for a range of possible distribution types. Sources and magnitude of errors Model predictions can never perfectly match observations or represent reality. Each step in the model construction process introduces some error as described CHAPTER 12 -PREDICTIVE MICROBIOLOGY 189 below (Cullen and Frey, 1999; Ross, McMeekin and Baranyi, 2014). • Homogeneity error arises because some foods are clearly not homogeneous. Current predictive models do not account for this inhomogeneity of foods. • Completeness error in predictive models arises because the model is a simplification, and other food effects and microbial ecology effects (structure, competition, etc.) that are difficult to quantify are not included in currently available models. • Model function error is similar to completeness error and arises mainly from the compromise made when using empirical models, namely that the model is only an approximation to reality. • Measurement error originates from inaccuracy in the measurement methods used to collect raw data that are used to estimate the parameters of a model. • Numerical procedure error includes all errors arising from procedures used for model fitting and evaluation, some of which are only methods of approximation. Like all statistical models, the fit of the model should be checked graphically against the actual observations. Sometimes the fitted model clearly does not match the data very well – if systematic deviations are observed then a different model formulation needs to be considered. However, it may be possible to simply add more variables in the specific dataset to a model to increase the “goodness of fit” (i.e. to reduce the apparent error; see also Section 13.3). However, there is a danger that the random variability in that specific data set is modelled, rather than the underlying biological processes that lead to the observations in that dataset. In other words, models should be parsimonious, that is, only those variables that explain a significant amount of variation in the response variable should be retained – various approaches are available to achieve a parsimonious model, including statistical tests and likelihood-based information criteria. However, the real test of the performance of a predictive model, rather than a descriptive model for a specific dataset, is whether it can accurately predict the observations for a data set not used to develop the model , i.e. the model is valid. As a rule of thumb, when constructing a predictive microbiology model, each variable that needs to be included in the model (considering the need for parsimony), increases the relative error in the prediction of the specific growth rate by approximately 10 percent (Ross, McMeekin and Baranyi, 2014). This phenomenon should not be confused with the statistical effects of variable selection and descriptive model fitting, which implies that every additional variable (whether significant or not) will reduce the standard error of the estimates. 190 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Instead, it needs to be recognised that the specific growth rate is not “estimated” from the data per se in a predictive model but is a function of the variables that affect it, i.e. the statistically significant variables of the secondary model. Each of these variables, and their associated parameter estimates, introduces additional variability and uncertainty, and the resulting standard error in the predicted growth rate is a function of independent variables (Wilks, 1962). As a result, confidence in the predicted growth rate, and thus total predicted growth, declines when more variables affect the growth rate. The practical importance of this for predicted exposure depends on the amount of growth predicted to occur. For a three-variable model the magnitude of the error in terms of growth rate and log number of cells would be around ±30 percent, irrespective of the amount of growth predicted. However, in many situations, probability of infection, and thus risk, is related to the absolute number of cells ingested, not the logarithm of dose. Thus, if one generation of growth (0.30 log10) were predicted (assuming the lag time and maximum population density are known exactly and not estimated), then the error in the predicted number of cells would be ±(0.30 log10 × 0.3) = ±0.09 log10, i.e. ±23 percent of the estimate. If 10 generations of growth were predicted, then the error would be ±(3.00 log10 × 0.3) = ±0.9 log10 which, in terms of numbers of cells would be ±700 percent. If lag time and MPD are also estimated, then these errors will be larger. CHAPTER 12 -PREDICTIVE MICROBIOLOGY 191 13 13. Dose–response Dose–response modelling requires a combination of mathematics, statistics, human biology (infection process, immune system), microbiology and epidemiology. Different approaches are available for model fitting and the assumptions underlying a dose–response model need to be understood, assessed and reported/ communicated. The focus of these sections is on infectious and toxico-infectious hazards, as this has been the area of most development. However, it should be noted that this chapter provides an overview of dose–response models and the interested reader is directed to the review by Haas (2015), which provides information on doseresponse models not only for foodborne hazards. 13.1 THE INFECTIOUS DISEASE PROCESSES The biological basis for dose–response models derives from the major steps in the disease process as they result from the interactions between the hazard, the host and the matrix. Figure 13 illustrates the major steps in the overall process, with each step being composed of many biological events. Colonization, toxin production, infection and illness can be seen as resulting from the hazard successfully passing multiple barriers in the host. These barriers are not all equally effective in eliminating or inactivating hazards and may have a range of effects, depending on the hazard and the individual. Each hazard has some particular probability to overcome a barrier, which is conditional on the previous step(s) being completed 192 successfully, similar to the hurdle concept in food processing. The disease process as a whole, and each of the component steps, may vary by hazard and by host. That is, at every stage in Figure 13, the ultimate result is affected by the characteristics of the hazard and the host and this should be evaluated cautiously and transparently. For example, stomach survival depends on the pH of the stomach of the host and on the acid resistance of the hazard. Then at the next stage the outcome depends on how the hazard can cope with the intestinal flora of the host. For intoxications (e.g. cereulide, botulinum toxin, staphylococcal enterotoxin) sequential probability processes do not exist, and a minimal toxic dose can exist (see Box 1). Very low levels will not result in a response in any person, since this is not an infectious agent, but a toxic component produced by a microorganism (van Leusden, 2000). This is a structural difference between an infective organism or a carcinogenic chemical component and a non-carcinogenic toxin. Infective organisms have a probability of infection and carcinogens give a probability of cancer – both result in probability increases as the dose increases, but no threshold exits. Other toxins can have a threshold, and this threshold can be host dependent, just like the probability of infection of an infectious organism. The difference is that, below the threshold, even of the most sensitive person has a zero probability of intoxication, while for infective organisms and carcinogens this is always a non-zero probability at any dose, even if this is sometimes extremely small. Consequently, even very small probabilities in large populations can give rise to a Intoxication e.g. botulinum toxin, Staphylococcus aureus enterotoxin or Bacillus cereus cereulide Response, if concentration above threshold toxin amount Toxico-Infection e.g. Bacillus cereus Clostridium perfringens, infant-botulism Stomach survival Bile, intestinal flora Colonisation Toxin production Infection e.g. Salmonella, Campylobacter Stomach survival Bile, intestinal flora Colonisation Invasion (immune response) FIGURE 13. The major steps in the foodborne disease process CHAPTER 13 -DOSE–RESPONSE 193 substantial public health burden. This distinction between infective organisms and carcinogens (where zero probability does not exist) and microbial toxins and noncarcinogenic or non-genotoxic compounds (where a zero probability exists below a threshold) is crucial. 13.1.1 Infection and illness Infection is usually measured as a quantal response, i.e. the presence or absence of infection by some criterion. The use of continuous response variables, e.g. an antibody titre, may be useful for further development of dose–response models. There are usually many different and simultaneous signs and symptoms of illness in any individual. The severity of symptoms varies between hazards, strains and between hosts infected with the same hazard. The extent of illness is therefore a process that can also be measured on a multidimensional, quantitative scale (e.g. number of stools passed per day, body temperature, laboratory measurements). A wide variety of case definitions for gastrointestinal illness are used in the literature, based on a variable list of symptoms, with or without a specified time window. These definitions sometimes include laboratory confirmation of etiological agents. This lack of standardization severely hampers integration of data from different sources. 13.1.2 Sequelae and mortality In a small fraction of ill persons, chronic infection or sequelae may occur. Some pathogens, such as Salmonella enterica serotype Typhi, are invasive and may cause bacteraemia and systemic infections. Other pathogens produce toxins that may result not only in enteric disease but also in severe damage in susceptible organs. An example is haemolytic uraemic syndrome, caused by damage to the kidneys from Shiga toxins of some E. coli strains. Complications may also arise by immune-mediated reactions: the immune response to the pathogen is then also directed against the host tissues. Reactive arthritis, including Reiter’s syndrome, and Guillain-Barré syndrome are well known examples of such sequelae. The complications from gastroenteritis normally require medical care, and frequently result in hospitalization. There may also be a risk of mortality in relation to sequelae, and not all patients may recover fully, but may suffer from residual symptoms, which may last a lifetime. Therefore, despite the low probability of complications, the public health burden may be significant. Also, there is a direct risk of mortality related to acute disease, in particular in the elderly, neonates and severely immunocompromised. 194 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD In the context of a risk assessment, the number of cases with sequelae and complications are usually ascertained on a proportional basis, similar to the approach used by the WHO Foodborne Disease Burden Epidemiology Reference Group (Section 4.2, WHO, 2015). 13.2 MODELLING CONCEPTS 13.2.1 The particulate nature of the inoculum It is commonly assumed that the organisms are randomly distributed in the inoculum. The Poisson distribution is then used to characterize the variability of the individual doses when pathogens are randomly distributed. While this assumption rarely holds completely, it often serves as a useful approximation. Compound distribution and over-dispersion may result from two different mechanisms: • A unit as detected by the measurement process (e.g. a CFU, a tissue culture infectious dose, or a Polymerase Chain Reaction (PCR) detectable unit) may, due to aggregation, consist of more than one particle. This is commonly observed for viruses (e.g. Teunis et al., 2008), but may also be the case for other pathogens (e.g. Jongenburger et al., 2011). The degree of aggregation strongly depends on the methods used for preparing the inoculum. It is important to know whether the aggregates remain intact during inoculum preparation or in the gastrointestinal tract. • In a well-homogenized liquid suspension, single disaggregated organisms will be more or less randomly distributed. If the inoculum consists of a solid or semisolid food matrix, however, spatial clustering may occur and result in over-dispersion of the inoculum (e.g. Jongenburger et al., 2012). This aspect of spatial clustering may differ between the data underlying the dose–response model and the actual exposure scenario. The reason why knowing about aggregation is important is that it can affect the dose–response model and thus the estimated 50 percent infectious dose, ID50. For example, for norovirus it was found that the ID50 was 1 015 genome copies for the aggregated inoculum, while for the disaggregated virus the ID50 was only 18 viruses (Teunis et al.., 2008) – approximately two orders of magnitude lower. 13.3 SELECTION OF MODELS Specific properties in the data become meaningful only within the context of a model. Different models may, however, lead to different interpretations of the same data, CHAPTER 13 -DOSE–RESPONSE 195 and so a rational basis for model selection is needed. This is true not only for doseresponse models, but all models that are fitted to data (e.g. predictive microbiology models). Different criteria may be applied when selecting mathematical models. For any model to be acceptable, it should satisfy the statistical criteria for goodness of fit, in particular, residual plots are essential tools for assessing goodness of fit. In the case of more than one model fitting equally well, goodness of fit statistics, such as the various likelihood-based Information Criteria, can be used to select “the best” (Dziak et al., 2020). However, many different models will usually fit a given data set (e.g. Holcomb et al., 1999) especially due to the large variability and uncertainty in the data and therefore goodness of fit is not a sufficient criterion for model selection. Additional criteria that might be used are conservativeness, flexibility, parsimony and biological plausibility, which are discussed below. Furthermore, it may be necessary to consider more than just one model. In this case it may be necessary to evaluate different models using what-if scenarios or to combine models using probability weights. A conservative model is one that tends to over-predict the response of interest (i.e. in the context of a dose-response model this is the probability of infection or illness). However, conservativeness can be approached in different ways: “Is the model structure conservative?”, “Are parameter estimates conservative?”, “Are specific properties, e.g. prediction at low doses, of the model conservative?” and so forth. It is not recommended to build conservativeness into the model structure itself. From a risk assessment perspective, a model should be restricted to describing the data and trying to discriminate the biological signal from the noise. Adding parameters usually improves the goodness of fit of a model but using a flexible model with many parameters may result in overfitting (Lever, Krzywinski and Altman, 2016; Steyerberg et al., 2010) – a lack of parsimony – and greater uncertainty of estimates, especially for extrapolated doses. It is recommended that dose–response models are biologically plausible. For example, a quadratic polynomial model may fit a given data set well, or even better than an alternative model, yet the quadratic model is not biologically plausible. Such a model will result in inappropriate predictions when extrapolated to very small or large doses. Note that it is generally not possible to “work back”, i.e. to deduce the assumptions underlying a given model formula. There is also a problem of identifiability: the same functional form may result from different assumptions, while two (or more) different functional forms (based on different assumptions) may describe the same dose–response data equally well. This can result either in 196 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD very different fitted curves if the data contain little information, or virtually the same curves if the data contain good information. However, even in the latter case, the model extrapolation may be very different. This means that a choice between different models or assumptions cannot be made on the basis of data alone (e.g. FAO and WHO, 2011b, Annex A1.1.1). 13.3.1 Dose-infection models Consider a host who ingests exactly one cell of a pathogenic microorganism. According to the singlehit assumption, the probability that this pathogen will survive all barriers and colonize the host has a nonzero value of p. Thus, the probability of the host not being infected is 1-p. If a second cell of the pathogen is ingested, and the hypothesis of independent action is valid, then the probability of the host not being infected is (1-p)2. For n pathogens, the probability of not being infected is (1p)n. Hence, the probability of infection (denoted as “Inf ” below) of a host that ingests exactly n cells can be expressed as: P(Inf |n,p)=1-(1-p)n 13.3.2 Dose-illness models The default assumption of constant probability models for illness given infection leads to the conclusion that the only difference between dose-infection and doseillness models is that the dose-illness models do not need to reach an asymptote of 1 because the probability of illness given infection can be something less than one when the probability of infection given dose approaches 1 (Teunis and Havelaar, 2000; Teunis, Nagelkerke and Haas, 1999). As such they essentially still belong to the family of hit-theory models. Alternatives to the constant probability assumption include an increase, or decrease, in the probability of illness with increasing dose (Teunis, Nagelkerke and Haas, 1999). 13.3.3 Sequelae and mortality Given illness, the probability of sequelae or mortality, or both, depends on the characteristics of the hazard, but more importantly on the characteristics of the host. Sequelae or mortality are usually rare events that affect specific subpopulations. These may be identified by factors such as age or immune status, but increasingly genetic factors are being recognized as important determinants. As for dose-illness models, the current possibilities are mainly restricted to constant probability models (e.g. FSIS, 2001). In the case of mortality, the proportion of infected patients who die is known as the mortality ratio. Stratification, e.g. by age, appears to be necessary in almost all cases where an acceptable description of risk grouping is available. CHAPTER 13 -DOSE–RESPONSE 197 13.4 EXTRAPOLATION 13.4.1 Low dose extrapolation Dose–response information is usually obtained in the range where the probability of observable effects is relatively high. In experimental studies using human or animal subjects, this is related to financial, ethical and logistical restrictions on group size. In some observational studies low dose effects can potentially be observed directly, but in these studies only major effects can be distinguished from background variation. The single-hit family of models (further exemplified in Section 13.5) is characterized by linear low-dose relationship on the log/log scale, or even on the arithmetic scale. That is, in the low dose range, the probability of infection or illness increases linearly with the dose and hence on the log-scale, these models have a slope of 1 at low doses (see for example Figure 8). Some examples of models with a linear low-dose relationship include: • The Binomial model • The linear model P(Inf | n, p1) =1 − ( 1 – p1)n P = r × D • The exponential model P = 1-exp(-r × D) • Beta-Poisson model P=1-[1+D/β]-α • The hypergeometric model P=1-1F1(α, α+ β,-D) P1=p1 P1=r P1=1-exp(-r) ≈ r P1 ≈ (α/β) P1 ≈ {α/(α+β)} where D = mean ingested dose and r, α and β are model parameters. Note that if α > β, the probability of infection predicted by the Beta-Poisson model is larger than one, which is not biologically plausible. 13.4.2 Extrapolation in the pathogen–host–matrix triangle Experimental datasets are usually obtained under carefully controlled conditions (e.g. using specific strains), and the data apply to a specific combination of pathogen, host and matrix. In actual exposure situations, there is more variability in each of these factors, and dose–response models need to be generalized. Assessing such variability requires the use of multiple datasets that capture the diversity of human populations (including differences in health status), pathogen strains and matrices. Failure to take such variation into account may lead to underestimation or overestimation of the actual risk of the outcome of interest. When developing dose–response models from multiple datasets, one should use all the pertinent data. This requires that the risk assessors make choices about how to use different datasets. Such choices should be based on objective scientific arguments but will inevitably include subjective arguments. Such arguments should be fully and transparently documented. Ideally they are discussed with the 198 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD risk manager and their significance and impact for risk management considered. The credibility of dose–response models increases significantly if dose–response relations derived from different data sources are consistent. When combining data from different sources, a common scale on both axes is needed. This often requires adjusting the reported data to make them comparable. For the dose, test accuracy, sample size, etc., need to be taken into account. For the response, a consistent case definition is needed, or the reported response needs to be adjusted to a common denominator (e.g. infection × conditional probability of illness given infection). Combining data from different sources within a single (multilevel) dose–response model requires thorough statistical skills and detailed insight into the biological processes that generated the data. An example is the multilevel dose–response model that has been developed for different isolates of Cryptosporidium parvum (Teunis, Chappell and Okhuysen, 2002a). The issue of combining data from different outbreak studies is discussed in the FAO/WHO risk assessments of Salmonella in eggs and broiler chickens (FAO and WHO, 2002a). Dose–response relations where the hazard only affects a portion of the population do require that subpopulations be separated from the general population to generate meaningful results. Using such stratified dose–response models in actual risk assessment studies requires that the percentage of the population that is actually susceptible can be estimated. Consideration of such subpopulations appears to be particularly important when attempting to develop dose–response relations for serious infections or mortality. However, it would also be pertinent when considering a hazard for which only a portion of the population can become infected, e.g. not all people are susceptible to norovirus infection (Teunis et al., 2008). A particular and highly relevant aspect of microbial dose–response models is the development of specific immunity in the host. Most volunteer experiments have been conducted with test subjects selected for absence of any previous contact with the pathogen, usually demonstrated by absence of specific antibodies. The actual population exposed to foodborne and waterborne pathogens will usually be a mixture of totally naïve persons and persons with varying degrees of protective immunity. No general statements can be made on the impact of these factors. This strongly depends on the pathogen and the host population. Some pathogens, such as many childhood diseases and the hepatitis A virus, will confer lifelong immunity upon first infection whether clinical or subclinical. In contrast, immunity to other pathogens may wane within a few months to a few years, or may be evaded by antigenic drift. At the same time, exposure to nonpathogenic strains may also CHAPTER 13 -DOSE–RESPONSE 199 protect against virulent variants. This principle is the basis for vaccination, but has also been demonstrated for natural exposure, e.g. to nonpathogenic strains of L. monocytogenes (Notermans et al., 1998). The degree to which the population is protected by immunity depends to a large extent on the general hygienic situation. In many developing countries, large parts of the population have built up high levels of immunity, and this is thought to be responsible for lower incidence or less serious forms of illness. Some examples are the predominantly watery form of diarrhoea by Campylobacter spp. infections in children and the lack of illness from this organism in young adults in developing countries. The apparent lack of E. coli O157:H7-related illness in Mexico has been explained as the result of crossimmunity following infections with other E. coli, such as enteropathogenic E. coli strains that are common there. Obviously, age is an important factor in this respect, as older people will have greater likelihood of prior exposure than children. In contrast, in some countries, contact with enteropathogens is less frequent and a larger part of the population is susceptible. This also highlights that dose–response models may not be globally applicable. Incorporating the effect of immunity in dose–response models has received little attention. The absence of accounting for immunity in dose–response models may complicate interpretations, and comparisons among geographic regions. This is particularly likely to be a problem with common infections such as Campylobacter spp., Salmonella spp. and pathogenic E. coli. Immunity may affect the probability of infection, the probability of illness given infection, or the severity of illness. There are currently only few data sets available on which to base model development. Where such data are available, a simple and possibly effective option would be to resort to stratified analysis and divide the population into groups with different susceptibility (e.g. Pouillot et al., 2015b; Teunis et al., 2008; USFDA/FSIS, 2003). Experimental work on infection of volunteers having different levels of acquired immunity to Cryptosporidium parvum was analysed with a dose–response model that includes the effects of immunity (Messner and Berger, 2016; Teunis, Chappell and Okhuysen, 2002b). In addition, recent work included an investigation of the effects of acquired immunity on risk estimates using a case study on exposure to Campylobacter jejuni (Havelaar and Swart, 2014). Stratified analysis can also be useful when dealing with seemingly outlying results, which may actually indicate a subpopulation with a different response. Removal of one or more outliers corresponds to removing (or separately analysing) the complete group from which the outlying result originated. Where a specific reason for the separation cannot be identified, there should be a bias toward being inclusive in relation to the data considered. As for all data analysis, any exclusion of 200 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD outlying data should be scientifically justified and clearly communicated to ensure the transparency of the assessment. 13.5 DOSE–RESPONSE MODEL FITTING APPROACHES According to the single-hit hypothesis (see Section 6.3 and 13.3.1), the probability of infection of a host that ingests exactly n pathogenic cells can be expressed as: P(Inf n , p) = 1 − (1 − p)n This model is also called the binomial dose–response model. Starting from this basic function and taking the discrete nature of pathogens into account, a broad family of dose–response models (hittheory models) can be derived. The most frequently used models are the exponential and the Beta-Poisson models, which are based on further assumptions on the distribution of pathogens in the inoculum, and on the distribution of p. When the distribution of the organisms in the inoculum is assumed to be random, and characterized by a Poisson distribution, it can be shown (Teunis and Havelaar, 2000) that the probability of infection as a function of the dose is given by: P(Inf D, r) = 1 − exp( − D p) where D is the mean ingested dose (while the n above is the exact number of organisms ingested). This model gives virtually the same result as the above binomial model. If p is assumed to have a constant value r for any given host and any given pathogen, the simple exponential model results: P(Inf D, r) = 1 − exp( − rD) When the dose is low and rD << 1, then this formula is approximated by a straight line, i.e. P(Inf | D, r) ≈ rD If the single-hit probability varies between organisms (e.g. different strains) or between hosts (e.g. due to susceptibility) and is assumed to follow a beta distribution, then: P(Inf | D, α, β ) = 1 −1 F1 (α, α +β, −D) CHAPTER 13 -DOSE–RESPONSE 201 Where 1F1() is the Kummer confluent hypergeometric function (Abramowitz and Stegun, 1972), which can also be found in the Digital Library of Mathematical Functions (Digital Library of Mathematical Function, 2021). For α << β and β >> 1, P is approximately equal to the Beta-Poisson formula: P(Inf D −α D, α, β) ≈ 1 − (1 + β ) As for the exponential model, when the dose is low and Da << b , this formula is approximated by a straight line (which also holds for the exact form involving 1F1()), i.e. P (Inf |D, α, β ) ≈ Dα β For both a → ∞ and b → ∞, while a/b → r, the Beta-Poisson formula approaches the exponential model. Other assumptions for n or p lead to other models. For example, spatial clustering of cells in the inoculum can be represented by a negative binomial distribution or any other contagious distribution. However, this has little effect on the shape of the dose–response relationship (Haas, Rose and Gerba, 2014) although the limiting curve for the confidence interval is affected (Teunis and Havelaar, 2000). It is also possible to model p as a function of covariables, such as immune status or age. Using these models, it is possible to determine the dose below which the dose– response relationship is approximately linear (Williams, Ebel and Vose, 2011a). If the exposure distribution is such that doses will be below this value, then the risk characterization is greatly simplified. 202 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 14 14. Variability and uncertainty Variability and uncertainty are frequently confused because both can be described by distributions. However, they have distinct meanings (Haas, Rose and Gerba, 2014; Nauta, 2000; Vose, 2008), and a common understanding between the risk manager and risk assessor of these concepts can greatly help in the risk assessment process. These topics are considered below. 14.1 VARIABILITY Variability, also sometimes referred to as inter-individual variability, refers to real differences in values of some property of the units from a population over time or space. The population could refer to people, units of food, a species of foodborne pathogen, etc. Examples of variable factors relevant to microbiological risk assessment include, but are not limited to the following: the storage temperatures of food products; seasonality of different food preparation methods (e.g. barbecuing); culinary practices; susceptibility to infection across subpopulations; consumption patterns across a region; differences in growth and inactivation characteristics and in virulence between strains; and product handling processes across different producers. In some cases, some of the variability in the population can be explained by observable individual attributes or explanatory factors. For example, while the human population is heterogeneous, there may be discernible differences between identifiable subpopulations because they are for some reason less frequently 203 exposed, or less susceptible, to the hazard of interest. Or there could be different methods of storing a food product, e.g. frozen, chilled and not chilled, leading to different potential for microbiological growth; the fractions of the food item that are stored in each manner need to be estimated, and they may vary over time. Hence, variability is inherent in the population being studied and describes by how much a specific attribute differs between the units in that population. As a result, variability cannot generally be reduced by more accurate measurement or by collecting more observations, it can only be estimated more precisely. However, some sources of variability may be explained by having more information, such as, knowing whether a food product was stored frozen, chilled or not chilled. In principle, variability can be described by listing the different values that the attribute can take. Often however, there are such a large number of values that it is more convenient to describe the variation using a probability distribution. For example, if an animal shedding an enteric bacterial pathogen, then there are only two possible values, that is, the animal is shedding or it is not. In contrast, the number of bacterial cells in a 10 g faecal sample has possible values 0, 1, 2, 3, etc. Instead of enumerating all possible values, and the probability with which these values can occur, it is usually preferred to describe the possible outcomes by a mathematical distribution, such as the Poisson or the negative binomial distributions. The use of some mathematical distributions is quite well established for some circumstances. For example, the binomial distribution is usually used to describe the number of infected animals sampled from a large herd; alternatively the hypergeometric distribution can be used for small herds. Similarly, the concentration of microbial cells in samples of a food product is often assumed to follow a lognormal distribution, although others may be more appropriate (e.g. Bassett et al., 2010; Haas, Rose and Gerba, 2014; Vose, 2008). Where possible, the fit of the mathematical distribution used to model a particular situation should be checked against empirical data. Tools for this include: • Density histograms with fitted distributions overlaid; • Cumulative distribution plots with fitted distributions overlaid; • Quantile–quantile plots; and • Skewness–kurtosis plots (Cullen and Frey, 1999). When there are discernible differences due to known factors, stratification can be a practical method of addressing the population variability by recognizing those populations as discrete within the risk assessment. The properties of each subpopulation, or stratum, may still be described as a variable quantity, but with a 204 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD different mean and spread of values. There are many ways of stratifying a human population using demographic, cultural and other variables, but in microbial risk assessment stratifications in human populations are usually done in one of two ways. One is based on differences in exposure and the other is due to differences in susceptibility, usually related to well recognized subpopulations such as the young, old, pregnant and immunocompromised (YOPI). Exposure and susceptibility strata may be combined, that is, if there is evidence of differences in susceptibility or differential exposure patterns in the population of interest, then consideration should be given to stratifying the risk accordingly. These ideas are illustrated in Figure 14. Here, it is assumed that exposure depends on season (A and B) and producer (1 and 2), leading to 4 different distributions of exposure (A1, A2, B1, B2). In addition, it is assumed that there are two subpopulations, each of which has its own dose–response curve. The figure shows how each exposure model is combined with the appropriate dose–response model if exposure and dose–response are stratified in this way. Source of variability Season Producer 1 A 2 Population x y x y x 1 B y x 2 y Exposure assessment Hazard characterization A,1 x A,1 y A,2 x A,2 y B,1 x B,1 y B,2 x B,2 y FIGURE 14. Linkage between exposure assessment and hazard characterization CHAPTER 14 -VARIABILITY AND UNCERTAINTY 205 With respect to qualitative and semi-quantitative risk assessments, one option for the inclusion of variability is to consider a number of scenarios that reflect the variability, e.g. near-optimal condition, normal situation and one or more adverse conditions. The risk scenarios are then evaluated separately, and the results are compared. The overall assessment of variability (and also uncertainty) will be evaluated in narrative terms such as “very small”, “small”, etc. This approach will make the effects of variability on the risk estimate more transparent. However, if the scenarios vary greatly in risk outcome, such an analysis may provide insufficient support for decision-making in the absence of any description of the relative likelihood of each scenario. It should be noted that risk can be dominated by, or at least strongly affected by, the more extreme scenarios, e.g. conditions leading to relatively high risk, despite their lower probability. It is important that the risk assessor identifies the likelihood with which such scenarios could occur. 14.2 UNCERTAINTY Uncertainty arises due to a lack of knowledge and is sometimes termed epistemic uncertainty, lack-of-knowledge uncertainty, or subjective uncertainty. It is often stated that variability is a property of the system being studied, whereas uncertainty is a property of the methodology and data used. Assessments with different methodologies and data will have different levels of uncertainty regarding their outputs. An understanding of uncertainty is important because it provides insight into how the lack of knowledge can affect decisions. In the EFSA opinion on the principles and methods behind EFSA’s Guidance on Uncertainty Analysis in Scientific Assessment (EFSA, 2018a) uncertainty is used as a general term referring to all types of limitations in available knowledge that affect the range and probability of possible answers to an assessment question. Available knowledge refers to the knowledge (evidence, data, etc.) available to assessors at the time the assessment is conducted and within the time and resources agreed for the assessment. When the uncertainty is large enough that there is ambiguity as to which risk management decision is preferred, then there may be value in collecting additional data or conducting additional research to reduce the uncertainty. It is the risk managers’ role to decide if the uncertainty of a risk assessment output allows for a decision to be made or not. These aspects apply equally to all types of risk assessment. In contrast to variability, uncertainty is not inherent in the population, but a result of limited information and lack of knowledge. Consequently, well targeted collection of data or information can usually help reduce uncertainty. For example, the uncertainty in the parameter estimates from a linear regression model can be 206 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD reduced when more data from the same population can be incorporated into the model fit. Similarly, uncertainty in the processing practices used to manufacture a food product can be reduced by visiting different manufacturing facilities (of different sizes) to gain a better understanding of what actually happens in practice. Uncertainty is associated not only with the inputs to an assessment model, but also regarding the scenarios assumed for the assessment and the model itself. Sources of scenario uncertainty include potential misspecification of the harmful agents of concern, exposure pathways and vectors, exposed populations, and the spatial and temporal dimensions of the problem. Sources of model uncertainty include model structure, detail, resolution, validation or lack thereof, extrapolation, and boundaries of what is included and what is excluded from the model. A list of most common types of uncertainty affecting scientific assessments associated with the inputs and the methodology was identified by EFSA (2018a) and these are presented in Table 40. In addition, Morgan and Henrion (1992) and Cullen and Frey (1999) provide examples of sources of uncertainty in risk assessment, and a more recent discussion is provided by (Spiegelhalter and Riesch, 2011). TABLE 40. List of most common types of uncertainty affecting risk assessments associated with the inputs and the methodology (EFSA, 2018a) Uncertainties associated with assessment inputs Ambiguity Accuracy and precision of the measures Sampling uncertainty Missing data within studies Missing studies Assumptions about inputs Statistical estimates Extrapolation uncertainty (i.e. limitations in external validity) Other uncertainties Uncertainties associated with assessment methodology Ambiguity Excluded factors Distributional assumptions Use of fixed values Relationship between parts of the assessment Evidence for the structure of the assessment Uncertainties relation to the process for dealing with evidence from the literature Expert judgement Calibration or validation with independent data Dependency between sources of uncertainty Other uncertainties CHAPTER 14 -VARIABILITY AND UNCERTAINTY 207 14.3 UNCERTAINTY ANALYSIS Uncertainty analysis is the process of identifying limitations in scientific knowledge and evaluating their implications for scientific conclusions (EFSA, 2018b). It is therefore relevant in all risk assessments to ensure that the conclusions provide the risk managers reliable information for making decisions. The form and extent of uncertainty analysis, and how the conclusions should be reported, vary widely depending on the nature and context of the assessment and the degree of uncertainty that is present. In a Guidance on Uncertainty Analysis (EFSA, 2018a), EFSA presented the main elements of an uncertainty analysis as the following: 1. Identifying uncertainties affecting the assessment. 2. Prioritizing uncertainties within the assessment 3. Dividing the uncertainty analysis into parts. 4. Ensuring the questions or quantities of interest are well-defined. 5. Characterising uncertainty for parts of the uncertainty analysis. 6. Combining uncertainty from different parts of the uncertainty analysis. 7. Characterising overall uncertainty. 8. Prioritizing uncertainties for future investigation. 9. Reporting uncertainty analysis. Identifying the various uncertainties affecting the risk assessment outputs is necessary in every assessment. This should be done in a structured way to minimize the chance of overlooking relevant uncertainties. Although it is often efficient to concentrate detailed analysis on the most important sources of uncertainty, the identification of uncertainties needs to be as comprehensive as possible. Risk assessors should examine in a systematic way every part of their assessment to identify all uncertainties (see Table 40 above). Prioritizing uncertainties within the risk assessment plays an important role in planning the uncertainty analysis, enabling the assessor to focus detailed analysis on the most important uncertainties and address others collectively when evaluating overall uncertainty. Prioritization can be done by expert judgement during the planning process. In more complex risk assessments uncertainties can be prioritized explicitly using sensitivity analysis (see Chapter 15). Depending on the methods and data used, it may be sufficient to characterise overall uncertainty for the whole assessment directly, by expert judgement. In other cases, it may be preferable to evaluate uncertainty for some or all parts of the assessment separately and then combine them to evaluate the overall uncertainty, either by calculation or expert judgement. 208 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Each parameter of interest must be well-defined. This is necessary to ensure the parameter can be estimated appropriately and to make it possible to express uncertainty clearly and unambiguously. Sometimes risk assessors choose or need to divide the uncertainty analysis into parts. In these cases, there may be a need to combine the different parts of the uncertainty analysis if an overall estimate of uncertainty is needed. The element of overall uncertainty analysis includes the quantitative expression of the overall effect of as many as possible of the identified uncertainties on the conclusions. Any unquantified uncertainties should be described qualitatively. In assessments where the effect of one or more uncertainties cannot be characterised, it must be reported that this is the case and that conclusions are conditional on assumptions about those uncertainties; these assumptions must also be specified. Prioritizing uncertainties for future investigation is implicit or explicit in any assessment where recommendations are made for future data collection or research. These priorities may be informed by the sensitivity analysis. The last step of the uncertainty analysis process is reporting. Uncertainty analysis is part of the risk assessment and should be reported in a transparent manner. It is important to list the sources of uncertainty that have been identified and document how they were identified; how each source of uncertainty has been evaluated and how they have been combined; where and how data and expert judgement have been used; what methodological approaches have been used and the rationale for choosing them; and what the results were. It is not necessary to use all the above elements in uncertainty analysis in all risk assessments. The extent and depth of the uncertainty analysis can be scaled to the needs of the assessment and the time and resources available. In addition, the approach to each element, as well as the order in which they are conducted, may vary depending on the nature or type of the risk assessment. 14.4 UNCERTAINTY AND VARIABILITY TOGETHER Most risk assessments will contain variable and uncertain inputs. In some cases, it may be difficult to decide whether information relates to uncertainty and/or variability, such as when model parameter estimates from the scientific literature are expressed as a mean value with an associated standard deviation. It may be unclear whether this standard deviation is an expression of variability or uncertainty, or both. For example, when a growth rate is estimated from a set of growth CHAPTER 14 -VARIABILITY AND UNCERTAINTY 209 experiments, it may not be clear whether the standard deviation in the growth rate – usually referred as a standard error, to denote that it refers to an estimate of a parameter – expresses uncertainty or variability. It is unknown whether the growth rate is actually fixed but cannot be determined precisely by growth experiments or varies between the experiments but can be determined precisely. Presumably, the standard deviation expresses both. In practice, it may be important to know which characteristic is represented, and to what extent (Nauta, 2000). When it is unclear how uncertainty and variability should be separated, there are several possible ways to proceed: • One could test the effect of separation, assuming different weights, i.e. proportional contributions, for uncertainty and variability and explore the effect on the model outputs in several scenarios (e.g. Nauta, 2000). This will show how important it is to separate uncertainty and variability in the given situation. • Alternatively, one might first assume that uncertainty is absent. An assumption of omniscience (pretending that everything is known) results in the remaining probability distributions necessarily describing variability. Once the variability is identified, uncertainty can then be reintroduced through scenarios by systematically varying the uncertain inputs and observing their effect on the model outputs. This approach may be quite cumbersome if there are many uncertain model inputs. • Another way to assess the potential effect of uncertainty is to identify the variable components, set their uncertain parameters to their expected value and run the model (similar to the approach described in the previous bullet point). Then the model is run as a mixed model where the uncertain and variable components are simulated together using distributions as inputs. The results of the two models can then be compared to assess the potential effect of uncertainty on the model outputs and the need or otherwise to separate the two by developing a second order model (see next bullet point). • Cullen and Frey (1999) suggested that the relative importance of variability and uncertainty can be assessed by inspecting a two-dimensional simulation result, i.e. using a second order model, plotted in the form of a cumulative distribution function (CDF) with confidence intervals. The mean CDF is a best estimate of variability. The confidence interval on the CDF is a best estimate of uncertainty. If the intervals are wide compared to the range of variation of the best estimate CDF, then uncertainty dominates. If the intervals are narrow, then variability dominates. • Alternatively, Thompson and Graham (1996) provide an overview of when to select various probabilistic analysis methods depending on the objectives. 210 In practice, a combination of the above approaches may be needed. For example, while uncertainty in parameter estimates can be assessed using a two-dimensional simulation model, differently structured food supply pathways may need to be assessed through different scenarios. To illustrate the effects of variability and uncertainty consider the following situations. In the simplest case, the risk measure may be a single point probabilistic measure, e.g. the probability of at least one illness per year or the expected number of cases per year (i.e. no variability is included). This means that, if no uncertainty has been included in the risk assessment model, then the outputs are fixed values (Figure 15, top left). If uncertainty has been included in the model, then the outputs are uncertainty distributions (Figure 15a). The risk measure may alternatively be a probability distribution capturing variability, e.g. a probability distribution of the number of adverse health events a random person might experience per year. This will be a first-order distribution if no uncertainty has been included in the model, or if uncertainty and variability have been combined (Figure 15b). If uncertainty has been included in the model and not combined with variability, then the output will be a second-order probability distribution (Figure 15c). Thirdly, the risk measure may describe the variation in risk across a population, e.g. in different strata. That risk can, for example, be characterized as the probability of illness per serving. This situation can result in a distribution of the variability in that probability across strata (see Section 14.1). The results can then also be stratified by graphing the variation in that probability per serving for each stratum. If the risk assessment did not include uncertainty, then a single probability measure can be used to describe the risk for each stratum (Figure 15d). If the risk assessment included uncertainty (not combined with variability), then the uncertainty in these estimates of probability per serving can be considered (Figure 15e). Whilst it is theoretically possible, it is difficult to graphically compare more than two second-order distributions. For example, if probability distributions of the number of illnesses per stratum over a period are second-order, then it will generally be clearer to make a comparison of an appropriate statistic (mean, 90th percentile, etc.) with attendant uncertainties. 211 Single-point probability measure A xed value Probability distribution (a) x = probability measure; y = con dence (b) x = number ill people(e.g.); y = probability Population variability (c) x,y same as (b). Multiple lines show uncertainty A B C D E F (d) x = sub-group; y = probability measure A B C D E F FIGURE 15. A matrix of various types of quantitative outputs one can produce from a risk assessment describing variability and uncertainty; variability only is shown in the graphs on the left and uncertainty and variability combined are shown in the graphs on the right To separate variability and uncertainty using Monte Carlo analysis, one can apply second order, or two-dimensional, Monte Carlo techniques. In one-dimensional simulation modelling the random realizations of the model inputs can be thought of as being arranged in a one-dimensional vector, with length equal to the number of iterations used for the model. In contrast, the two-dimensional approach can be considered as a series of such vectors, making a two-dimensional array or matrix of size (Nv×Nu); the row dimension (Nv) then captures the variability in the input while the column dimension (Nu) captures the uncertainty (see Figure 2 in Pouillot et al., 2007; and Figure 6 in Pouillot and Delignette-Muller, 2010). It should be noted that twodimensional modelling is not a necessity for dealing with variability and uncertainty. In fact, “manually” investigating uncertainty and variability using, for example, scenario analysis can be more informative than “blindly” applying second order modelling. 212 15 15. Sensitivity analysis Complex risk assessments may have many input and output variables that are linked by a system of equations or other model structures. Sensitivity analysis is a broad set of tools that can provide insights to risk assessors and risk managers about the relative importance of the components of a risk assessment to the risk management question (Frey, Mokhtari and Danish, 2003; Frey, Mokhtari and Zheng, 2004; Saltelli, Chan and Scott, 2008). The plausibility of important components is essential to the overall quality of the risk assessment. Changes in important components also can be expressed in terms of the effect that these inputs have on the answers to risk management questions. A key criterion for sensitivity analysis is that it must be relevant to a decision. Sensitivity analysis evaluates the effect of changes in model input values and assumptions on the model output, and thus on decisions that would be based on the model output. It can be used during model development to evaluate and refine model performance and can play an important role in model verification and validation. Sensitivity analysis can also be used to provide insight into the robustness of model results when making decisions. Sensitivity analysis can also aid in identifying risk mitigation strategies or monitoring points and to focus research activities for purposes of prioritizing additional data collection or research (Lamboni, Sanaa and Tenenhaus-Aziza, 2014). For these purposes, Value of Information (Laxminarayan and Macauley, 2012) analysis can complement sensitivity analysis methods. 213 Microbiological risk assessment models typically have the following characteristics, which can pose substantial challenges to the application of sensitivity analysis methods: • nonlinearities; • thresholds, e.g. below which there is no growth of a microbiological pathogen; • discrete inputs, e.g. integer numbers of animals or yes/no indicators of contamination; • incorporation of measurement error; • variation in the scale (units and range) and shape of distributions of model inputs; and • temporal and spatial dimensions, including dynamics, seasonality or inter- annual variability. The relationship between model inputs and outputs should be one-to-one for effective application of sensitivity analysis methods. Ideally, a sensitivity analysis method should provide not just a rank ordering of key inputs, but also some discriminatory quantitative measure of sensitivity, such that it is possible to clearly distinguish the relative importance of different inputs (e.g. correlation). For example, are there groups of inputs among which several inputs are of comparable importance, and is there clearly a difference in importance between such groups? Statistical-based methods such as regression analysis or analysis of variance (ANOVA) produce quantitative indicators of the relative importance of different inputs, e.g. using normalized or standardized regression coefficients. Moreover, techniques such as regression analysis also provide an indication of the statistical significance of differences in sensitivity among inputs, based on confidence intervals for regression coefficients. However, it should be noted that statistical tests may be able to detect very small effects, especially if the number of iterations is large, and hence any significant effect should be assessed as to its practical importance, i.e. is the effect large enough to affect risk management decisions? Irrespective of the risk assessment approach used, the utility of well-constructed what-if scenarios should not be underestimated, e.g. covering different exposure pathways or dose-response models. 15.1 SENSITIVITY ANALYSIS IN QUALITATIVE RISK ASSESSMENT In examining an association between a hazard and an adverse health effect, widely accepted criteria (e.g. Hill’s Criteria of causation) have been established for determining whether the evidence is weak, moderate or compelling (e.g. Tomatsis, 1990). Narrative criteria may be inherently subjective, and therefore difficult to 214 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD reproduce. To the extent that the criteria can be evaluated objectively, however, different assessors using the same information should be able to independently reproduce a determination of whether the criteria have been satisfied. For example, the weight of evidence for causality is stronger when detection of the association has been independently reported from multiple sources, when the strength of association is related to the level of exposure to the agent, or when changes in the hazard precede changes in the observed effect. Determining whether such criteria are satisfied requires scientific evidence. If the results of a qualitative assessment are invariant to an accumulation of evidence regarding an association or, alternatively, to contradictory evidence, then the assessment is insensitive to the established criteria for evaluating causality. For example, in a qualitative hazard characterization, an assessment based solely on the criteria of acute health outcomes could be insensitive to information regarding known chronic sequelae. Alternatively, a qualitative hazard characterization may be highly sensitive to weak evidence regarding chronic sequelae associated with an opportunistic pathogen that rarely causes acute illness. If a qualitative risk assessment finds that a pathogen poses a negligible risk based on the assumption that the pathogen does not grow under certain environmental conditions, and new information indicates that the pathogen is capable of growing under these conditions, then the sensitivity of the findings of the risk assessment to this new information may depend on prespecified criteria. Such criteria may be based on whether the results have been independently reproduced or the methods have been exposed to peer review. At a minimum, the scientific basis and criteria for characterization of a qualitative risk assessment need to be sufficiently transparent to permit assessment of the effect of new information or plausible alternative assumptions on the findings. 15.2 SENSITIVITY ANALYSIS IN QUANTITATIVE RISK ASSESSMENT There are several approaches to sensitivity analysis in quantitative risk assessment models. Saltelli et al. (2008) provide a thorough exploration of the topic, summarized below, as do Frey et al. (2003; 2004). 15.2.1 Statistical methods Examples of statistical sensitivity analysis methods (also referred to as variancebased methods) include rank order correlations, regression analysis, ANOVA, response surface methods, Fourier amplitude sensitivity test (FAST), mutual information index (MII), and classification and regression trees (Frey, Mokhtari and Danish, 2003; Frey, Mokhtari and Zheng, 2004; Frey and Patil, 2002; Mokhtari, CHAPTER 15 -SENSITIVITY ANALYSIS 215 Frey and Jaykus, 2006). Most of these methods are applied in conjunction with, or after, a Monte Carlo simulation. Regression analysis, ANOVA, FAST and MII provide quantitative measures of the sensitivity for each input. Regression analysis requires the assumption of a model form. 15.2.2 Graphical methods Graphical methods represent sensitivity typically in the form of graphs, such as scatter plots and spider plots (Eschenbach, 1992; Frey, Mokhtari and Danish, 2003). The results of other sensitivity analysis methods also may be summarized graphically, e.g. tornado charts for displaying rank order correlation. These methods can be used as a screening method before further analysis of a model, or to represent complex dependencies between inputs and outputs. For example, such complex dependencies could include thresholds or nonlinearities that might not be appropriately captured by other techniques. 15.2.3 Evaluation of sensitivity analysis methods Each sensitivity analysis method provides different information (e.g. Table 5-1 in Frey, Mokhtari and Zheng, 2004) regarding sensitivities of the inputs such as the joint effect of inputs versus individual effects, small perturbations of inputs versus the effect of a range of variation, or apportionment of variance versus mutual information. Nonparametric methods, such as Spearman’s rank correlation, are applicable to monotonic, nonlinear models, and Vose (2008) recommends the use of spider plots to illustrate the effect of individual input variables on the uncertainty of the model output. Because agreement among multiple methods implies robust findings, two or more different types of sensitivity methods should be applied where practicable. This allows the results of each method to be compared and conclusions to be drawn about the robustness of rank ordering of key inputs. 216 16 16. Quality assurance The validity of any risk assessment is based on the soundness of the model structure, its inputs, the underlying assumptions and the interpretation of results. Therefore, quality assurance is a crucial element of risk assessment. 16.1 DATA EVALUATION Risk assessors must evaluate the quality of the data used in the analysis (see also Chapter 10), and the means of characterizing the uncertainty of any data used. The aspects listed in this section are not primarily intended for differentiating “good” from “bad” data, but rather to guide the subsequent analysis and their use in a risk assessment model. Formalized quality control of raw data and its subsequent treatment is desirable, but also dependent on data availability and how the data are used. There is no formalized system for evaluation of data. Few generalizations can be made, but how data are collected and interpreted needs to be clear. Good data are complete, relevant and valid; complete data are objective; relevant data are case-specific; and validation is context specific. Data which are complete include such things as the data source and the related study information (e.g. sample size, species or strain, immune status, etc.). Characteristics of relevant data can include age of data; region or country of origin; purpose of study; analytical or data collection methods. Observations in a 217 database should be “model free”, i.e. reported without interpretation by a particular model, to allow data to be used in ways that the original investigator might not have considered. Ideally this implies that the raw data can be accessed, which may be difficult to achieve in practice. Scientific publishers are encouraging the sharing of data associated with publications, where possible, and independent data repositories have also been created; see for example http://foodrisk.org or https:// www.combase.cc. Valid data are those that agree with others in terms of comparable methods and test development. In general, for dose–response modelling, for example, human data need less extrapolation and are preferred to animal data, which in turn are preferable to in vitro data. Data on the pathogen of concern are preferred to data on surrogate organisms, which should only be used when proven to be valid (USNACMCF, 2010). The current recommended practice is to consider any available data as potentially useful. Whether data should be eliminated from the risk assessment depends on the purpose and stage of the assessment. Small data sets or those with qualitative values may be useful in the early stages of a risk assessment. The later stages of risk assessment may include only those data that meet a particular quality standard. Excluding data from the analysis should be based on predefined criteria, e.g. age of the data set and geographic representativeness, and not based solely on statistical criteria (e.g. Section 16.1.2). If the data are extremely heterogenous or contain outliers, then they may be stratified according to suitable criteria. This practice should provide increased insight rather than information loss. Sources of data may come from the peer-reviewed or non-peer-reviewed literature. Although peerreviewed data are generally preferable, they also have some important drawbacks (see also Section 10.1). Access to the peerreviewed literature may be restricted especially for developing countries, although open-access publications and Research4Life (see Section 3.5.2), for example, are helping to address some of these limitations. Peerreviewed data may be missing important methodological details (e.g. sample preparation and characteristics), are usually presented in an aggregated form, and may not provide the level of detail necessary for uncertainty analysis. Quality control of the measurement process may be poorly documented. The potential for publication bias should not be ignored, as “replication studies” may not provide enough novelty for publishers and hence may only get published through conference presentations, reports or other formats. The analyst might wish to add information from other sources for any of these reasons. The quality of any data used should be explicitly reviewed, preferably by independent experts, and any concerns regarding data quality should be explicitly noted. 218 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD The results of any risk assessment are conditional on the data and information used to develop the risk model. Any risk assessment should summarize the primary strengths and limitations of the data, methods, and analyses used. Typically, these analyses require risk assessors to synthesize and draw inferences from disparate data sources not specifically or originally intended for use in risk assessment. In some cases, this requires the use of unconventional or nonroutine methods that might be highlighted for close scrutiny, to ensure that they are reasonable and correctly applied. 16.1.1 Data collection Suitable data for microbiological risk assessment may be sparse. Assessors should initially collect all reasonably obtainable data consistent with the assessment objective, and subsequently investigate the quality of different data sources. When collecting data, several issues should be considered to evaluate data quality. The following considerations apply to any data, including information elicited from experts. Risk assessors should ideally have access to raw, rather than to data summaries (e.g. EcoSure, 2008). Statistical methods such as quantile–quantile plots and skewness– kurtosis plots (Cullen and Frey, 1999) can be useful to identify suitable parametric distributions, if the raw data contain sufficient observations. Alternatively, empirical distributions or nonparametric simulation methods can be used to characterize input distributions. Graphical methods are generally preferred over statistical tests (e.g. Goodness-of-Fit) which are affected by the size of the data set. Large sample sizes can identify statistically significant deviations from the hypothesized distribution, even though these deviations may be of little practical importance. Raw data are frequently inaccessible and results are often reported as aggregated summary statistics (e.g. estimated mean, standard deviation or standard error). It may be necessary to obtain information on the assumed distribution of the underlying data, together with the sample size to develop a distribution from data summary statistics. It is useful to collect as much background information on the data sources as possible, such as the year of completion, country of origin, the type of sample, possible transformation of the data, methods of analysis, microbiological strain and population demographics. This information could be important about treatment or use of the data or to support the decision on whether to include these data in the model. CHAPTER 16 -QUALITY ASSURANCE 219 As an example, consider the Danish risk assessment for Campylobacter jejuni in chicken (Christensen et al., 2001). Quantitative data were needed to describe the relative change in pathogen concentration over a given processing step in a poultry slaughterhouse. Because Danish data were unavailable, data from foreign studies were applied to assess the efficacy of the wash and chiller process in reducing the pathogen levels on chicken carcases. Data for the microorganism of interest were available, but the data from the different studies were obtained from different sample units (neck skin samples, whole carcase wash, and swab samples). This mix of sample types all reflected surface contamination of chicken carcases. The risk assessors assumed that the relative reduction in pathogen concentration over the process was independent of the type of surface measured. The slopes of the lines shown in Figure 16 reflect differences in log-concentration over the process. Since all the slopes appear to be similar (though not identical), all data sets were used in describing the reduction over the “wash + chiller” process. Mead et al. (1995) Concentration of Campylobacter (log10cfu/carcase) Oosterrom et al. (1983) Izat et al. (1988) before wash+chiller a er wash+chiller Cason et al. (1997) FIGURE 16. The effect of a selected slaughterhouse process on the Campylobacter concentration on chicken carcases. The change in pathogen concentrations (expressed as log CFU per carcase) before and after the process is represented by a line connecting data points originating from the same study (adapted from Christensen et al., 2001) Data for the specific microorganism under study may not always be available or of suitable quantity and quality. Data from a surrogate microorganism may be used, provided that the surrogate behaves similarly under the process of interest, e.g. generic E. coli to estimate cross-contamination during slaughter procedures. 220 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Data from different surrogate organisms could be used to model different steps in the same model, based on data availability and suitability. Sampled data with different units, e.g. absolute concentration or change in concentration, can be used to describe the same process, as the example above illustrates. Depending on how the data are used in the model, e.g. describing a change in concentration over a step or describing the concentration level, different parameters may be evaluated in a sensitivity analysis to ensure data quality objectives are satisfied. In some cases, the available data may not be representative of the population of interest. These data may be excluded from the analysis or incorporated with appropriate adjustment. The bases for decisions regarding the treatment of unrepresentative data are context specific and need to be clearly articulated. For example, data from a particular source may be considered unrepresentative for the purposes of providing an estimate of central tendency (e.g. the mean) but may nevertheless be useful for the purposes of characterizing the spread of an input distribution (e.g. the standard deviation). 16.1.2 Sorting and selecting data sources After collecting potentially suitable data sets, the risk assessor should evaluate each critically and select the data sets that will provide the most appropriate model input for the specific purpose (e.g. contamination level, contamination prevalence or changes during processing). Plotting the parameter of interest with the 95 percent confidence intervals provides a useful overview (see Figure 17). Studies 1 2 3 4 5 6 7 8 9 10 95% con dence interval FIGURE 17. Example of an overview of data from different studies, with 95 percent confidence intervals for a model input parameter CHAPTER 16 -QUALITY ASSURANCE 221 Both subjective and statistical criteria may be applied in selecting the suitable data for incorporation into the risk assessment. Subjective evaluation criteria may include the representativeness of the geographical and temporal properties of the study. For example, if study 1 in Figure 17 is the only study conducted outside the country of interest, and it is significantly different from the rest (based on statistical criteria), this data set could be excluded. If the 10 studies all originate from the same country, but are reported by different laboratories, the differences may be due to variability between the laboratories or specific sampling context and the assessor might decide to incorporate all studies in the model. Irrespective of the decision taken, the rationale should be documented. 16.2 MODEL QUALITY ASSURANCE Models should be verified and validated, and they may also be anchored (calibrated). Model verification is achieved by auditing the model to ensure that it operates as intended by the developer. Anchoring and calibration are techniques to adjust the model to approximate observed data. Model validation can be defined as demonstrating the accuracy of the model for a specified use. Model verification should precede model validation. If the model is to be both anchored and validated, using a withheld test portion of the independent data, then anchoring should precede model validation. 16.2.1 Model verification Verification involves checking the software code used to implement the model. Verification requires that the model be suitably documented. All data, methods, assumptions and tools used should be clearly described, so that the model can be independently reproduced. A well-organized model structure facilitates verification. The following questions may be useful for those seeking to verify a model: • Are the analytical equations correctly derived and free of error? If approximations are used, then under what assumptions do they hold and are those assumptions always met? • Is the computerized version of the analytical model correctly implemented? What, if any, are the limits of the implementation? • Are the inputs correctly specified? • Do the units of measurement (e.g. CFU or log CFU) propagate correctly through the model? 222 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD • Is the model internally consistent? For example, if an assumption is made in one part of the model, is it consistently applied throughout the model? Is there consistency within the model between the intermediate outputs and inputs? • Are errors in any computational step flagged appropriately, or could they result in inappropriate values being propagated through the model? • Are the intermediate outcomes and end results evaluated to be realistic? It may be difficult in some cases to do a line-by-line verification of computer code, especially for large models. The verification of any computer code will be facilitated if good software engineering practices (e.g. Pressman, 2005) are followed. This may include clear specification of databases; development of a software structure design prior to coding; version control; modular design; clear specification of interfaces between components of a model; and good communication among project teams when different individuals are developing different components of a model. Literate programming techniques (Knuth, 1992) can also be useful for this purpose as they allow embedding of the model code in the documentation; a range of tools for various programming languages and environments are available (WikiPedia, 2021). Model documentation and peer review are critical aspects of the verification process. 16.2.2 Model anchoring or calibration Anchoring is a technique in which the model is adjusted, or calibrated, to be more compatible with observed data. For example, model parameters may be adjusted to achieve agreement between model predictions and observed data, such as the predicted versus actual number of illnesses per year attributed to the hazard and the food. As noted above, if the model is to be both anchored and validated, using a withheld, independent test portion of the data, then anchoring should precede model validation. Anchoring is a generally accepted practice in health risk assessment and environmental modelling, and has been employed in one fashion or another in various risk assessments (FAO and WHO, 2005; FSIS, 2001, 2005; USFDA/FSIS, 2003). Data from outbreaks could be considered as the ultimate anchor for dose– response models and they are also an important way to validate risk assessments. This is because the dose ingested by different consumers involved in an outbreak is likely to be more similar than the doses associated with sporadic cases. Since anchoring requires some data, it may compromise efforts to validate the model in situations without sufficient data to support both activities. A common approach in statistics and machine learning is to separate a data set into two independent components: training and test data. The training data are used to fit the model and CHAPTER 16 -QUALITY ASSURANCE 223 estimate the model parameters, while the test data are used to independently check the predictions of the model against previously unseen observations. In general, anchoring approaches that weigh model inputs in proportion to their likelihood in light of the observed data are superior to using simple adjustment factors or censoring input values that are incompatible with the observed data (Institute of Medicine, 2002; Williams, Ebel and Vose, 2011b). Whatever the anchoring approach, considerable care must be taken to ensure that the adjustment procedure is well reasoned and transparent. 16.2.3 Model validation Risk assessment, like any type of problem solving is cyclical in nature. Defining the problem, considering alternative solutions, and implementing a solution all lead to the need to assess the effectiveness of the chosen solution. The cycle may repeat based on that assessment. No risk assessor should think their job is done after a risk management decision is made. The risk assessor may begin planning how they will assess the validity of the predictions of their model in the context of the risk management option selected. This assessment of validity may not occur until years after risk management options are implemented. Model validation can be defined as demonstrating the accuracy of the model for a specified use. Accuracy is the absence of systematic and random error, commonly known as trueness and precision, respectively. Models are always incomplete representations of the system they are intended to model, but they can still be useful. General information on working with mathematical models can be found in various theoretical and applied textbooks. Doucet and Sloep (1992) give a good introduction to model testing. These authors discriminate between models shown to be plausible and models shown to be true. McCullagh and Nelder’s book on generalized linear models (McCullagh and Nelder, 1999) is a valuable resource on statistical modelling methods, and describes some general principles of applying mathematical models, underlining three key principles: • All models are wrong, but some are more useful than others; • Do not fall in love with one model to the exclusion of others; and • Thoroughly check the fit of a model to the data. Law (2014), in addressing the issue of building valid, credible and appropriately detailed simulation models, considers techniques for increasing model validity and credibility. Model validation procedures should be aimed at answering questions such as the following. 224 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 1. Does the model make sense? 2. Does the model respond in an appropriate manner to changes in input assumptions? 3. Do predictions respond in an appropriate manner to changes in the structure of the analysis? These processes are also referred to by some as a “reality check”, “laugh test” or “confidence building”. Model validation is highly dependent on the risk management question, and the degree of validation required should be proportionate to the stakes of the decision. Model validation involves demonstrating the accuracy of the model for a specified use and there are different aspects to model validation. Dee (1994, 1995) identified four major aspects associated with model validation: (i) Conceptual validation; (ii) Validation of algorithms; (iii) Validation of software code; and (iv) Functional validation. These are described below. Conceptual validation concerns the question of whether the model accurately represents the system under study. Was the simplification of the underlying biological process in model steps realistic, i.e. were the model assumptions credible? Usually, conceptual validation is largely qualitative and is best tested against the opinion of experts with different scientific backgrounds. Different models with various conceptual bases can be tested against each other within a Bayesian framework, using Bayes factors, or some other information criterion (Kass and Raftery, 1995). Experimental or observational data in support of the principles and assumptions should be presented and discussed. With respect to dose–response modelling, the concepts described in Section 6.3 represent the consensus opinion of a broad group of experts who contributed to the original FAO guidelines on hazard characterization (FAO and WHO, 2003). These are based on mechanistic reasoning and are supported by some experimental evidence. As such, they are considered to be currently the best basis for dose–response modelling studies. Algorithm validation concerns the translation of model concepts into mathematical formulae. It addresses questions such as: • Do the model equations represent the conceptual model? • Under which conditions can simplifying assumptions be justified? • What effect does the choice of numerical methods for model solving have on the results? • Is there agreement among the results from use of different methods to solve the model? CHAPTER 16 -QUALITY ASSURANCE 225 Software code validation concerns the implementation of the model in a computer language. Good programming practice (i.e. modular and fully documented) is an essential prerequisite. Specific points for attention are the possible effects of machine precision and software-specific factors on the model output. For this reason, open-source software and models implemented in a computing language (e.g. R, Python, C++, etc.) may be preferable to those implemented in a proprietary software program, as all computational steps can be inspected if needed. Internal error reports of the software are important sources of information, as well as evaluation of intermediate output. Functional validation concerns checking the model against independently obtained observations. Ideally, it is evaluated by obtaining pertinent real-world data, and performing a statistical comparison of simulated outcomes and observations (Ebel and Williams, 2019). This requires more detailed information than is usually available, especially if data are also used for anchoring (Section 16.2.2). It may be possible to compare results from risk assessment studies with independently obtained epidemiological estimates of disease incidence. Such data cannot validate a dose–response model per se but may produce valuable insights, especially if the predictions do not closely match epidemiological observations. Most studies to date have considered that a range check of estimated risks and observed incidences were sufficient validation of the model. Credibility of results can also be established by demonstrating that different sources of data are consistent with output values. These might include intermediate outputs. Cassin et al. (1998) provide a good example of such comparisons. When making such comparisons, the different nature of the food, microbiological hazard and processes must be accounted for. It should be noted that if the model output does not agree with the observations, it might not necessarily be that the model is wrong. It may be that the observations themselves were affected by an unknown factor (e.g. microbiological methodological insensitivity) or that foodborne illness were underestimated based on current epidemiological data. There may also be a variety of different effects acting in concert to cause the differences in the results. Close agreement between an initial risk-modelling effort and independent validation data would be fortuitous. Agreement between the model output and validation data may be coincidental, however, and would not necessarily indicate that all of the intermediate model components are accurate. Typically, model development and refinement are iterative. Whether model validation or anchoring is considered, the credibility of the model may be strengthened by having multiple points at which the model can be compared to observed data. In general, the 226 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD scientific credibility of a model is strengthened if consistent results are derived from different relevant data sources (e.g. laboratories, regions) or types (observational or experimental), or a combination. The required degree of relevance and consistency is a context-specific judgement. The tolerance for inconsistent answers depends on what constitutes an important difference with respect to changes in model results. In the risk assessment context, an important difference in model results is one that would significantly modify the risk management decision under the relevant decision criteria. There are situations in which it may be difficult, or practically impossible, to completely validate a model. For example, because risk assessment models are often attempting to predict low probability events, it can be difficult to obtain an independent data set of sufficient sample size to make a sensible comparison of predictions versus observations. It may be possible to validate components of the model even in such situations. For example, it may be possible to validate portions of the model that deal with a particular exposure pathway by making measurements of hazard levels in specific foods. In many cases, there may be insufficient or no independent data with which to compare model predictions. In these situations, alternatives to validation include: • Screening procedures to identify the most important model inputs and pathways • Sensitivity analysis to identify the most important inputs or groups of inputs • Uncertainty analysis to evaluate the effect of uncertainty in model inputs with respect to predictions • Comparison among predictions of different models • Evaluation of sensitivity of results to different assumptions regarding scenarios, model boundaries, model resolution and level of detail While none of these techniques provides a direct validation of the model, each of these techniques provides insight into the sensitivity of the model predictions to key assumptions regarding the analysis. The response of the predictions to these procedures can be evaluated with respect to prior expectations, comparison with analogous systems, and theoretical justifications. 16.3 COMPARISON WITH EPIDEMIOLOGICAL DATA To make a valid comparison with a foodborne pathogen risk estimate, at least three factors need to be considered when deriving an epidemiological estimate CHAPTER 16 -QUALITY ASSURANCE 227 from human surveillance data (Powell, Ebel and Schlosser, 2001). These factors, discussed in more detail below, are: 1. Cluster-weighted rate of illness; 2. Adjustment of surveillance data to account for underreporting; and 3. Etiological fraction attributable to food products. If the risk assessment estimates the incidence of illness at the national level, the epidemiological estimate will need to extrapolate the rate of illness beyond the surveillance area to permit comparison at the national level. In this case, the raw reported rate in each surveillance area may be weighted by the population of the region represented by the area (e.g. state population size) to obtain a weighted average rate of illness (e.g. cases per 100 000 in the national population). If multiple years of surveillance data are available, then the data can be used to characterize year-to-year variability in the rate of illness. Estimating the actual incidence of illness requires adjustment for recognized sources of underreporting in human surveillance data (Scallan et al., 2011; Williams, Ebel and Vose, 2011b). For example, some ill persons do not seek medical care, physicians do not obtain stool specimens from all patients, laboratories do not culture all stool samples for the pathogen of concern, and some proportion of the lab results are false negatives. If estimates are available on the proportion of cases at each step in the reporting process, the negative binomial distribution can be used in sequential fashion to estimate the number of cases missed at each step. In some cases, the proportions may be dependent on the nature or severity of symptoms. For example, a person with bloody diarrhoea may be more likely to seek medical care than one with non-bloody diarrhoea. In this case, the proportion of cases with different levels of symptoms must be estimated prior to accounting for the number of cases missed at each step, and the adjusted symptom-specific estimates are summed to estimate the total number of cases (Hall et al., 2008). In general, the degree of underreporting tends to be substantial (WHO, 2015), and varies among countries and between regions within countries (Scallan et al., 2011). The etiological fraction refers to the proportion of cases attributable to an exposure pathway or a specific food product (Greig and Ravel, 2009; Mullner et al., 2009; Painter et al., 2013; Pires, 2013; Pires et al., 2009). If the scope of the risk assessment is limited to a particular food product, then the proportion of cases due to other exposure pathways (e.g. other foods, drinking water) needs to be subtracted from the overall estimate of illness obtained from the human surveillance data. In general, empirical data on the etiological fraction are scarce. It may be possible, however, to specify a range of uncertainty on the basis of expert judgement (e.g. Vally et al., 2014). 228 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 16.4 EXTRAPOLATION AND ROBUSTNESS Model robustness refers to the performance of the model when its assumptions are violated. In this context, assumptions include model form and model inputs. Extrapolating model results to other settings may involve many forms of extrapolation. Examples include extrapolation from the present to the future; from one geographical region to another; from one microorganism to another; from animals to humans; from clinical trial subjects to the general population; from one population to another; from available data to values beyond the observed range; and from experimental settings to operational environments. Some extrapolations can be made with relative confidence, while others cannot. Some degree of extrapolation may be inevitable, since the demands of risk management may outstrip the supply of relevant science. The importance of various forms of extrapolation made in risk assessment needs to be considered and, to the extent feasible and relevant, characterized in a clear manner, either quantitatively or qualitatively. Extrapolation is explicit when the selected values of model inputs are outside the range of values used to calibrate or validate the model, or both. However, there can also be hidden extrapolation. A hidden extrapolation occurs for a combination of values of two or more model inputs, such that these values individually are enclosed by ranges used for calibration and validation, but for which the specific combination was not included or approximated during calibration or validation. Thus, simple range checks on each input will not guarantee that a hidden extrapolation does not occur. Hidden extrapolation would typically be more of a problem for a system in which there are highly sensitive interactions among inputs or when model inputs are highly correlated. A model that is calibrated to a narrow range of values for each input may not be robust when applied to sensitivity or uncertainty analysis. The use of ranges or distributions, rather than point estimates, could lead to hidden or implicit extrapolations of the model. Situations may also arise in which some iteration of Monte Carlo simulation give division by zero or unbounded result errors. Such problems can often be solved by investigating model assumptions, checking model inputs, and adding error trapping in the software code. Problems such as these can arise in practice, particularly when working with a model or code that someone else developed and for which documentation may be inadequate. A model is considered to be robust if it responds in a reasonable manner to variation in input values. At the same time, such a model will not easily be subject to singularity points or other structural issues that lead to substantial magnification of errors in input values, whether because of uncertainty or user error. A model CHAPTER 16 -QUALITY ASSURANCE 229 that is based on sound theory might be used with more confidence compared with a purely empirical model that is essentially “curve fitting”. 16.5 CREDIBILITY OF THE RISK ASSESSMENT Documentation, validation, and review are necessary criteria for the credibility of a risk assessment. None of these criteria is sufficient by itself, however, as credibility depends on all three criteria being satisfied in a manner that is proportionate to the stakes of the decision. Documentation and scientific review are discussed below and validation has already been discussed in Section 16.2.3. 16.5.1 Risk assessment documentation Risk assessment documentation should serve both technical and nontechnical readers. One way to address this need is to provide a technical document with all modelling details and a less technical interpretive summary. Risk assessment documentation must enable the analysis to be independently reproduced. Modern programming tools, free and open-source software, and sharing of risk assessment model code may assist in this aim. The principle of transparency also requires that the basis for model inputs and assumptions is clearly stated, e.g. by references to scientific literature, evaluation criteria or expert judgement. The expectation for risk assessment documentation should be reasonable, however, because in some cases, assumptions may be based on common knowledge or generally accepted practices in the field. For example, the lognormal distribution is commonly assumed for modelling variables that are the product of several other variables. Because risk assessments are difficult to fully validate, and because such assessments are used to inform public health decision-making, it is critically important that the information used for the assessment, including the model, are accessible for review by experts and the lay public (e.g. FAO and WHO, 2009c, 2009d). The information in the documentation of a risk assessment should include: 1. Data or references to data sources 2. Data analysis and estimation, including methods and results 3. Scenarios, including the temporal and spatial aspects of the exposure scenarios, the specific hazards addressed, exposed populations and exposure pathways 4. The analytical model used for analysis, including the theoretical or empirical basis 5. Discussion and comparison of alternative model formulations, and justification for choices made regarding model structure 230 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 6. Assumptions regarding values assigned to model inputs, including point estimates, ranges and distributions 7. Model verification, including assessment of results from sensitivity and uncertainty analysis 8. Model anchoring (calibration) 9. Model validation 10. Computer implementation of the analytical model, including software design 11. An interpretive summary that is understandable by the risk manager 16.5.2 Scientific peer-review The credibility of risk assessment results can be improved by the process used to develop the results. Peer and public review of risk assessments are an essential part of the process, but each type of review generates distinct and sometimes conflicting demands that should be addressed on their own terms. Morgan and Henrion (1992) identify exposure to peer review as a basic tenet of good policy analysis. The focus of a scientific peer review is highly dependent on the risk management question that the risk assessment is intended to inform. Without reference to a well-defined and specific risk management question, peer review of a risk assessment may fail to focus on the particular uncertainties that are most likely to affect the risk management decision. For example, if the risk management question is “What is the likelihood that a specific pathogen occurs in a particular food production process?”, then data gaps and other uncertainties regarding postproduction processes are irrelevant to the decision. Peerreview comments regarding the scope of the risk assessment, while potentially useful for future risk assessments, are not relevant to the adequacy of the risk assessment under review. If a risk assessment has multiple objectives, peer review may help to identify which objectives an assessment satisfies, since an assessment that is adequate to inform one decision may be insufficient to support another. A thorough review can be difficult and time consuming for a complex risk assessment, even if the documentation is adequate. In the case of large, complex risk assessments, a thorough review may require a multidisciplinary team and a significant budget, e.g. the review by the Institute of Medicine (2002) of the FSIS risk assessment of E. coli O157:H7 in ground beef (FSIS, 2001). The substantive and procedural benefits of peer review should therefore be balanced by time and resource considerations. The level, extent and timing of review should be proportionate to the stakes of the decision, taking into consideration the need for immediate action in the event of actual public health emergencies. CHAPTER 16 -QUALITY ASSURANCE 231 Bibliography den Aantrekker, E.D., Beumer, R.R., van Gerwen, S.J.C., Zwietering, M.H., van Schothorst, M. & Boom, R.M. 2003. Estimating the probability of recontamination via the air using Monte Carlo simulations. International Journal of Food Microbiology, 87(1–2): 1–15. Abramowitz, M. & Stegun, I.A. 1972. Handbook of mathematical functions. Ninth edition. New York, USA, Dover. ACMSF. 2012. ACMSF Approaches to microbiological risk assessment: Discussion paper -- risk assessment outputs. No. ACM/1065. Advisory Committee on the Microbiological Specifications of Food. (also available at https://acmsf.food.gov. uk/sites/default/files/mnt/drupal_data/sources/files/multimedia/pdfs/committee/ acm_1065.pdf ). Alber, S.A. & Schaffner, D.W. 1992. Evaluation of data transformations used with the square root and schoolfield models for predicting bacterial growth rate. Applied and Environmental Microbiology, 58(10): 3337–3342. Al-Zeyara, S.A., Jarvis, B. & Mackey, B.M. 2011. The inhibitory effect of natural microflora of food on growth of Listeria monocytogenes in enrichment broths. International Journal of Food Microbiology, 145(1): 98–105. https://doi. org/10.1016/j.ijfoodmicro.2010.11.036 Anderson, R.M. & May, R.M. 1992. Infectious Diseases of Humans: Dynamics and Control. OUP Oxford. 772 pp. Arrow, K.J. 1963. Uncertainty and the Welfare Economics of Medical Care. The American Economic Review, 53(5): 941–973. Arthur, T.M., Bosilevac, J.M., Brichta-Harhay, D.M., Guerini, M.N., Kalchayanand, N., Shackelford, S.D., Wheeler, T.L. & Koohmaraie, M. 2007. Transportation and lairage environment effects on prevalence, numbers, and diversity of Escherichia coli O157:H7 on hides and carcasses of beef cattle at processing. Journal of Food Protection, 70(2): 280–286. https://doi.org/10.4315/0362-028X-70.2.280 Bailey, N.T.J. 1975. The mathematical theory of infectious diseases and its applications. Second edition. New York, Hafner Press. Baranyi, J., Buss da Silva, N. & Ellouze, M. 2017. Rethinking tertiary models: Relationships between growth parameters of Bacillus cereus strains. Frontiers in Microbiology, 8. https://doi.org/10.3389/fmicb.2017.01890 Baranyi, J., Ross, T., McMeekin, T.A. & Roberts, T.A. 1996. Effects of parameterization on the performance of empirical models used in `predictive microbiology’. Food Microbiology, 13(1): 83–91. https://doi.org/10.1006/fmic.1996.0011 232 Bassett, J., Jackson, T., Jewell, K., Jongenburger, I. & Zwietering, M.H. 2010. Impact of microbial distributions on food safety. International Life Sciences Institute (ILSI) Europe. (also available at http://ilsi.org/publication/impact-of-microbialdistributions-on-food-safety/). Bassett, J., Nauta, M.J., Lindqvist, R. & Zwietering, M.H. 2012. Tools for microbiological risk assessment. International Life Sciences Institute (ILSI) Europe. (also available at http://ilsi.org/publication/tools-for-microbiological-risk-assessment/). Batz, M., Hoffmann, S. & Morris, J.G. 2014. Disease-outcome trees, EQ-5D Scores, and estimated annual losses of Quality-Adjusted Life Years (QALYs) for 14 foodborne pathogens in the United States. Foodborne Pathogens and Disease, 11(5): 395–402. https://doi.org/10.1089/fpd.2013.1658 Berends, B.R., Van Knapen, F., Mossel, D.A.A., Burt, S.A. & Snijders, J.M.A. 1998. Impact on human health of Salmonella spp. on pork in The Netherlands and the anticipated effects of some currently proposed control strategies. International Journal of Food Microbiology, 44(3): 219–229. https://doi.org/10.1016/S01681605(98)00121-4 Bergold, J. & Thomas, S. 2012. Participatory research methods: A methodological approach in motion. Historical Social Research / Historische Sozialforschung, 37(4): 191–222. Berjia, F.L., Poulsen, M. & Nauta, M.J. 2014. Burden of diseases estimates associated to different red meat cooking practices. Food and Chemical Toxicology, 66: 237–244. https://doi.org/10.1016/j.fct.2014.01.045 Biosecurity Australia. 2016. Biosecurity import risk analysis guidelines 2016: Managing biosecurity risks for imports into Australia. Canberra, Australia Government, Department of Agriculture, Water and the Environment. (also available at https://www.agriculture.gov.au/sites/default/files/sitecollectiondocuments/biraguidelines-2016.pdf ). Bogaardt, M.J., Mangen, M.-J.J., de Wit, G.A., Nauta, M.J. & Havelaar, A.H. 2004. Controlling Campylobacter in the chicken meat chain - Towards a decision support model [Beheersing van campylobacter in de ippenvleesketen-naar een eslissingsondersteunend model]. , p. 56. No. 250911005. Bilthoven, The Netherlands, RIVM [Rijksinstituut voor Volksgezondheid en Milieu]. (also available at https:// www.rivm.nl/bibliotheek/rapporten/250911005.html). Bogen, K.T. 1994. A note on compounded conservatism. Risk Analysis, 14(4): 379–381. https://doi.org/10.1111/j.1539-6924.1994.tb00255.x de Boor, C. 2001. A practical guide to splines. New York, Springer. 348 pp. Brul, S., van Gerwen, S.J.C. & Zwietering, M.H., eds. 2007. Modelling microorganisms in food. Cambridge, England, Woodhead Publishing Limited. 294 pp. BIBLIOGRAPHY 233 Buchanan, R.L. 1993. Predictive food microbiology. Trends in Food Science & Technology, 4(1): 6–11. https://doi.org/10.1016/S0924-2244(05)80004-4 Buchanan, R.L., Damert, W.G., Whiting, R.C. & van Schothorst, M. 1997. Use of epidemiologic and food survey data to estimate a purposefully conservative doseresponse relationship for Listeria monocytogenes levels and incidence of listeriosis. Journal of Food Protection, 60(8): 918–922. https://doi.org/10.4315/0362028X-60.8.918 Buchanan, R.L., Gorris, L.G.M., Hayman, M.M., Jackson, T.C. & Whiting, R.C. 2017. A review of Listeria monocytogenes: An update on outbreaks, virulence, dose-response, ecology, and risk assessments. Food Control, 75: 1–13. https://doi. org/10.1016/j.foodcont.2016.12.016 Buchanan, R.L., Smith, J.L. & Long, W. 2000. Microbial risk assessment: dose-response relations and risk characterization. International Journal of Food Microbiology, 58(3): 159–172. https://doi.org/10.1016/S0168-1605(00)00270-1 Burgman, M.A. 2015. Trusting judgements: How to get the best out of experts. Cambridge, UK, Cambridge University Press. 215 pp. Burmaster, D.E. & Harris, R.H. 1993. The magnitude of compounding conservatisms in superfund risk assessments. Risk Analysis, 13(2): 131–134. https://doi. org/10.1111/j.1539-6924.1993.tb01058.x Busschaert, P., Geeraerd, A.H., Uyttendaele, M. & Van Impe, J.F. 2011. Hierarchical Bayesian analysis of censored microbiological contamination data for use in risk assessment and mitigation. Food Microbiology, 28(4): 712–719. CAC. 1969. CAC/RCP 1-1969: General Principles of Food Hygiene (2020 Revision). Codex Alimantarius Commission. CAC. 1999. CAC/GL 30-1999: Principles and guidelines for the gonduct of microbiological risk assessment (Amendments 2012, 2014). Codex Alimentarius Commission. (also available at http://www.fao.org/fao-who-codexalimentarius/home/en/). CAC. 2000. Report of the 33rd Session of the Codex Committee on Food Hygiene (CCFH). Washington, DC, USA, Codex Alimentarius Commission. (also available at http://www.fao.org/fao-who-codexalimentarius/meetings/archives/en/). CAC. 2008. CAC/GL 63-2007: Principles and guidelines for the conduct of microbiological rRisk management (MRM). Codex Alimentarius Commission. (also available at http://www.fao.org/fao-who-codexalimentarius/home/en/). CAC. 2019. Codex Alimentarius Commission Procedural Manual. 27th edition. Codex Alimantarius Commission. (also available at http://www.fao.org/fao-whocodexalimentarius/home/en/). 234 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Cadavez, V.A.P., Campagnollo, F.B., Silva, R.A., Duffner, C.M., Schaffner, D.W., Sant’Ana, A.S. & Gonzales-Barron, U. 2019. A comparison of dynamic tertiary and competition models for describing the fate of Listeria monocytogenes in Minas fresh cheese during refrigerated storage. Food Microbiology, 79: 48–60. https://doi. org/10.1016/j.fm.2018.11.004 Carbone, K.M., Luftig, R.B. & Buckley, M. 2005. Microbial triggers of chronic human Illness. American Academy of Microbiology. Cardoen, S., Van Huffel, X., Berkvens, D., Quoilin, S., Ducoffre, G., Saegerman, C., Speybroeck, N., Imberechts, H., Herman, L., Ducatelle, R. & Dierick, K. 2009. Evidence-based semiquantitative methodology for prioritization of foodborne zoonoses. Foodborne Pathogens and Disease, 6(9): 1083–1096. https://doi. org/10.1089/fpd.2009.0291 Cassin, M.H., Lammerding, A.M., Todd, E.C.D., Ross, W. & McColl, R.S. 1998. Quantitative risk assessment for Escherichia coli O157:H7 in ground beef hamburgers. International Journal of Food Microbiology, 41: 21–44. Cassin, M.H., Paoli, G.M. & Lammerding, A.M. 1998. Simulation modeling for microbial risk assessment. Journal of Food Protection, 61(11): 1560–1566. https:// doi.org/10.4315/0362-028X-61.11.1560 Cassin, M.H., Paoli, G.M., McColl, R.S. & Lammerding, A.M. 1996. A comment on ‘Hazard assessment of Listeria monocytogenes in the processing of bovine milk’. Journal of Food Protection, 59(4): 341–343. CB Premium. 2021. RiskRanger [Online]. Tasmania. [Cited 30 May 2021]. https://www. cbpremium.org CCFFP. 2011. Agenda Item 3 (Comment) - Thailand information paper: On estimating the risk of developing histamine poisoning from the consumption Thai fish sauces. No. FFP/31 CRD 18. Codex Alimentarius Commission. (also available at http:// www.fao.org/tempref/codex/Meetings/CCFFP/ccffp31/CRD/CRD_18_Thailand. pdf ). CDC. 2021a. Foodborne Diseases Active Surveillance Network (FoodNet) [Online]. [Cited 30 May 2021]. http://www.cdc.gov/foodnet/ CDC. 2021b. PulseNet [Online]. [Cited 30 May 2021]. http://www.cdc.gov/pulsenet/ CFIA. 2019. Food incident response process [online]. Last Modified: 2019-11-13. [Cited 25 August 2020]. https://www.inspection.gc.ca/food-safety-for-industry/compliancecontinuum/guidance-for-food-inspection-activities/incident-response/foodincident-response-process/eng/1544737953805/1544737954088 Chardon, J. & Swart, A. 2016. Food consumption and handling survey for quantitative microbiological consumer phase risk assessments. Journal of Food Protection, 79(7): 1221–1233. https://doi.org/10.4315/0362-028X.JFP-15-448 BIBLIOGRAPHY 235 Chen, Y., Jackson, K.M., Chea, F.P. & Schaffner, D.W. 2001. Quantification and variability analysis of bacterial cross-contamination rates in common food service tasks. Journal of Food Protection, 64(1): 72–80. https://doi.org/10.4315/0362028X-64.1.72 Chereau, F., Opatowski, L., Tourdjman, M. & Vong, S. 2017. Risk assessment for antibiotic resistance in South East Asia. BMJ: j3393. https://doi.org/10.1136/bmj. j3393 Christensen, B., Sommer, H., Rosenquist, H. & Nielsen, N. 2001. Risk assessment on Campylobacter jejuni in chicken products. Danish Veterinary and Food Administration. (also available at http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.118.7218). Cochran, W.G. 1950. Estimation of bacterial densities by means of the “most probable number”. Biometrics, 6(2): 105–116. Coleman, M. & Marks, H. 1998. Topics in dose-response modeling. Journal of Food Protection, 61(11): 1550–1559. https://doi.org/10.4315/0362-028X-61.11.1550 Coleman, M.E., Marks, H.M., Golden, N.J. & Latimer, H.K. 2004. Discerning strain effects in microbial dose-response data. Journal of Toxicology and Environmental Health, Part A, 67(8–10): 667–685. https://doi.org/10.1080/15287390490428134 Coleman, M.E., Sandberg, S. & Anderson, S.A. 2003. Impact of microbial ecology of meat and poultry products on predictions from exposure assessment scenarios for refrigerated storage. Risk Analysis, 23(1): 215–228. https://doi.org/10.1111/15396924.00301 ComBase. 2021a. ComBase [Online]. [Cited 30 May 2021]. http://www.combase.cc ComBase. 2021b. Tool [Online]. [Cited 30 May 2021]. https://www.combase.cc/index. php/en/8-category-en-gb/21-tools Cox Jr., L.A. 2008. What’s wrong with risk matrices? Risk Analysis, 28(2): 497–512. https://doi.org/10.1111/j.1539-6924.2008.01030.x Cox Jr., L.A., Babayev, D. & Huber, W. 2005. Some limitations of qualitative risk rating systems. Risk Analysis, 25(3): 651–662. https://doi.org/10.1111/j.15396924.2005.00615.x Cullen, A.C. 1994. Measures of compounding conservatism in probabilistic risk assessment. Risk Analysis, 14(4): 389–393. https://doi.org/10.1111/j.1539-6924.1994. tb00257.x Cullen, A.C. & Frey, H.C. 1999. Probabilistic techniques in exposure assessment: A handbook for dealing with variability and uncertainty in models and inputs. Springer US. (also available at http://www.springer.com/gp/book/9780306459566). Dalgaard, P., Ross, T., Kamperman, L., Neumeyer, K. & McMeekin, T.A. 1994. Estimation of bacterial growth rates from turbidimetric and viable count data. International Journal of Food Microbiology, 23(3): 391–404. https://doi. org/10.1016/0168-1605(94)90165-1 236 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD De Knegt, L.V., Pires, S.M. & Hald, T. 2015. Attributing foodborne salmonellosis in humans to animal reservoirs in the European Union using a multi-country stochastic model. Epidemiology & Infection, 143(6): 1175–1186. https://doi. org/10.1017/S0950268814001903 DeDonder, S., Jacob, C.J., Surgeoner, B.V., Chapman, B., Phebus, R. & Powell, D.A. 2009. Self‐reported and observed behavior of primary meal preparers and adolescents during preparation of frozen, uncooked, breaded chicken products. British Food Journal, 111(9): 915–929. https://doi.org/10.1108/00070700910992844 Dee, D.P. 1994. Guidelines for documenting the validity of computational modelling software. Delft, The Netherlands, IAHR/AIRH. 24 pp. Dee, D.P. 1995. A pragmatic approach to model validation. In D.R. Lynch & A.M. Davies, eds. Quantitative skill assessment for coastal ocean models, p. American Geophysical Union. Dias, L.C., Morton, A. & Quigley, J., eds. 2018. Elicitation: The science and art of structuring judgement. Springer International Publishing. (also available at http:// link.springer.com/10.1007/978-3-319-65052-4). Digital Library of Mathematical Functions. 2021. NIST Digital Library of Mathematical Functions [Online]. [Cited 30 May 2021]. https://dlmf.nist.gov Dogan, O.B., Clarke, J., Mattos, F. & Wang, B. 2019. A quantitative microbial risk assessment model of Campylobacter in broiler chickens: Evaluating processing interventions. Food Control, 100: 97–110. https://doi.org/10.1016/j. foodcont.2019.01.003 Dolan, K.D. & Mishra, D.K. 2013. Parameter estimation in food science. Annual Review of Food Science and Technology, 4(1): 401–422. https://doi.org/10.1146/annurevfood-022811-101247 Dorny, P. & Praet, N. 2007. Taenia saginata in Europe. Veterinary Parasitology, 149(1–2): 22–24. https://doi.org/10.1016/j.vetpar.2007.07.004 Doucet, P. & Sloep, P.B. 1992. Mathematical modeling in the life sciences. Ellis Horwood Series in Mathematics and Its Applications. New York, Ellis Horwood Ltd. 450 pp. Duncan, G.E. 2014. Determining the health benefits of poultry industry compliance measures: the case of campylobacteriosis regulation in New Zealand. The New Zealand Medical Journal, 127(1391): 22–37. Dziak, J.J., Coffman, D.L., Lanza, S.T., Li, R. & Jermiin, L.S. 2020. Sensitivity and specificity of information criteria. Briefings in Bioinformatics, 21(2): 553–565. https://doi.org/10.1093/bib/bbz016 Ebel, E.D. & Williams, M.S. 2019. Assessing the effectiveness of revised performance standards for Salmonella contamination of comminuted poultry. Microbial Risk Analysis: 100076. https://doi.org/10.1016/j.mran.2019.05.002 BIBLIOGRAPHY 237 Ebel, E.D., Williams, M.S., Golden, N.J. & Marks, H.M. 2012. Simplified framework for predicting changes in public health from performance standards applied in slaughter establishments. Food Control, 28(2): 250–257. https://doi.org/10.1016/j. foodcont.2012.05.016 EcoSure. 2008. 2007 U.S. cold temperature evaluation: Design and summary pages. EcoSure. (also available at http://foodrisk.org/exclusives/EcoSure/). EFSA. 2004a. Statement on BSE/TSE and the health risks of the consumption of milk and milk derived products from goats by the Scientific Panel on Biological Hazards (BIOHAZ). EFSA Journal, 2(11). https://doi.org/10.2903/j.efsa.2004.136 EFSA. 2004b. EFSA publishes geographical BSE-Risk (GBR) assessments for Australia, Canada, Mexico, Norway, South Africa, Sweden and the United States of America. In: European Food Safety Authority [online]. [Cited 23 July 2020]. https://www.efsa. europa.eu/en/press/news/040820 EFSA. 2012a. Scientific opinion on risk assessment terminology. EFSA Journal, 10(5): 2664. https://doi.org/10.2903/j.efsa.2012.2664 EFSA. 2012b. Scientific opinion on the development of a risk ranking framework on biological hazards. EFSA Journal, 10(6): 2724. https://doi.org/10.2903/j. efsa.2012.2724 EFSA. 2014a. Guidance on expert knowledge elicitation in food and feed safety risk assessment: guidance on expert knowledge elicitation. EFSA Journal, 12(6): 3734. https://doi.org/10.2903/j.efsa.2014.3734 EFSA. 2014b. EFSA Panel on Biological Hazards (BIOHAZ) scientific opinion on the public health risks of table eggs due to deterioration and development of pathogens. EFSA Journal, 12(7): 3782. https://doi.org/10.2903/j.efsa.2014.3782 EFSA. 2018a. Guidance on uncertainty analysis in scientific assessments. EFSA Journal, 16(1): e05123. https://doi.org/10.2903/j.efsa.2018.5123 EFSA. 2018b. The principles and methods behind EFSA’s guidance on uncertainty analysis in scientific assessment. EFSA Journal, 16(1): e05122. https://doi.org/10.2903/j. efsa.2018.5122 EFSA. 2021. EFSA provides preliminary advice on health risks of goat milk and derived products with regards to BSE/TSE [Online]. [Cited 30 May 2021]. https://www. efsa.europa.eu/en/news/efsa-provides-preliminary-advice-health-risks-goat-milkand-derived-products Egorov, A.I., Griffin, S.M., Ward, H.D., Reilly, K., Fout, G.S. & Wade, T.J. 2018. Application of a salivary immunoassay in a prospective community study of waterborne infections. Water Research, 142: 289–300. https://doi.org/10.1016/j. watres.2018.05.030 Eschenbach, T.G. 1992. Spiderplots versus tornado diagrams for sensitivity analysis. Interfaces, 22(6): 40–46. https://doi.org/10.1287/inte.22.6.40 238 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD European Commission. 2021. RASFF - Food and Feed Safety Alerts [Online]. [Cited 30 May 2021]. https://ec.europa.eu/food/safety/rasff_en Evers, E.G., Post, J., Putirulan, F.F. & Wal, F.J. van der. 2010. Detection probability of Campylobacter. Food Control, 21(3): 247–252. https://doi.org/10.1016/j. foodcont.2009.06.004 FAO. 2001. Joint FAO/WHO expert consultation on risk Assessment of microbiological hazards in foods: Risk characterization of Salmonella spp. in eggs and broiler chickens and Listeria monocytogenes in ready-to-eat foods. Food and Nutrition Paper No. 72. Rome, Italy, FAO. (also available at http://www.fao.org/docrep/008/ y1332e/y1332e00.htm). FAO. 2017. Food Safety Risk Management: Evidence-informed policies and decisions, considering multiple factors. Food Safety and Quality No. 4. Rome, Italy, FAO. (also available at http://www.fao.org/3/i8240en/I8240EN.pdf). FAO. 2021a. Joint FAO/WHO Expert meetings on microbiological risk assessment (JEMRA) [Online]. Rome. [Cited 30 May 2021]. http://www.fao.org/food-safety/ scientific-advice/microbiological-risks-and-jemra/en/ FAO. 2021b. Joint FAO/WHO Expert Committee on Food Additives (JECFA) [Online]. Rome. [Cited 30 May 2021]. http://www.fao.org/food-safety/scientific-advice/ jecfa/en/ FAO. 2021c. FAOSTAT [Online]. Rome. [Cited 30 May 2021]. http://www.fao.org/ faostat/en/#home FAO & WHO. 2002a. Risk assessments of Salmonella in eggs and broiler chickens. Microbiological Risk Assessment Series No. 2. Rome, Italy, FAO and WHO. (also available at http://www.who.int/foodsafety/publications/salmonella/en/). FAO & WHO. 2002b. Principles and guidelines for incorporating microbiological risk assessment in the development of food safety standards, guidelines and related texts. Report of a Joint FAO/WHO Consultation. Kiel, Germany, FAO and WHO. (also available at http://www.fao.org/3/a-y4302e.pdf). FAO & WHO. 2003. Hazard characterization for pathogens in food and water. , p. 76. Microbiological Risk Assessment Series No. 3. Rome, Italy, FAO and WHO. (also available at http://www.who.int/foodsafety/publications/mra_3/en/). FAO & WHO. 2004. Risk assessment of Listeria monocytogenes in ready-to-eat foods. Microbiological Risk Assessment Series No. 5. Rome, Italy, FAO and WHO. (also available at http://www.who.int/foodsafety/publications/mra_listeria/en/). FAO & WHO. 2005. Risk assessment of Vibrio vulnificus in raw oysters. Microbiological Risk Assessment Series No. 8. Rome, Italy, FAO and WHO. (also available at http:// www.who.int/foodsafety/publications/mra8/en/). FAO & WHO. 2006a. Food safety risk analysis: a guide for national food safety authorities. FAO food and nutrition paper No. 87. Rome, FAO and WHO. 102 pp. (also available at http://www.fao.org/3/a0822e/a0822e.pdf). BIBLIOGRAPHY 239 FAO & WHO. 2006b. Enterobacter sakazakii and Salmonella in powdered infant formula. Microbiological Risk Assessment Series No. 10. Rome, Italy, FAO and WHO. (also available at https://www.who.int/foodsafety/publications/mra10/en/). FAO & WHO. 2006c. The use of microbiological risk assessment outputs to develop practical risk management strategies: Metrics to improve food safety. Kiel, Germany, FAO and WHO. FAO & WHO. 2007. Enterobacter sakazakii and other microorganisms in powdered infant formula. Microbiological Risk Assessment Series No. 6. Rome, Italy, FAO and WHO. (also available at https://www.who.int/foodsafety/publications/mra6enterobacter-sakazakii/en/). FAO & WHO. 2008. Exposure assessment of microbiological hazards in food. Microbiological Risk Assessment Series No. 7. Rome, Italy, FAO and WHO. (also available at http://www.who.int/foodsafety/publications/mra7/en/). FAO & WHO. 2009a. Risk characterization of microbiological hazards in food. Microbiological Risk Assessment Series No. 17. Rome, Italy, FAO and WHO. (also available at www.who.int/foodsafety/publications/risk-characterization/en/). FAO & WHO. 2009b. Salmonella and Campylobacter in chicken meat: Meeting report. Microbiological Risk Assessment Series No. 19. Rome, Italy, FAO and WHO. (also available at http://www.who.int/foodsafety/publications/mra19/en/). FAO & WHO. 2009c. Risk assessment of Campylobacter spp. in broiler chickens: Interpretative summary. Microbiological Risk Assessment Series No. 11. Rome, Italy, FAO and WHO. (also available at http://www.fao.org/fileadmin/templates/ agns/pdf/jemra/MRA_11.pdf ). FAO & WHO. 2009d. Risk assessment of Campylobacter spp. in broiler chickens. Microbiological Risk Assessment Series No. 12. Rome, Italy, FAO and WHO. (also available at http://www.fao.org/fileadmin/templates/agns/pdf/jemra/MRA_12. pdf ). FAO & WHO. 2009e. Principles and methods for the risk assessment of chemicals in food. Environmental health criteria No. 240. Geneva, Switzerland, WHO. (also available at https://www.who.int/publications-detail/principles-and-methods-for-the-riskassessment-of-chemicals-in-food). FAO & WHO. 2011a. Risk assessment of Vibrio parahaemolyticus in seafood. Microbiological Risk Assessment Series No. 16. Rome, Italy, FAO and WHO. (also available at https://www.who.int/foodsafety/publications/mra_16/en/). FAO & WHO. 2011b. Enterohaemorrhagic Escherichia coli in raw beef and beef products: approaches for the provision of scientific advice meeting report. FAO and WHO. FAO & WHO. 2014. Multicriteria-based ranking for risk management of food-borne parasites. Microbiological Risk Assessment Series No. 23. Rome, Italy, FAO and WHO. (also available at https://www.who.int/foodsafety/publications/mra_23/en/). 240 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD FAO & WHO. 2016. Risk communication applied to food safety: Handbook. Food safety and quality series No. 2. Rome, Italy, FAO and WHO. (also available at http://www. who.int/foodsafety/Risk-Communication/en/). FAO & WHO. 2018a. Shiga toxin-producing Escherichia coli (STEC) and food: attribution, characterization, and monitoring. Microbiological Risk Assessment Series No. 31. Rome, Italy, FAO and WHO. (also available at https://www.who.int/ foodsafety/publications/mra_31/en/). FAO & WHO. 2018b. FAO/WHO framework for the provision of scientific advice on food safety and nutrition. Rome, Italy, FAO and WHO. (also available at http://www.fao. org/3/i7494en/I7494EN.pdf ). FAO & WHO. 2021. Food safety risk analysis tools [Online]. Rome. [Cited 30 May 2021]. http://www.fstools.org Fazil, A., Lowman, R., Stern, N. & Lammerding, A.M. 1999. Quantitative risk assessment model for Campylobacter jejuni in chicken. p. 65. Paper presented at 10th Int. Workshop on CHRO (Campylobacter, Helicobacter and Related Organisms), 1999, Baltimore, MD, USA. Fazil, A.M. 2005. A primer on risk assessment modelling: focus on seafood products. Fisheries Technical Paper No. 462. Rome, FAO. FDA. 2021. FDA-iRISK [Online]. Maryland. [Cited 30 May 2021]. https://irisk.foodrisk. org Frank, C., Milde-Busch, A. & Werber, D. 2014. Results of surveillance for infections with Shiga toxin-producing Escherichia coli (STEC) of serotype O104:H4 after the large outbreak in Germany, July to December 2011. Eurosurveillance, 19(14). (also available at https://www.eurosurveillance.org/content/10.2807/1560-7917. ES2014.19.14.20760). Frey, H.C., Mokhtari, A. & Danish, T. 2003. Evaluation of selected sensitivity analysis methods based upon applications to two food safety process risk models. Washington, DC, USA, Office of Risk Assessment and Cost-Benefit Analysis, U.S. Department of Agriculture. (also available at https://www.ccee.ncsu.edu/wpcontent/uploads/2015/08/risk-phase-2-final.pdf ). Frey, H.C., Mokhtari, A. & Zheng, J. 2004. Recommended practice regarding selection, application, and interpretation of sensitivity analysis methods applied to food safety process risk models. Washington, DC, USA, Office of Risk Assessment and Cost-Benefit Analysis, U.S. Department of Agriculture. (also available at https:// www.ccee.ncsu.edu/wp-content/uploads/2015/08/risk-phase-3-final.pdf ). Frey, H.C. & Patil, S.R. 2002. Identification and review of sensitivity analysis methods. Risk Analysis, 22(3): 553–578. https://doi.org/10.1111/0272-4332.00039 FRISBEE project. 2020. FRISBEE project [Online]. [Cited 27 August 2020]. https:// frisbee-project.eu/ BIBLIOGRAPHY 241 FSIS. 2001. Risk assessment of the public health impact of Escherichia coli O157:H7 in ground beef. U.S. Department of Agriculture, Food Safety and Inspection Service. (also available at https://www.fsis.usda.gov/wps/wcm/connect/1db13d79-1cd94e4d-b6ca-16ad89a085a1/00-023NReport.pdf ?MOD=AJPERES). FSIS. 2002. Comparative risk assessment for intact (non-tenderized) and non-intact (tenderized) beef: Technical report. Washington, DC, USA, U.S. Department of Agriculture, Food Safety and Inspection Service. (also available at https://www.fsis. usda.gov/shared/PDF/Beef_Risk_Assess_Report_Mar2002.pdf ). FSIS. 2005. Risk assessments of Salmonella Enteritidis in shell eggs and Salmonella spp. in egg products. U.S. Department of Agriculture, Food Safety and Inspection Service. (also available at https://www.fsis.usda.gov/wps/wcm/connect/16abcb5f-21fc4731-9346-fd8401582ba4/SE_Risk_Assess_Oct2005.pdf ?MOD=AJPERES). Funk, J.A., Davies, P.R. & Nichols, M.A. 2000. The effect of fecal sample weight on detection of Salmonella enterica in swine feces. Journal of Veterinary Diagnostics, 12: 412–418. Gallagher, D., Pouillot, R., Hoelzer, K., Tang, J., Dennis, S.B. & Kause, J.R. 2016. Listeria monocytogenes in retail delicatessens: An interagency risk assessment— Risk mitigations. Journal of Food Protection, 79(7): 1076–1088. https://doi. org/10.4315/0362-028X.JFP-15-336 Gallagher, E., Ryan, J., Kelly, L., Leforban, Y. & Wooldridge, M. 2002. Estimating the risk of importation of foot-and-mouth disease into Europe. Veterinary Record, 150: 769–772. https://doi.org/10.1136/vr.150.25.769 Gelman, A. & Carlin, J. 2014. Beyond power calculations: Assessing Type S (Sign) and Type M (Magnitude) errors. Perspectives on Psychological Science, 9(6): 641–651. https://doi.org/10.1177/1745691614551642 Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A. & Rubin, D.B. 2013. Bayesian data analysis. Third edition. Boca Raton, USA, Chapman and Hall/CRC. 675 pp. van Gerwen, S.J.C., te Giffel, M.C., van’t Riet, K., Beumer, R.R. & Zwietering, M.H. 2000. Stepwise quantitative risk assessment as a tool for characterization of microbiological food safety. Journal of Applied Microbiology, 88(6): 938–951. https://doi.org/10.1046/j.1365-2672.2000.01059.x van Gerwen, S.J.C. & Gorris, L.G.M. 2004. Application of elements of microbiological risk assessment in the food industry via a tiered approach. Journal of Food Protection, 67(9): 2033–2040. https://doi.org/10.4315/0362-028X-67.9.2033 Giménez, B. & Dalgaard, P. 2004. Modelling and predicting the simultaneous growth of Listeria monocytogenes and spoilage micro-organisms in cold-smoked salmon. Journal of Applied Microbiology, 96(1): 96–109. https://doi.org/10.1046/j.13652672.2003.02137.x 242 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Gkogka, E., Reij, M.W., Gorris, L.G.M. & Zwietering, M.H. 2013a. The application of the Appropriate Level of Protection (ALOP) and Food Safety Objective (FSO) concepts in food safety management, using Listeria monocytogenes in deli meats as a case study. Food Control, 29(2): 382–393. https://doi.org/10.1016/j. foodcont.2012.04.020 Gkogka, E., Reij, M.W., Gorris, L.G.M. & Zwietering, M.H. 2013b. Risk assessment strategies as a tool in the application of the Appropriate Level of Protection (ALOP) and Food Safety Objective (FSO) by risk managers. International Journal of Food Microbiology, 167(1): 8–28. https://doi.org/10.1016/j.ijfoodmicro.2013.04.013 Gold, M.R., Stevenson, D. & Fryback, D.G. 2002. HALYs and QALYs and DALYs, Oh my: Similarities and differences in summary measures of population health. Annual Review of Public Health, 23(1): 115–134. https://doi.org/10.1146/annurev. publhealth.23.100901.140513 Gonzales-Barron, U., Gonçalves-Tenório, A., Rodrigues, V. & Cadavez, V. 2017. Foodborne pathogens in raw milk and cheese of sheep and goat origin: a metaanalysis approach. Current Opinion in Food Science, 18: 7–13. https://doi. org/10.1016/j.cofs.2017.10.002 Gonzales-Barron, U., Kerr, M., Sheridan, J.J. & Butler, F. 2010. Count data distributions and their zero-modified equivalents as a framework for modelling microbial data with a relatively high occurrence of zero counts. International Journal of Food Microbiology, 136(3): 268–277. Gonzales-Barron, U., Piza, L., Xavier, C., Costa, E. & Cadavez, V. 2016. An exposure assessment model of the prevalence of Salmonella spp. along the processing stages of Brazilian beef. Food Science and Technology International, 22(1): 10–20. https:// doi.org/10.1177/1082013214560446 Greig, J.D. & Ravel, A. 2009. Analysis of foodborne outbreak data reported internationally for source attribution. International Journal of Food Microbiology, 130(2): 77–87. https://doi.org/10.1016/j.ijfoodmicro.2008.12.031 Greppi, A. & Rantsiou, K. 2016. Methodological advancements in foodborne pathogen determination: from presence to behavior. Current Opinion in Food Science, 8: 80– 88. https://doi.org/10.1016/j.cofs.2016.04.005 Guo, C., Hoekstra, R.M., Schroeder, C.M., Pires, S.M., Ong, K.L., Hartnett, E., Naugle, A., Harman, J., Bennett, P., Cieslak, P., Scallan, E., Rose, B., Holt, K.G., Kissler, B., Mbandi, E., Roodsari, R., Angulo, F.J. & Cole, D. 2011. Application of Bayesian techniques to model the burden of human salmonellosis attributable to U.S. food commodities at the point of processing: Adaptation of a Danish model. Foodborne Pathogens and Disease, 8(4): 509–516. https://doi.org/10.1089/fpd.2010.0714 Guo, M., Buchanan, R.L., Dubey, J.P., Hill, D.E., Lambertini, E., Ying, Y., Gamble, H.R., Jones, J.L. & Pradhan, A.K. 2015. Qualitative assessment for Toxoplasma gondii exposure risk associated with meat products in the United States. Journal of Food Protection, 78(12): 2207–2219. https://doi.org/10.4315/0362-028X.JFP-15-270 BIBLIOGRAPHY 243 Gurman, P.M., Ross, T., Holds, G.L., Jarrett, R.G. & Kiermeier, A. 2016. Thermal inactivation of Salmonella spp. in pork burger patties. International Journal of Food Microbiology, 219: 12–21. https://doi.org/10.1016/j.ijfoodmicro.2015.11.014 Gustavsson, J., Cederberg, C. & Sonesson, U. 2011. Global food losses and food waste. Paper presented at Save Food Congress, 16 May 2011, Duesseldorf, Germany. [Cited 31 March 2020]. https://www.researchgate.net/publication/267919405_ Global_Food_Losses_and_Food_Waste Haas, C.N. 1983. Estimation of risk due to low doses of microorganisms: A comparison of alternative methodolodies. American Journal of Epidemiology, 118(4): 573–582. Haas, C.N. 2002. Conditional dose-response relationships for microorganisms: Development and application. Risk Analysis, 22(3): 455–463. https://doi. org/10.1111/0272-4332.00035 Haas, C.N. 2015. Microbial dose response modeling: Past, present, and future. Environmental Science & Technology, 49(3): 1245–1259. https://doi.org/10.1021/ es504422q Haas, C.N., Rose, J.B. & Gerba, C.P. 2014. Quantitative microbial risk assessment. Second edition. Hoboken, USA, Wiley. 440 pp. (also available at https://www.wiley.com/en-au/ Quantitative+Microbial+Risk+Assessment%2C+2nd+Edition-p-9781118910023). Haddad, N., Johnson, N., Kathariou, S., Métris, A., Phister, T., Pielaat, A., Tassou, C., Wells-Bennik, M.H.J. & Zwietering, M.H. 2018. Next generation microbiological risk assessment—Potential of omics data for hazard characterisation. International Journal of Food Microbiology, 287: 28–39. https://doi.org/10.1016/j. ijfoodmicro.2018.04.015 Hald, T., Vose, D., Wegener, H.C. & Koupeev, T. 2004. A Bayesian approach to quantify the contribution of animal-food sources to human salmonellosis. Risk Analysis, 24(1): 255–269. https://doi.org/10.1111/j.0272-4332.2004.00427.x Hald, T., Wingstrand, A., Brøndsted, T. & Wong, D.M.A.L.F. 2006. Human health impact of Salmonella contamination in imported soybean products: A semiquantitative risk assessment. Foodborne Pathogens and Disease, 3(4): 422–431. https://doi.org/10.1089/fpd.2006.3.422 Hall, G., Keflemariam, Y., Raupach, J., Becker, N. & Kirk, M. 2008. Estimating community incidence of Salmonella, Campylobacter, and Shiga toxin-producing Escherichia coli infections, Australia. Emerging Infectious Diseases, 14(10): 1601– 1609. https://doi.org/10.3201/eid1410.071042 Hansen, L.P. 1982. Large sample properties of generalized method of moments estimators. Econometrica, 50(4): 1029–1054. https://doi.org/10.2307/1912775 Hartnett, E. 2002. Campylobacter in broilers: a quantitative risk assessment approach. Glasgow, United Kingdom, University of Strathclyde. (PhD Thesis) 244 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Hartnett, E., Kelly, L., Newell, D., Wooldridge, M. & Gettinby, G. 2001. A quantitative risk assessment for the occurrence of Campylobacter in chickens at the point of slaughter. Epidemiology & Infection, 127(2): 195–206. https://doi.org/10.1017/ S0950268801005866 Havelaar, A.H. & Swart, A.N. 2014. Impact of acquired immunity and dose-dependent probability of illness on quantitative microbial risk assessment. Risk Analysis, 34(10): 1807–1819. https://doi.org/10.1111/risa.12214 Havelaar, A.H., de Wit, M.A.S., van Koningsveld, R. & van Kempen, E. 2000. Health burden in the Netherlands due to infection with thermophilic Campylobacter spp. Epidemiology & Infection, 125(3): 505–522. Hayama, Y., Yamamoto, T., Kasuga, F. & Tsutsui, T. 2011. Simulation model for Campylobacter cross-contamination during poultry processing at slaughterhouses. Zoonoses and Public Health, 58(6): 399–406. https://doi.org/10.1111/j.18632378.2010.01385.x Health Evidence. 2021. Health Evidence [Online]. Ontario. [Cited 30 May 2021]. https:// www.healthevidence.org Hemming, V., Burgman, M.A., Hanea, A.M., McBride, M.F. & Wintle, B.C. 2018. A practical guide to structured expert elicitation using the IDEA protocol. Methods in Ecology and Evolution, 9(1): 169–180. https://doi.org/10.1111/2041-210X.12857 Holcomb, D.L., Smith, M.A., Ware, G.O., Hung, Y.-C., Brackett, R.E. & Doyle, M.P. 1999. Comparison of six dose-response models for use with food-borne pathogens. Risk Analysis, 19(6): 1091–1100. Hsi, D.J., Ebel, E.D., Williams, M.S., Golden, N.J. & Schlosser, W.D. 2015. Comparing foodborne illness risks among meat commodities in the United States. Food Control, 54: 353–359. https://doi.org/10.1016/j.foodcont.2015.02.018 Huang, L. 2017. IPMP Global Fit – A one-step direct data analysis tool for predictive microbiology. International Journal of Food Microbiology, 262: 38–48. https://doi. org/10.1016/j.ijfoodmicro.2017.09.010 ICMSF. 1996. Microorganisms in foods 5: Characteristics of microbial pathogens. Kluwer Academic / Plenum Publishers. ILSI. 2000. Revised framework for microbial risk assessment. Washington, DC, International Life Science Institute Press. (also available at https://archive.epa.gov/ raf/web/pdf/ilsi-ra-framework.pdf ). Institute of Medicine. 2002. Escherichia coli O157:H7 in ground beef: Review of a draft risk assessment. Washington, DC, USA, The National Academies Press. (also available at https://www.nap.edu/catalog/10528/escherichia-coli-o157h7-inground-beef-review-of-a-draft). BIBLIOGRAPHY 245 IPCC. 2001. Good practice guidance and uncertainty management in national greenhouse gas inventories. Intergovernmental Panel on Climate Change. (also available at https://www.ipcc-nggip.iges.or.jp/public/gp/english/gpgaum_en.html). Jameson, J.E. 1962. A discussion of the dynamics of Salmonella enrichment. Epidemiology & Infection, 60(2): 193–207. https://doi.org/10.1017/S0022172400039462 Jaykus, L.A. 1996. The application of quantitative risk assessment to microbial food safety risks. Critical Reviews in Microbiology, 22(4): 279–293. https://doi. org/10.3109/10408419609105483 Jenkinson, D. 2005. The elicitation of probabilities: A review of the statistical literature. University of Sheffield. Jensen, D.A., Friedrich, L.M., Harris, L.J., Danyluk, M.D. & Schaffner, D.W. 2015. Cross contamination of Escherichia coli O157:H7 between lettuce and wash water during home-scale washing. Food Microbiology, 46: 428–433. https://doi. org/10.1016/j.fm.2014.08.025 Jewell, K. 2012. Comparison of 1-step and 2-step methods of fitting microbiological models. International Journal of Food Microbiology, 160(2): 145–161. Johnson, N.L., Kemp, A.W. & Kotz, S. 2008. Continuous univariate distributions. Third edition. Wiley. Jongenburger, I., Bassett, J., Jackson, T., Zwietering, M.H. & Jewell, K. 2012. Impact of microbial distributions on food safety I. Factors influencing microbial distributions and modelling aspects. Food Control, 26(2): 601–609. Jongenburger, I., Reij, M.W., Boer, E.P.J., Gorris, L.G.M. & Zwietering, M.H. 2011. Actual distribution of Cronobacterspp. in industrial batches of powdered infant formula and consequences for performance of sampling strategies. International Journal of Food Microbiology, 151(1): 62–69. Kaplan, S. & Garrick, B.J. 1981. On the quantitative definition of risk. Risk Analysis, 1(1): 11–27. https://doi.org/10.1111/j.1539-6924.1981.tb01350.x Kass, R.E. & Raftery, A.E. 1995. Bayes factors. Journal of the American Statistical Association, 90(430): 773–795. https://doi.org/10.1080/01621459.1995.10476572 Kasuga, F., Hirota, M., Wada, M., Yunokawa, T., Toyofuku, H., Shibatsuji, M., Michino, H., Kuwasaki, T., Yamamoto, S. & Kumagai, S. 2004. Archiving of food samples from restaurants and caterers—Quantitative profiling of outbreaks of foodborne salmonellosis in Japan. Journal of Food Protection, 67(9): 2024–2032. https://doi.org/10.4315/0362-028X-67.9.2024 Kiermeier, A., Jenson, I. & Sumner, J. 2015. Risk assessment of Escherichia coli O157 illness from consumption of hamburgers in the United States made from Australian manufacturing beef. Risk Analysis, 35(1): 77–89. https://doi.org/10.1111/risa.12248 Kiermeier, A., Mellor, G., Barlow, R. & Jenson, I. 2011. Assumptions of acceptance sampling and the implications for lot contamination: E. coli O157 in lots of Australian manufacturing beef. Journal of Food Protection, 74(4): 539–544. 246 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Kim, S.A., Yun, S.J., Lee, S.H., Hwang, I.G. & Rhee, M.S. 2013. Temperature increase of foods in car trunk and the potential hazard for microbial growth. Food Control, 29: 66–70. King, N., Lake, R. & Cressey, P. 2011. Risk profile: Salmonella (non-typhoid) in poultry (whole and pieces). No. FW11044. Environmental Sciences and Research Limited. (also available at https://www.mpi.govt.nz/food-safety/food-safety-and-suitabilityresearch/food-risk-assessment/food-risk-profiles/). Knuth, D.E. 1992. Literate programming. CSLI lecture notes No. no. 27. Stanford, Calif., Center for the Study of Language and Information. 368 pp. Kosa, K.M., Cates, S.C., Bradley, S., Chambers, E. & Godwin, S. 2015. Consumerreported handling of raw poultry products at home: Results from a national survey. Journal of Food Protection, 78(1): 180–186. https://doi.org/10.4315/0362-028X.JFP14-231 Koutsoumanis, K.P., Lianou, A. & Gougouli, M. 2016. Latest developments in foodborne pathogens modeling. Current Opinion in Food Science, 8: 89–98. https:// doi.org/10.1016/j.cofs.2016.04.006 Krapf, T. & Gantenbein-Demarchi, C. 2010. Thermal inactivation of Salmonella spp. during conching. LWT - Food Science and Technology, 43(4): 720–723. https://doi. org/10.1016/j.lwt.2009.10.009 Kruschke, J. 2014. Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan. Second edition. Boston, USA, Academic Press. 776 pp. Lake, R. & Cressey, P. 2013. Risk profile: Campylobacter jejuni/coli in poultry (whole and pieces). No. FW11065. Environmental Sciences and Research Limited. (also available at the Ministry for Primary Industries website: https://www.mpi.govt.nz/ food-safety/food-safety-and-suitability-research/food-risk-assessment/food-riskprofiles/). Lake, R., Gilbert, S., Wong, T.L. & Cressey, P. 2009. Risk profile: Mycobacterium bovis (non-typhoid) in milk. No. FW08031. Environmental Sciences and Research Limited. (also available at the Ministry for Primary Industries website: https:// www.mpi.govt.nz/food-safety/food-safety-and-suitability-research/food-riskassessment/food-risk-profiles/). Lake, R., Hudson, A., Cressey, P. & Gilbert, S. 2005. Risk profile: Listeria Monocytogenes in ready-to-eat salads. No. FW0446. Environmental Sciences and Research Limited. (also available at the Ministry for Primary Industries website: https://www.mpi. govt.nz/dmsdocument/25850-Risk-Profile-Listeria-monocytogenes-in-ready-toeat-salads). Lake, R.J., Cressey, P.J., Campbell, D.M. & Oakley, E. 2010. Risk ranking for foodborne microbial hazards in New Zealand: Burden of disease estimates. Risk Analysis, 30(5): 743–752. https://doi.org/10.1111/j.1539-6924.2009.01269.x BIBLIOGRAPHY 247 Lamboni, M., Sanaa, M. & Tenenhaus-Aziza, F. 2014. Sensitivity analysis for Critical Control Points determination and uncertainty analysis to link FSO and process criteria: Application to Listeria monocytogenes in soft cheese made from pasteurized milk. Risk Analysis, 34(4): 751–764. https://doi.org/10.1111/risa.12134 Lammerding, A.M. & Fazil, A. 2000. Hazard identification and exposure assessment for microbial food safety risk assessment. International Journal of Food Microbiology, 58(3): 147–157. https://doi.org/10.1016/S0168-1605(00)00269-5 Law, A.M. 2014. Simulation modeling and analysis. Fifth edition. Dubuque, USA, McGraw-Hill Education. 800 pp. Lawson, L.G., Jensen, J.D., Christiansen, P. & Lund, M. 2009. Cost-effectiveness of Salmonella reduction in Danish abattoirs. International Journal of Food Microbiology, 134(1–2): 126–132. https://doi.org/10.1016/j.ijfoodmicro.2009.03.024 Laxminarayan, R. & Macauley, M.K., eds. 2012. The value of information. Dordrecht, The Netherlands, Springer Netherlands. (also available at http://link.springer. com/10.1007/978-94-007-4839-2). Le Marc, Y., Valík, L. & Medveďová, A. 2009. Modelling the effect of the starter culture on the growth of Staphylococcus aureus in milk. International Journal of Food Microbiology, 129(3): 306–311. https://doi.org/10.1016/j.ijfoodmicro.2008.12.015 Lefkowitz, A., Fout, G.S., Losonsky, G., Wasserman, S.S., Israel, E. & Morris Jr., J.G. 1992. A serosurvey of pathogens associated with shellfish: Prevalence of antibodies to Vibrio species and Norwalk virus in the Chesapeake Bay region. American Journal of Epidemiology, 135(4): 369–380. https://doi.org/10.1093/oxfordjournals. aje.a116298 van Leusden, F.M. 2000. Hazard identification and characterisation, and dose reponse assessment of spore forming pathogsn in cooked chilled food containing vegetables. No. 149106 008. Bilthoven, The Netherlands, RIVM [Rijksinstituut voor Volksgezondheid en Milieu]. (also available at https://www.rivm.nl/bibliotheek/ rapporten/149106008.pdf ). Lever, J., Krzywinski, M. & Altman, N. 2016. Model selection and overfitting. Nature Methods, 13(9): 703–704. https://doi.org/10.1038/nmeth.3968 Levine, E.S. 2012. Improving risk matrices: the advantages of logarithmically scaled axes. Journal of Risk Research, 15(2): 209–222. https://doi.org/10.1080/13669877.2011.6 34514 Linstone, H.A. & Turoff, M., eds. 2002. The Delphi method: Techniques and application. Second edition. (also available at https://web.njit.edu/~turoff/pubs/delphibook/ index.html). Lorimer, M.F. & Kiermeier, A. 2007. Analysing microbiological data: Tobit or not Tobit? International Journal of Food Microbiology, 116(3): 313–318. 248 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Mangen, M.-J.J., Batz, M.B., Käsbohrer, A., Hald, T., Morris, J.G., Taylor, M. & Havelaar, A.H. 2010. Integrated approaches for the public health prioritization of foodborne and zoonotic pathogens. Risk Analysis, 30(5): 782–797. https://doi. org/10.1111/j.1539-6924.2009.01291.x Mangen, M.-J.J., Bouwknegt, M., Friesema, I.H.M., Haagsma, J.A., Kortbeek, L.M., Tariq, L., Wilson, M., van Pelt, W. & Havelaar, A.H. 2015. Cost-ofillness and disease burden of food-related pathogens in The Netherlands, 2011. International Journal of Food Microbiology, 196: 84–93. https://doi.org/10.1016/j. ijfoodmicro.2014.11.022 Mariner, J.C. & Paskin, R. 2000. Manual on participatory epidemiology: methods for the collection of action-oriented epidemiological intelligence. FAO Animal Health Manual. Rome, Italy, FAO. 86 pp. (also available at http://www.fao.org/3/X8833E/ X8833E00.htm). Marks, H.M., Coleman, M.E., Lin, C.-T.J. & Roberts, T. 1998. Topics in microbial risk assessment: dynamic flow tree process. Risk Analysis, 18(3): 309–328. https://doi. org/10.1111/j.1539-6924.1998.tb01298.x Merriam-Webster. 2021. Model [Online]. [Cited 30 May 2021]. https://www.merriamwebster.com/dictionary/model McCullagh, P. & Nelder, J.A. 1999. Generalized linear models. Second edition. Boca Raton, USA, Chapman and Hall. 532 pp. (also available at https://doi. org/10.1201/9780203753736). McKellar, R.J. & Lu, X., eds. 2003. Modeling microbial responses in food. Contemporary Food Science. CRC Press. (also available at https://www.crcpress.com/ModelingMicrobial-Responses-in-Food/McKellar-Lu/p/book/9780849312373). McLinden, T., Sargeant, J.M., Thomas, M.K., Papadopoulos, A. & Fazil, A. 2014. Component costs of foodborne illness: A scoping review. BMC Public Health, 14(1): 509. https://doi.org/10.1186/1471-2458-14-509 McMeekin, T., Bowman, J., McQuestin, O., Mellefont, L., Ross, T. & Tamplin, M. 2008. The future of predictive microbiology: Strategic research, innovative applications and great expectations. International Journal of Food Microbiology, 128(1): 2–9. https://doi.org/10.1016/j.ijfoodmicro.2008.06.026 McMeekin, T.A., Olley, J., Ross, T. & Ratkowsky, D.A. 1993. Predictive microbiology: Theory and application. Innovation in Microbiology No. 5. Taunton, UK, Research studies press. McQuestin, O.J., Shadbolt, C.T. & Ross, T. 2009. Quantification of the relative effects of temperature, pH, and water activity on inactivation of Escherichia coli in fermented meat by meta-analysis. Applied and Environmental Microbiology, 75(22): 6963– 6972. https://doi.org/10.1128/AEM.00291-09 BIBLIOGRAPHY 249 Mellefont, L.A., McMeekin, T.A. & Ross, T. 2008. Effect of relative inoculum concentration on Listeria monocytogenes growth in co-culture. International Journal of Food Microbiology, 121(2): 157–168. https://doi.org/10.1016/j.ijfoodmicro.2007.10.010 Membré, J.-M. & Boué, G. 2018. Quantitative microbiological risk assessment in food industry: Theory and practical application. Food Research International, 106: 1132– 1139. https://doi.org/10.1016/j.foodres.2017.11.025 Merrell, D.S., Butler, S.M., Qadri, F., Dolganov, N.A., Alam, A., Cohen, M.B., Calderwood, S.B., Schoolnik, G.K. & Camilli, A. 2002. Host-induced epidemic spread of the cholera bacterium. Nature, 417(6889): 642–645. https://doi. org/10.1038/nature00778 Messner, M.J. & Berger, P. 2016. Cryptosporidium infection risk: Results of new doseresponse modeling. Risk Analysis, 36(10): 1969–1982. https://doi.org/10.1111/ risa.12541 Mokhtari, A., Frey, H.C. & Jaykus, L.-A. 2006. Application of classification and regression trees for sensitivity analysis of the Escherichia coli O157:H7 food safety process risk model. Journal of Food Protection, 69(3): 609–618. Monge, S., Pijnacker, R., van Pelt, W., Franz, E., Kortbeek, L.M. & Mangen, M.-J.J. 2019. Accounting for long-term manifestations of Cryptosporidium spp infection in burden of disease and cost-of-illness estimations, the Netherlands (2013–2017). PLOS ONE, 14(3): e0213752. https://doi.org/10.1371/journal.pone.0213752 Moore, C.M., Sheldon, B.W. & Jaykus, L.-A. 2003. Transfer of Salmonella and Campylobacter from stainless steel to romaine lettuce. Journal of Food Protection, 66(12): 2231–2236. https://doi.org/10.4315/0362-028X-66.12.2231 Morgan, M.G., Henrion, M. & Small, M. 1992. Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press. 354 pp. (also available at https://www.cambridge.org/gb/academic/subjects/ psychology/cognition/uncertainty-guide-dealing-uncertainty-quantitative-riskand-policy-analysis?format=HB&isbn=9780521365420). Mullner, P., Spencer, S.E.F., Wilson, D.J., Jones, G., Noble, A.D., Midwinter, A.C., Collins-Emerson, J.M., Carter, P., Hathaway, S. & French, N.P. 2009. Assigning the source of human campylobacteriosis in New Zealand: A comparative genetic and epidemiological approach. Infection, Genetics and Evolution, 9(6): 1311–1319. https://doi.org/10.1016/j.meegid.2009.09.003 Murray, R., Glass-Kaastra, S., Gardhouse, C., Marshall, B., Ciampa, N., Franklin, K., Hurst, M., Thomas, M.K. & Nesbitt, A. 2017. Canadian consumer food safety practices and knowledge: Foodbook study. Journal of Food Protection, 80(10): 1711–1718. https://doi.org/10.4315/0362-028X.JFP-17-108 Mylius, S.D., Nauta, M.J. & Havelaar, A.H. 2007. Cross-contamination during food preparation: A mechanistic model applied to chicken-borne Campylobacter. Risk Analysis, 27(4): 803–813. https://doi.org/10.1111/j.1539-6924.2006.00872.x 250 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD National Research Council. 1989. Biologic markers in reproductive toxicology. Washington, DC, USA, The National Academies Press. (also available at https:// doi.org/10.17226/774). Nauta, M.J. 2000. Separation of uncertainty and variability in quantitative microbial risk assessment models. International Journal of Food Microbiology, 57(1): 9–18. https:// doi.org/10.1016/S0168-1605(00)00225-7 Nauta, M.J. 2001. A modular process risk model structure for quantiative microbiolgical risk assessment and its application in an exposure assessment of Bacillus cereus in a REPFED. , p. 100. RIVM [Rijksinstituut voor Volksgezondheid en Milieu]. (also available at www.rivm.nl/bibliotheek/rapporten/149106007.html). Nauta, M.J. 2008. The modular process risk model (MPRM): A structured approach to food chain exposure assessment (Ch 4). In D.W. Schaffner, ed. Microbial Risk Analysis of Foods, pp. 99–136. (also available at http://www.asmscience.org/ content/book/10.1128/9781555815752.ch04). Nauta, M.J., Evers, E.G., Takumi, K. & Havelaar, A.H. 2001. Risk assessment of Shigatoxin producing Escherichia coli O157 in steak tartare in The Netherlands. , p. 169. Bilthoven, The Netherlands, RIVM [Rijksinstituut voor Volksgezondheid en Milieu]. (also available at https://www.rivm.nl/bibliotheek/rapporten/257851003. html). Nauta, M.J., Van der Fels-Klerx, I. & Havelaar, A. 2005. A poultry-processing model for quantitative microbiological risk assessment. Risk Analysis, 25(1): 85–98. https:// doi.org/10.1111/j.0272-4332.2005.00569.x Nauta, M.J., van der Wal, F.J., Putirulan, F.F., Post, J., van de Kassteele, J. & Bolder, N.M. 2009. Evaluation of the “testing and scheduling” strategy for control of Campylobacter in broiler meat in The Netherlands. International Journal of Food Microbiology, 134: 216–222. NHMRC. 2011. Australian drinking water guidelines 6 (Update 2019). National Health and Medical Research Council. (also available at https://nhmrc.gov.au/aboutus/publications/australian-drinking-water-guidelines#block-views-block-fileattachments-content-block-1). Notermans, S., Dufrenne, J., Teunis, P. & Chackraborty, T. 1998. Studies on the risk assessment of Listeria monocytogenes. Journal of Food Protection, 61(2): 244–248. https://doi.org/10.4315/0362-028X-61.2.244 OIE. 2018. Terrestial animal health code. 27th edition. World Organization for Animal Health. (also available at http://www.oie.int/standard-setting/terrestrial-code/ access-online/). Omurtag, I., Paulsen, P., Hilbert, F. & Smulders, F.J.M. 2013. The risk of transfer of foodborne bacterial hazards in Turkey through the consumption of meat; risk ranking of muscle foods with the potential to transfer Campylobacter spp. Food Security, 5(1): 117–127. https://doi.org/10.1007/s12571-012-0230-z BIBLIOGRAPHY 251 Painter, J.A., Hoekstra, R.M., Ayers, T., Tauxe, R.V., Braden, C.R., Angulo, F.J. & Griffin, P.M. 2013. Attribution of foodborne illnesses, hospitalizations, and deaths to food commodities by using outbreak data, United States, 1998–2008. Emerging Infectious Diseases, 19(3): 407–415. Pang, H., Lambertini, E., Buchanan, R.L., Schaffner, D.W. & Pradhan, A.K. 2017. Quantitative microbial risk assessment for Escherichia coli O157:H7 in fresh-cut lettuce. Journal of Food Protection, 80(2): 302–311. https://doi.org/10.4315/0362028X.JFP-16-246 Pérez-Rodríguez, F., Campos, D., Ryser, E.T., Buchholz, A.L., Posada-Izquierdo, G.D., Marks, B.P., Zurera, G. & Todd, E. 2011. A mathematical risk model for Escherichia coli O157:H7 cross-contamination of lettuce during processing. Food Microbiology, 28(4): 694–701. Pérez-Rodríguez, F. & Valero, A. 2013. Predictive microbiology in foods. SpringerBriefs in food, health, and nutrition No. 5. New York, Springer. 128 pp. Pérez-Rodríguez, F., Valero, A., Carrasco, E., García, R.M. & Zurera, G. 2008. Understanding and modelling bacterial transfer to foods: a review. Trends in Food Science & Technology, 19(3): 131–144. https://doi.org/10.1016/j.tifs.2007.08.003 Perrin, F., Tenenhaus-Aziza, F., Michel, V., Miszczycha, S., Bel, N. & Sanaa, M. 2015. Quantitative risk assessment of haemolytic and uremic syndrome linked to O157:H7 and non-O157:H7 Shiga-toxin producing Escherichia coli strains in raw milk soft cheeses. Risk Analysis, 35(1): 109–128. https://doi.org/10.1111/risa.12267 Petitti, D.B. 2000. Meta-analysis, decision analysis, and cost-effectiveness analysis: methods for quantitative synthesis in medicine. Second edition. New York, Oxford University Press. 306 pp. Pichler, J., Ziegler, J., Aldrian, U. & Allerberger, F. 2014. Evaluating levels of knowledge on food safety among food handlers from restaurants and various catering businesses in Vienna, Austria 2011/2012. Food Control, 35(1): 33–40. https://doi. org/10.1016/j.foodcont.2013.06.034 Pires, S.M. 2013. Assessing the applicability of currently available methods for attributing foodborne disease to sources, including food and food commodities. Foodborne Pathogens and Disease, 10(3): 206–13. Pires, S.M., Evers, E.G., van Pelt, W., Ayers, T., Scallan, E., Angulo, F.J., Havelaar, A. & Hald, T. 2009. Attributing the human disease burden of foodborne infections to specific sources. Foodborne Pathogens and Disease, 6(4): 417–424. https://doi. org/10.1089/fpd.2008.0208 Possas, A., Carrasco, E., García-Gimeno, R.M. & Valero, A. 2017. Models of microbial cross-contamination dynamics. Current Opinion in Food Science, 14: 43–49. https:// doi.org/10.1016/j.cofs.2017.01.006 252 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Pouillot, R., Chen, Y. & Hoelzer, K. 2015. Modeling number of bacteria per food unit in comparison to bacterial concentration in quantitative risk assessment: Impact on risk estimates. Food Microbiology, 45: 245–253. https://doi.org/10.1016/j. fm.2014.05.008 Pouillot, R. & Delignette-Muller, M.L. 2010. Evaluating variability and uncertainty separately in microbial quantitative risk assessment using two R packages. International Journal of Food Microbiology, 142(3): 330–340. Pouillot, R., Gallagher, D., Tang, J., Hoelzer, K., Kause, J. & Dennis, S.B. 2015a. Listeria monocytogenes in retail delicatessens: An interagency risk assessment— Model and baseline results. Journal of Food Protection, 78(1): 134–145. https://doi. org/10.4315/0362-028X.JFP-14-235 Pouillot, R., Garin, B., Ravaonindrina, N., Diop, K., Ratsitorahina, M., Ramanantsoa, D. & Rocourt, J. 2012. A risk assessment of campylobacteriosis and salmonellosis linked to chicken meals prepared in households in Dakar, Senegal. Risk Analysis, 32(10): 1798–1819. https://doi.org/10.1111/j.1539-6924.2012.01796.x Pouillot, R., Hoelzer, K., Chen, Y. & Dennis, S.B. 2015b. Listeria monocytogenes dose response revisited—Incorporating adjustments for variability in strain virulence and host susceptibility. Risk Analysis, 35(1): 90–108. https://doi.org/10.1111/ risa.12235 Pouillot, R., Klontz, K.C., Chen, Y., Burall, L.S., Macarisin, D., Doyle, M., Bally, K.M., Strain, E., Datta, A.R., Hammack, T.S. & Van Doren, J.M. 2016. Infectious dose of Listeria monocytogenes in outbreak linked to ice cream, United States, 2015. Emerging Infectious Diseases, 22(12): 2113–2119. https://doi.org/10.3201/ eid2212.160165 Pouillot, R., Miconnet, N., Afchain, A.-L., Delignette-Muller, M.L., Beaufort, A., Rosso, L., J.-B, D. & Cornu, M. 2007. Quantitative risk assessment of Listeria monocytogenes in french cold-smoked salmon: I. Quantitative exposure assessment. Risk Analysis, 27(3): 683–700. https://doi.org/10.1111/j.1539-6924.2007.00921.x Powell, M., Ebel, E. & Schlosser, W. 2001. Considering uncertainty in comparing the burden of illness due to foodborne microbial pathogens. International Journal of Food Microbiology, 69(3): 209–215. https://doi.org/10.1016/S0168-1605(01)00495-0 Powell, M., Schlosser, W. & Ebel, E. 2004. Considering the complexity of microbial community dynamics in food safety risk assessment. International Journal of Food Microbiology, 90(2): 171–179. https://doi.org/10.1016/S0168-1605(03)00106-5 Prado-Silva, L., Cadavez, V., Gonzales-Barron, U., Rezende, A.C.B. & Sant’Ana, A.S. 2015. Meta-analysis of the effects of sanitizing treatments on Salmonella, Escherichia coli O157:H7, and Listeria monocytogenes inactivation in fresh produce. Applied and Environmental Microbiology, 81(23): 8008–8021. https://doi.org/10.1128/ AEM.02216-15 BIBLIOGRAPHY 253 Pressman, R.S. 2005. Software engineering: A practitioner’s approach. Palgrave Macmillan. 922 pp. Pujol, L., Albert, I., Johnson, N.B. & Membré, J.-M. 2013. Potential application of quantitative microbiological risk assessment techniques to an aseptic-UHT process in the food industry. International Journal of Food Microbiology, 162(3): 283–296. https://doi.org/10.1016/j.ijfoodmicro.2013.01.021 PulseNet International. 2021. PulseNet International [Online]. [Cited 30 May 2021]. https://pulsenetinternational.org/ QMRA wiki. 2021. Quantitative Microbial Risk Assessment (QMRA) Wiki [Online]. Michigan. [Cited 30 May 2021]. http://qmrawiki.canr.msu.edu/index.php Ranta, J., Lindqvist, R., Hansson, I., Tuominen, P. & Nauta, M. 2015. A Bayesian approach to the evaluation of risk-based microbiological criteria for Campylobacter in broiler meat. Annals of Applied Statistics, 9(3): 1415–1432. https://doi. org/10.1214/15-AOAS845 Rantsiou, K., Kathariou, S., Winkler, A., Skandamis, P., Saint-Cyr, M.J., RouzeauSzynalski, K. & Amézquita, A. 2018. Next generation microbiological risk assessment: opportunities of whole genome sequencing (WGS) for foodborne pathogen surveillance, source tracking and risk assessment. International Journal of Food Microbiology, 287: 3–9. https://doi.org/10.1016/j.ijfoodmicro.2017.11.007 Ratkowsky, D.A. 1992. Predicting response times in predictive food microbiology. No. 1992/1. Hobart, Australia, Department of Primary Industry, Fisheries and Energy (Tasmania). Ratkowsky, D.A., Ross, T., Macario, N., Dommett, T.W. & Kamperman, L. 1996. Choosing probability distributions for modelling generation time variability. Journal of Applied Bacteriology, 80(2): 131–137. https://doi.org/10.1111/j.1365-2672.1996. tb03200.x Ratkowsky, D.A., Ross, T., McMeekin, T.A. & Olley, J. 1991. Comparison of Arrheniustype and Bêlehrádek-type models for prediction of bacterial growth in foods. Journal of Applied Bacteriology, 71(5): 452–459. https://doi.org/10.1111/j.1365-2672.1991. tb03816.x Roberts, T., Ahl, A. & McDowell, R. 1995. Risk assessment for foodborne microbial hazards. Tracking foodborne pathogens from farm to table, p. Miscellaneous Publications No. 1532. Washington, DC, USA, U.S. Department of Agriculture. Ross, T. 2008. Microbial ecology in food safety risk assessment (Ch 3). In D.W. Schaffner, ed. Microbial Risk Analysis of Foods, pp. 51–97. (also available at http://www. asmscience.org/content/book/10.1128/9781555815752.ch03). Ross, T., Dalgaard, P. & Tienungoon, S. 2000. Predictive modelling of the growth and survival of Listeria in fishery products. International Journal of Food Microbiology, 62(3): 231–245. https://doi.org/10.1016/S0168-1605(00)00340-8 254 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Ross, T. & McMeekin, T.A. 2003. Modeling microbial growth within food safety risk assessments. Risk Analysis, 23(1): 179–197. https://doi.org/10.1111/15396924.00299 Ross, T., McMeekin, T.A. & Baranyi, J. 2014. Predictive microbiology and food safety. Encyclopedia of Food Microbiology. 2nd edition, pp. 59–68. Elsevier. (also available at https://linkinghub.elsevier.com/retrieve/pii/B9780123847300002561). Ross, T., Rasmussen, S., Fazil, A., Paoli, G. & Sumner, J. 2009. Quantitative risk assessment of Listeria monocytogenes in ready-to-eat meats in Australia. International Journal of Food Microbiology, 131(2): 128–137. https://doi. org/10.1016/j.ijfoodmicro.2009.02.007 Ross, T. & Sumner, J. 2002. A simple, spreadsheet-based, food safety risk assessment tool. International Journal of Food Microbiology, 77(1): 39–53. https://doi.org/10.1016/ S0168-1605(02)00061-2 van Rossum, C.T.M., Fransen, H.P., Verkaik-Kloosterman, J., Buurma-Rethans, E.J.M. & Ocké, M.C. 2011. Dutch national food consumption survey 2007-2010 : Diet of children and adults aged 7 to 69 years. (also available at https://rivm. openrepository.com/rivm/handle/10029/261553). Rothstein, H.R., Sutton, A.J. & Borenstein, M. 2006. Publication bias in metaanalysis: Prevention, assessment and adjustments. H.R. Rothstein, A.J. Sutton & M. Borenstein, eds. Chichester, UK, John Wiley & Sons, Ltd. (also available at http:// doi.wiley.com/10.1002/0470870168). Roulo, R.M., Fishburn, J.D., Amosu, M., Etchison, A.R. & Smith, M.A. 2014. Dose response of Listeria monocytogenes invasion, fetal morbidity, and fetal mortality after oral challenge in pregnant and nonpregnant mongolian gerbils. Infection and Immunity, 82(11): 4834–4841. https://doi.org/10.1128/IAI.01514-14 Ruzante, J.M., Whiting, R.C., Dennis, S.B. & Buchanan, R.L. 2013. Microbial risk assessment. In M.P. Doyle & R.L. Buchanan, eds. Food Microbiology. Fourth edition, pp. 1023–1037. (also available at http://www.asmscience.org/content/ book/10.1128/9781555818463.chap41). Ryan, M.O., Gurian, P.L., Haas, C.N., Rose, J.B. & Duzinski, P.J. 2013a. Acceptable microbial risk: Cost-benefit analysis of a boil water order for Cryptosporidium. Journal - American Water Works Association, 105(4): E189–E194. https://doi. org/10.5942/jawwa.2013.105.0020 Ryan, M.O., Gurian, P.L., Haas, C.N., Rose, J.B. & Duzinski, P.J. 2013b. Errata: Acceptable microbial risk: Cost-benefit analysis of a boil water order for Cryptosporidium. Journal - American Water Works Association, 105(7): 87–87. https://doi.org/10.1002/j.1551-8833.2013.tb08895.x Saltelli, A., Chan, K. & Scott, E.M. 2008. Sensitivity analysis. Second edition. Wiley. 494 pp. (also available at https://www.wiley.com/en-au/ Sensitivity+Analysis-p-9780470743829). BIBLIOGRAPHY 255 Samapundo, S., Climat, R., Xhaferi, R. & Devlieghere, F. 2015. Food safety knowledge, attitudes and practices of street food vendors and consumers in Port-au-Prince, Haiti. Food Control, 50: 457–466. https://doi.org/10.1016/j.foodcont.2014.09.010 Scallan, E., Hoekstra, R.M., Angulo, F.J., Tauxe, R.V., Widdowson, M.-A., Roy, S.L., Jones, J.L. & Griffin, P.M. 2011. Foodborne illness aquired in the United States— Major pathogens. Emerging Infectious Diseases, 17(1): 7–15. Scallan, E., Hoekstra, R.M., Mahon, B.E., Jones, T.F. & Griffin, P.M. 2015. An assessment of the human health impact of seven leading foodborne pathogens in the United States using disability adjusted life years. Epidemiology & Infection, 143(13): 2795–2804. https://doi.org/10.1017/S0950268814003185 Schaffner, D.W. 2003. Models: what comes after the next generation. In R.J. McKellar & X. Lu, eds. Modeling Microbial Responses in Food, p. CRC Press. (also available at https://www.crcpress.com/Modeling-Microbial-Responses-in-Food/McKellarLu/p/book/9780849312373). Schaffner, D.W. 2004. Mathematical frameworks for modeling Listeria crosscontamination in food. Journal of Food Science, 69(6): R155–R159. https://doi. org/10.1111/j.1365-2621.2004.tb11004.x Shinagawa, K., Hu, D. & Yoshida, S. 1997. Correspondence and problem for enterohemorrhagic E. coli O157 outbreak in Morioka city, Iwate. Koshu Eisei Kenkyu, 46: 104–112. Shorten, P.R., Pleasants, A.B. & Soboleva, T.K. 2006. Estimation of microbial growth using population measurements subject to a detection limit. International Journal of Food Microbiology, 108: 369–375. Skjerve, E. 1999. Possible increase of human Taenia saginata infections through import of beef to norway from a high prevalence area. Journal of Food Protection, 62(11): 1314–1319. https://doi.org/10.4315/0362-028X-62.11.1314 Slikker Jr., W. 2018. Biomarkers and their impact on precision medicine. Experimental Biology and Medicine, 243(3): 211–212. https://doi.org/10.1177/1535370217733426 Smid, J.H., de Jonge, R., Havelaar, A.H. & Pielaat, A. 2013. Variability and uncertainty analysis of the cross-contamination ratios of Salmonella during pork cutting. Risk Analysis, 33(6): 1100–1115. Smith, A.M., Tau, N.P., Smouse, S.L., Allam, M., Ismail, A., Ramalwa, N.R., Disenyeng, B., Ngomane, M. & Thomas, J. 2019. Outbreak of Listeria monocytogenes in South Africa, 2017–2018: Laboratory activities and experiences associated with wholegenome sequencing analysis of isolates. Foodborne Pathogens and Disease. https:// doi.org/10.1089/fpd.2018.2586 Smith, B.A., Fazil, A. & Lammerding, A.M. 2013. A risk assessment model for Escherichia coli O157:H7 in ground beef and beef cuts in Canada: Evaluating the effects of interventions. Food Control, 29(2): 364–381. 256 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Smith, M.A., Takeuchi, K., Anderson, G., Ware, G.O., McClure, H.M., Raybourne, R.B., Mytle, N. & Doyle, M.P. 2008. Dose-response model for Listeria monocytogenesinduced stillbirths in nonhuman primates. Infection and Immunity, 76(2): 726–731. https://doi.org/10.1128/IAI.01366-06 Smith, M.A., Takeuchi, K., Brackett, R.E., McClure, H.M., Raybourne, R.B., Williams, K.M., Babu, U.S., Ware, G.O., Broderson, J.R. & Doyle, M.P. 2003. Nonhuman primate model for Listeria monocytogenes-induced stillbirths. Infection and Immunity, 71(3): 1574–1579. https://doi.org/10.1128/IAI.71.3.1574-1579.2003 SourceForge. 2021. OpenML for Predictive Modelling in Food [Online]. [Cited 30 May 2021]. https://sourceforge.net/p/microbialmodelingexchange/wiki/Tools/ Spiegelhalter, D.J. & Riesch, H. 2011. Don’t know, can’t know: embracing deeper uncertainties when analysing risks. Phil. Trans. R. Soc. A, 369(1956): 4730–4750. https://doi.org/10.1098/rsta.2011.0163 Ssemanda, J.N. 2018. Towards microbial safety of fresh vegetables in Rwanda. The Netherlands, Wageningen University. (PhD Thesis). (also available at https://www. wur.nl/en/activity/Towards-microbial-safety-of-fresh-vegetables-in-Rwanda-1. htm). Ssemanda, J.N., Reij, M.W., Bagabe, M.C., Muvunyi, C.M., Nyamusore, J., Joosten, H. & Zwietering, M.H. 2018. Estimates of the burden of illnesses related to foodborne pathogens as from the syndromic surveillance data of 2013 in Rwanda. Microbial Risk Analysis, 9: 55–63. https://doi.org/10.1016/j.mran.2018.02.002 Stephens, P.J., Joynson, J.A., Davies, K.W., Holbrook, R., Lappin‐Scott, H.M. & Humphrey, T.J. 1997. The use of an automated growth analyser to measure recovery times of single heat-injured Salmonella cells. Journal of Applied Microbiology, 83(4): 445–455. https://doi.org/10.1046/j.1365-2672.1997.00255.x Steyerberg, E.W., Vickers, A.J., Cook, N.R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M.J. & Kattan, M.W. 2010. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology, 21(1): 128–138. https://doi.org/10.1097/ede.0b013e3181c30fb2 Strachan, N.J.C., Doyle, M.P., Kasuga, F., Rotariu, O. & Ogden, I.D. 2005. Dose response modelling of Escherichia coli O157 incorporating data from foodborne and environmental outbreaks. International Journal of Food Microbiology, 103: 35–47. Sumner, J. 2016. The impact of transport to Australia’s distant markets on the shelf-life of beef and sheep primals. No. 2016.1075. North Sydney, Australian Meat Processor Corporation. (also available at https://www.ampc.com.au/uploads/cgblog/ id10/2016-1075-The-impact-of-transport-to-Australias-distant-markets-FinalReport.pdf ). BIBLIOGRAPHY 257 Sumner, J. & Ross, T. 2002. A semi-quantitative seafood safety risk assessment. International Journal of Food Microbiology, 77(1): 55–59. Sumner, J., Ross, T. & Ababouch, L. 2004. Application of risk assessment in the fish industry. Fisheries Technical Paper No. 442. Rome, Italy, FAO. (also available at http://www.fao.org/3/y4722e/y4722e00.htm). Tenenhaus-Aziza, F. & Ellouze, M. 2015. Software for predictive microbiology and risk assessment: A description and comparison of tools presented at the ICPMF8 software fair. Food Microbiology, 45: 290–299. https://doi.org/10.1016/j. fm.2014.06.026 Tessema, A.G., Gelaye, K.A. & Chercos, D.H. 2014. Factors affecting food handling practices among food handlers of Dangila town food and drink establishments, North West Ethiopia. BMC Public Health, 14(1): 571. https://doi.org/10.1186/14712458-14-571 Teunis, P.F.M. 1997. Infectious gastro-enteritis - opportunities for dose response modelling. Bilthoven, RIVM. (also available at https://www.rivm.nl/bibliotheek/ rapporten/284550003.html). Teunis, P.F.M., Chappell, C.L. & Okhuysen, P.C. 2002a. Cryptosporidium dose response studies: Variation between isolates. Risk Analysis, 22(1): 175–185. https://doi. org/10.1111/0272-4332.00014 Teunis, P.F.M., Chappell, C.L. & Okhuysen, P.C. 2002b. Cryptosporidium dose-response studies: Variation between hosts. Risk Analysis, 22(3): 475–485. https://doi. org/10.1111/0272-4332.00046 Teunis, P.F.M. & Havelaar, A.H. 2000. The Beta Poisson dose-response model is not a single-hit model. Risk Analysis, 20(4): 513–520. Teunis, P.F.M., van der Heijden, O.G., van der Giessen, J.W.B. & Havelaar, A.H. 1996. The dose-response relation in human volunteers for gastro-intestinal pathogens. Bilthoven, RIVM. (also available at https://www.rivm.nl/bibliotheek/ rapporten/284550002.html). Teunis, P.F.M., Kasuga, F., Fazil, A., Ogden, I.D., Rotariu, O. & Strachan, N.J.C. 2010. Dose–response modeling of Salmonella using outbreak data. International Journal of Food Microbiology, 144(2): 243–249. https://doi.org/10.1016/j. ijfoodmicro.2010.09.026 Teunis, P.F.M., Moe, C.L., Liu, P., Miller, S.E., Lindesmith, L., Baric, R.S., Le Pendu, J. & Calderon, R.L. 2008. Norwalk virus: How infectious is it? Journal of Medical Virology, 80(8): 1468–1476. Teunis, P.F.M., Nagelkerke, N.J.D. & Haas, C.N. 1999. Dose response models for infectious gastroenteritis. Risk Analysis, 19(6): 1251–1260. 258 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Teunis, P.F.M. & Schijven, J.F. 2019. Generic guidance to quantitative microbial risk assessment for food and water. RIVM [Rijksinstituut voor Volksgezondheid en Milieu]. (also available at https://rivm.openrepository.com/handle/10029/623001). Thomas, C.J., Daughtry, B.J., Padula, D., Jordan, D., Arzey, G., Davey, K.R., Holds, G., Slade, J. & Pointon, A. 2006. An egg: Salmonella quantitative risk assessment model for the Australian egg industry. Australian Egg Corporation Limited. (also available at https://www.australianeggs.org.au/what-we-do/leading-research/anegg-salmonella-quantitative-risk-assessment-model/). Thompson, K.M. & Graham, J.D. 1996. Going beyond the single number: Using probabilistic risk assessment to improve risk management. Human and Ecological Risk Assessment, 2(4): 1008–1034. https://doi.org/10.1080/10807039609383660 Thomsen, S.T., de Boer, W., Pires, S.M., Devleesschauwer, B., Fagt, S., Andersen, R., Poulsen, M. & van der Voet, H. 2019. A probabilistic approach for risk-benefit assessment of food substitutions: A case study on substituting meat by fish. Food and Chemical Toxicology, 126: 79–96. https://doi.org/10.1016/j.fct.2019.02.018 Tomatsis, L., ed. 1990. Cancer: Causes, occurrence and control. Lyon, France, International Agency for Research on Cancer. (also available at http://doi.wiley.com/10.1002/ sim.4780110216). USDA. 2021. Predictive Microbiology Information Portal [Online]. [Cited 30 May 2021]. https://portal.errc.ars.usda.gov/ USEPA. 2006. Peer review and peer involvement at the U.S. Environmental Protection Agency. United States Environmental Protection Agency. [Cited 12 October 2018]. https://www.epa.gov/sites/production/files/2015-01/documents/peer_ review_policy_and_memo.pdf USFDA. 2005. Quantitative risk assessment on the public health impact of pathogenic Vibrio parahaemolyticus in raw oysters. U.S. Department of Health and Human Services / Food and Drug Administration. (also available at https://www.fda.gov/ food/foodscienceresearch/risksafetyassessment/ucm050421.htm). USFDA/FSIS. 2003. Quantitative assessment of relative risk to public health from foodborne Listeria monocytogenes among selected categories of ready-to-eat foods. Washington, DC, USA, U.S. Department of Health and Human Services - Food and Drug Administration / U.S. Department of Agriculture - Food Safety and Inspection Service. (also available at https://www.fda.gov/Food/FoodScienceResearch/ RiskSafetyAssessment/ucm183966.htm). USNACMCF. 2004. Response to the questions posed by FSIS regarding performance standards with particular reference to raw ground chicken. Atlanta, USA, US National Advisory Committee on Microbiological Criteria for Foods. (also available at https://www.fsis.usda.gov/wps/wcm/connect/e2ac5b0d-0c6a-4075adb5-eb1ccbc1b1ff/NACMCF_Ground_Chicken_082704.pdf ?MOD=AJPERES). BIBLIOGRAPHY 259 USNACMCF. 2010. Parameters for determining inoculated pack/challenge study protocols. Journal of Food Protection, 73(1): 140–202. https://doi.org/10.4315/0362028X-73.1.140 Vally, H., Glass, K., Ford, L., Hall, G., Kirk, M.D., Shadbolt, C., Veitch, M., Fullerton, K.E., Musto, J. & Becker, N. 2014. Proportion of illness acquired by foodborne transmission for nine enteric pathogens in Australia: An expert elicitation. Foodborne Pathogens and Disease, 11(9): 727–733. https://doi.org/10.1089/ fpd.2014.1746 Van Abel, N., Schoen, M.E., Kissel, J.C. & Meschke, J.S. 2017. Comparison of risk predicted by multiple Norovirus dose–response models and implications for quantitative microbial risk assessment. Risk Analysis, 37(2): 245–264. https://doi. org/10.1111/risa.12616 Van der Fels-Klerx, H.J., Van Asselt, E.D., Raley, M., Poulsen, M., Korsgaard, H., Bredsdorff, L., Nauta, M., D’agostino, M., Coles, D., Marvin, H.J.P. & Frewer, L.J. 2018. Critical review of methods for risk ranking of food-related hazards, based on risks for human health. Critical Reviews in Food Science and Nutrition, 58(2): 178–193. https://doi.org/10.1080/10408398.2016.1141165 Vatanpour, S., Hrudey, S. & Dinu, I. 2015. Can public health risk assessment using risk matrices be misleading? International Journal of Environmental Research and Public Health, 12(8): 9575–9588. https://doi.org/10.3390/ijerph120809575 Vimont, A.J.M., Kiermeier, A., Padula, D.J., Holds, G.L. & Pointon, A.M. 2005. The adequacy of sample type/weight and incubation period on detection of Salmonella spp. in slaughter cattle. Journal of Veterinary Diagnostic Investigation, 17(5): 430– 435. Vose, D. 2008. Risk analysis: A quantitative guide. Third edition. Wiley. 752 pp. (also available at https://www.wiley.com/en-au/Risk+Analysis%3A+A+Quantitative+ Guide%2C+3rd+Edition-p-9780470512845). Whittemore, A.S. 1983. Facts and values in risk analysis for environmental toxicants. Risk Analysis, 3(1): 23–33. https://doi.org/10.1111/j.1539-6924.1983.tb00103.x WHO. 1999. Health-based monitoring of recreational waters: the feasibility of a new approach (the “Annapolis Protocol”). No. WHO/SDE/WSH/99.1. Geneva, Switzerland, WHO. (also available at http://www.who.int/water_sanitation_health/ bathing/annapolis.pdf ). WHO. 2000. The world health report 2000 - Health systems: improving performance. Geneva, Switzerland, WHO. (also available at http://www.who.int/whr/2000/en/). WHO. 2001. Joint FAO/WHO Expert Consultation on Risk Assessment of Microbiological Hazards in Foods : hazard identification, exposure assessment and hazard characterization of Campylobacter spp. in broiler chickens and Vibrio spp. 260 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD in seafood, WHO headquarters, Geneva, Switzerland, 23-27 July 2001. No. WHO/ SDE/PHE/FOS/01.4. Geneva, Switzerland, WHO48 pp. (also available at http:// apps.who.int/iris/handle/10665/67090). WHO. 2003. Guidelines for safe recreational water environments: Volume 1, Coastal and fresh water. Geneva, Switzerland, WHO. WHO. 2015. WHO estimates of the global burden of foodborne diseases: Foodborne disease burden epidemiology reference group 2007-2015. Geneva, Switzerland, WHO. (also available at http://www.who.int/foodsafety/areas_work/foodborne-diseases/ferg/ en/). WHO. 2016. Quantitative microbial risk assessment: Application for water safety management. Geneva, Switzerland, WHO. (also available at https://www.who.int/ water_sanitation_health/publications/qmra/en/). WHO. 2018. Whole genome sequencing for foodborne disease surveillance: landscape paper. Geneva, Switzerland, WHO. (also available at http://www.who.int/ foodsafety/publications/foodborne_disease/wgs_landscape/en/). WHO & OECD. 2003. Assessing microbial safety of drinking water: Improving approaches and methods. IWA Publishing. (also available at http://www.who.int/water_ sanitation_health/publications/9241546301/en/). WHO. 2021. GEMS/Food consumption database. Geneva. [Cited 30 May 2021]. https:// www.who.int/nutrition/landscape_analysis/nlis_gem_food/en/ Wieland, B., Dhollander, S., Salman, M. & Koenen, F. 2011. Qualitative risk assessment in a data-scarce environment: A model to assess the impact of control measures on spread of African Swine Fever. Preventive Veterinary Medicine, 99(1): 4–14. https:// doi.org/10.1016/j.prevetmed.2011.01.001 WikiPedia. 2021. Literate programming [Online]. [Cited 30 May 2021]. https:// en.wikipedia.org/wiki/Literate_programming Wilks, S.S. 1962. Mathematical Statistics. Second edition. New York, John Wiley & Sons. Williams, D., Castleman, J., Lee, C.-C., Mote, B. & Smith, M.A. 2009. Risk of fetal mortality after exposure to Listeria monocytogenes based on dose-response data from pregnant guinea pigs and primates. Risk Analysis, 29(11): 1495–1505. https:// doi.org/10.1111/j.1539-6924.2009.01308.x Williams, D., Irvin, E.A., Chmielewski, R.A., Frank, J.F. & Smith, M.A. 2007. Doseresponse of Listeria monocytogenes after oral exposure in pregnant guinea pigs. Journal of Food Protection, 70(5): 1122–1128. https://doi.org/10.4315/0362028X-70.5.1122 Williams, M.S. & Ebel, E.D. 2014. Fitting a distribution to censored contamination data using Markov Chain Monte Carlo methods and samples selected with unequal BIBLIOGRAPHY 261 probabilities. Environmental Science & Technology, 48(22): 13316–13322. https:// doi.org/10.1021/es5035574 Williams, M.S., Ebel, E.D. & Vose, D. 2011a. Methodology for determining the appropriateness of a linear dose-response function. Risk Analysis, 31(3): 345–350. https://doi.org/10.1111/j.1539-6924.2010.01518.x Williams, M.S., Ebel, E.D. & Vose, D. 2011b. Framework for microbial food-safety risk assessments amenable to Bayesian modeling. Risk Analysis, 31(4): 548–565. https:// doi.org/10.1111/j.1539-6924.2010.01532.x Wilson, P.D.G., Brocklehurst, T.F., Arino, S., Thuault, D., Jakobsen, M., Lange, M., Farkas, J., Wimpenny, J.W.T. & Van Impe, J.F. 2002. Modelling microbial growth in structured foods: towards a unified approach. International Journal of Food Microbiology, 73(2): 275–289. https://doi.org/10.1016/S0168-1605(01)00660-2 Woods, C.R. 2013. False-positive results for Immunoglobulin M serologic results: Explanations and examples. Journal of the Pediatric Infectious Diseases Society, 2(1): 87–90. https://doi.org/10.1093/jpids/pis133 Wooldridge, M. 2008. Qualitative risk assessment. Microbial Risk Analysis of Foods: 1–28. https://doi.org/10.1128/9781555815752.ch1 WTO. 1998. Australia - Measures affecting importation of salmon - Report of the panel. , p. 267. No. WT/DS18/R. Brussels, Belgium, World Trade Organisztion. (also available at https://www.wto.org/english/tratop_e/dispu_e/cases_e/ds18_e.htm). WTO. 2000. Committee on Sanitary and Phytosanitary Measures - Guidelines to further the practical implementation of Article 5.5 [G/SPS/15]. World Trade Organization. [Cited 6 December 2018]. https://docsonline.wto.org/Dol2FE/ Pages/FormerScriptedSearch/directdoc.aspx?DDFDocuments/t/G/SPS/15.DOC Wu, F.M., Doyle, M.P., Beuchat, L.R., Wells, J.G., Mintz, E.D. & Swaminathan, B. 2000. Fate of Shigella sonnei on parsley and methods of disinfection. Journal of Food Protection, 63(5): 568–572. Young, I., Reimer, D., Greig, J., Meldrum, R., Turgeon, P. & Waddell, L. 2017a. Explaining consumer safe food handling through behavior-change theories: A systematic review. Foodborne Pathogens and Disease, 14(11): 609–622. https://doi. org/10.1089/fpd.2017.2288 Young, I., Thaivalappil, A., Reimer, D. & Greig, J. 2017b. Food safety at farmers’ markets: A knowledge synthesis of published research. Journal of Food Protection, 80(12): 2033–2047. https://doi.org/10.4315/0362-028X.JFP-17-193 Zhang, D., McQuestin, O.J., Mellefont, L.A. & Ross, T. 2010. The influence of nonlethal temperature on the rate of inactivation of vegetative bacteria in inimical environments may be independent of bacterial species. Food Microbiology, 27(4): 453–459. https://doi.org/10.1016/j.fm.2009.12.006 262 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD Zilelidou, E.A., Tsourou, V., Poimenidou, S., Loukou, A. & Skandamis, P.N. 2015. Modeling transfer of Escherichia coli O157:H7 and Listeria monocytogenes during preparation of fresh-cut salads: Impact of cutting and shredding practices. Food Microbiology, 45: 254–265. https://doi.org/10.1016/j.fm.2014.06.019 Zoellner, C., Jennings, R., Wiedmann, M. & Ivanek, R. 2019. EnABLe: An agent-based model to understand Listeria dynamics in food processing facilities. Scientific Reports, 9(1): 1–14. https://doi.org/10.1038/s41598-018-36654-z Zwietering, M.H., Cuppers, H.G.A.M., de Wit, J.C. & van’t Riet, K. 1994. Evaluation of data transformations and validation of a model for the effect of temperature on bacterial growth. Appl. Environ. Microbiol., 60(1): 195–203. Zwietering, M.H. & Hasting, A.P.M. 1997a. Modelling the hygienic processing of foods – a global process overview. IChemE Transactions Part C: Food Bioproducts Processing, 75(C3): 159–167. Zwietering, M.H. & Hasting, A.P.M. 1997b. Modelling the hygienic processing of foods – influence of individual process stages. IChemE Transactions Part C: Food Bioproducts Processing, 75(C3): 168–173. Zwietering, M.H., Jongenburger, I., Rombouts, F.M. & Van ’T Riet, K. 1990. Modeling of the bacterial growth curve. Applied and Environmental Microbiology, 56(6): 1875–1881. GLOSSARY 263 Glossary Baseline risk: The level of food safety risk posed by a hazard in a food supply chain without any changes to the current system, i.e. without additional risk management options being implemented. Dose–response assessment: The determination of the relationship between the magnitude of exposure (dose) to a chemical, biological or physical agent and the severity and/or frequency of associated adverse health effects (response). (CAC, 2019) Exposure assessment: The qualitative and/or quantitative evaluation of the likely intake of biological, chemical, and physical agents via food as well as exposures from other sources if relevant. (CAC, 2019) HACCP system: The development of a HACCP plan and the implementation of the procedures in accordance with that plan (CAC, 1969). The HACCP plan is defined as “documentation or set of documents, prepared in accordance with the principles of HACCP to ensure control of significant hazards in the food business” (CAC, 1969). Hazard: a biological, chemical or physical agent in, or condition of, food with the potential to cause an adverse health effect. (CAC, 2019) Hazard characterization: The qualitative and/or quantitative evaluation of the nature of the adverse health effects associated with biological, chemical and physical agents which may be present in food. (CAC, 2019) Hazard identification: The identification of biological, chemical, and physical agents capable of causing adverse health effects and which may be present in a particular food or group of foods. (CAC, 2019) Qualitative risk assessment: A risk assessment based on data which, while forming an inadequate basis for numerical risk estimations, nonetheless, when conditioned by prior expert knowledge and identification of attendant uncertainties permits risk ranking or separation into descriptive categories of risk. (CAC, 1999) Quantitative risk assessment: A risk assessment that provides numerical expressions of risk and indication of the attendant uncertainties. (CAC, 1999) Ranking: The process of ordering different hazard–food product combinations according to risk for risk assessment and/or risk management priority. Risk: A function of the probability of an adverse health effect and the severity of that effect, consequential to a hazard(s) in food. (CAC, 2019) Risk analysis: A process consisting of three components: risk assessment, risk management and risk communication. (CAC, 2019) 264 Risk assessment: A scientifically based process consisting of the following steps: (i) hazard identification, (ii) hazard characterization, (iii) exposure assessment, and (iv) risk characterization. (CAC, 2019) Risk characterization: The process of determining the qualitative and/or quantitative estimation, including attendant uncertainties, of the probability of occurrence and severity of known or potential adverse health effects in a given population based on hazard identification, hazard characterization and exposure assessment. (CAC, 2019) Risk communication: The interactive exchange of information and opinions throughout the risk analysis process concerning risks, risk-related factors and risk perceptions, among risk assessors, risk managers, consumers, industry, the academic community and other interested parties, including the explanation of risk assessment findings and the basis of risk management decisions. (CAC, 2019) Risk estimate: The qualitative and/or quantitative estimation of risk resulting from risk characterization. (CAC, 2019) Risk management: The process, distinct from risk assessment, of weighing policy alternatives, in consultation with all interested parties, considering risk assessment and other factors relevant for the health protection of consumers and for the promotion of fair trade practices, and, if needed, selecting appropriate prevention and control options. (CAC, 2019) Risk profile: The description of the food safety problem and its context. (CAC, 2019) Semi-quantitative risk assessment: A risk assessment conducted by assigning numbers to qualitative estimates of exposure and the dose–response relationship, in the form of probability ranges, weights or scores, and combining them by addition, multiplication, or other mathematical operation, to arrive at a risk estimate with the objective of achieving a greater level of objectivity compared to a qualitative risk assessment approach. Sensitivity analysis: A method used to examine the behaviour of a model by measuring the variation in its outputs resulting from changes to its inputs. (CAC, 1999) Transparent: Characteristics of a process where the rationale, the logic of development, constraints, assumptions, value judgements, decisions, limitations and uncertainties of the expressed determination are fully and systematically stated, documented, and accessible for review. (CAC, 1999) Uncertainty analysis: A method used to estimate the uncertainty associated with model inputs, assumptions and structure/form. (CAC, 1999) GLOSSARY 265 FAO/WHO Microbiological Risk Assessment Series 1 Risk assessments of Salmonella in eggs and broiler chickens: Interpretative Summary, 2002 2 Risk assessments of Salmonella in eggs and broiler chickens, 2002 3 Hazard characterization for pathogens in food and water: Guidelines, 2003 4 Risk assessment of Listeria monocytogenes in ready-to-eat foods: Interpretative Summary, 2004 5 Risk assessment of Listeria monocytogenes in ready-to-eat foods: Technical Report, 2004 6 Enterobacter sakazakii and microorganisms in powdered infant formula: Meeting Report, 2004 7 Exposure assessment of microbiological hazards in food: Guidelines, 2008 8 Risk assessment of Vibrio vulnificus in raw oysters: Interpretative Summary and Technical Report, 2005 9 Risk assessment of choleragenic Vibrio cholerae 01 and 0139 in warm-water shrimp in international trade: Interpretative Summary and Technical Report, 2005 10 Enterobacter sakazakii and Salmonella in powdered infant formula: Meeting Report, 2006 11 Risk assessment of Campylobacter spp. in broiler chickens: Interpretative Summary, 2008 12 Risk assessment of Campylobacter spp. in broiler chickens: Technical Report, 2008 13 Viruses in food: Scientific Advice to Support Risk Management Activities: Meeting Report, 2008 14 Microbiological hazards in fresh leafy vegetables and herbs: Meeting Report, 2008 15 Enterobacter sakazakii (Cronobacter spp.) in powdered follow-up formula: Meeting Report, 2008 16 Risk assessment of Vibrio parahaemolyticus in seafood: Interpretative Summary and Technical Report, 2011 17 Risk characterization of microbiological hazards in food: Guidelines, 2009. 18 Enterohaemorrhagic Escherichia coli in raw beef and beef products: approaches for the provision of scientific advice: Meeting Report, 2010 19 Salmonella and Campylobacter in chicken meat: Meeting Report, 2009 266 MICROBIOLOGICAL RISK ASSESSMENT GUIDANCE FOR FOOD 20 Risk assessment tools for Vibrio parahaemolyticus and Vibrio vulnificus associated with seafood: Meeting Report, 2020 21 Salmonella spp. In bivalve molluscs: Risk Assessment and Meeting Report, In press 22 Selection and application of methods for the detection and enumeration of human pathogenic halophilic Vibrio spp. in seafood: Guidance, 2016 23 Multicriteria-based ranking for risk management of food-borne parasites, 2014 24 Statistical aspects of microbiological criteria related to foods: A risk managers guide, 2016 25 Risk-based examples and approach for control of Trichinella spp. and Taenia saginata in meat: Meeting Report, 2020 26 Ranking of low moisture foods in support of microbiological risk management: Meeting Report and Systematic Review, In press 27 Microbiological hazards associated with spices and dried aromatic herbs: Meeting Report, In press 28 Microbial safety of lipid based ready-to-use foods for management of moderate acute malnutrition and severe acute malnutrition: First meeting report, 2016 29 Microbial safety of lipid based ready-to-use foods for management of moderate acute malnutrition and severe acute malnutrition: Second meeting report, 2021 30 Interventions for the control of non-typhoidal Salmonella spp. in Beef and Pork: Meeting Report and Systematic Review, 2016 31 Shiga toxin-producing Escherichia coli (STEC) and food: attribution, characterization, and monitoring, 2018 32 Attributing illness caused by Shiga toxin-producing Escherichia coli (STEC) to specific foods, 2019 33 Safety and quality of water used in food production a nd processing, 2019 34 Foodborne antimicrobial resistance: Role of the environment, crops and biocides, 2019. 35 Developments of science and risk assessment tools for Vibrio parahaemolyticus and V. vulnificus associated with seafood: Meeting report, 2021. 36. Microbiological risk assessment guidance for food: Guidance, 2021 FAO/WHO MICROBIOLOGICAL RISK ASSESSMENT SERIES 267 Microbiological Risk Assessment Guidance for Food (MRA 36) provides a structured framework for assessing the risk of microbiological hazards in food. It updates three previous FAO and WHO guidance documents (MRA 3, MRA 7 and MRA 17) and brings them into a single volume, providing an overall umbrella for microbiological risk assessment. In doing so it captures the growth and experience in this field, which continues to evolve in-line with science and risk management demands. The Microbiological Risk Assessment Guidance for Food was developed for the global community of scientists and risk assessors, both experienced and inexperienced in risk assessment, and the risk managers or others responsible for risk decision-making and/or communication so that they can: i) identify the key issues and features of a microbiological risk; ii) recognize the properties of a best-practice risk assessment; iii) avoid some common pitfalls of risk assessment; and iv) perform risk assessments that are responsive to the needs of risk managers. Food Systems and Food Safety - Economic and Social Development jemra@fao.org http://www.fao.org/food-safety Food and Agriculture Organization of the United Nations Viale delle Terme di Caracalla 00153 Rome, Italy Department of Nutrition and Food Safety jemra@who.int www.who.int/health-topics/food-safety/ World Health Organization 20 Avenue Appia 1211 Geneva 27, Switzerland ISBN 978-92-5-134518-4 ISSN 1726-5274 9 789251 345184 CB5006EN/1/06.21
关注微信
