Introduction
Public health information available in humanitarian crises is, in general, inadequate and … its application is [often] secondary to reasoning and incentives of a political nature, thus contributing to the recurrent failings of humanitarian action.1
After a decade in which the threat seemed to recede, famine recurred with a vengeance in Somalia in 2011. Between 2016 and 2020, at least three additional countries suffered – or teetered on the edge of – famine, while rapidly increasing numbers of people faced acute food insecurity of a slightly less severe degree (FEWS NET, 2017; IPC, 2020). ‘Famine’ has both human and political connotations. In human terms, it means destruction of livelihoods – to the point of destitution with large numbers of food-insecure people, increased severe malnutrition, disease epidemics, excess death and, frequently, the breakdown of institutions and social norms. Politically, it means a failure of governance – a failure to provide the most basic of protections. Technically, it has come to mean the simultaneous and unambiguous breaching of thresholds for food insecurity, malnutrition and mortality in a given location and time period for a population of at least 10,000 people (IPC Global Partners, 2019). Globally, even before the COVID-19 pandemic, the number of acutely food-insecure people was rapidly increasing and has risen sharply since early 2020 (FAO/WFP, 2020).
‘Famine’ retains the power to shock. On the one hand, mention of ‘famine’ awakens humanitarian actors to a serious food/nutrition/health crisis that has been ignored or under-funded: the risk of famine in Somalia, South Sudan, Nigeria and Yemen prompted the US Congress to allocate an additional $990 million in 2017, despite pressure to reduce – not increase – foreign-assistance budgets (Washington Post, 2017). On the other hand, both states and agencies are extremely reluctant to use the word ‘famine’ (de Waal, 1997; Howe and Devereux, 2007; Lautze and Maxwell, 2007). Broadly speaking, states and governments don’t want to admit that crises have deteriorated to the point of widespread malnutrition and death under their administrations (neither do armed-opposition groups). Donors may have political objectives but may also be reluctant to admit that, even with major humanitarian funding, conditions continue to deteriorate. For humanitarian actors, famine is the dramatic manifestation of response failure (Maxwell and Majid, 2016). With increasing recognition of the linkage between conflict and famine, the United Nations Security Council passed Resolution 2417 in 2018 that condemns both the use of starvation as a weapon and the denial of humanitarian access in conflict (de Waal, 2018). This makes the ability to predict and analyse famine an even more urgent need, but given the connotations of the word, there is significant pressure not to use it, to cover it up, or to cast it as something else. That is to say, any use of the term ‘famine’ is highly political – and so is the analysis of famine. The contemporary analysis of famine comes at a time when evidence and evidence-based decision making is increasingly challenged by the rise of ‘fake news’ and populist governance (Winston and Winston, 2020) and by the deterioration of the multilateral institutions on which famine prevention and humanitarian response has been based (Hopgood, 2019).
Famine has always had political consequences. In perhaps the most infamous contemporary case, the famine in Ethiopia in 1984–85 was deliberately kept out of view by the governing regime because it coincided with the celebrations of the tenth anniversary of the overthrow of Haile Selassie (Burg, 2008; Desportes et al., 2019). Information was suppressed about conflict (Vaux, 2001), forced migration and the extent of the crisis itself (Clay and Holcomb, 1985). The reason was clearly political: the famine of 1972–74 was the triggering event (if not the underlying cause) for the overthrow of Haile Selassie’s government. The regime was very aware of the political consequences of failing to prevent famine, so kept it out of the public eye. This is not a recent phenomenon (Dikötter, 2010; O’Grada, 2015). In many contemporary crises, there is little attempt to analyse how data-collection or analyses processes are undermined or influenced by political factors rather than (or in addition to) being guided by the evidence. These problems are especially pronounced where the risk of famine is high.
Much of contemporary food-security analysis, and virtually all contemporary analysis of famine, has been consolidated under the rubric of the Integrated Food Security Phase Classification tool (IPC) or the analytically identical Cadre Harmonisé (CH) protocols in West Africa. While IPC/CH was not specifically designed to be the main tool for analysing famine, it has assumed that role and has become invaluable in acutely food-insecure contexts. But recent experience suggests that information gathering and analytical processes may be subject to considerable political influences that prevent or limit good analysis (Bailey, 2012; Buchanan-Smith et al., 2019).
Lurking in the background is the age-old humanitarian dilemma of sovereignty: do sovereign states have the sole right to declare crises (and famines) within their own boundaries? What is the role and obligation of the international community? Humanitarian agencies are often caught between waiting for a government or an ‘official’ process to declare an emergency and the humanitarian imperative to push ahead with a response. All of this leads to considerable pressure on data-collection and analysis processes that the system has been extremely challenged to handle.
Research Questions
This study came to largely similar conclusions about the impact of the word ‘famine’, but the point of this study was not just to confirm the intrusion of politics into humanitarian analysis but to understand the dynamics of the politisation of data collection and analysis, and to suggest better ways of managing these intrusions – ridding the humanitarian information system of political interference is simply not a realistic goal. Four main questions drove the research: the first was about the available data, the reasons for gaps in the data and how missing or poor information is managed. The second was about constraints on data-collection processes. The third was about constraints and influences on analysis processes. The last was about processes for managing the political interference in the analysis of severe humanitarian emergencies. The overall objectives of this study were to understand the constraints to collection and analysis in famines and food-security crises and to suggest methods to ensure independent and objective analysis of humanitarian emergencies.
This study compared six case studies for the availability and quality of information and the independence, rigour and quality of analysis, including Ethiopia, Kenya, Nigeria, Somalia, South Sudan and Yemen (Maxwell and Hailey, 2020). All these countries, with the exception of Kenya, have been subject to famine or near famine conditions in recent years (FEWS NET, 2017). Due to constraints on space, individual cases are not described here but are analysed in detail elsewhere.2 Each included a comprehensive desk review and key-informant interviews with government officials and staff of donor and UN agencies as well as international and local NGOs. In all, 339 key informants were interviewed.3
Given the emphasis on the analysis of famine, much of the study addressed the process of Integrated Food Security Phase Classification (IPC) analysis (IPC Global Partners, 2019), but all information and analysis systems in each country were analysed. IPC/CH analysis is based on information from a number of sources – typically World Food Programme surveys for food security and SMART (Standardised Methods for Assessment of Relief and Transition) surveys for nutrition and mortality information.4
At the core of IPC analysis is a reference table that aggregates information about food consumption, livelihoods, nutrition and mortality outcomes into categories of severity – or ‘phases’. The analysis is intended to follow a ‘technical consensus’, based on the convergence of evidence from various sources to classify the severity of the crisis, and estimate numbers of people affected. These results are transcribed by colour coding to a map of the affected area or country. ‘Contributing factors’ or causes are noted, but IPC is fundamentally a classification system emphasising the degree of severity of current humanitarian status. It includes projections – a snapshot of the future.
The Politics of Information and Analysis: Evidence from Six Cases
The comparative analysis of the six cases includes examination of the constraints and influences on data and data collection, on the analysis process, the impact of these influences and lessons learned for better managing these influences.
Influences on Data and Data Collection
Numerous factors – some blatant and some subtle – put pressure on the independent assessment and information collection of famine or near-famine emergencies, particularly in conflict crises. Some are technical, related to data quality, the timing of data collection, the lack of data sharing protocols and the limited ability to utilise qualitative data. Some are related to resources and the use of analyses as ‘report cards’ on humanitarian response. But some are more political – including direct interference, minders, intimidation of field teams, limiting or prohibiting access, creating real and imagined security obstacles and bureaucratic hindrances.
These come from several sources: governments who do not want the depth of a crisis to be exposed, donors who do not wish to investigate deeply the impact of counter-terrorism restrictions or who expect to see ‘results’ from the money devoted to humanitarian response over the previous period, or agencies who also want the analysis to reflect the positive impact of programmes. Managing influences requires the ability to understand where they come from and the motivations behind them.
Table 1 notes the dominant constraints related to data and information, and the cases in which these significantly limited the analysis.
Data and data-collection constraint, by country case study
South Sudan | Nigeria | Somalia | Yemen | Ethiopia | Kenya | |
---|---|---|---|---|---|---|
Missing data | X | X | X | X | ||
Uneven data quality and reliability | X | X | X | X | X | |
Timing, frequency and coordination | X | X | X | X | ||
Lack of data sharing | X | X | X | X | X | |
Constraints on the collection and use of qualitative data | X | X | X | X | X |
Data are frequently missing from analyses – in particular nutrition/mortality data, population data and displacement data. Mortality information is the most politically charged kind, and very frequently not available because authorities do not permit its collection, especially in conflict-driven crises, such as Yemen, South Sudan and Nigeria. Population information is crucial to turning assessment data (which estimates prevalence and rates) into actual numbers of people who require assistance – the ‘population in need’ (PIN) number that drives decisions after the analysis process. But population information is almost universally subject to significant doubt in extreme emergencies, and lack of baseline data is a significant constraint. Of course, in conflict (all case studies except Kenya) people are displaced and displaced people among the most vulnerable groups. ‘Trapped’ or besieged populations are less understood: in Nigeria, while the number of displaced people was known, considerable political controversy swirled around numbers of people trapped inside Boko Haram–controlled areas. Often nutrition data were missing from the analysis, sometimes because there was no data but also because the existing data did not meet the relatively high standards for nutrition analysis. In some cases, the data were too outdated. Finally, across all cases, food-security information dominates other sectors: health and WASH (water, sanitation and hygiene) insecurity are both drivers and outcomes of food insecurity but are only occasionally included in the analysis.
All parties clearly understand that, without mortality data, nothing can be said about famine. So, if the political pressure is to prevent concrete talk about famine, one certain way of ensuring this is to prevent collection of mortality data (South Sudan) or to argue that mortality data must be collected in such a rush that it is of insufficient rigour and reliability (Nigeria). Other frequently missing pieces of information (population and displacement) made analysis more difficult but weren’t an absolute constraint in the same way that absence of mortality data was.
Data quality is variable across cases and between sectors. Nutrition data have been standardised by SMART methodology, which has clear data quality requirements, standards for enumerator training and field checks. Food-security data are more mixed. In some cases, data quality is limited by the technical capacity of field teams. But time is often a more significant constraint to data collection. New technical guidance (IPC Technical Manual Version 3.0) addresses some of these concerns, but much remains to be improved.
A consistent challenge is the timing and frequency of data collection. In some cases, the timing of data collection and the timing of analysis are so different that data is outdated by the time it is analysed. Food-security information tends to be collected on a seasonal basis in bimodal rainfall areas and once a year elsewhere, and peaks in malnutrition may be different from peaks in food security (Young et al., 2019). The emphasis is primarily on the collection of outcome indicator data, relegating causal information to a secondary (and sometimes quite diminished) place in the analysis. This especially constrains good analysis with regard to conflict and conflict dynamics.
Across the board, agencies are reluctant to share data. In some cases, this has led to major disagreements between parties over how food insecurity should be interpreted (the nutrition sector, for the most part, has data-sharing protocols). Agencies are reluctant for three reasons. First, whoever controls the data controls the narrative – in some cases, the level of competition among information systems can be intense and maintaining control of data provides an edge. Second is the fear of another party interpreting the data differently – and thus publishing contradictory findings and recommendations. Third is the fear that, given the often extremely difficult circumstances under which data are collected (resulting in less than perfect data quality), the agency collecting the data will be attacked over methodological rigour if the data are shared. This is a constraint across all cases, although recent agreements in Kenya have reduced the problem.
Finally, with few exceptions, all contemporary data-collection and analysis protocols are oriented towards the use of quantitative data – no guidelines exist using qualitative information in the analysis, even though sometimes anecdotal evidence or even what can only be described as ‘hearsay’ can sway a judgement about famine. In one particularly egregious example, the detailed field notes of a qualitative assessment (under circumstances where conducting a random-sample survey was out of the question) were denied as ‘evidence’, but an anecdote about a colleague who ‘drove through that district last month and didn’t see any crisis’ was admitted as evidence.
Other concerns related to data and data collection include differences in units of analysis between different components of the analysis, limited sectors of data, limited ability to identify ‘hotspots’ and the emphasis on ‘outcomes’ (at the expense of causal factors). These findings are echoed in studies considering other cases of famine mortality (Office of the UN Deputy Humanitarian Coordinator, 2016), and public health in emergencies (Colombo and Checchi, 2018).
Influences on Analysis
Table 2 depicts the constraints on the analysis process itself, and the cases in which these constituted significant limitations or distortions to the understanding of the overall crisis. These include the extent to which analyses that are intended to be a ‘technical consensus’ (IPC Global Partners, 2019) are actually participatory and transparent (and the role of government leadership of these processes), constraints to the consensus process itself and some of the forms these constraints can take, safeguards in the analysis process (and the extent to which preventing false positives is prioritised over false negatives) and the politics of the numbers of the ‘population in need’.
Analysis issues, by country case study
South Sudan | Nigeria | Somalia | Yemen | Ethiopia | Kenya | |
---|---|---|---|---|---|---|
Limited participation/transparency | X | X | X | X | X | X |
Consensus-based analysis | X | X | X | X | X | X |
• The ‘loudest voice in the room’ problem | X | X | X | X | X | X |
• The ‘Goldilocks’ solution | X | X | X | X | ||
Safeguards in the analysis: the risk of false negatives | X | X | X | X | X | |
The politics of numbers: ‘population in need’ | X | X | X | X |
Famine analyses are intended to be a transparent, participatory process, led by government. But in some cases, some parties may be excluded – usually smaller and local agencies – and the processes may not be as participatory as intended. Managing a participatory process is effectively a coordination task added to an already difficult analysis task. When compounded by attempts to undermine or control the analysis for political purposes, it can overwhelm technical staff charged with leading the process. Whether information systems are government owned and led (as in Ethiopia or Kenya), are IPC/CH partnerships convened by government (South Sudan, Nigeria, Yemen), or were set up as independent units (Somalia), almost all parties agree that information systems should be government-led. One view is that anything less than this amounts to undermining national sovereignty and borders on the worst neo-colonial tendencies of the humanitarian community. However, it can also undermine the independence of the analysis. In fact, how leadership of these systems plays out is highly variable. In countries where conflict or civil war is a major driver of famine or food insecurity and governments are party to those wars, there is a clear conflict of interest in having the same government judging the impact of the humanitarian crises that result.5
In some cases – most strikingly, South Sudan – government-led processes have resulted in entire analyses being quashed, reversed, or kept from publication; groups being expelled from the analysis; and not-so-subtle efforts to sway determinations away from outcomes that might reflect badly on the government. On the other hand, government-led systems in Kenya and Ethiopia function more independently but have their own issues, for instance, during election years, when evidence of a food security crisis might be used by opposition politicians.6 Although the information system for Somalia has long been an independent unit (based in Nairobi), the ownership and management of this unit has become a contentious issue with the government of Somalia.
A fundamental tenet of the IPC is the notion that humanitarian analysis should be a technical consensus: the best analysts in a country garnering the best data, working together to hammer out their best analysis of a complex situation that threatens people’s right to basic needs. But several issues constrain this consensus analysis. The first is the attempt of more powerful actors to control the analysis, labelled the ‘loudest voice in the room’. This phenomenon was observed in nearly every consensus process. In short, some members assert their authority over a consensus-based process and overtly influence the outcome beyond consideration of the evidence. This may be based on the political power of the agency or the reputation or experience of the individual. In some cases, another influential member may be able to pull a consensus process back on track if it is going astray, but more frequently powerful actors influence the analysis towards a particular outcome that is more suited their purposes, facilitated by gaps in the data, poor-quality data and uncertainties about how to use qualitative information.
The second tendency is towards ‘less risky’ outcomes to the analysis. Risks are inherent in famine analysis whether of a humanitarian or a political nature. A frequent accusation is that big agencies are simply trying to protect their reputations, their budgets and their privileges (but also trying to protect pipelines to vulnerable populations against shortfalls). The second risk is of angering a host government: almost without exception, governments do not like to hear the terms ‘disaster’ or ‘emergency’ without very strong evidence (i.e. a full-blown emergency is already happening), and none likes to hear the word ‘famine’. The third risk is that agencies need to manage expectations and reputations vis-à-vis donors. Note that the one party for whom these analyses take place – affected or at-risk populations – do not have a voice at the analysis table. These pressures come to bear on consensus processes leading to an outcome that has been labelled the ‘Goldilocks solution’7 or a politically negotiated outcome to the analysis that is ‘just right’ – that is, all parties can live with it, even if it does not reflect the evidence (for specific examples, see next section) and does not serve the affected population.
There are well-developed guidelines for judging the reliability of quantitative data, and rules for whether data can be admitted into the analysis. In several cases (South Sudan, Yemen, Nigeria), meeting the data requirements for famine is often impossible given the obstacles to data collection and independent analysis. While there are valid reasons for requiring rigorous and reliable evidence for a famine declaration, these stringent requirements (see introduction) all safeguard against the likelihood of a false positive: that is, determining a famine when in fact no famine is occurring. However, this configuration of requirements does little to safeguard against the opposite error of a false negative: failing to declare a famine when one is actually occurring. Indeed, in recent cases of famine or near-famine conditions, much of the evidence has not been of sufficient rigour and reliability to make firm statements. These safeguards against ‘false positives’ have frequently been invoked to prevent any statement – a case of rigorous data and analytical requirements aligning with political interests which prefer that ‘famine’ not be mentioned.
And finally, of course, the pressures to inflate or decrease the number of people in need (PIN) are practically endemic in famines and humanitarian emergencies. The evidence suggests that numbers are often inflated where resource allocation is concerned but may be decreased – at least for certain categories, notably populations in IPC Phase 5 – in the analysis of famine or extreme emergencies. The numbers, of course, have a major impact on Humanitarian Needs Overviews (HNOs) and other donor requirements that have a major influence on resource allocation, the ability of agencies to intervene and governments’ claims of legitimately protecting their citizenry. The PIN number is also sometimes used as a ‘scorecard’, with declining numbers reflecting well on the investment in the humanitarian response plan just completed (or even, in one case, on the state of the conflict).
The Impact of These Influences
These influences manifest in several ways. Frequently, they may appear as technical issues, but addressing the technical solutions may be equally subject to political constraints, and interpretation of analyses is most susceptible to political pressures when the technical quality of the data and the analytical capacity is weakest. In some cases, political interference was direct and flagrant. In other cases, the impact may be more subtle, and some may amount to ‘self-censorship’ on the part of analysis teams.
Table 3 presents some of the main categories of the manner in which the influences are manifested in analyses and analytical processes. Overt government interference included reports being quashed, analyses being stopped and individuals being threatened with deportation (if international) or removed from their jobs (if national government employees). On several occasions, reports from whole analyses were quashed; on many occasions, changes to reports were required. National government technical staff involved in the famine declaration were fired from their government jobs and expatriate analysts threated with expulsion and, in at least one case, directly attacked.
Political constraints and influences, by country case study
South Sudan | Nigeria | Somalia | Yemen | Ethiopia | Kenya | |
---|---|---|---|---|---|---|
Political interference: government, agencies, donors | X | X | X | X | X | X |
Access (and how lack of access is depicted) | X | X | X | X | X | |
Missing information (especially mortality) | X | X | X | X | ||
Numbers in need | X | X | X | X | ||
Self-censorship | X | X | X | X | ||
Right-skewed but truncated population distributions | X | X |
Denial of movement, long delays in approvals and other tactics were reported in case studies. More subtle influences include the way in which access (or inaccessibility) is mapped and influences on PIN numbers. While examples of these manifestations were found across the case studies, the most extreme examples were noted in the South Sudan case study (Maxwell et al. 2018).
A major constraint on data collection – and frequently the reason for missing data – is difficulty accessing affected populations. Access was an absolute constraint in Nigeria and a major constraint in al Shabaab-controlled areas in Somalia. In other cases, access constraints are not permanent, but delays in getting permissions often meant that the analysis of a given situation was incomplete – and the missing geographies were the very places where conditions might logically be expected to be most severe. When permissions for assessments are not granted, it is frequently not clear if the reason is genuinely because of insecurity or whether insecurity is a convenient excuse. When areas cannot be accessed, there is no standard means of analysis. Sometimes no attempt is made to analyse, and the area is mapped to show it has not been analysed (usually coloured grey). In some cases, inaccessible areas have been given the same classification as adjacent areas, which may misleadingly under-classify those areas. In Somalia, key-informant interviews are the main source of information on al Shabaab-controlled areas – which are mapped the same way they would be if based on survey data.
The other major source of interference is over the PIN numbers; in some contexts numbers are unquestionably politically negotiated, not evidence-derived. Ethiopia is probably the most extreme case with regard to the politics of numbers (Desportes et al., 2019; Kimetrica, 2020) but the PIN is contentious in many cases. And of course, without detailed knowledge of the behind-the-scenes politics, it is difficult to determine if the numbers are being pushed upwards (usually in search of greater resource allocation) or downwards (usually to minimise the extent of a crisis and improve appearances to domestic and international political stakeholders).
Agencies, too, sometimes have interest in the numbers, because resources are tied to the reported PIN number. This usually doesn’t result in flagrant attempts to manipulate the numbers, but many respondents noted a general sensitivity about numbers even in humanitarian agencies. Donors on the other hand, may be sceptical and push back on analyses, but they have their own interests in numbers. Analysts were aware of these pressures in all the case studies, but these pressures do influence analyses. As the Nigeria case study report noted: ‘If the outcomes of the Cadre Harmonisé don’t improve, donors will question the impact of the ongoing response – perhaps endangering future programs. On the other hand, if Cadre Harmonisé outcomes improve too much, it would support the conclusion that the crisis has abated and be a reason for scaling back the response’ (Maxwell et al., 2018: 29).
Perhaps the most insidious way in which these pressures manifest themselves is through self-censorship on the part of analysis teams. In some cases, analysts began to change the analysis to deflect criticism or push-back from the politically powerful. Some were intimidated by people with ‘the loudest voice in the room’, some simply for self-preservation: these teams not only have to think about the data in front of them, but also of future access to the field for assessment, future streams of funding, their own security and their own sanity. Self-censorship manifests in various ways, including delaying data collection, revising schedules or protocols, or not pushing back very hard on denials of access. Analysts may simply avoid extremely sensitive areas or topics of conversation in the analysis. Or they may not push back with evidence to the contrary when it comes to numbers – particularly if dealing with armed groups. There are several additional ways in which self-censorship is manifested and several responses to it. The most graphically evident case of apparent self-censorship is the ‘Goldilocks’ solution outlined above – a left-skewed but truncated distribution of population by IPC category (referred to by some field analysts as ‘over-loading Phase 4’).
Populations in IPC Phase 3 are deemed to require food assistance, though not as urgently – particularly in a resource-constrained situation. Phase 5 implies famine or famine-like conditions, which raises political problems with governments. A frequent compromise ‘consensus’ is to put a large part of a population into Phase 4, but none in Phase 5 – resulting in an oddly skewed distribution of population (Figure 1).
‘Left-skewed/truncated’ distributions of population of selected counties, by IPC phase classification (South Sudan, various years)
Source: Author’s analysis, data from South Sudan IPC Technical Working Group.
Citation: Journal of Humanitarian Affairs 3, 1; 10.7227/JHA.055
While there is no a priori expectation that the distribution of people across IPC phases are normally distributed, the graphs on the left side of Figure 1 (a, b, c) give examples of distributions that might be expected. For example, under relatively ‘normal’ conditions, a ‘right-skewed’ distribution might be expected (1a), with declining proportions of the population in each higher phase. In a crisis situation, some kind of ‘central tendency’ across several phases might be expected (1b), with a small proportion of the population in Phase 5. In very severe situations, a ‘left-skewed’ distribution with increasing proportions of the populations in each higher phase might be anticipated (1c), as occurred in Leer county in South Sudan in early 2017. Figures 1a–c are all actual population distributions from South Sudan.
The distributions on the right side of Figure 1 (d, e and f) depict increasing proportions of the population in each higher phase, until Phase 5, where no population is noted. This gives a ‘left-skewed but truncated’ distribution of the affected population. This was first noted in South Sudan with regard to several cases in 2015–16 but continued even afterward. In theory, population distributions could be expected in any of the depictions on the left side of Figure 1, but, it is highly unlikely that the distribution would be left-skewed but then truncated at Phase 5: the only feasible explanation is that extremely well-targeted food assistance is going only to the absolutely most vulnerable – a situation well-known not to exist in South Sudan (Maxwell and Burns, 2008). Several dozen cases of this distribution were noted over several years in South Sudan. In Yemen in 2018, nearly half the districts analysed for famine risk were depicted with this kind of population distribution (more illustrative examples are in Figure 2).
‘Left-skewed/truncated’ distributions of population of selected districts, by IPC phase classification (Yemen, 2018)
Source: Author’s analysis, data from Yemen IPC Technical Working Group.
Citation: Journal of Humanitarian Affairs 3, 1; 10.7227/JHA.055
Since there is no alternative measure against which to compare these distributions, the only thing that can be said is that they are highly unlikely – and are likely underestimating the gravity of the situation. But this distribution serves everyone’s interest whether it is correct or not: it suggests a sufficient degree of severity (Phase 4) to keep funds flowing, while avoiding any talk of famine (Phase 5) and the political consequences that might bring. The strong safeguards noted above against false positives tend to reinforce the tendency towards this kind of outcome.
This is an example of the kind of data-quality check needed for standardised analysis. Of course, in the absence of alternative or comparative data, nothing can be said except that this is a highly unlikely distribution that, even if a few cases exist, would not exist in half of the districts analysed. At a minimum it would suggest the need for a re-assessment of the data to look for biases that might result in this kind of distribution.
Skilled and committed individuals and teams are working on these problems, and some good practice and emergent possibilities for mitigating or managing these pressures have been noted (and even suggested) in the course of this study. The next section briefly reviews some of the most salient ones.
Lessons Learned: Emerging Good Practice to Manage Political Influences
Several categories of good practice emerge from the evidence across the six case studies. These include strengthening technical capacity, clarifying the role of government, developing protocols for data sharing, broadening participation, building buy-in at higher levels, the use of qualitative information, incorporating innovative ideas in analysis and new technology. Table 4 summarises these by case study.
Managing the influences, by country case study
South Sudan | Nigeria | Somalia | Yemen | Ethiopia | Kenya | |
---|---|---|---|---|---|---|
Improving capacity (analysis teams and government) | X | X | X | X | X | X |
Clarifying the role of government | X | X | X | X | X | |
Sharing data | X | X | X | X | X | X |
Broadening participation | X | X | X | X | X | |
Minority reports ‘speaking outside the consensus’ | X | X | X | |||
Maintaining ‘two sets of books’ | X | X | X | |||
Building buy-in and support at higher levels | X | X | X | |||
Improving causal analysis/use of qualitative information | X | X | X | X | ||
Integrating the analysis | X | X | X | X | X | |
Trying new technology? | ? | ? | ? | ? | ? | ? |
This study observed that political influences are the most flagrant where the data collection and the technical capacity of analysis teams are the weakest. Therefore, it makes sense to focus on strengthening capacity. This is already happening in many ways: new versions of technical guidelines are being developed and rolled out;8 new organisations have appeared in recent years devoted specifically to information capacity (ACAPS, 2020, REACH, 2020) and established organisations are developing new initiatives to improve information and analysis (World Bank, 2019). But these efforts face obstacles: staff turnover is high so investment in staff training may be short-lived, motivation may be mixed and inevitably new challenges arise for which capacity and technical guidance do not yet exist. Much work also remains to be done to build the skillset around consensus building, ensuring participation and other ‘soft’ skills, but the rising tendency to simply create ‘alternative narratives’ undermines much of the attempt to reinforce technical skills.
The role of national governments in humanitarian information systems varies and, in some cases, should be reviewed. The normative view is that governments should lead these processes. Indeed, the initial purpose of the IPC/CH process was both to provide a consensus analysis and to build the capacity of government to lead that analysis. But as noted, this becomes a much more complicated issue where governments are party to conflicts that are at least one of the causes of the emergency. Even when government is not directly involved in a conflict that is driving famine risk, different levels of government may try to alter the numbers in need or in other ways change an analysis. These are not technical questions or an issue of capacity but, ultimately, must be addressed to protect independent analysis and must have the buy-in and support of all parties. The relationship with government needs to fit the context (Buchanan-Smith et al., 2019).
Some analyses are seriously crippled because data is not shared. Nearly all parties have a policy of ‘data transparency’ but those policies don’t specify a time frame for this transparency – sometimes it can be months or years before a data set is available for public use, by which time its humanitarian usefulness is limited to retrospective analysis. To be useful for humanitarian analysis, data has to be shared in near real time. Various interim arrangements have been worked out for at least some modicum of data sharing in the field. In some cases, staff from different organisations agreed to work together on real-time analysis. In other cases, donors have brought pressure on agencies to share data. Better data transparency and sharing of data cleaning among various partners in the analysis would mean that the analysis could be cross-checked. This may not be in the short-term interests of some parties (who have an interest in controlling a specific narrative) but would be in the long-term interests of all because it would strengthen observers’ trust in the analysis. This is one area where donors can help by specifying the time period within which data must be available to other analysis teams before assessments are funded. The Nutrition Cluster already has protocols to this effect for SMART surveys (Michalska et al., 2015).
Levelling the playing field to broaden participation is of paramount importance because most of these processes are built on consensus. The 2019 evaluation of the IPC Global Strategic Programme (Buchanan-Smith et al., 2019) showed that the collective/consensus-based nature of IPC analysis is its greatest asset. However, political influences affect the results of the analysis when technical teams can’t manage participation well. Ensuring a wide base of participation and ensuring that not only the specialised or large agencies have a ‘voice’ in the process, counterbalances the ‘loudest voice in the room’ phenomenon. In the same vein, nutrition working groups must be fully integrated into IPC processes, especially as nutrition and mortality hotspots are often not the same as food-security hotspots. While analysis is intended to include staff from affected locations, there remains no participation in either data collection or analysis from the actual affected populations.
In some cases, a party to the analysis either issued a separate analysis or registered a minority opinion – ‘speaking outside a consensus analysis’. It may be the only means of avoiding censorship, but involves considerable risk. In Nigeria in 2016, although CH analysis had not suggested that a famine occurred, FEWS NET analysed the rapid assessments that took place in the first weeks after several towns were recaptured from Boko Haram and concluded that a famine had likely been occurring in those towns at the time they were recaptured by the army – and that famine was likely continuing inside the Boko Haram-controlled territory (FEWS NET, 2016). Despite the intent of consensus analysis, ‘speaking outside the consensus’, while annoying to some, is actually important under some circumstances. In some countries, regular sessions have been instituted to evaluate the independence and rigour of the analysis process once the actual process is complete. This provides opportunities to learn from and help correct technical mistakes and overcome political influences. At a minimum, some kind of alternative is required to consensus-based analysis processes in the event that the ‘consensus’ itself is undermined by external influences.
In several cases, both analysts and some donors openly admitted that they maintain two sets of ‘books’ or figures on populations in need – one that is ‘official’ and can be talked about publicly and the other private, but which contains one’s best estimates as to ‘real’ figures. While the ‘official’ figures should be the ‘real’ figures, where political pressures distort official figures, keeping a second ‘set of books’ may be the only way to try to retain some sense of reality. This practice was noted in several countries, but has a long history in Ethiopia (Burg, 2008; Kimetrica, 2020). Openly keeping two sets of books might endanger individuals, but might eventually pull the ‘official’ and the ‘real’ together into the same set of data – and thus the same analysis, especially if it is clear that donors are doing it.
Technical managers of data-collection and analysis agencies and teams already have enough problems on their plates, and they need better buy-in and support at higher levels. Where political challenges need to be managed, having the buy-in and support at the higher levels of the humanitarian system is essential. In contentious IPC analyses, having support or presence from the Global Support Unit during the analysis process has proven helpful (Buchanan-Smith et al., 2019).
Much of current analysis relies heavily on outcome data and virtually entirely on quantitative data. Better incorporation of qualitative information – about the context but especially about drivers or contributing factors – improves the analysis. Statistical information is critical to classification of severity, but other kinds of analysis discussed here require different kinds of information – particularly causal information. And finally (and perhaps most problematically), incorporating conflict analysis into the process would improve projections so that better prevention and mitigation efforts could be launched. Limiting the incorporation of conflict analysis is one form of political influence.
Despite efforts to continuously improve the technical analysis, inevitably situations arise for which existing means of analysis are inadequate. Improvisation is viewed as the antithesis of ‘rules-based’ analysis, but some room must be left in the process for allowing new or different means of analysis. Examples include famine analyses where there are well-founded fears that famine might be occurring, even though it couldn’t be declared following strict ‘rules’. Rapid assessment methods have been devised for situations of extremely limited access or for conducting analyses outside of the usual timetable of seasonal analysis. Other attempts to address access constraints include greater use of remote sensing, Delphic processes, or other innovations including predictive analytics, machine learning and artificial intelligence.
To many observers, if humans are the source of political interference, analysis that has less human participation might be less political, more accurate and more trustworthy. This has led to several attempts to automate analysis and rely more heavily on public sources of data. New technologies are competing to improve humanitarian analysis, including the analysis of famine. But these also carry risks and costs. Many are equally if not more data hungry than current approaches. New technologies may be highly extractive or introduce biases of their own (Hernandez and Roberts, 2020). So, while new technologies are one additional tool for improving analysis, they are not necessarily a panacea (Lentz et al., 2020).
Towards Rigorous and Independent Analysis
Despite all the problems enumerated, progress has been made in the past decade and a half in building evidence-based responses to famine and extreme humanitarian emergencies. Much of this has been accomplished by improving, streamlining and regularising methods of data collection and analysis. Nevertheless, numerous parties have an interest in shaping, influencing and sometimes blocking or suppressing information about these emergencies for political reasons. A recent evaluation of IPC (Buchanan-Smith et al., 2019) noted that field teams are often not able to manage the politics that have been noted above – hence the need for all parties to agree on measures to address these major influences. The threats can come from nearly all quarters – governments and armed opposition groups, donors and humanitarian agencies and in some cases local government or representatives of affected communities. In many cases, the word ‘famine’ is forbidden. Attempts to influence resource allocation also means influencing both analysis and data-collection processes.
In all these cases, humanitarian information systems are operating in a highly political environment. To think that the analysis of such crises can take place in a completely independent and influence-free environment is unrealistic. Far more practical is the search for better methods to manage the politics, rather than trying to erase them altogether. Expert analysts in food security and nutrition are not necessarily equipped to also manage political tensions. Higher-level leadership within the humanitarian community must provide the space for technical experts to do their job. In other words, political tensions with government officials need to be defused; addressing these tensions should be the task of UN Humanitarian Country Team or agency leadership, but an additional independent check may be needed (external and independent technical checks are already built into the system, in the form of the Famine Review Committee or a Real Time Quality Review (IPC Global Partners, 2019).
In the declaration of famine or other extremely severe emergency, a balance must be struck between the fear of a false positive (declaring a famine or emergency when there actually isn’t famine – or when an emergency isn’t that serious) versus the fear of a false negative (failing to declare a famine or emergency when there actually is one). There are direct trade-offs between these two – protecting against one necessarily means a higher risk of the other. Current systems tend to prioritise safeguarding against false positives. However, the risk of a false negative is much higher in humanitarian terms – potential loss of life, livelihoods and dignity. In recent years, a ‘no regrets’ approach has been promoted to prevent the risk of false negatives, but even a ‘no regrets’ approach is dependent on having good information and independent analysis.
Famine and extreme emergencies have multiple causes, but conflict is the common thread among causes of contemporary famine. Information on conflict is frequently either missing completely or else relegated to a brief mention as a ‘contributing factor’. However, considering that conflict information or analysis is also the most highly politicised, it is the most problematic to include (Lentz et al., 2020).
Famine analysis is dominated by food-security and nutrition information. However, given the predominance of the analytical processes for famine and food-security crises, these types of information tend to dominate humanitarian analysis more generally. In general, information on health, WASH, displacement and other sectors or outcomes is less available. Where such information is available, uncertainties exist on how to use it in the analysis.
The process of building technical consensus is a major strength, but a better understanding of what ‘consensus’ means must be built: it does not mean ‘unanimity’; nor does it mean a conclusion forced by the most powerful party to the analysis. And the system needs to incorporate a means of dissent, and a process to resolve disputes. System learning is evident in these analytical processes, particularly where it is specifically fostered. This is constrained by high turnover in the staff who collect data in the field and/or conduct the analyses; but in many ways high turnover and other constraints only emphasise the need for system learning and documentation.
More study is needed on how machine learning can be used to supplement or complement conventional forms of data collection and analysis, but care must be taken to ensure that these processes don’t simply replace one kind of influence with another.
While doing away with the politics of famine is probably impossible, recommendations from this study could improve analysis by reducing the influence of political influences on data collection, analysis and interpretation. Humanitarian food-security and nutrition analysis in general has improved markedly over the past decade (Choularton and Krishnamurthy, 2019). Nevertheless, analyses continue to be influenced by political factors. This study took place over the course of three years and included efforts made in each country case study to provide detailed feedback to all stakeholders in the assessment and analysis process. Over the course of the study, analysis teams have already begun to implement some of the recommendations but many of these practices still need to be strengthened. Those who lead these analyses, fund them, rely on them to make decisions must work to minimise the influences on independent and rigorous analysis.
Notes
Colombo and Checchi (2018: 214), emphasis added.
For the individual case study reports, see: https://fic.tufts.edu/research-item/the-constraints-and-complexities-of-information-and-analysis/
The Tufts University Social, Behavioral, and Educational Research Institutional Review Board granted clearance for the overall research programme on 31 May 2017, renewed on 25 May 2018 and renewed again on 24 May 2019.
Although this study involved examination of IPC analyses, it was not an evaluation of the IPC.
Fear about the way in which systems are led, particularly in conflict emergencies in which governments are party to the conflict, was one of the factors that led to this study.
This paper was written and submitted prior to the outbreak of war in Ethiopia’s northern region of Tigray.
Based on the nineteenth-century English folktale about a little girl who inadvertently enters the home of three bears while they are away, and finds the porridge, bed and chair of the parent bears too extreme (too hot, too cold, too hard, too soft) but finds baby bear’s amenities to be ‘just right’.
Notably Version 3.0 of the IPC Technical Manual (IPC Global Partners, 2019).
Works Cited
ACAPS (2020), South Sudan Analysis Ecosystem (Geneva: ACAPS).
Bailey, R. (2012), Famine Early Warning and Early Action: The Cost of Delay (London: Chatham House).
Buchanan-Smith, M., Mohamed, S. and Carrazon (2019), Final Evaluation of the Integrated Food Security Phase Classification (IPC) Global Strategic Programme (GSP) – 2014–2018 (Rome: FAO).
Burg, S. (2008), ‘Fixing Famine: The Politics of Information in Famine Early Warning’ (PhD dissertation), University of California, San Diego.
Choularton, R. and Krishnamurthy, P. K. (2019), ‘How Accurate Is Food Security Early Warning? Evaluation of FEWS NET Accuracy in Ethiopia’, Food Security, 11:2, 333–44.
Clay, J. and Holcomb, B. (1985), ‘The Politics of Famine in Ethiopia’, Cultural Survival Quarterly Magazine (June).
Colombo, S. and Checchi, F. (2018), ‘Decision-Making in Humanitarian Crises: Politics, and not only Evidence, Is the Problem’, Epidemiologia e Prevenzione 42:(3–4), 214–25.
Desportes, I., Mandefro, H. and Hilhorst, D. (2019), ‘The Humanitarian Theatre: Drought Response during Ethiopia’s Low-Intensity Conflict of 2016’, Journal of Modern African Studies 57:1, 31–59.
de Waal, A. (1997), Famine Crimes: Politics and the Disaster Relief Industry in Africa, African Issues (London: James Currey).
de Waal, A. (2018), Mass Starvation: The History and Future of Famine (Cambridge: Polity Press).
Dikötter, F. (2010), Mao’s Great Famine: The History of China’s Most Devastating Catastrophe, 1958–1962 (London: Bloomsbury).
FAO/WFP (2020), ‘2020 Global Report on Food Crises’, FAO/WFP, Rome.
FEWS NET (2016), ‘Special Alert on Borno State, Nigeria’. Evidence and Standards for Better Food Security Decisions, December 16, FEWS NET, Washington DC.
FEWS NET (2017), ‘Food Assistance Outlook Brief, May’, FEWS NET, Washington DC.
Fiori, J. (2019), ‘Humanitarianism and the End of Liberal Order’, Journal of Humanitarian Affairs 1:1, 1–3.
Hernandez, K. and Roberts, T. (2020), Predictive Analytics in Humanitarian Action: A Preliminary Mapping and Analysis, K4D Emerging Issues Report 33 (Brighton, UK: Institute of Development Studies).
Hopgood, S. (2019), ‘When the Music Stops: Humanitarianism in a Post-Liberal World Order’, Journal of Humanitarian Affairs, 1:1, 4–14.
Howe, P. and Devereux, S. (2007), ‘Famine scales: Towards an Instrumental Definition of “famine”’, in S. Devereux (ed.), The New Famines: Why Famines Persist in an Era of Globalization (London: Routledge), pp. 27–49.
IPC (2020), South Sudan: Consolidated Findings from the IPC Technical Working Group and External Reviews (Rome: Global Support Unit for Integrated Food Security Phase Classification), www.ipcinfo.org/fileadmin/user_upload/ipcinfo/docs/South_Sudan_Combined_IPC_Results_2020Oct_2021July.pdf (accessed 15 February 2021).
IPC Global Partners (2019), The Integrated Food Security Phase Classification Technical Manual Version 3.0 (Rome: FAO).
Kimetrica (2020), Phase II Report. Ethiopia Early Warning System Enhancement: Executive Summary (Nairobi: Kimetrica).
Lautze, S. and Maxwell, D. (2007), ‘Why do famines persist in the Horn of Africa? Ethiopia, 1999–2003’, in S. Devereux (ed.), The New Famines: Why Famines Persist in an Era of Globalization (London: Routledge), pp. 222–44.
Lentz, E., Gottlieb, G., Simmons, C. and Maxwell, D. (2020), 2020 Hindsight. The Ecosystem of Humanitarian Diagnostics and Its Application to Anticipatory Action (Boston, MA: Feinstein International Center).
Maxwell, D. and Burns, J. (2008), Targeting in Complex Emergencies. South Sudan Case Study (Medford, MA: Feinstein International Center).
Maxwell, D. and Hailey, P. (2020), The Politics of Information and Analysis: Humanitarian Information Systems and Analysis in Famines and Extreme Emergencies (Medford, MA: Feinstein International Center, and Nairobi: Centre for Humanitarian Change).
Maxwell, D. and Majid, N. (2016), Famine in Somalia: Competing Imperatives, Collective Failures, 2011–12 (London: Hurst Publishers).
Maxwell, D., Hailey, P., Kim, J. J., McCloskey, E. and Wrabel, M. (2018), The Constraints and Complexities of Information and Analysis in Humanitarian Emergencies: Evidence from South Sudan (Medford, MA: Feinstein International Center, and Nairobi: Centre for Humanitarian Change).
Michalska, A., Leidman, E., Fuhrman, S., Mwirigi, L., Bilukha, O. and Basquin, C. (2015), ‘Nutrition Surveillance in Emergency Contexts: South Sudan Case Study’, Field Exchange, August, 50: 73.
Office of the UN Deputy Humanitarian Coordinator (2016), Crisis Impacts on Households in Unity State, South Sudan, 2014–2015: Initial Results of a Survey (Juba: Office of the UN Deputy Humanitarian Coordinator).
O’Grada, C. (2015), Eating People Is Wrong, and Other Essays on Famine, Its Past, and Its Future (Princeton, NJ: Princeton University Press).
REACH (2020), ‘Humanitarian Situation Monitoring’ (Monthly) (Juba: REACH).
Vaux, T. (2001), The Selfish Altruist (New York: Earthscan).
Washington Post (2017), ‘More than 20 Million People Are at Risk of Starving to Death. Will the World Step up?’, 17 July, www.washingtonpost.com/opinions/global-opinions/more-than-20-million-people-are-at-risk-of-starving-to-death-will-the-world-step-up/2017/07/17/c7d7703c-6b14–11e7-b9e2–2056e768a7e5_story.html (accessed 15 December 2019).
Winston, B. and Winston, M. (2020), The Roots of Fake News: Objecting to Objective Journalism (London: Taylor and Francis).
World Bank (2019), The Famine Action Mechanism (FAM). Famine Action Plans – Overview and Process of Engagement (Washington DC: World Bank).
Young, H., Marshak, A. and Venkat, A. (2019), Twin Peaks: The Seasonality of Acute Malnutrition, Conflict, and Environmental Factors (Medford, MA: Feinstein International Center, and Nairobi: Centre for Humanitarian Change).