Introduction
In 2017, at the opening event of The Centre for Humanitarian Data (or Centre for Humdata) in The Hague, Mark Lowcock, the former United Nations Under-Secretary-General for Humanitarian Affairs, praised the potential of quantitative data:1
We have the opportunity to see things as they are happening, but also crucially to predict what’s going to happen next … that is where we see a really big opportunity for using data: better solving of humanitarian problems. If we use the techniques of predictive data analytics, we will be able to work out the next problem before it crystallizes and then we will act faster, cheaper, better, we will protect more people and we will do it cheaper … I think it is a fantastic challenge for humanitarians to engage with.
The promise of more efficient action resulting from the use of quantitative data, often referred to as ‘evidence-based humanitarianism’ (EBH), is not new. In fact, it is one of the reasons why quantification pervaded the humanitarian field in the first place, notably through the ‘need-system’. Yet, the hope associated with the use of quantitative data has clearly grown these last few years, and a more innovative and extensive EBH has been encouraged.
The expansion of EBH relies on two main elements. First, as mentioned by Mark Lowcock, technological advances are crucial to developing EBH. The Centre for Humdata is at the forefront of this trend. It makes data more accessible through open platforms, such as the Humanitarian Data Exchange (HDX). It also provides data training to humanitarian workers and develops predictive data analytics. Second, the multiplication of quantitative data sources also plays a huge role. For centuries, the data used for humanitarian actions were mainly first-hand data, that is, data collected by the humanitarian organisations’ own means and resources. However, as second-hand data, that is, data collected by other actors (international organisations, institutes, think tanks) have been made more and more available, they have become new potential materials for humanitarian workers.
As part of the DATAWAR research project, this article focuses on a specific object: quantitative data coming from armed conflict databases (ACDs). On the one hand, quantitative data are considered as information that can be counted or measured (‘quantified’) and given a numerical value. On the other hand, ACDs were defined by the project members as open-source databases that can act as sources of quantitative data on issues directly related to conflicts, including, for example, those documenting the direct consequences of conflict (number of deaths, injuries, violent events), those addressing the environment of conflict (defence expenditures, military capabilities), and those marking the risk of conflict itself (early warning ranking systems, risk indicators).
This particular focus is explained by the fact that many institutes, think tanks and political institutions have, since the beginning of the twenty-first century, taken a greater interest in quantitative studies (De Franco and Meyer, 2011; Ward et al., 2013; Colonomos, 2016; Meyer et al., 2019).2 These ACDs are presented as a means to help practitioners make better decisions, a decisive asset for apprehending the international environment. Some have developed specific expertise in conflict-related data, such as the Armed Conflict Location & Event Data Project (ACLED), which focuses on producing data on political violence, and the Military Balance+ of the International Institute for Security Studies (IISS), which offers detailed data on military capabilities. Others have implemented specific tools to allow for a kind of ‘objective’ comparison between states in the international arena, such as the Global Peace Index, the Fragile States Index, and the Index for Risk Management (INFORM) from the European Union. Considering the greater dissemination of conflict-related data, especially with open-source availability, and the influence of EBH in the humanitarian field, members of the DATAWAR research project assumed that these ACDs could have been incorporated into humanitarian workers’ practices. Here, humanitarian workers are considered as employees of humanitarian organisations involved, in any way, in the organisation’s international operations (helping, from the organisation’s headquarters or on the ground, people affected by man-made and natural disasters like wars, outbreaks of disease, floods or earthquakes). Thus, this study’s aim was twofold: gauging whether these initiatives were attractive to practitioners,3 and if yes, what were the effects on their perception of the situation.
However, our result points in the opposite direction. Starting with the limited use of ACDs, it expands more broadly on how quantitative data are considered by humanitarian workers. The conclusion qualifies (for the time being) Mr Lowcock’s ambitions: EBH may still have a long way to go before becoming embedded in humanitarian actors’ practices.
In fact, we argue that the situation has not changed much since the introduction of quantitative data within the humanitarian field: quantification mainly serves bottom-up accountability for decision-makers and donors, rather than day-to-day project management. This study discusses three main reasons to explain this limited use of data. First, there is a criticism of the need-system among humanitarian workers, who consider it as a control tool implemented by donors. As a result, they don’t want to expand the logic of quantification. Second, there is a lack of resources for humanitarian workers to truly become data literate. Without this data literacy, much of the data produced externally won’t be used by these workers. Third, there is a broader refusal of the quantification of humanitarian practices, both in its substance and in its modalities.
This article is divided into four parts. The first part provides a brief background on the quantification of the humanitarian field to show how quantitative data have become so prevalent in this context. The second part explains the methodology chosen and details our main materials for this study. The third part offers an assessment of the quantitative data mobilised by humanitarian workers and shows how it remains orientated towards the need-system. The fourth and final part tries to untangle the three factors blocking the use of quantitative data mentioned above.
On the Importance of Quantification within the Humanitarian Field
Understanding the way quantitative data have become central in humanitarian practices is the first step to better grasp their current uses – especially since the use of quantitative data in this field is not new. Traces of quantitative data and the use of statistics can be found at the beginning of contemporary humanitarian practices, that is since the nineteenth century. What has changed in the last century, however, is the importance that quantitative data have been given: as Glasman argues, ‘during the twentieth century, decisive shifts took place in the role played by humanitarian numbers – their producers, their scale, and their use for justifying action’ (2020: 5). Given the link between evidence and action in EBH, the use of data in justifying action is of specific interest here.
Indeed, EBH relies on the ambition to make ‘informed and responsible decisions’, that is, decisions that are based on ‘reliable’, ‘timely’ (UN Secretary-General, 2017: 14) and objective data. As a result, it optimises ‘the efficiency, suitability and flexibility of logistics’ (Lawson, 2021: 57) surrounding humanitarian interventions. This discourse has slowly become prevalent since the 1990s and one of its main assumptions is that reliable and objective data are often synonymous with quantitative data: as Glasman highlights, ‘“evidence-based” humanitarianism ha[s] mainly been interpreted in a narrow way; “evidence” meaning only “numbers”, and “data” only “quantitative data”’ (2020: 248).
Three main factors seem to have driven this desire for quantitative data: the emergence of ‘needology’, which stems from political economy, health policy and the post-Second World War context (Glasman, 2020); the ‘technocratic turn’ experienced by the third sector from the 1980s onwards (Read et al., 2016), which has resulted in the adoption of private-sector logics; and finally, the emergence of new technologies and the possibilities associated with them. These factors reinforce each other.
At the end of the Second World War, quantification was presented as an advantage to incorporate the logic of ‘needs’, which quickly resulted in the adoption of mathematical standards as a prerequisite for any humanitarian intervention (Glasman, 2020). The adoption of private-sector logics, based on the rationale of new public management (NPM), then reinforced the role given to numbers by placing them at the centre of humanitarian practices. This is especially true regarding matters of diagnosis, monitoring and evaluation (DME). NPM places a strong emphasis on outcomes, which is often synonymous with the adoption of numerical targets and indicators to evaluate the efficiency of humanitarian practices. What is more, the growing expectations of donors since the 1990s, notably because of the failure of certain humanitarian development projects, has put ever-increasing pressure on humanitarian organisations to account for their actions. As Read et al. explain, this ‘more corporate orientation was reinforced by increased pressure from donors and publics for efficiency and transparency, and by a developing political economy of competition between organisations for “market share”’ (2016: 5). Finally, this efficiency supposes constant adaptation from humanitarian organisations, hence the role played by the adoption of new technologies. The requirement for adaptation is even more central knowing the context in which organisations intervene:
Both the day-to-day and strategic issues facing humanitarian organisations make technological ‘solutions’ attractive. These issues include the need to collect information in hazardous and hard-to-access areas; to make decisions quickly in situations of imperfect information; to coordinate and sequence with other service providers; to be seen to be effective and compete for ‘market share’; to manage increasingly complex back-office operations; and to manage increasing demands for transparency internationally and in the area of operations. (Read et al., 2016: 6)
Thus, the logic of quantification in humanitarian practices is both specific to the humanitarian field – because of the emergence of the need-system – and the result of broader, external dynamics, such as NPM and technological advances.
The contemporary use of quantitative data is therefore central in humanitarian decision-making (Campbell and Knox Clarke, 2019; Lawson, 2021). The literature shows how numbers are used to legitimise humanitarian interventions and confer trust to the organisations producing and communicating them (Barnett, 2013; Jacobsen and Fast, 2019), notably through the need-system. It certainly echoes what ‘quantitative politics’ have highlighted concerning the importance of numbers in contemporary societies and their use for governance. In this regard, the trust humanitarian workers seem to put in numbers is embedded in a broader societal transformation linked to the privileged status of quantitative data in contemporary societies (Porter, 1995; but see also Supiot, 2015 or Desrosières, 2013a, 2013b, 2014).
One of the most prominent findings on the consequences of quantification in humanitarianism is what has been coined as the shift from ‘proximity to distance’ (Olivius, 2016; Beerli, 2017; Lokot, 2019). This refers more precisely to the way a humanitarian worker becomes, through the application of quantitative logics, ‘a distant, detached observer who creates knowledge based on the application of standardized models rather than on experience in the particular local context’ (Olivius, 2016: 280). What is more, some observers agree that these new practices have led to a new kind of humanitarianism called ‘digital humanitarianism’ (Burns, 2014, 2015, 2019; Benton and Glennie, 2016; Jacobsen and Fast, 2019; Roth and Luczak-Roesch, 2020; Rothe et al., 2021). A slight nuance should however be underlined. As Glasman admits, the role played by new technologies in humanitarian practices remains far from smooth:
[I]f there is really an affinity between the numbers and high technology, the two are far from being synonymous. Obviously the Excel spreadsheets, the databases, the cell phones, the emails, and the clouds. But most humanitarian quantification is not ‘digital’ at all. (2020: 200)
That being said, the trust of ‘data optimists’ (Fast, 2017) regarding humanitarian practices and quantitative data has not diminished. On the contrary, it continues to grow. Arguments supporting this optimism are addressed by only a handful of humanitarian organisations and international institutions, often major producers and consumers of quantitative data. The recent emergence of data-specialised non-governmental organisations (NGOs), such as CartONG, ACAPS, MapAction, REACH, IMPACT and iMMAP, has played a key role in the spread of this dynamic through the production of reports and factsheets. This literature, based on the premise that quantitative data is an asset, aims to enhance the development of EBH. For instance, it provides advice on information management and inventories what could be done with current data and how it could be better managed. Following the objectives of the Centre for Humdata, the ambition is to persuade practitioners that building data skills and strengthening the data literacy of humanitarian workers is necessary. However, it also shows that humanitarian organisations may still have a long way to go to fully embrace the potential associated with quantitative data.
The current literature does not allow us to fully grasp the real importance of quantitative data within the humanitarian field. First, a careful investigation of the type of data used by humanitarian workers is clearly missing. Defining a scope of use would make it easier to apprehend the place quantitative data is accorded in daily humanitarian practices. It would also help gauge humanitarian workers’ data literacy. Second, there is a lack of understanding of humanitarian workers’ perception of quantification. As referred to above, current research has mainly focused on what quantification does to humanitarian practices, not on what humanitarian workers may feel about it. Yet, if these workers are reluctant to the logics and effects of quantification, it could diminish the importance they give to quantitative data and thus to the development of EBH. This study is a first attempt to fill those gaps.
Methodology and Materials: Annual Reports and Practitioners’ Interviews
Several questions have guided this research: which quantitative data seem to be useful to humanitarian workers? Are these mainly first-hand data or second-hand data? Why are these data most helpful? How is the relationship with external knowledge organised? And what does all of this say about the broader system of ‘evidence-based humanitarianism’?
To answer these questions, we have brought together two main materials. First, we undertook an in-depth analysis of annual reports from twenty humanitarian organisations,4 found on their websites. This resulted in approximately two hundred documents read and analysed, notably through a manual analysis with a list of keywords (Appendix 1). Since annual reports present the activities carried out during the previous year, the main objective of this analysis was to detail the quantitative materials used and highlighted by each humanitarian organisation. For instance, since humanitarian organisations are expected to justify their actions and interventions, we expected that their context analyses may have incorporated quantitative elements, notably from open-source ACDs. This methodology entails some difficulties regarding its concrete application. For instance, identifying a shared period for all annual reports was not really possible:5 some humanitarian organisations have published archives from the end of the nineteenth century to last year (for the International Committee of the Red Cross, ICRC), while others only began to make them available online in the 2010s (for example Médecins du Monde). What is more, relying on open-source availability means being subject to potential gaps through the years not only because of false or expired URL links but also because some organisations were not up to date on their publication of annual reports. However, this inaccessibility has no significant effect on this study, since the investigation’s main aim is to undertake a general overview of the use of quantitative data by humanitarian organisations, which would then be enriched with other materials.
Second, we conducted fifteen interviews and two workshops with a range of practitioners (on the ground and within headquarters) and representatives from the same humanitarian organisations to deepen our understanding of the way quantitative data was incorporated in their everyday work. Knowing that not all humanitarian workers need quantified data and that it can prove useful to some personnel but not every humanitarian worker, we have targeted different profiles – all potential users of quantitative data. Thus, we met with project managers, desk officers, humanitarian workers on the ground, and training officers from France and internationally (mainly working in the Horn of Africa). This allowed us to have a better grasp of the character of the issues surrounding the use of quantitative data within the humanitarian field and to appreciate how quantitative data were really integrated in the process of humanitarian work compared to what was shown in annual reports and official discourses. Finally, it was also important to discuss whether open-source ACDs were used in internal processes and if not, to figure out the reasons why. Transcripts were read and analysed by hand. Information on the data used were put in perspective with the analysis of annual reports. Recurring themes were identified and discussed within the team. We then manually colour-coded quotes that were linked to these themes in order to use them within our argumentation.
An Extensive Use of Quantitative Data: A Detailed Assessment
This part details general findings on the data consumption of humanitarian workers. It offers, first and foremost, a brief summary of the study of annual reports and interviews. It then discusses how these results can be interpreted through the lasting influence of the need-system.
What Data Is Useful for Humanitarian Workers?
This study has confirmed an extensive use of quantitative data in the annual reports of all humanitarian organisations considered, regardless of their size. Such data are mainly mobilised for their visual aspect (tables, charts, country sheets, maps).
To be more specific in our analysis, we divide quantitative data into two categories related to their source: first hand and second hand. As mentioned above, first-hand quantitative data are quantitative data that are collected, treated and mobilised by the humanitarian organisation in question. Second-hand quantitative data are used by humanitarian organisations but originate from external sources. These include international organisations or sub-organisations (such as United Nations (UN) bodies), regional organisations (such as the European Union), the academic field of international relations, think tanks and institutes, as well as other humanitarian organisations.
The vast majority of data included in annual reports are first-hand quantitative data and they have two main roles. First, in a logic of diagnosis, monitoring and evaluation (DME) they account for humanitarian action and detail the number of projects and programmes implemented, as well as the assistance given (for example by underlining the number of beneficiaries of an action to the nearest decimal). These figures are often placed at the beginning of the annual report in an ‘our organisation in figures’ page, or given for regional areas in the most detailed reports:
In 2009, the ICRC expatriate and national engineers and technicians were involved in water, sanitation and construction work in 39 countries. These projects catered for the needs of some 14,249,000 people worldwide (IDPs [internally displaced persons], returnees, residents – in general, people living in rural areas and/or areas difficult to reach owing to insecurity and/or lack of infrastructure – and people deprived of their freedom). Around 32% and 41% of the beneficiaries were women and children respectively. (ICRC, 2010: 93)
1,421 people benefited from physical therapy in Abidjan hospitals – 50 beneficiaries were fitted with equipment – 701 injured and/or hospitalized people benefited from mobility assistance. (Handicap International, 2012: 6)
Second, they itemise the administrative management of the humanitarian organisation, for example by giving information on human resources or the budget of the organisation. This section is usually found at the end of the report.
Second-hand quantitative data only began to be used in the 2010s and are mobilised for purposes of contextualisation. They often take the form of statistics or large numbers related to trends in the population in need of assistance or to broader societal, economic, political and social issues. The ambition is to report on the magnitude of the humanitarian crisis and its consequences on people in need of assistance more than the conflict itself:
Some 660,000 people who had fled Syria, and were registered by the UNHCR, remained in Jordan, along with thousands of unregistered migrants. (ICRC, 2020: 463)
Between October 2010 and April 2012, the severe drought and subsequent food crisis caused the death of more than 250,000 Somalis (source: FAO). According to FSNAU (the Food, Security and Nutrition Analysis Unit in Somalia) analyses published in February 2012, 2.34 million people were in need of humanitarian assistance and 1.29 million of them were in need of emergency response in the southern regions. (Solidarités international, 2012: 19)
However, this first layer of analysis is not sufficient to grasp the complexity of the data mobilised by humanitarian workers. Our analysis of annual reports, coupled with interviews, allowed us to organise our findings to point out three broad categories relating to the data’s main potential uses: contextual analyses (data needed to ‘understand the situation’), intervention analyses (data needed ‘to take action’ and related to DME) and finally, analyses of the security and safety of humanitarian workers. Table 1 gives examples as well as potential sources.6
Quantitative data used by humanitarian workers depending on their uses
| For contextualisation | For intervention, depending on the sector | For security and safety |
|---|---|---|
| Data on the situation | Data on access to services | Data on incidents |
|
|
|
| Sources: internal and external (UN or UN bodies) | Sources: internal and external (UN or UN bodies) | Sources: internal and external (International NGO Safety Organisation (INSO), Armed Conflict Location Event Data Project (ACLED)) |
| Data on the field – geography and infrastructure | Data on affected people and humanitarian needs | Data on fatalities |
|
|
|
| Sources: internal and external (UN or UN bodies) | Sources: internal and external (UN or UN bodies) | Sources: internal and external (INSO, ACLED) |
| Data on NGO present on the ground and type of action (cluster systems, coordination, etc.) | ||
| Sources: external (other NGOs, UN Office of the Coordination of Humanitarian Affairs (OCHA)) |
This analysis shows that, in terms of volume, quantitative data are predominantly mobilised for DME issues. They need to be hyper-detailed, based on ‘need’ standards, and geographically located with a high level of granularity. Here, quantitative data are mandatory and often produced by humanitarian organisations themselves. As one humanitarian worker summarised:
The biggest problem for humanitarian aid is to arrive in an area and say: in these areas, what damage has been done, who are the people affected, where are they, etc. It is this assessment that is complicated. It is an important and fundamental issue. Assessing? That’s why we produce a lot of data, but we produce data more locally, generally on a more local scale. It’s rare that we produce on a national scale.7
When it comes to contextual analysis, quantitative data do not seem to be the preferred material for analysis and are often marginalised in favour of qualitative elements. When they are mobilised, they are not chosen per se, because of their exactness, but rather when they support the qualitative analysis. In this regard, one practitioner mentioned the principle of ‘adapted inaccuracy’8 when choosing quantitative data. Thus, they mainly relate to international and regional trends. They often come from UN reports and deal with the consequences of conflict more than the conflict itself. Practitioners mentioned, for instance, that the ‘severity of a conflict or crisis will be seen in terms of population displacement. We are not so much interested in the number of incidents, but especially in the displacement of people’.9
For security and safety issues, quantitative data seem to combine all requirements. International, national and regional trends are useful to contextualise the conflict environment. However, what matters most is very detailed data on incidents (type of incident, location and date) to implement security plans. In this regard, quantitative data are mandatory.
The Lasting Influence of the ‘Need-System’
Table 1 shows how influential the need-system still is in the data consumption of humanitarian workers. All quantitative data mobilised in this sector relate to needs, whether directly or indirectly. Within this framework, there is little room for ACDs. In fact, ACDs were almost never mentioned, even though they could have been used to improve humanitarian workers’ daily tasks, such as contextual analysis. For example, the Uppsala Conflict Data Program/Peace Research Institute Oslo (UCDP/PRIO), the Correlates of War Project (COW), the Fragile States Index, the Global Peace Index, or even quantitative data from the Stockholm International Peace Research Institute (SIPRI) (knowing that defence expenditures are still mobilised as an indicator for geopolitical tensions) could be used for contextual analysis.10 Event databases such as ACLED could be used for contextual analysis or security matters. Early warning systems such as ACLED’s Conflict Pulse, the INFORM Risk Index, the PRIO Conflict Prediction or the Violence Early-Warning System (ViEWS) from Uppsala University/PRIO could be mobilised for predictive analysis or for coordination among humanitarian workers. Many of these are mentioned in the Centre for Humdata’s ‘Catalogue of Predictive Models in the Humanitarian Sector’.11 However, these potential uses were mostly dismissed by both our interviewees and the content of annual reports.
As a result, ACDs are not considered by humanitarian workers because the way they mobilise quantitative data is mainly predetermined by a specific, restrictive, hierarchically oriented evidence-based system. When the discourse around EBH expanded, the main way quantitative data were used was through the quantification of needs (Glasman, 2020). At the time of writing, in 2022, this is still the case: although quantitative data could serve other purposes, this study has found that they are mainly mobilised for matters related to need. Thus, projects of a more extensive and innovative EBH seem, for the time being, to be only a promise.
The Stagnation of the EBH System
This study identifies three related factors to explain the stagnation of the EBH system: the criticism of the quantified need-system, considered to be suited to donors; the lack of data literacy of humanitarian workers; and finally, the refusal to dehumanise the profession.
A Need-System Suited to Donors
A first factor to explain the stagnation of EBH is the criticism (or, at the very least, scepticism) of the quantified need-system which is at the root of EBH. In the literature, this criticism is often associated with the imperfection of the data used. As Glasman mentions in the conclusion of his study, ‘[a] first reservoir of criticism is of course to be found in humanitarian expertise itself’ (2020: 250). Humanitarian workers are often critical of the quality of the quantitative data their organisation may collect and mobilise. However, the same kind of criticism arises with second-hand data. UN figures are a case in point.
As mentioned above, a careful examination of annual reports has led us to observe a fairly extensive use of quantitative data from international organisations. Among the most mobilised sources were the Food and Agriculture Organization (FAO), the Humanitarian Response Plan (HRP), the United Nations Development Programme (UNDP), the United Nations Population Fund (UNFPA), the United Nations High Commissioner for Refugees (UNHCR), the United Nations Children’s Fund (UNICEF), the United Nations Office on Drugs and Crime (UNODC), the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA), the World Food Programme (WFP) and the World Health Organization (WHO). All promote the need-based approach: thus, their quantitative data have the advantage of being suited to humanitarian action both in terms of purpose and potential uses, even more so since they are easy to access for humanitarian workers. Yet, humanitarian workers seem quite sceptical of these numbers as well:
I was reading a very serious report, a UN publication, in which they said: ‘Approximately, we have 973,000 people…’. For me, we can’t be ‘approximate’. This is a battle that has been going on for years. And I dug into how these people had produced their figures… [rolls his eyes and sighs].12
We can also use the UN datasets but if you are in the field, you can have reservations about what it really measures…13
What is interesting with UN data is that the situation always seems to get worse despite our interventions…14
Behind these discourses lies a paradox: how to explain humanitarian workers’ massive use of quantitative data when they are so critical about it? In fact, it is necessary to understand that the problem does not lie with the quality of data. They are known to be imperfect. It is rather the system in which they are embedded that is criticised. In other words, what humanitarian workers really question is the quantification system as it stands today more than the data themselves. Criticism often refers to this system’s purpose. As one humanitarian worker explained, there is still some resistance within humanitarian organisations regarding the collection of data:
There is a debate every time a process is introduced to collect data. The question that comes up like a leitmotif is: what is it going to be used for and what is it for? And this is a legitimate question.15
In fact, a major part of the quantitative data produced by humanitarian organisations is intended for – and therefore, determined by – donors. Humanitarian organisations may use these data for their own internal monitoring, but it is donors who ultimately verify these figures. This system is considered as a very strong means of control:
Donors are putting more and more pressure on us to analyze data properly, to provide evidence that aid is being delivered and to assess our impact, because we will be funded based on this impact, and we can be held accountable for it…. This is their way of having control over us, over what we do, and to avoid the money being misspent…. So there is a lot of pressure today … we have to collect the data correctly.16
I would say that there is a lot of financial pressure. We can disappear overnight if something goes wrong. It’s such a huge pressure that it can make NGOs not being super picky and making numbers speak in their favour.17
Quantitative data produced by humanitarian organisations are therefore considered as a strategic asset orientated towards donors. One practitioner illustrated it in these terms:
According to me, these quantitative databases, this desire to standardize and have indicators, it’s important for people at a global level, that is to say important for the donors. They want to know where to invest…. It’s not important at all for people at the local level because at the local level you don’t think with data. You know what’s going on in the conflict, you know who the people are who are affected by it. In fact, the information is already there.18
In this case, it may not seem relevant to use second-hand data, let alone one ACDs: ‘the war to get data is also a war to gain access to funding. This is not open source’.19 This is different for UN datasets, since they ‘define the funding priorities. As we are completely dependent on UN money, if there are things that are not identified as priorities, it can be a problem. It’s very frustrating, and it prevented us from doing our work’.20
The Lack of Data Literacy among Humanitarian Workers
A second factor of stagnation relates to humanitarian workers’ data (il) literacy. As Glasman explains, EBH ‘overestimates the control that humanitarian agencies have over their tools’ (2020: 2) and over technology as a whole. In line with this claim, this study has found that, contrary to what is often taken for granted, the quantification of the humanitarian field has not been accompanied by an acculturation to the data. This means that although the use of quantitative data has grown, humanitarian workers have not processed and incorporated this use into their daily practices (beyond the need-system). This was put forward by humanitarian workers themselves, in their general observation of their lack of training on quantification matters:
The level of competence in NGOs has improved in recent years but is still very limited…. As people do not master the biases associated with data, they end up not being confident enough, not comfortable enough with the data to make it relevant.21
Let’s say that French NGOs are a bit behind … for instance, one of the obstacles is the lack of training and competence of the staff who go into the field.22
For me, pure quantitative work is a dream that will never be achieved in the humanitarian field.23
This lack of training has certainly narrowed humanitarian workers’ perspective on the potential of quantitative data. When discussing how quantitative data may be appropriate for contextual analyses, humanitarian workers always dismissed the argument:
I’m not against this global data, I find it very interesting…. In UN reports that are 50 pages long, you will only use 10 per cent of the content for operational purposes.… I remember going through reports and seeing that the data is at the level of the region, and it’s a shame. You’re always looking for the little nugget that will give you super precise data. It almost never happens.24
The UN and the big organizations will collect data – but global data. So they will visit each area, but they will create global percentages. And we are interested in the micro…. The data we get from the UN, although interesting, will never allow us to set up projects. It is not operational.25
What is more, as the collection and processing of humanitarian organisations’ own quantitative data are often externalised, there is little chance that humanitarian workers will gain experience in using data. For example, IMPACT Initiatives is ACTED’s sister organisation, specialising in ‘humanitarian assessments and monitoring and evaluation activities …, information management solutions and … organizational capacity building programmes that support aid stakeholders to plan and respond to crises’.26 Together, these organisations have created REACH, which aims to provide ‘granular data, timely information and in-depth analysis from contexts of crisis, disaster and displacement’.27 NGOs specialising in mapping, such as CartONG, MAPAction and iMMAP, are often linked to bigger humanitarian organisations. Many of them produce reports on information management to present the assets of quantitative data, mainly because they consider that this issue may not be taken seriously enough by humanitarian organisations. For instance, CartONG did a recent study entitled Program Data: The Silver Bullet of the Humanitarian and Development Sectors? Panorama of the Practices and Needs of Francophone CSOs (2020), in which it concludes:
Although data management has become essential to the management of operations, it seems to remain somewhat invisible in the [humanitarian field] reflections and orientations, despite its numerous ethical, financial and human implications, and especially its consequences on the quality of projects. In the field and at headquarters, project teams are devoting more and more time to data management, to the detriment of other activities. Poorly trained and ill-tooled, they are sometimes even suffering from these tasks, even though the subject is not considered an organizational issue by most CSOs [civil society organisations]. (CartONG, 2020: 8, emphasis added)
Humanitarian workers generally agree that the gap in their knowledge is too wide and will continue to be so because of a lack of financial, human, or even time resources:
We are humanitarians, and we know that we will never claim the same level of analysis as IMPACT or REACH. There is no money, there are no resources.28
There is a problem of time, of pressure on the workload, producing research and analysis is complicated. We can’t have the freedom to lose ourselves. What we also see in the field is that, yes, you need data, you need it to analyze, but not everyone has this skill. Understanding the difference between an average and a median is already complicated. So, think of statistical details or machine-learning algorithms or more complicated mathematical approaches … It’s possible, but we don’t have the skills and people who have these skills don’t come.29
In fact, as Glasman and Lawson mention in their introduction of this special section, producing good quality data does imply for humanitarian organisations to make long-lasting investments in bureaucratic capacities and qualified staff.
The Refusal to Dehumanise the Profession
Finally, explaining the lack of progress of EBH lies in analysing humanitarian workers’ refusal to dehumanise their profession. During interviews, they often mentioned their reluctance to use quantitative data so as not to expand the logics of the need-system.
The first aspect of this reluctance is related to beneficiaries themselves. As the root of the word ‘humanitarian’ suggests, humanity lies at the heart of humanitarian action – and, needless to say, is commonly recognised as one of its core principles. However, because of the introduction of quantification, the role of narratives and individual stories has declined. Agencies and humanitarian organisations’ heads now want ‘to know “the real facts” of the recovery process’ and, in that respect, ‘people’s experiences [are] not considered valuable knowledge’ (Brun and Lund, 2010: 822). In the end, ‘some of the original motivations of humanitarianism … have become less of a priority than the drive towards generating evidence and data’ (Lokot, 2019: 468). As one humanitarian worker mentioned, the result ‘is a reduction of mankind, it is a reduction of the human being in his singularity’.30
This feeling of dehumanisation is even more prominent considering that quantification went hand in hand with the appearance of counting categories (Glasman, 2020). These categories tend to be highly criticised not only because of their all-encompassing aspect (Andreas and Greenhill, 2010), but also because of their political dimension. The choices underlying the denomination of a category and the way it will be counted is, in this respect, one main point of criticism. For instance, who counts as a ‘missing person’? Who counts as a ‘wounded person’? Who counts as a ‘prisoner’? As Andreas and Greenhill (2010) have shown, these definitions are often highly politicised and reflect issues that go far beyond their apparent neutrality.
Beyond their impersonal nature, quantitative data are also seen as dehumanising because they are considered to be disconnected from any sense of humanity. As one humanitarian worker conceded, ‘numbers, for me, take away the emotional side and puts you on orders of magnitude’.31 Another added that ‘figures give seriousness, testimonies give emotion. Figures give credibility, they are not a testimony. It’s a research job’.32
This can be partly explained by the fact that quantitative data are considered to be the result of a rational and scientific process free of subjectivity, as opposed to a reasoning based on emotions. As Porter has shown, individuals grant legitimacy to numbers because of their apparent objectivity, and this objectivity has been established as an alternative to personal trust. In his own words, this ‘form of trust supporting objectivity is anonymous and institutional rather than personal and face to face’ (1995: 224, emphasis added). Practitioners share this assumption: ‘the data creates a kind of distance between you and the human impact. Because it’s at the aggregate, neutral, distanced level, and so on’.33
Finally, this fear of dehumanisation is also valid for practices themselves. In fact, as Read et al. note, ‘techno-optimism has not spread to all corners of the humanitarian sector’ (2016: 6). Numbers, quantitative data and technology are so intricate that an extensive use of data is often considered by humanitarian workers as a risk of dehumanisation. As an example, several of the interviews and workshops organised by our research programme drifted pretty quickly to the question of the role of artificial intelligence in decision-making and the dispossession of human beings of their capacities. For instance, one interviewee explained:
There is a fantasy in humanitarianism that quantification will allow us to ignore the inherent uncertainty we face. Thanks to algorithms, we will be able to relieve ourselves of the difficulty of interpreting ambiguous events about which we have incomplete and unreliable information … and we will forget the fact that the very essence of decision-making in humanitarian situations is to make bets in the face of uncertainty.34
Thus, part of the resistance to databases and quantitative data lies in the desire to keep the decision-making process as human as possible, based on staff expertise. Current EBH is seen as promoting the opposite:
So these systems that want to do a lot of data in a short amount of time forget that the value of humanitarian information is linked to the trust that exists between the person who gives the information and the person who collects it.35
Conclusion
Do humanitarian workers trust numbers – meaning, do they believe they are reliable and can be used in their daily practices? Investigating the limited use of ACDs by humanitarian workers enables us to better understand the current importance of quantitative data within the humanitarian field – and it is, in fact, much less developed than what advocates of EBH currently suggest. It is often taken for granted that the introduction of quantification, through the need-system, has made humanitarian workers more aware of the merits of quantitative data. Yet in reality, the way humanitarian workers use and even consider quantitative materials is still the same since the beginning and always relates to need assessments in one way or another. Promoting better collecting and processing of data for humanitarian workers, as EBH does, only reinforces their accountability towards donors. As a result, the current dynamic is self-sustaining.
What is more, this study has found that the use of quantitative data has not expanded for three main reasons: criticism of the need-system by humanitarian workers, their lack of data literacy and their refusal to dehumanise their profession. We argue that the lack of data literacy acts as a catalyst for the two others, which are actual symptoms of a lack of trust. If humanitarian workers feel that their use of quantitative data is only imposed from above (both from donors and the hierarchy), they will continue to see it as a means of control. They will thus refuse to expand the use of quantitative data – and even more since it is considered as dehumanising the core of their work. However, the situation cannot evolve if humanitarian workers don’t feel competent enough to use quantitative data. Moreover, if they do not clearly understand the benefits of quantitative data beyond the need-system that they denounce, they cannot be expected to incorporate these in their daily practices. Dedicating time to train humanitarian workers to use quantitative data with confidence and full understanding of their meaning, by allowing them to participate in training organised by other NGOs or even by the Centre for Humdata, could be a first step towards improving the situation. Internalising some of the collecting or processing of quantitative data could also be a great way for humanitarian workers to gain data literacy. We realise that this requires resources that humanitarian organisations may not have: further research may thus have to think of organisational factors explaining these difficulties. It might also be necessary to analyse to what extent such training could be important for humanitarian workers.
Finally, a question worth asking is: do humanitarian workers need to trust numbers?36 It could be tempting to answer that complying with donor expectations does not oblige them to expand their trust in any way. However, this study also shows that the issues surrounding both the need-system and quantitative data are debated within organisations and requires some answers – and that a lack of trust may complicate the process. What is more, this is still a work in progress: given the current discourses on the growing importance of quantitative data within the field, especially regarding EBH, one could expect these uses to multiply and diversify in the following decades. If this were to happen, anticipating practitioners’ discourses on the matter may prove useful.
Declarations
Availability of data and materials: all data generated or analysed during this study are included in this published article and its supplementary information files.
Competing interests: the author declares that she has no competing interests.
Funding: this research benefits from funding by the French Agence Nationale de la Recherche (ANR) allocated to the DATAWAR research project (ANR 19-CE39-0013).
Author’s contributions: Not applicable.
Acknowledgements: I would like to thank the members of the DATAWAR research project for their guidance, Brendan Lawson for his helpful comments on the first version of this article, and finally, Caitlin Gordon Walker for her meticulous proofreading.
Appendix 1: Keywords used in manual content analysis
Data, figures, indicators, investigation, statistics, survey, rates, %.
Armed Conflict Databases (ACD): ACLED, aggle, Conflict barometer, Conflict Early Warning System (ICEWS), Correlate of war, dataminr, Eurostat, Fragile states index, Global Database of Events, Language and Tone (GDELT), Global peace index, Global terrorism database, Heidelberg, Humanitarian data exchange, index for risk management, INFORM, International Crisis behaviour, Open Situation Room Exchange, SIPRI, Uppsala conflict data program, Ushahidi, World Event/Interaction Survey (WEIS).
Notes
The starting point of this study is linked to a broader research project, DATAWAR, which has the ambition to conduct the first investigation on the effects of quantitative conflict analysis on practitioners’ representations of war.
Initially confined to the academic field, the first ‘conflict databases’ are commonly recognised as the Uppsala Conflict Data Program (UCDP), the Peace Research Institute of Oslo (PRIO), the Correlates of War (COW) from Michigan, or even the Conflict Barometer from the Heidelberg Institute for International Conflict Resolution (HIIK).
The research project has also conducted interviews with military personnel, diplomats and journalists, as they may contribute to practitioners’ perception of conflict.
Chosen from the ‘Liste des Organisations de Solidarité International françaises engagées dans l’action humanitaire’, published by the French Ministry of Foreign Affairs, www.diplomatie.gouv.fr/IMG/pdf/Liste_des_OSI_humanitaires_francaises-2_cle841b4b.pdf (accessed 1 December 2022).
At first, the period defined by the DATAWAR programme was 1989–2020, in line with other studies of the project. However, for the twenty humanitarian organisations chosen, only one – the ICRC – had published reports for the whole period. Thus, we decided to gather all annual reports we could without defining a specific period – hence the difficulty of providing a graph presenting the evolution of the use of quantitative data. Most of the annual reports available date back to the 2010s.
This attempt to categorise the uses of quantitative data and the examples given is a proposition and is by no means an absolute. This table has been realised thanks to a careful analysis of annual reports and interviews and does not claim to be exhaustive.
Interview H7 (2021).
Interview H7 (2021).
Interview H26 (2021).
This is the case for other conflict practitioners studied within the DATAWAR project. See Beaumais and Ramel (2023).
https://centre.humdata.org/catalogue-for-predictive-models-in-the-humanitarian-sector/ (accessed 31 July 2023).
Interview H5 (2021).
Interview H7 (2021).
Interview H8 (2021).
Interview H7 (2021).
Interview H26 (2021).
Interview H26 (2021).
Interview H20 (2021).
Interview H5 (2021).
Interview H26 (2021).
Interview H23 (2021).
Interview H26 (2021).
Interview H20 (2021).
Interview H20 (2021).
Interview H26 (2021).
‘What we do’, IMPACT Initiatives website, November 2021.
‘What we do’, REACH website, November 2021.
Interview H26 (2021).
Interview H7 (2021).
Interview H7 (2021).
Interview H20 (2021).
Interview H7 (2021).
Interview H20 (2021).
DATAWAR Workshop no.1 (2020).
Interview H5 (2021).
I would like to thank one of the reviewers for this very important reflection.
Works Cited
Andreas, P. and Greenhill, K. M. (eds) (2010), Sex, Drugs, and Body Counts: The Politics of Numbers in Global Crime and Conflict (Ithaca, NY and London: Cornell University Press).
Barnett, M. (2013), ‘Humanitarian Governance’, Annual Review of Political Science, 16, 379–98.
Beaumais, L. and Ramel, F. (2023), ‘Diplomats, Soldiers, and Armed Conflict Databases: Another French Exception?’, Global Studies Quarterly, 3:2, doi:10.1093/isagsq/ksad027.
Beerli, M. J. (2017), ‘The Power to Count and the Stakes of Counting: An Inquiry into the Quantified Production of Humanitarian Insecurity’, Global Governance, 23:1, 57–70.
Benton, M. and Glennie, A. (2016). Digital Humanitarianism: How Tech Entrepreneurs Are Supporting Refugee Integration (Washington, DC: Migration Policy Institute), www.migrationpolicy.org/research/digital-humanitarianism-how-tech-entrepreneurs-are-supporting-refugee-integration.
Brun, C. and Lund, R. (2010), ‘Real-time Research: Decolonising Research Practices – or Just Another Spectacle of Researcher – Practitioner Collaboration?’, Development in Practice, 20:7, 812–26.
Burns, R. (2014), ‘Moments of Closure in the Knowledge Politics of Digital Humanitarianism’, Geoforum, 53, 51–62.
Burns, R. (2015), ‘Rethinking Big Data in Digital Humanitarianism: Practices, Epistemologies, and Social Relations’, GeoJournal, 80:4, 477–90.
Burns, R. (2019), ‘New Frontiers of Philanthro-capitalism: Digital Technologies and Humanitarianism’, Antipode, 51:4, 1101–22.
Campbell, L. and Knox Clarke, P. (2019), Beyond Assumptions: How Humanitarians Make Operational Decisions, ALNAP Study (London: ODI/ALNAP).
CartONG (2020), Program Data: The Silver Bullet of the Humanitarian and Development Sectors? Panorama of the Practices and Needs of Francophone CSOs, https://cartong.org/sites/cartong/files/2020_CartONG_Study_Program_Data_CSOs_EN.pdf (accessed 1 December 2022).
Colonomos, A. (2016), Selling the Future: The Perils of Predicting Global Politics (Oxford: Oxford University Press).
De Franco, C. and Meyer, C. O. (eds) (2011), Forecasting, Warning and Responding to Transnational Risks (Houndmills, Basingstoke: Palgrave Macmillan).
Desrosières, A. (2013a), Pour une sociologie historique de la quantification: l’argument statistique I (Paris: Presses de l’Ecole des Mines).
Desrosières, A. (2013b), Gouverner par les nombres: l’argument statistique II (Paris: Presses de l’Ecole des Mines).
Desrosières, A. (2014), ‘Chapitre 1. Le gouvernement de la cité néolibérale: quand la quantification rétroagit sur les acteurs’, in Desrosières, A., Prouver et gouverner: Une analyse politique des statistiques publiques (Paris: La Découverte), pp. 31–59.
Fast, L. (2017), ‘Diverging Data: Exploring the Epistemologies of Data Collection and Use among Those Working on and in Conflict’, International Peacekeeping, 24:5, 706–32.
Glasman, J. (2020), Humanitarianism and the Quantification of Human Needs: Minimal Humanity (London and New York: Routledge).
Handicap International (2012), Rapport d’activité International 2011, www.handicap-international.lu/fr/rapports-d-activite-nationaux-et-internationaux (accessed 1 July 2023).
ICRC (2010), Annual Report 2009 (Geneva: International Committee of the Red Cross), https://icrc.org/en/doc/resources/documents/annual-report/icrc-annual-report-2009.htm (accessed 1 July 2023).
ICRC (2020), Annual Report 2019: Volume II (Geneva: International Committee of the Red Cross), https://icrc.org/en/document/annual-report-2019 (accessed 1 July 2023).
Jacobsen, K. L. and Fast, L. (2019), ‘Rethinking Access: How Humanitarian Technology Governance Blurs Control and Care’, Disasters, 43:S2, S151–S168.
Lawson, B. T. (2021), ‘Quantification and Humanitarianism’, Journal of Humanitarian Affairs, 3:1, 53–60.
Lokot, M. (2019), ‘The Space between Us: Feminist Values and Humanitarian Power Dynamics in Research with Refugees’, Gender & Development, 27:3, 467–84.
Meyer, C. O., De Franco, C. and Otto, F. (2019), Warning about War: Conflict, Persuasion and Foreign Policy (Cambridge: Cambridge University Press).
Olivius, E. (2016), ‘Constructing Humanitarian Selves and Refugee Others: Gender Equality and the Global Governance of Refugees’, International Feminist Journal of Politics, 18:2, 270–90.
Porter, T. M. (1995), Trust in Numbers: The Pursuit of Objectivity in Science and Public Life (Princeton, NJ: Princeton University Press).
Read, R., Taithe, B. and Mac Ginty, R. (2016), ‘Data Hubris? Humanitarian Information Systems and the Mirage of Technology’, Third World Quarterly, 37:8, doi:10.1080/01436597.2015.1136208.
Roth, S. and Luczak-Roesch, M. (2020), ‘Deconstructing the Data Life-Cycle in Digital Humanitarianism’, Information, Communication & Society, 23:4, 555–71.
Rothe, D., Fröhlich, C. and Rodriguez Lopez, J. M. (2021), ‘Digital Humanitarianism and the Visual Politics of the Refugee Camp: (Un) Seeing Control’, International Political Sociology, 15:1, 41–62.
Solidarités International (2012), Annual Report, https://solidarites.org/wp-content/uploads/2017/05/2012-Annual-Report-solidarites-International.pdf (accessed 1 December 2022).
Supiot, A. (2015), La Gouvernance par les nombres (Paris: Fayard).
UN Secretary-General (2017), ‘Secretary-General’s remarks at the Opening of the Centre for Humanitarian Data [as delivered]’, United Nations, www.un.org/sg/en/content/sg/statement/2017-12-22/secretary-generals-remarks-opening-centre-humanitarian-data (accessed 1 July 2023).
Ward, M. D., Metternich, N. W., Dorff, C. L., Gallop, M., Hollenbach, F. M., Schultz, A. and Weschle, S. (2013), ‘Learning from the Past and Stepping into the Future: Toward a New Generation of Conflict Prediction’, International Studies Review, 15:4, 473–90.
