Measuring 'subjective resilience': using peoples' perceptions to quantify household resilience

Submitted by Lindsey Jones 4th August 2015 11:35
Image of an individual working in field

Credit: Neil Palmer/CIAT

Introduction

Resilience has rapidly risen to the top of the development agenda (Burnard, 2011; Frankenberger, 2014; Béné et al., 2013), and is now seen as a valuable conceptual tool in furthering the understanding of how people respond and adapt to the many changing shocks and stresses that affect livelihood outcomes (Manyena, 2006; Miller et al., 2010; Nelson et al., 2007). Inevitably, a push for resilience-building within the development and humanitarian communities has led to increased demand for ways of measuring levels of resilience amongst people and communities (Brooks et al., 2011). In theory, more accurate measurement and tracking of resilience can help to ensure that resilience-related policies and programmes are supporting the right activities and targeting the right people (Oddsdottir et al., 2013). Monitoring the effectiveness of resilience programming, rapid post-disaster assessment, and targeted social protection activities all involve the tracking of resilience on different scales (Frankenburger et al., 2014).

Unfortunately, the assessment of resilience is fraught with complexity: both the definition of resilience and the methodologies used to measure it are heavily contested (Cumming et al., 2005). Confounding factors, such as what mix of indicators to choose, which systems and scale of analysis to apply, and how to recognise the context-specific nature of resilience each muddy the waters. Indeed, despite growing global interest in supporting resilience-building activities, existing approaches to the measurement and tracking of resilience have generally not been able to the deliver the desired policy support (Levine, 2014).

A large number of frameworks and approaches have been proposed for quantifying household resilience (Bahadur et al., 2015). Most concentrate on ‘objective’ indicators by identifying key socio-economic variables and other capitals that support people’s livelihoods. The selection of these variables is often value-laden and contested (Carpenter et al., 2001; Bahadur et al., 2015). They also typically require large and robust socio-economic datasets, which are not easily found in many developing country contexts. However, a complementary means of assessing household resilience that can add value to and complement objective forms of measurement has largely been overlooked: ‘subjective’ household resilience. Subjective resilience stems from the premise that people have a good understanding of their own capacities, capabilities and limits. In leveraging more bottom-up processes of data collection, ‘subjective’ indicators of perceived resilience are based on behaviours, attitudes and psychology: factors not easily captured by traditional ‘objective’ indicators. The measurement of perceived resilience is therefore about how people rate their own resilience, and the resilience of the wider community of which they form a part.

In this paper, we call for the tracking and measurement of subjective resilience at the household level. We describe why new approaches to measuring resilience are needed, what can be learned from other disciplines that use subjective indicators, and how subjective resilience can be adequately incorporated into methods of measuring and tracking resilience on the ground. We argue that efforts to measure resilience need to take people’s perceptions of their own capabilities and capacities into account, either in combination with, or separate to, objective forms of resilience measurements. In order to narrow the context, we have chosen disaster resilience as the entry point for this paper, specifically the resilience of households to weather and climate extremes. However, the same principles would apply equally to other aspects of resilience, such as livelihood, community or social resilience, all of which possess many of the same characteristics. 

Challenges to the assessment of subjective resilience and how to address them

Subjective approaches to the measurement of socioeconomic characteristics such as poverty, well-being or resilience are each affected by a number of well-documented biases and methodological challenges. While many of them can be accounted for through thorough careful design of methods, they nonetheless require consideration.

A commonly cited concern is whether self-reported responses can be considered valid (Diener et al., 2002). After all, reports of subjective resilience do not reflect a stable state of household resilience. Rather, they are a judgement that individuals provide on the spot, based on information that is available to them at the time and influenced by myriad contextual and emotive factors (Schwarz & Strack, 1999). Traits such as personality, values and beliefs may have a significant effect on how two people with the same levels of resilience self-report themselves. This is particularly the case with measures of subjective well-being, where scores can vary during the day depending on a number of endogenous and exogenous factors such as the weather, time of day, or where the question appears in relation to others (Strack et al., 1991).

Understanding how they affect scores is important, as we would not necessarily expect a household’s resilience to disasters to fluctuate largely on a day-to-day basis without considerable forcing. Yet, these influencing factors do not present grounds for dismissing subjective measures altogether as ‘the idiosyncratic effects of recent, irrelevant events are likely to average out over representative samples’ (Kahneman & Krueger, 2006: 7). Thus, with careful survey design, multiple surveys over time and adequate sampling methods, many of these biases can be reduced and largely accounted for.

More widely, people tend to compare themselves with those around them. Therefore, respondents exposed to one culture may report a different score than compared to others from another culture, irrespective of any differences in their overall resilience (e.g. perhaps one culture has a more optimist take on life than another). This makes cross-cultural comparison difficult. One method of trying to reduce this bias is to ask people to respond relative to a reference point, such as their neighbours or an average person in their village/town. However, the effects of cross- cultural bias require careful consideration in interpreting higher-level data, such as comparison between different countries. One option is the use of anchoring vignettes (Hopkins & King, 2010), which provides people with a hypothetical example (such as the characterisation of a person highly vulnerable to flooding) and asks them to provide a subjective rating. Given that cultural values do not tend to shift quickly, it may also be more insightful to focus on longitudinal analyses (tracking resilience-scores over time in the same place) than cross-comparisons between other cultures or locales (tracking the difference between one place and another at the same moment in time).

A bias that is particularly important to account for is tactical reporting. For example, in areas that receive considerable development or humanitarian assistance in meeting people’s basic livelihood needs, it is possible that respondents may choose to respond in their own self-interest, i.e. claiming to be more vulnerable than they actually are in the hope of securing sustained or increased levels of assistance. The opposite may equally be true, whereby people do not want to be considered as having low levels of resilience – perhaps due to the social stigma attached with the label – and deliberately claim that their household has a higher level of resilience than in reality. This is where a thorough understanding of the context and political economy of the surveyed area can be of immense value. Clear and neutral wording can also be important. Above all, it showcases the need to consider subjective information not as a stand-alone but as a useful complement to objective forms of data collection. 

Conclusion

In this paper, we outline the rationale for assessing subjective disaster resilience at the household level. While it is clear that any approach to subjective assessment will face significant methodological and conceptual challenges, we show these to be far from insurmountable. Most importantly, measuring subjective resilience offers a valuable opportunity to capture the perspectives of those who know most about their own resilience and the factors that contribute to it: the people themselves. Moreover, this type of information has a number of unique practical applications, such as helping to improve our understanding of what works and doesn’t with regards to resilience- building activities; enhanced targeting of resilience-related programmes and resources; as well as providing a useful bottom-up tool for capturing the voice of beneficiaries and local communities.

Establishing the feasibility and methodological robustness of a subjective approach to measuring disaster resilience will inevitably take time. However, a tremendous amount of knowledge can already be drawn from current understandings of household disaster resilience, as well as insights gained through gathering subjective information in related fields, such as subjective well-being and psychological resilience. Care should nonetheless be taken in examining the merits and limitations of various different approaches to measuring subjective resilience. It is likely that a range of methods, surveying tools and applications will be required to satisfy the diversity of user needs and resources available.

Ultimately, the aim here is not to entirely replace traditional methods of resilience measurement. On the contrary, objective measures are a vital component of the measurement process. Rather, if shown to be effective, we argue that bottom-up subjective methods should be used alongside objective methods, helping to capture many of the components of resilience that are difficult to observe and allowing people’s perspectives to be heard in a systematic manner. Getting the process right will be an important step forward in gaining a more holistic understanding of what it takes for a household to be resilient to disaster risk. 

Comments

lindsey jones copy - climate adaptation.

A new paper on subjective resilience. Hopefully of use to those interested in measuring resilience using alternative methods.

Systematic approaches to subjective resilience are key to solve several of the problems affective adaptive capacity of local groups of stakeholders. Bottom up approaches to the problem often encounter considerable resistance due to non-objective, albeit complex, obstacles.

Modeling the interactions occurring among stakeholders is in turn essential for resilience planning and risk prioritization in a community.