What evidence do we have for building safety in shelter self-recovery?

The humanitarian sector is increasingly aware of the role that good quality evidence plays in the underpinning of effective and accountable practice. An evidence review by Louise Harriss (visiting researcher CENDEP) Charles Parrack (CENDEP) and Zoe Jordan (CENDEP) addresses the need for reliable evidence by evaluating current knowledge about the intersection of two key outcome targets of post-disaster shelter response: supporting shelter self-recovery and building back safer. Evidence about post-disaster shelter programmes that aim to improve hazard resistance while supporting shelter self-recovery has been systematically analysed and evaluated. Technical support, especially training in safer construction techniques, was found to be a central programme feature, but the impact of this and other programme attributes on building safety was largely not ascertainable. Programme reports and studies lack sufficient detail, especially on the hazard resistance of repaired houses. Accounts of shelter programmes need to include more reliable reporting of key activities and assessment of outcomes, in order to contribute to the growing evidence base in this field. 

Read more here

https://www.ncbi.nlm.nih.gov/pubmed/31343759

Humanitarian Evidence Week: Long-term impacts of shelter programmes

Charles Parrack writes:

Earlier this year, in 2018, CENDEP hosted a one-day workshop on longitudinal studies in shelter at Oxford Brookes University. Attendees came from a wide range of backgrounds with experience in the humanitarian sector, shelter and academia.

The topic has been raised in various shelter cluster and non-cluster meetings over several years. At the Global Shelter Cluster meeting in Geneva 2017, the Shelter Projects working group found strong interest in longitudinal studies and, as a result, proposed to advance discussions on a more organised approach to methodology, procurement, dissemination and sectoral learning/evidence.

Longitudinal studies are by no means a new concept; indeed, evidence shows them to be a part of other sectoral practice, however they are something that the shelter sector does not yet embed into its own practice. Why is this? Some thoughts from the workshop suggested the answer to be a combination of rapid agency staff turnover, funding challenges, a lack of accountability and perhaps a fear of sharing programme failures.

Photograph: © Patrick Brown/UNICEF/Panos Pictures

Why are they needed?

Discussion ranged over the following questions: How would impact measurement benefit the sector? How would it benefit affected communities? There was a consensus that the outcomes of impact studies would encourage institutional learning particularly if disseminated and shared as best practice. There is a need within the sector to see the long-term consequences of our actions; to understand how beneficiaries take ownership of shelter and how safety is enhanced or reduced after implementation. Following up on programmes, two years, five years or even ten years after completion can enable agencies to better understand and engage with issues of prevention, sustainability, resilience and cost-efficiency which can feed back into and improve future programming. Whether it is the original agency who carries out the study, or an independent organisation, it is important that the results consider the context both past and present and are not a biased representation. Consideration of the learning opportunities for affected populations, implementers and donors drew an extensive list in the workshop highlighting a strong benefit for many actors.

Methodology

Longitudinal studies involve repeatedly gathering data on the same focus area over a long period of time. The unit of analysis, that is what is being measured, can be a person, shelter, household or community. It is important the unit of analysis is confirmed and remains the focus of the study as the methodology’s most important aspect is being able to compare in order to fully understand change overtime. Time is the best test of a programme’s viability, effectiveness and sustainability.

Shelter is delivered in various forms, both soft and hard interventions, and it is not necessarily a structure in the physical sense of the word. Perhaps 90% of what the shelter sector delivers is provided to meet short-term emergency needs. It does not take into account the future long-term consequences and may therefore, be difficult to locate, let alone measure. With this in mind, discussion looked towards certain shelter interventions that are quick and common enough for us to be able to find out what their impact has been, for example, transitional shelters. Transitional shelters are intended to be transformative in their nature, therefore it may be possible for us to track their transformation. The question of how their transformation may be followed and what exactly we may want or need to understand generated further conversation. What was voiced by many was the role of the affected communities in longitudinal studies; that perhaps it is the residents themselves who document their own recovery and their qualitative data can be understood by agencies in a more quantitative way if preferred. Participation from affected communities must continue to happen to institutionalise the process of learning so we, as a sector, can continue to develop and improve programming that meets the needs of the user. For shelter in the context of conflict, the journey of displaced populations may offer the opportunity to study soft modalities in the form of long-term displacement.

Questions and challenges

As long-term studies are not standard practice in the shelter sector challenges and questions surrounding the implementation of such studies were raised:

-          What are the criteria for success?

-          Do long-term studies link with the localism agenda?

-          Changing the building culture takes time and is not part of the emergency     programme

-          How to fund studies with long-term intentions when the programmes are inherently short-term?

Conclusions

Long-term effects of shelter programming can be both positive and negative but these effects cannot be quantified in the end of a project evaluation; they require time to develop and change in response to the intended user. It is clear from this workshop and Global Shelter Cluster findings that many organisations have the willingness to carry out long-term studies, but do not have the time or indeed, the funding. Understanding the consequences of our actions will enable better design and implementation of improved shelter programmes for reduced costs and greater effectiveness.

The group suggested many ways we can measure shelter, from technical standards like the Sphere Standards to quantifiable measures like value for money; to less tangible, but just as important, indicators such as level of happiness and sense of ownership. The shortlist of these was; safe shelter and settlements, user satisfaction, enabling informed decisions, adaptability, saving lives and value for money. Uninterrupted sleep and beauty were shortlisted as overall proxy indicators of a successful shelter. In general, the discussants agreed that the Sphere Standards offered good indicators for identifying success when looking at; security, peace, dignity, security of tenure and protection from forced eviction.

Based on the day’s thoughts, it is likely that these studies would have a varied audience including funders, agencies and university courses with wide opportunities for learning and improving practice for all. Whether you prefer to call them ‘longitudinal studies’ or ‘long-term impacts’, the interest for putting them into action is strong and we look forward to continuing the conversation from the workshop.

Launching the humanitarian evidence guide

Photograph: © Patrick Brown/UNICEF/Panos Pictures

The Health in Humanitarian Crisis Centre (London School of Hygiene), the Alliance for Useful Evidence (Nesta), and Evidence Aid – with input with many collaborators from across the humanitarian and research community - have written a new practice guide to help humanitarian actors navigate the production, assessment and use of evidence in humanitarian settings.

                                                                                           

To celebrate the launch of the guide, we would be delighted if you could join us at one of three events on 21-22 November (Humanitarian Evidence Week).

21 November, 2pm: Save the Children, 1 St John's Lane, London EC1M 4AR. Speakers: Prisca Benelli (Save the Children), Jonathan Breckon (Nesta, Alliance for Useful Evidence) and Ben Heaven Taylor (Evidence Aid).

22 November, 11am: Parliamentary Launch, Houses of Parliament, Westminster, London. Speakers: Baroness Sheehan, Ben Heaven Taylor (Evidence Aid), Phil Davies (Evidence Aid) (tbc), and Jonathan Breckon (Nesta, Alliance for Useful Evidence) (tbc).

22 November, 5pm: University of Oxford (Kellogg College), 60-62 Banbury Road, Oxford, OX2 6PN. Speakers: Prisca Benelli (Save the Children), Ben Heaven Taylor (Evidence Aid), and Kamal Mahtani (Centre for Evidence-Based Medicine). Chaired by Professor Carl Heneghan.

The Oxford event coincides with the timing of our Work in Progress seminar. This week's seminar is therefore cancelled and we urge those who are interested to attend the event at Kellogg College. Watch this space for further discussion of Humanitarian Evidence Week.