Charles Parrack writes:
Earlier this year, in 2018, CENDEP hosted a one-day workshop
on longitudinal studies in shelter at Oxford Brookes University. Attendees came
from a wide range of backgrounds with experience in the humanitarian sector,
shelter and academia.
The topic has been raised in various shelter cluster and
non-cluster meetings over several years. At the Global Shelter Cluster meeting
in Geneva 2017, the Shelter Projects working group found strong interest in
longitudinal studies and, as a result, proposed to advance discussions on a
more organised approach to methodology, procurement, dissemination and sectoral
learning/evidence.
Longitudinal studies are by no means a new concept; indeed,
evidence shows them to be a part of other sectoral practice, however they are something
that the shelter sector does not yet embed into its own practice. Why is this?
Some thoughts from the workshop suggested the answer to be a combination of
rapid agency staff turnover, funding challenges, a lack of accountability and
perhaps a fear of sharing programme failures.
 |
Photograph: © Patrick Brown/UNICEF/Panos Pictures |
Why are they needed?
Discussion ranged over the following questions: How would impact
measurement benefit the sector? How would it benefit affected communities? There
was a consensus that the outcomes of impact studies would encourage
institutional learning particularly if disseminated and shared as best
practice. There is a need within the sector to see the long-term consequences
of our actions; to understand how beneficiaries take ownership of shelter and
how safety is enhanced or reduced after implementation. Following up on
programmes, two years, five years or even ten years after completion can enable
agencies to better understand and engage with issues of prevention,
sustainability, resilience and cost-efficiency which can feed back into and
improve future programming. Whether it is the original agency who carries out
the study, or an independent organisation, it is important that the results consider
the context both past and present and are not a biased representation. Consideration
of the learning opportunities for affected populations, implementers and donors
drew an extensive list in the workshop highlighting a strong benefit for many actors.
Methodology
Longitudinal studies involve repeatedly gathering data on
the same focus area over a long period of time. The unit of analysis, that is
what is being measured, can be a person, shelter, household or community. It is
important the unit of analysis is confirmed and remains the focus of the study
as the methodology’s most important aspect is being able to compare in order to
fully understand change overtime. Time is the best test of a programme’s
viability, effectiveness and sustainability.
Shelter is delivered in various forms, both soft and hard
interventions, and it is not necessarily a structure in the physical sense of
the word. Perhaps 90% of what the shelter sector delivers is provided to meet
short-term emergency needs. It does not take into account the future long-term
consequences and may therefore, be difficult to locate, let alone measure. With
this in mind, discussion looked towards certain shelter interventions that are
quick and common enough for us to be able to find out what their impact has
been, for example, transitional shelters. Transitional shelters are intended to
be transformative in their nature, therefore it may be possible for us to track
their transformation. The question of how their transformation may be followed and
what exactly we may want or need to understand generated further conversation. What
was voiced by many was the role of the affected communities in longitudinal
studies; that perhaps it is the residents themselves who document their own
recovery and their qualitative data can be understood by agencies in a more
quantitative way if preferred. Participation from affected communities must
continue to happen to institutionalise the process of learning so we, as a
sector, can continue to develop and improve programming that meets the needs of
the user. For shelter in the context of conflict, the journey of displaced
populations may offer the opportunity to study soft modalities in the form of
long-term displacement.
Questions and
challenges
As long-term studies are not standard practice in the
shelter sector challenges and questions surrounding the implementation of such
studies were raised:
-
What are the criteria for success?
-
Do long-term studies link with the localism
agenda?
-
Changing the building culture takes time and is
not part of the emergency programme
-
How to fund studies with long-term intentions
when the programmes are inherently short-term?
Conclusions
Long-term effects of shelter programming can be both
positive and negative but these effects cannot be quantified in the end of a
project evaluation; they require time to develop and change in response to the
intended user. It is clear from this workshop and Global Shelter Cluster
findings that many organisations have the willingness to carry out long-term
studies, but do not have the time or indeed, the funding. Understanding the
consequences of our actions will enable better design and implementation of improved
shelter programmes for reduced costs and greater effectiveness.
The group suggested many ways we can measure shelter, from
technical standards like the Sphere Standards to quantifiable measures like
value for money; to less tangible, but just as important, indicators such as
level of happiness and sense of ownership. The shortlist of these was; safe
shelter and settlements, user satisfaction, enabling informed decisions,
adaptability, saving lives and value for money. Uninterrupted sleep and beauty
were shortlisted as overall proxy indicators of a successful shelter. In
general, the discussants agreed that the Sphere Standards offered good
indicators for identifying success when looking at; security, peace, dignity,
security of tenure and protection from forced eviction.
Based on the day’s thoughts, it is likely that these studies
would have a varied audience including funders, agencies and university courses
with wide opportunities for learning and improving practice for all. Whether
you prefer to call them ‘longitudinal studies’ or ‘long-term impacts’, the
interest for putting them into action is strong and we look forward to
continuing the conversation from the workshop.