Scholarly article on topic 'GoWell: The challenges of evaluating regeneration as a population health intervention'

GoWell: The challenges of evaluating regeneration as a population health intervention Academic research paper on "Sociology"

CC BY-NC-ND
0
0
Share paper
Academic journal
Preventive Medicine
OECD Field of science
Keywords
{"Population health intervention research" / "Urban regeneration" / "Evaluation methods"}

Abstract of research paper on Sociology, author of scientific article — Lyndal Bond, Matt Egan, Ade Kearns, Carol Tannahill

Abstract Objective Urban regeneration can be considered a population health intervention (PHI). It is expected to impact on population health but the evidence is limited or weak, in part due to the difficulties of evaluating PHIs. We explore these challenges using GoWell as a case study. Method A 10-year evaluation of housing improvement and urban regeneration in 15 deprived areas in Glasgow, Scotland (2005–2015). Results Challenges faced include: definition and changing nature of the intervention; identifying the recipients of the intervention; and constraints of study design affecting capacity to attribute effects. We have met these challenges by: adapting the evaluation to take account of changing intervention plans and delivery; making pragmatic choices about which populations to focus on for different parts of the study; and taking advantage of delayed delivery of some components to identify controls. Conclusion Commitment to a long-term evaluation by the Scottish Government and other partners has enabled us to develop a package of studies to investigate health and other outcomes, and the processes of a PHI. GoWell will contribute to the evidence base for interventions focused on tackling the wider determinants of health and help policymakers to be more explicit and realistic about what regeneration might achieve.

Academic research paper on topic "GoWell: The challenges of evaluating regeneration as a population health intervention"

GoWell: The challenges of evaluating regeneration as a population health intervention^

Lyndal Bond a,*< Matt Egan a, Ade Kearns b, Carol Tannahillc

a MRC/CSO Social and Public Health Sciences Unit, Glasgow, UK b Urban Studies, University of Glasgow, UK c Glasgow Centre for Population Health, UK

ARTICLE INFO

Available online 14 August 2013 Keywords:

Population health intervention research Urban regeneration Evaluation methods

ABSTRACT

Objective. Urban regeneration can be considered a population health intervention (PHI). It is expected to impact on population health but the evidence is limited or weak, in part due to the difficulties of evaluating PHIs. We explore these challenges using GoWell as a case study.

Method. A 10-year evaluation of housing improvement and urban regeneration in 15 deprived areas in Glasgow, Scotland (2005-2015).

Results. Challenges faced include: definition and changing nature of the intervention; identifying the recipients of the intervention; and constraints of study design affecting capacity to attribute effects. We have met these challenges by: adapting the evaluation to take account of changing intervention plans and delivery; making pragmatic choices about which populations to focus on for different parts of the study; and taking advantage of delayed delivery of some components to identify controls.

Conclusion. Commitment to a long-term evaluation by the Scottish Government and other partners has enabled us to develop a package of studies to investigate health and other outcomes, and the processes of a PHI. GoWell will contribute to the evidence base for interventions focused on tackling the wider determinants of health and help policymakers to be more explicit and realistic about what regeneration might achieve.

© 2013 The Authors. Published by Elsevier Inc. All rights reserved.

Introduction

Poor health is associated with poorer living circumstances (Clark et al., 2007; Croucher et al., 2007; Davison and Lawson, 2006; Ellaway et al., 2012; Meijer et al., 2012; Renalds et al., 2010; Truong and Ma, 2006; Yen et al., 2009) and there is therefore, an expectation that housing improvements and area regeneration in disadvantaged urban areas will improve health and reduce social inequalities in health (Kearns et al., 2009; WHO Commission on Social Determinants of Health, 2008). Urban regeneration can thus be considered a public health intervention (PHI) whereby improvements in health and wellbeing are stated as specific aims of regeneration strategies (Beck et al., 2010). Regeneration generally includes a range of activities that may potentially improve the interlinked dimensions of household, dwelling, community and neighborhood environment in urban areas, thereby impacting on many of the social determinants of health (Dahlgren and Whitehead, 2007). However, to date the evidence that regeneration activities achieve these health benefits is limited or weak

☆ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

* Corresponding author at: Centre of Excellence in Intervention and Prevention Science, 15-31 Pelham St, Carlton South, Vic 3070, Australia. Fax: +61 3 9667 1375. E-mail address: lyndalbond@ceips.org.au (L. Bond).

and any health effects are small (Jacobs et al., 2010; Thomson et al., 2009). Evidence for long-term effects and the mechanisms by which different interventions or combinations of interventions might lead to positive health outcomes tend also to be absent (Atkinson et al., 2006; Jacobs et al., 2010; Lindberg et al., 2010; Thomson et al., 2006). There are also concerns that regeneration activities may have unintended consequences of social disruption and displacement through gentri-fication (Fullilove, 2004; Huxley et al., 2004; Lindberg et al., 2010; Paris and Blackaby, 1979).

Undertaking an evaluation of regeneration is difficult — these are complex interventions not easily suited to being assessed using RCT methods. In the USA two well-researched regeneration programs have used random allocation. The Gautreaux 1 Program used a quasi-random allocation of households to suburban locations (Rubinowitz and Rosenbaum, 2000). Informed by this program the Moving to Opportunity Demonstration used random allocation to experimental, comparison and control groups for relocation purposes (Briggs et al., 2010). Studies of these programs have focused mostly on outcomes related to employment, earnings, education/ college, and crime or victimization, with some studies considering health behaviors such as smoking and sexual activity among young people (Briggs et al., 2010; Rubinowitz and Rosenbaum, 2000). However these two studies were not strictly evaluations of urban regeneration but rather of relocation with the combined objectives

0091-7435/$ - see front matter © 2013 The Authors. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.! 016/j.ypmed.2013.08.007

of moving people away from concentrated poverty as well as away from racially segregated places. The focus on relocation and the combination of poverty and racism in US society means that it is difficult to transfer the findings to other national contexts where these problems are less extreme and where the response to such problems tends to be focused on regeneration of areas rather than relocation, so-called 'dilution' rather than 'dispersal', as in the UK (Kearns, 2002). Looking more specifically at interventions focused on housing improvement or area regeneration, there have been four published studies that have used RCTs to evaluate warmth improvements (Jacobs et al., 2010; Ludwig et al., 2012; Thomson et al., 2009), interventions that are much easier to randomize than such things as demolition of tower blocks. Most other evaluations of regeneration or housing improvement have used quasi-experimental methods, with relatively short follow-up periods and, while not necessarily having small numbers they are often not powered to find small effects and suffer from sample bias and low levels of recruitment and follow-up (Thomson et al., 2013).

The lack of good quality evaluations is not just an issue for investigating the effects of urban regeneration but is rather a problem for many PHls (Craig et al., 2008; Egan et al., 2010; Petticrew et al., 2004; Thomson, 2008; Weitzman et al., 2009; Whitehead et al., 2004). PHls are challenging to evaluate but we argue that it is important to do so. Not doing so leads to less research in this field, and therefore contributes to the so-called inverse evidence law, which suggests that policies more geared towards tackling the wider determinants of health often have little or no robust evidence upon which to base decisions that may (a) potentially have long term impacts on individuals and communities; and (b) cost a lot of money (Hawe and Potvin, 2009; Morabia and Costanza, 2012; Ogilvie et al., 2005; Petticrew et al., 2004). Much of the discussion of these challenges in the current literature tends to be at a rather abstract level. In contrast, this paper uses a worked example of a large scale regeneration evaluation (GoWell) to explore in detail the challenges of evaluating natural experiments involving complex social interventions (Craig et al., 2012), and some ways of overcoming those challenges. Here we use GoWell to illustrate the challenges of evaluating public health interventions enacted in or through non-health sectors. The following provides a brief description of regeneration in Glasgow, the focus and study components of GoWell and then, the challenges of evaluating this type of intervention.

Glasgow and regeneration

Glasgow is the largest city in Scotland. lt has high concentrations of poverty, disadvantage and poor health. There are stark area-based health inequalities with life expectancy in the most disadvantaged areas estimated to be at least 15 years less than in the least disadvantaged (Hanlon et al., 2006; Palmer et al., 2006; Walsh, 2008; WHO, 2008).

Glasgow's socially disadvantaged areas include:

• post-second world war housing estates situated on the edges of Glasgow city (referred to as peripheral estates). These largely comprise low-rise and medium-rise tenement flats (large buildings divided into flats off a common stairwell) and houses.

• inner-city estates comprising post-war multi-storey flats and tenement flats, gardened estates of houses and flats mostly dating from the 1930s, and old neighborhoods dominated by 19th and early 20th century tenement flats.

The intervention(s)

Social or council housing remains a dominant form of housing in Glasgow with about 40% of housing being socially rented. (This compares to about 17% socially rented UK-wide). In 2003, over 80,000 socially rented homes in the city were transferred from public ownership to Glasgow Housing Association (GHA), a third sector social

landlord. Most of these 80,000 homes needed improvement to meet the Scottish Housing Quality Standard (Communities Scotland, 2007)1 and a major regeneration program was developed which included housing improvements, building new socially rented and private sector homes, demolition (approximately 20,000 homes), improvements to the physical neighborhood environment, new/improved amenities and services, and community interventions (see Box 1 for details).

GoWell

ln GoWell we are studying this large, multi-faceted program of housing investment and area regeneration in 15 areas across Glasgow. The GoWell Program began in 2005 and was a planned 10-year evaluation aimed at exploring the links between regeneration and the health and wellbeing of individuals, families and communities. lt also aimed to establish the nature and extent of these impacts and the processes that have brought them about, to learn about the relative effectiveness of different approaches, and to inform policy and practice.

GoWell is a research and learning program comprising multiple components, and multiple research methods and uses a pragmatic comparative design and mixed methods. The components of the evaluation are shown in Box 2. GoWell also has a strong focus on dissemination and community engagement activities including: regular community newsletters to residents and presentations of local data to community resident groups, briefing papers primarily for policymakers and practitioners, website, blogs and twitter and an annual event with participation from housing associations, Glasgow City Council, Scottish Government, community and voluntary sector organizations, residents and academics.

Challenges for evaluating regeneration

The regeneration of areas of Glasgow meets most definitions of a complex intervention and we have faced (and sometimes overcome) multiple challenges in this evaluation. We present these challenges under four headings:

1. Interventions: definition, changing phasing, nature of the interventions over time and likely effects on health and its social determinants

2. Recipients: identification of the recipients of the intervention and participation in the evaluation

3. Evaluation: attribution of effect, evaluation of moving targets, definition of pragmatic controls

4. Stakeholders: tensions and changing policy and practitioner priorities.

Challenges with the intervention

The intervention is difficult to define. lt comprises multiple, interrelated activities (demolition, new builds aimed at tenure diversification, housing improvements, and social and community interventions), delivered in different ways to different people in different places and at different time points. The precise mixture and sequencing of interventions delivered to the areas and communities are not always preplanned or delivered according to plan, particularly when regeneration is implemented by a range of public sector partners without a strong governing structure in place to oversee regeneration in any one area or across the city.

The boundaries of the interventions can be 'fuzzy', as can be the boundaries of the affected areas. For example, we have found it challenging to delimit the areas affected by relocations or define a receiving community; to assess how much of a large peripheral estate can be

1 The Scottish Housing Quality Standard consists of five broad housing criteria, which must all be met if the property is to pass. These are: 1) must be compliant with the tolerable standard 2) must be free from serious disrepair 3) must be energy efficient 4) must have modern facilities and services and 5) must be healthy, safe and secure.

Key elements of Glasgow's regeneration program, 2005.

• Housing improvement: including repairs or replacements to roofs, external cladding, doors, windows, kitchens, bathrooms, electrics, heating, common areas, etc., based on surveyor's assessments of each property.

• New builds: building socially rented and private sector new homes in green and brown field sites. Plans include encouraging private sector building of affordable owner occupied homes in areas dominated in the past by socially rented property (referred to as tenure diversification) and a dedicated "reprovisioning" programme to provide new homes for residents relocated out of tower blocks into low-rise housing.

• Transformational regeneration: a neighborhood-wide approach to regeneration planning involving several or all of the following: relocation of residents, demolitions, new builds, housing improvements, improvements to the physical neighborhood environment, new/improved amenities and services, and community interventions, to create sustainable, mixed-tenure communities. Eight areas across the city are targeted for this type of intervention, three of which are in the GoWell study.

• Local regeneration: similar to transformational regeneration but targeting smaller pockets of disadvantage situated in larger neighborhoods, and typically involving less demolition. Seven locations across the city are targeted for this approach, three of which are in the study.

• Community interventions (referred to by planners as 'wider actions'): include employment and training initiatives, activities for young people, improved play areas, support for vulnerable people, addiction support, parent and child groups, financial advice and debt management, services for elderly residents, community buildings and resources, and other investments intended to strengthen and support communities in specific localities or across the city.

• Community engagement and empowerment: improving the way tenants are informed and consulted and involved in decisions affecting their homes, neighborhoods, communities and public services. Includes provision of information, surveys, consultation exercises and changes in housing management processes (including the gradual splitting of GHA into smaller local housing organizations working towards becoming independent Registered Social Landlords). These activities may take place in any social housing area, but with more regularity and intensity in regeneration areas.

• Wider effects: it is hoped that transforming highly disadvantaged neighborhoods and reducing social problems in those areas will benefit adjoining neighborhoods.

Study components of the GoWell Program, Glasgow 2006-2014.

Quantitative studies

Community health and wellbeing survey: a cross-sectional study of change in the 1 5 communities, involving about 4000 to 6000 residents repeated every 2-3 years (Wave 1 2006, Wave 2 2008, Wave 3 2011 and Wave 4 planned for 2014). GoWell longitudinal study: this is nested within the community health and wellbeing survey to study the impacts of housing improvements and area regeneration upon residents. It comprises: i) a 'remainers' cohort i.e. those people who were interviewed in Wave 1 or 2 of the survey and are still living in the same study area, divided into those in regeneration areas and those in other areas ii) an 'outmovers' cohort i.e. those people who move voluntarily or who are relocated out of regeneration areas, either permanently or temporarily, and iii) an 'inmovers' cohort of people who move into one of the regeneration areas.

Ecological study to monitor changes across Glasgow: This component involves investigating the wider context within which neighborhood regeneration is taking place. This includes researching the expectations of policy-makers and practitioners and analyzing of routine data and data linkage to i) monitor the changes relating to housing and health throughout Glasgow so that the changes in the study areas can be looked at in the context of wider trends, and ii) investigate whether area-based inequalities in health and deprivation across the city are reduced over time through regeneration.

Qualitative studies

Governance, empowerment and participation: using focus groups and in-depth interviews with residents, policy-makers and practitioners to gain an understanding of how the governance of neighborhood change is working out in practice, this component enables us to identify those aspects of change most valued by residents and to suggest the most successful approaches to co-operation and engagement.

Lived realities: a longitudinal study of families living through regeneration. These families have been moved from multi-storey flats due for demolition into surrounding areas and in depth interviews are conducted with adults and children.

Evaluations of 'wider action' interventions and aspects of regeneration policy: focusing on specific initiatives aimed at improving particular aspects of communities or in-depth evaluations of certain policies or aspects of regeneration, such as play area improvements and youth diversionary program.

thought to be affected by private sector housing developments or to clearly categorize different approaches to community consultation.

The plans for some areas are unclear and have been revised several times during the period of our study, resulting in the desired end-state being somewhat unknown. Masterplans have been produced but seem not to form a fixed reference point for interventions.

Timings of components of the intervention are variable and flexible so that measuring actual against intended progress is difficult. Plans have changed over time for a variety of reasons including: response to findings from the GoWell study (e.g. residents' use of GoWell data to reverse GHA's decision to demolish a number of multi-storey flats; GoWell data being used to inform strategic plans); the slowing of activity due to the economic recession post-2008; and most recently a

bid by Glasgow City Council for the 2018 Youth Olympics. The recession has had differential effects on the implementation of components of the intervention (see Table 1) and the bid for the Youth Olympics has seen a major change in the planned demolition, regeneration and timing of rebuilding of one of GoWell's study areas — all multi-storey flats now to be demolished and rapid rebuilding/regeneration of the area is to take place.

In response to these challenges we have adapted the evaluation to take account of changing intervention plans and delivery. For example, at baseline we had proceeded on the premise that two neighborhoods dominated by social rented homes would experience intensive private sector home building to encourage a greater mix of tenures. However, by the second and third waves it was clear that the private sector

Table 1

Changes to the intervention and implementation of urban regeneration in Glasgow, 2006-2014.

Planned interventions (2006)

Emerging/changing interventions (2008,2011)

Demolition (clearances/relocations)

Neighborhood redesign New build/tenure diversification

Housing improvements Social/community interventions

Partial clearance and demolition in 4 neighborhoods. Changes to demolition plans in one study area due to bid for Youth Olympics in 2018 with the new plan involving demolition of all tower blocks and rapid regeneration. Largely at planning and demolition stage

1st phase for social sector, but lagging behind demolition, thus requiring more relocatees to be housed elsewhere. Recession slows private sector activity in peripheral estates and demolition areas. Good progress made with improvement of existing homes in many areas. Many smaller interventions, difficult to track, no overall strategy for specific areas

homes had not been built to the anticipated scale, and in fact the dominant form of housing intervention in these neighborhoods turned out to have been housing improvement rather than tenure diversification. As a result, we have been able to comment on the barriers to delivering tenure diversification during a recession, while our longitudinal analysis for these neighborhoods has focused on the effects of housing improvement. This adaptation to the evaluation also required us to reconsider the categorization of our study areas and our analytical treatment of the different areas has had to be flexible and to change between study waves.

In order take account of new or unexpected circumstances, we have put a lot of effort into finding out how the regeneration plans have changed over time, and into monitoring progress with ongoing interventions. This has become a major research task in a way we had not anticipated; one which has required good contacts with the city's key service providers and significant assistance from them. Nonetheless we are conscious that some service providers have proved more willing or able to provide us with information than others, and so our knowledge of intervention delivery is, we think, substantial but not complete (see Table 1).

Possible effects on health of the interventions

Through a review of the relevant policy literature, as well as interviews with key respondents in national and local roles in Scotland, we established that there were no clear theories of change or logic models helping to make explicit the health or social outcomes expected to be affected by regeneration, and/or the mechanisms by which these outcomes would be achieved (Beck et al., 2010). For example, diversification of tenure (one aim of regeneration in Glasgow and elsewhere) is purported to bring a range of social, environmental and residential benefits to residents although how this will occur is rarely made explicit nor is there good evidence that it occurs (Bond et al., 2011; Sautkina et al., 2012). Therefore, we have focused on a range of plausible health outcomes from regeneration including: mental wellbeing; health behaviors; and health-related quality of life. However, in addition to health and wellbeing outcomes, we have also examined residential (housing and neighborhood) outcomes and social and community outcomes. As Petticrew (2013) argues there is often no primary outcome for social interventions and the ones chosen reflect both the researchers' and the stakeholders' perspectives. Focusing only on health and well-being outcomes in the case of housing and regeneration interventions 'may result in biased conclusions about their value' (p.91).

As the study has progressed we have developed our own sense of what some of the key mechanisms of change might be, including monitoring the relevant research literature produced in the years that followed our baseline survey. To this end, we are currently testing through our analysis the efficacy of several pathways to outcomes including: environmental; psychosocial; social; and empowerment. Moderators within these pathways include such issues as place attachment and resident attitudes to change. The pathways, mediators and moderators included in our analysis vary depending on the particular aspect of the intervention being studied. Through this approach we have been able to turn the variability of the intervention into a strength.

Single, tightly defined interventions do not allow for this sort of detailed look at different mechanisms.

Challenges with the recipients of the intervention?

GoWell aims to investigate the impact of housing improvements and regeneration at the individual level, the community level and city-wide. In each case there are difficulties in defining both the numerator (those receiving the interventions) and the denominator (the total population of interest). This can be illustrated particularly clearly at the community level. While interventions designed to foster community empowerment, cohesion and sustainability are aimed at 'the community', this is not properly constituted as a policy target group, so rather than being an active participant, the community can be considered an absent or passive recipient of the intervention.

Residents may be the direct or indirect recipients of regeneration interventions, and it is possible that those most likely to benefit from regeneration activities may be the children and young people in these communities or indeed future generations.

To some extent, our 'solution' to these challenges rests on making pragmatic but we hope, justifiable choices about which populations to focus on for different parts of the study. Once again, these decisions may change over time as they draw on our own growing knowledge of the interventions, their spatial and social reach, and their possible pathways and outcomes. We have attempted to spatially delimit the areas affected by an intervention, or the area in which residents may take advantage of a new service or program, even if the residents themselves are not all aware of its operation or existence. As GoWell has progressed we have added components focused on family's (Egan and Lawson, 2012), young people's (Neary et al., 2012) and asylum seekers' experience of regeneration (GoWell, 2009a).

Challenges in studies in areas of deprivation

We have identified two major challenges in studying areas of deprivation: diversity of residents, and instability of households. Residents in our study areas are diverse and many areas are not the stable, working class communities, which were the focus of urban regeneration in the past. ln particular, residents vary according to their nationality (tremendous diversity and numbers of refugees and asylum seekers in some areas) and their degree of support needs for issues like substance dependencies (GoWell, 2009b). We have found great instability of households, in part due to the nature of the interventions (decanting and relocating some residents) and the prevalence of significant lifeevent complications such as relationship breakdown, victimization, hospitalization and bereavement (Egan and Lawson, 2012). Methodological challenges result in relation to examining differences between comparison groups (adjusting for known confounders can help address this problem but does not fully 'solve' it) and difficulty tracking participants over time. On the other hand, both are features of the study population that can be explored in more detail to better understand intervention effects including the social patterning of those effects. Hence, our approach to such issues is to do what we can to limit, but not eradicate, bias, while also treating challenging population dynamics as an

opportunity to learn more about how the interventions play out within this fluid and heterogeneous context. This is one of the values of GoWell, namely that it looks at how the effects of interventions can differ depending on a variety of challenging social circumstances; comparisons with stable residential areas will not tell us that.

A further challenge lies in engaging residents in the research and thereby obtaining good response rates and representative samples. GoWell has achieved response rates of about 50% over the three waves of data collected so far, which we consider reasonable given the challenges described above combined with police safety campaigns in many of our study areas urging residents not to open their doors to unexpected callers. To help us maintain our response rate we have adopted a number of techniques, including newsletters and neighborhood awareness raising, prize draws and vouchers for participants.

Challenges with evaluation: attribution of effect, and definition of pragmatic controls

Regeneration can be considered a natural experiment (Craig et al., 2012). Researchers have no control over the planning, delivery or allocation of the intervention(s), which are not neatly contained within a certain period of time, nor necessarily mutually exclusive. Further the residents in study areas may have been exposed to previous urban renewal activities. Guidance for the evaluation of natural experiments states that evaluations are best undertaken when the implementation is 'immediate' and the effects are likely to be large and happen soon after the event (e.g. smoking ban legislation) (Craig et al., 2012). Urban regeneration can be thought of as a natural experiment but it does not meet these guidelines: it does not happen overnight; effects are not likely to be large or immediate. Evaluation of a slow natural experiment raises particular problems with attributing effects and defining controls.

Difficult to establish or attribute effects to the intervention(s)

When evaluating an intervention whose effects may take many years to be realized it is often not possible to identify control or comparison areas that will not also be exposed to some regeneration activities during that time. Thus it is difficult to disentangle intervention effects from confounding variables. We have tried to address this challenge in a number of ways. First, by comparing experiences of different types of regeneration to look for differential effects and pathways rather than a single 'intervention' effect and second, comparing GoWell health and social outcomes with Glasgow-wide data. Across the city, it is possible to identify areas for comparison, which have not had the same extent or mix of interventions as our study areas, but which are comparable in other ways, thus enabling us to tease out and attribute intervention effects using ecological data. Again, this relies upon the careful identification of the nature and extent of regeneration activity in different places.

Our approach to the analysis of survey data contributes to the assessment of attribution. We have taken advantage of unexpected delays in the implementation program due to an economic recession. For example, by 2008 many participants had not experienced demolition or housing improvement and these we have used as a pragmatic control group to examine short to medium term effects of these interventions on current recipients (Bond et al., 2012; Egan et al., 2013). Thus, while unpredictable change presents a major challenge, we have tried to take advantage of it where possible by identifying different ways (at different time points) in which intervention exposure varies across our sample of participants. Without intending to do so, practitioners have created a 'waiting list' effect within the interventions that can help us assess intervention impacts and dose-response relationships.

Our ability to do this type of analysis is the result of efforts to link practitioner-held information on the interventions, including the dates and exact nature of actions taken, to our survey data on a case-by-case basis through property addresses. This is a time-consuming exercise

as the data held by practitioners is not readily user-friendly for research purposes. It is also uncommon in regeneration evaluations to do this, as much analysis is only conducted on an area basis, but it adds another level to our ability to identify the effects of regeneration on residents, and relies upon a high degree of trust between the researchers and practitioners for individual-level data to be shared in this way.

Our use of several time points in longitudinal analysis (eventually four-time points) is another way of using the analysis of the survey data to test pathways to outcomes and establish whether changes in health and wellbeing outcomes can be attributed to more immediate changes in residential circumstances brought about by housing and regeneration interventions. We can also use repeated analysis following subsequent survey waves to address unanswered questions arising from previous analysis. For example, after the first two survey waves, we found an absence of health decline among residents of demolition areas (Egan et al., 2013), as a result of which we are exploring several potential explanations for this apparent 'protective' effect on health in our analysis of the third wave of survey data (linked longitudinally to the previous two waves).

Finally, our mixed methods approach can help with the issue of attribution of effect. For example, our survey findings indicate relatively negative trends in social outcomes in areas that have received relo-catees from regeneration areas. We cannot tell through the survey evidence whether or not this is due to the arrival of 'incomers' from elsewhere, so-called 'negative spill over effects' (Kleinhans and Varady, 2011), but we are embarking on qualitative research in these areas to ascertain whether this appears to be the case from residents' accounts of social change.

Stakeholder challenges: multiple funders, multiple interests, changing priorities

GoWell has multiple stakeholders: the Scottish Government and other funders, housing associations, community and voluntary sector organizations, communities, residents and academics social service practitioners and academics. GoWell is funded by the Scottish Government, NHS Health Scotland, NHS Greater Glasgow and Clyde, Glasgow Centre for Population Health and supported in kind by the University of Glasgow and the MRC/CSO Social and Public Health Sciences Unit. GHA, the organization responsible for much of the housing-led regeneration activity, funds the Community Health and Wellbeing Survey. All have vested, but sometimes different, interests in the study. It is a long term investment for all funders, and there is a reasonable expectation that GoWell can and should respond to changing stakeholder interests/focus and research questions which were not part of the original plans. This presents challenges or tension for the researchers —being responsive without abandoning the initial, primary research questions or diminishing the quality of established research streams.

Undertaking PH1R like GoWell is also a challenge for academic careers. Such research is inherently long-term and risky. While it is more acceptable now to publish negative or null results, these results are often based on somewhat less than perfect study designs and low response rates and are therefore difficult to 'sell' to peer reviewers and academic journals. Moreover, the cross-disciplinary and system-based nature of the research means that outputs sit less neatly within specific academic domains.

We have used our study design to advantage where we can: although we do not include non-deprived control areas, we have been able to show, firstly, that assumptions about what will work in more affluent areas do not always apply in deprived areas; and, secondly, that there is a great deal of variation in circumstances that mediates and moderates impacts even within a group of deprived areas.

There is also a tension between the types of outputs that are valued and considered useful. On the one hand the timeframe for publishing peer-reviewed journal articles (sometimes 12 months or more between submission and final publication) is not particularly useful for other

stakeholders; on the other hand, reports and briefing papers for the policy-makers are often not valued by academia. We have moved to produce more syntheses of findings on particular issues so as to consolidate our academic work, and make it more usable for policy-makers and practitioners.

Conclusion

In this paper we have outlined a number of challenges to evaluating a PHI delivered through non-health sectors. These challenges include consideration of what the intervention comprises, the nature of the recipients, the difficulty of attribution of effect due to limitations in possible study designs, specific challenges in studying areas of deprivation, and the challenges and risks related to different agendas of funders, stakeholders and researchers. GoWell has taken a mixed methods approach and sought to synthesize findings from these various methods to compare the health and wellbeing effects of different approaches to regeneration, by generating theory on pathways from regeneration to health and exploring the attitudes and responses of residents and other stakeholders to neighborhood change. Our approach has parallels with contribution analysis, whereby we develop the contribution story as an iterative process, examining further theories of change and contributory factors as we go along (Mayne, 2008).

We work closely with our stakeholders and we have been able to be responsive to changes in circumstances with respect to the implementation and policy focus. Having a stated commitment to a long-term evaluation by the Scottish Government and others (with 3-yearly review cycles) has enabled us to develop an ambitious and extensive package of studies to investigate not just the health outcomes of a PHl, but also multiple outcomes, on many groups experiencing these activities and the processes of the intervention. By doing so, we hope GoWell will contribute to the evidence base for interventions focused on tackling the wider determinants of health and importantly, help policymakers to be more explicit and realistic about what regeneration might achieve.

Conflict of interest

The authors declare that there are no conflicts of interests.

Acknowledgments

GoWell is funded by the Scottish Government, NHS Health Scotland, Glasgow Housing Association, Glasgow Centre for Population Health and NHS Greater Glasgow & Clyde. LB & ME are funded by the Chief Scientist Office at the Scottish Government Health Directorate as part of the Evaluating Social Interventions program at the MRC Social and Public Health Sciences Unit (U.130059812).

References

Atkinson, R., Thomson, H., Kearns, A., Petticrew, M., 2006. Giving urban policy its 'medical': assessing the place of health in area-based regeneration. Policy & Politics 34, 5-26.

Beck, S.A., Hanlon, P.W., Tannahill, C.E., Crawford, F.A., Ogilvie, R.M., Kearns, A.J., 2010. How will area regeneration impact on health? Learning from the GoWell study. Public Health 124 (3), 125-130. Bond, L., Sautkina, E., Kearns, A., 2011. Mixed messages about mixed tenure: do reviews

tell the real story? Hous. Stud. 26 (1), 69-94. Bond, L., Egan, M., Kearns, A., Clark, J., Tannahill, C., 2012. Smoking and intention to quit in deprived areas of Glasgow: is it related to housing improvements and neighborhood regeneration because of improved mental health? J. Epidemiol. Community Health. http://dx.doi.org/10.1136/jech-2012-201828. Briggs, X.D., Popkin, S.J., Goering, J., 2010. Moving to Opportunity: The Story of an

American Experiment to Fight Ghetto Poverty. Oxford University, Oxford. Clark, C., Myron, R., Stansfeld, S., Candy, B., 2007. A systematic review of the evidence on the effect of the built and physical environment on mental health. J. Public Ment. Health 6,14-27.

Communities Scotland, 2007. Scottish Housing Quality Standards Progress Report.

Communities Scotland, Edinburgh. Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M., 2008. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ337, a1655.

Craig, P., Cooper, C., Gunnell, D., et al., 2012. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J. Epidemiol. Community Health 66 (12), 1182-1186. http://dx.doi.org/10.1136/jech-2011-200375.

Croucher, K., Myers, L., Jones, R., Ellaway, A., Beck, S., 2007. Health and the Physical Characteristics of Urban Neighborhoods: A Critical Literature Review. Glasgow Centre for Population Health, Glasgow.

Dahlgren, G., Whitehead, M., 2007. European Strategies for Tackling Social Inequities in Health: Levelling Up. Part 2. WHO Regional Office for Europe, Copenhagen.

Davison, K.K., Lawson, C.L., 2006. Do attributes in the physical environment influence children's physical activity? A review of the literature. Int. J. Behav. Nutr. Phys. Act. 3 (19).

Egan, M., Lawson, L., 2012. Residents' Perspectives of Health and Its Social Contexts: Qualitative Findings from Three of Glasgow's Transformational Regeneration Areas. GoWell, Glasgow.

Egan, M., Beck S., Bond, L., et al., 2010. Protocol for a mixed methods study investigating the impact of investment in housing, regeneration and neighborhood renewal on the health and wellbeing of residents: the GoWell programme. BMC Med. Res. Methodol. 10,10-41.

Egan, M., Katikireddi, V.S., Kearns, A., Tannahill, C., Bond, L., 2013. Impacts of neighborhood demolition and housing improvements on residents' health: a prospective controlled study of two natural experiments in urban renewal. Am. J. Public Health 103 (6), 47-53.

Ellaway, A., Benzeval, M., Green, M., Leyland, A., Macintyre, S., 2012. 'Getting sicker quicker': does living in a more deprived neighborhood mean your health deteriorates faster? Health Place 18,132-137.

Fullilove, M.T., 2004. Root Shock: How Tearing up City Neighborhoods Hurts America, and What We Can Do About It. Ballantine Books, New York.

GoWell, 2009a. GoWell Findings: Asylum Seekers and Refugees in Glasgow's Regeneration Areas, 2006-7. (GoWell Briefing Paper No 3) GoWell, Glasgow.

GoWell, 2009b. Progress for People and Places: Monitoring Change in Glasgow's Communities. Evidence from the GoWell Surveys 2006 and 2008. GoWell, Glasgow.

Hanlon, P., Walsh, D., Whyte, B., 2006. Let Glasgow Flourish. Glasgow Centre for Population Health, Glasgow.

Hawe, P., Potvin, L., 2009. What is population health intervention research? Can. J. Public Health 100 (1 Suppl.), 18-114.

Huxley, P., Evans, S., Leese, M., et al., 2004. Urban regeneration and mental health. Soc. Psychiatry Psychiatr. Epidemiol. 39 (4), 280-285.

Jacobs, D.E., Brown, M.J., Baeder, A., et al., 2010. A systematic review of housing interventions and health: introduction, methods, and summary findings. J. Public Health Manage. Pract 16 (5 Suppl.), S5-S10.

Kearns, A., 2002. Response: from residential disadvantage to opportunity? Reflections on British and European policy and research. Hous. Stud. 17 (1), 145-150.

Kearns, A., Tannahill, C., Bond, L., 2009. Regeneration and health: conceptualising the connections. J. Urban Regen. Renewal 3, 56-76.

Kleinhans, R., Varady, D., 2011. Moving out and going down? A review of recent evidence on negative spillover effects of housing restructuring programmes in the United States and the Netherlands. Hous. Stud. 11 (2), 155-174.

Lindberg, R., Shenassa, E., Acevedo-Garcia, D., Popkin, S., Villaveces, A., Morley, R., 2010. Housing interventions at the neighborhood-level and health: a review of the evidence. J. Public Health Manage. Pract. 16 (5S), S42-S50.

Ludwig, J., Duncan, G.J., Gennetian, L.A., et al., 2012. Neighborhood effects on the long-term well-being of low-income adults. Science 337,1505-1510.

Mayne, J., 2008. Contribution analysis: an approach to exploring cause and effect. lLAC Brief 16.

Meijer, M., Rohl, J., Bloomfield, K., Grittner, U., 2012. Do neighborhoods affect individual mortality? A systematic review and meta-analysis of multilevel studies. Soc. Sci. Med. 74 (8), 1204-1212.

Morabia, A., Costanza, M., 2012. Population Health Intervention Research (PH1R): should we fear the "P"? Prev. Med. 54, 291-292.

Neary, J., Egan, M., Keenan, P.J., Bond, L., 2012. "Damned if they do, damned if they don't": negotiating the tricky context of anti-social behavior and keeping safe in disadvan-taged urban neighborhoods. J. Youth Stud. http://dx.doi.org/10.1080/13676261. 2012.710745.

Ogilvie, D., Egan, M., Hamilton, V., Petticrew, M., 2005. Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go? J. Epidemiol. Community Health 59, 886-892.

Palmer, G., Maclnnes, T., Kenway, P., 2006. Monitoring Poverty and Social Exclusion in Scotland. York Publishing Services for Joseph Rowntree.

Paris, C., Blackaby, B., 1979. Not Much Improvement: Urban Renewal Policy in Birmingham. Heinemann, London.

Petticrew, M., 2013. Public health evaluation: epistemological challenges to evidence production and use. Evid. Policy 9 (1), 87-95.

Petticrew, M., Whitehead, M., Macintyre, S., Graham, H., Egan, M., 2004. Evidence for public health policy on inequalities: 1: the reality according to policymakers. J. Epidemiol. Community Health 58, 811-816.

Renalds, A., Smith, T., Hale, P., 2010. A systematic review of built environment and health. Fam. Community Health 33, 68-78.

Rubinowitz, L.S., Rosenbaum, J.E., 2000. Crossing the Class and Colour Line: From Public Housing to White Suburbia. University of Chicago Press, Chicago.

Sautkina, E., Bond, L., Kearns, A., 2012. Mixed evidence on the effects of Mixed Tenure: findings from a systematic review of UK studies, 1995-2009. Hous. Stud. 27,748-782.

Thomson, H., 2008. A dose of realism for healthy urban policy: lessons from area based initiatives in the UK. J. Epidemiol. Community Health 62,932-936.

Thomson, H., Atkinson, R., Petticrew, M., Kearns, A., 2006. Do urban regeneration programmes improve public health and reduce health inequalities? A synthesis of the evidence from UK policy and practice (1980-2004). J. Epidemiol. Community Health 60,108-115.

Thomson, H., Thomas, S., Sellstrom, E., Petticrew, M., 2009. The health impacts of housing improvement: a systematic review of intervention studies from 1887-2007. Am. J. Public Health 99, S681-S692.

Thomson, H., Thomas, S., Sellstrom, E., Petticrew, M., 2013. Housing Improvements for Health and Associated Socio-economic Outcomes. (Review, The Cochrane Library, 2013 Issue 3) The Cochrane Collaboration and John Wiley & Sons, London.

Truong, K.D., Ma, S., 2006. A systematic review of relations between neighborhood and mental health. J. Ment Health Policy Econ. 9,137-154.

Walsh, D., 2008. Health and Wellbeing in Glasgow and the GoWell Areas — Deprivation Based Analyses. Glasgow Centre for Population Health, Glasgow.

Weitzman, B.C., Mijanovich, T., Silver, D., 2009. Finding the impact in a messy intervention: using an integrated design to evaluate a comprehensive citywide health initiative. Am. J. Eval. 30, 495-514.

Whitehead, M., Petticrew, M., Graham, H., Macintyre, S., Bambra, C., Egan, M., 2004. Evidence for public health policy on inequalities: 2: assembling the evidence jigsaw. J. Epidemiol. Community Health 58, 817-821. WHO Commission on Social Determinants of Health, 2008. Closing the Gap in a Generation: Health Equity Through Action on the Social Determinants of Health. World Health Organization, Geneva.

Yen, I., Michael, Y., Perdue, L., 2009. Neighborhood environment in studies of health of older adults: a systematic review. Am. J. Prev. Med. 37 (5), 455-463.