0European Journal of Information Systems (2008) 17, 352-370 & 2008 Operational Research Society Ltd. All rights reserved 0960-085X/08
www.palgrave-journals.com/ejis
Improving the impact of IT development projects: the benefits realization capability model
Colin Ashurst1, Neil F. Doherty2 and Joe Peppard3
1Durham Business School, Durham University, Durham, U.K.; 2The Business School, Loughborough University, Loughborough, Leicestershire, U.K.; 3Information Systems Research Centre, Cranfield School of Management, Cranfield, Bedford, U.K.
Correspondence: Neil F. Doherty, The Business School, Loughborough University, Ashby Road, Loughborough, Leicestershire LE11 3TU, U.K. Tel: + 44 1908 223128; Fax: + 441509 223960; E-mail: n.f.doherty@lboro.ac.uk
Abstract
The return that organizations derive from investments in information systems and technology continues to disappoint. While there are many theoretical prescriptions as to how the planned benefits from an information technology (IT) project might best be realized, there is very little empirical evidence, as to whether such advice is being heeded in practice. Drawing on the resource-based view of the firm, a conceptual model of a benefits realization capability is presented and developed. In this model, the benefits realization capability is operationalized through four distinct competences, each of which is underpinned by a variety of socially defined practices. The model was populated by using a thorough review of the literature to identify and categorize those specific practices that have the potential to contribute to the effective achievement of benefits from IT investment projects. These practices are then studied in an empirical examination of 25 IT projects. The analysis finds no evidence of benefits realization practices being adopted in any consistent, comprehensive or coherent manner. Effective benefits realization requires an ongoing commitment to, and focus upon, the benefits, rather than the technology, throughout a system's development, implementation and operation.
European Journal of Information Systems (2008) 17, 352-370. doi:I0.I057/ejis.2008.33
Keywords: IT benefits realization; organizational change; resource-based view; practices; systems development
Received: 9 August 2007 Revised: 8 May 2008 2nd Revision: 21 August 2008 Accepted: 22 August 2008
Introduction
Information technology (IT) has become an increasingly ubiquitous and integral part of the modern organization as it has the potential to enhance performance, at the operational and strategic levels. However, as Zuboff (1988, p. 7) notes, this widespread adoption of IT is not 'neutral', as it embodies 'essential characteristics that are bound to alter the nature of work within factories and offices, and among workers, managers and professionals'. Indeed, a steady stream of research, over the past 20 years, has confirmed that IT implementations are typically associated with very significant amounts of organizational change (e.g. Markus & Robey, 1988;Robey & Boudreau, 1999;Markus, 2004;Davidson & Chiasson, 2005;Peppard & Ward, 2005). For example, whether by happenstance or design, the introduction of a complex and highly integrated technology, such as ERP, is likely to have significant impacts on an organization's business processes, structure, culture and enterprise level performance, as well as
the motivation, job specifications and performance of individual employees (Markus, 2004).
Despite its recognized tendency to act as a catalyst for change, IT cannot be viewed as a deterministic artefact (Grint & Woolgar, 1997;Orlikowski & Hoffman, 1997), as even when deployed in very similar organizational settings, identical information systems (IS) can give rise to significantly different outcomes (Orlikowski, 1992; Sahay & Robey, 1996;Doherty et al., 2006). Consequently, predicting and managing the social and organizational impacts of a system's implementation is by no means a straightforward endeavour (Clegg et al., 1997; Doherty & King, 1998). Moreover, in far too many instances the planned organizational impacts fail to materialize, while the actual impacts can result in user resistance and, in extreme cases, possibly even system rejection (Martinsons & Chong, 1999). Indeed, there is a growing consensus that the high incidence of systems development projects that fail is primarily due to the inability of organizations to effectively predict and manage IT-enabled organizational change (e.g. Lyytinen & Hirschheim, 1987;Lederer & Nath, 1991; Ewusi-Mensah & Przasnyski, 1994;Doherty et al., 2003; Peppard & Ward, 2005).
The establishment of a link between the unpredictable nature of organizational impacts and unsuccessful IT projects has significant implications because a considerable amount of time, money, effort and opportunity can be wasted upon IT investments that ultimately fail to deliver benefits. Estimates of the level of failure may vary, but over the past 30 years they have tended to stay uncomfortably high. More specifically, it has been suggested that in the late 1970s only 20% of projects 'achieved something like their intended benefits' (Eason, 1988), and by the late 1980s, it was estimated that up to 70% of IS projects failed (Hochstrasser & Griffiths, 1991). By the late 1990s, Clegg etal. (1997) reported that 'up to 90% of all IT projects fail to meet their goals', while more recently the British Computer Society (British Computer Society, 2004) concluded that ' only around 16 per cent of IT projects can be considered truly successful'. Against this backdrop, it is important that more reliable ways of managing the organizational change associated with IT projects should be found, to help reduce the incidence of IS failure. Although it may be widely acknowledged that the unpredictable nature of organizational change is a key contributor to IS failure, paradoxically, it is also recognized that the benefits of IT typically come from the organizational change that accompanies its introduction (Peppard & Ward, 2005). However, the explanation for this apparent paradox is not difficult to discern, as the typical IT project team will generally focus upon delivering a technical solution, and only worry about its organizational impacts, once it is operational, rather than managing organizational change as an integral part of the project (Ahn & Skudlark, 1997;Clegg, 2000;Eason, 2001;Markus, 2004).
One potentially important mechanism for proactively managing the social and organizational impacts of an IT project is through an explicit benefits realization programme, which can be defined as 'the process of organising and managing, such that the potential benefits arising from the use of IT are actually realised' (Ward & Elvin, 1999). Indeed, a number of previous studies have attempted to promote the role of formal and explicit benefits realization' approaches, for improving the outcomes of IS development projects, through the proactive management of organizational change (e.g. Farbey et al., 1993; Ward et al., 1996;Remenyi et al., 1997;Ward & Elvin,
1999). However, to date, there is little evidence that organizations have been able to translate these academic prescriptions into effective working practices (National Audit Office, 2006). Benefits realization appears to be a good example of the often substantial gap between management theory and practice (Pfeffer & Sutton,
2000). Consequently, there is a pressing need for novel contributions that present insights into how an explicit focus on benefits realization might best be incorporated into the actual routines of systems development and implementation. Such contributions would also address calls for increasing the relevance of research in the IS discipline for practitioners (Keen, 1991; Robey & Markus, 1998;Benbasat & Zmud, 1999;Breu & Peppard, 2003).
In this paper, we address the question of how an organization embarking upon a new IT investment project can increase the likelihood of its projected benefits being ultimately realized. Drawing on the resource-based view (RBV) of the firm, we argue that organizations should have a benefits realization capability. We build a conceptual model of this capability, and suggest that the notion of practice' provides a foundation to operationalize this capability, and also provides specific guidance for practitioners. The remainder of the paper is organized into four parts. First, we provide a brief review of literature related to the concepts of resources, capabilities, competences and practices, before applying these to the task of IT benefits realization. We then outline the research method adopted for the empirical part of this study and summarize the key findings. Finally, we explore the theoretical and practical implications of this work, paying particular attention to the value of the competences/practices approach, in the realization of business benefits through IT.
Capabilities, competences and practices
Resources, capabilities, competences and practices are all important concepts that have already received much attention in the general and strategic management literatures (e.g. Barney, 1991;Grant, 1996a;Teece et al., 1997;Brown & Duguid, 2000;Pfeffer & Sutton, 2000; Helfat & Peteraf, 2003). In this section, we illustrate how these theoretical constructs can be applied to the task of delivering specified benefits from IT investments.
Competences and capabilities for IT
Over the past 25 years there has been significant interest in the process by which organizations can assemble a unique portfolio of resources that will render them a competitive advantage. The RBV of the firm (Wernerfelt, 1984;Barney, 1991) suggests that organizations should invest in those assets and resources that they believe will best assist them in successfully gaining a sustainable competitive advantage. In this context, resources have been defined as ' stocks of available factors that are owned or controlled by the firm' (Amit & Schoemaker, 1993). However, from a competitive perspective not all resources are equally valuable, as it has been argued that an organization's primary source of competitive advantage will be through those resources that are simultaneously valuable, rare, imperfectly imitable and non-substitutable - the so called VRIN conditions (Barney, 1991). While resources are clearly a critical element of the RBV, there is a growing recognition that resources, per se, do not create value. Rather, value is created by an organization's ability to mobilize, marshal and utilize these resources, through the application of capabilities and competences (Black & Boal, 1994;Grant, 1996b;Bowman & Ambrosini, 2000). Consequently, it can be argued that organizations will only attain a sustainable competitive advantage if they can assemble a set of competences that can be consistently applied (Teece & Pisano, 1994) and that competitors find difficult to imitate (Prahalad & Hamel, 1990; Barney, 1991).
This analysis and logic can be applied to the realization of benefits from IT investments. We argue that all organizations should establish a benefits realization capability, whether or not IT is a source of competitive advantage. However, this capability cannot be developed within the boundaries of the IS function, as research demonstrates the need for enterprise-wide cooperation and engagement to realize the benefits from IT investments (Ward & Peppard, 2002). In delivering value through IT, the key resource is not technology but knowledge and this knowledge will be distributed throughout the organization. As Newell et al. (2004) have noted, the primary challenge for project teams, set up to design and implement a large-scope IT system, is how to coordinate and integrate such distributed knowledge.
While we draw on the RBV in our argument, there is a lack of precision in the usage of terms and concepts surrounding this perspective, which needs to be addressed, particularly with respect to the distinction between a competence and a capability:
• Competence refers to a ' firm's capacity to deploy resources, usually in combination, using organizational processes, to effect a desired end' (Amit & Shoemaker, 1993, p. 35). A competence is thus an attribute of a team, function or even the entire organization. Each competence is underpinned by the skills, knowledge and experiences of employees, that is, people resources, who are likely to be distributed enterprise-wide, and deployed in
combination with specific organizational processes and resources (McGrath et al., 1995). • Capability is a higher level construct than a competence (Stalk et al., 1992), defined and enacted through the application of a set of competences (Teece et al., 1997;Moingeon et al., 1998;Kangas, 1999). More specifically, a capability can be defined as an organization's ability to 'perform a set of co-ordinated tasks, utilizing organizational resources, for the purposes of achieving a particular end result' (Helfat & Peteraf, 2003, p. 1000).
Benefits realization from IT investments can therefore be conceptualized as an organizational capability that has the express purpose of ensuring that investments made in IT consistently generate value, through the enactment of a number of distinct, yet complementary, competences. However, while it appears to make sense to conceptualize benefits realization as a capability underpinned by a number of distinct competences, such a model is still at a relatively high level of granularity. Competences have been referred to as an ' amorphous heap' (Wernerfelt, 1984) as little is generally known about the knowledge resources that underpin them, nor how this knowledge should be coordinated and integrated. Consequently, the practitioner will almost certainly be left asking questions as to how specific benefits realization competences might best be developed, and ultimately managed, while the researcher will want to know how they can observe and measure such high-level constructs, when conducting empirical research (Black & Boal, 1994;Miller & Shamsie, 1996).
Practices - a way of operationalizing competences?
One potentially rewarding way of adding granularity to a benefits realization competence is by decomposing it into a number of constituent practices, each of which is underpinned by the skills, knowledge and experiences of organizational employees and sometimes those employees of external entities. The concept of practice is increasingly used within the organizational sciences literature and a range of descriptions and definitions have inevitably emerged. Wenger et al. (2002) define practices as ' a set of socially defined ways of doing things in a specific domain: a set of common approaches and shared standards that create a basis for action, problem solving, performance and accountability'. In a similar vein, Carlile (2002) contends that practices are strongly focused upon their 'objects' and 'ends', which makes practices far more concrete and observable than competences. Not only does the concept of a practice appear to be very closely aligned with how people actually work, it is also particularly relevant for IS projects, where much of the effort is based upon the knowledge and experiences of individuals and teams (Newell etal., 2004). Moreover, the concept of practice relates to the informal organization and how individuals and teams discharge their responsibilities. In contrast, most management literature tends to
emphasize processes and procedures, defined by the formal organization, which focus upon prescriptions of how the work should be done, and in so doing, often ignore many critical factors that affect performance (Brown & Duguid, 2000). For example, from an IS development perspective, Nandhakumar & Avison (1999) highlight the limitations of formal methodologies, arguing that they often represent a convenient fiction', to provide an appearance of control, but bear little relationship to how work actually gets done.
Based primarily on Wenger et al's (2002) definition, but also taking account of other appropriate sources (e.g. Grant, 1996a;Brown & Duguid, 2000;Schultze & Boland, 2000; Carlile, 2002), we established the following working definition of 'practice', for the purpose of this study:
a set of socially defined ways of doing things, in a specific domain, to achieve a defined - and generally measurable -outcome, and create the basis for responding appropriately to individual circumstances.
From this definition, a number of phrases require further clarification, as they have a significant impact upon the way in which it can be used to identify appropriate practices.
• Socially defined ways of doing things: 'Socially defined' implies that a practice is inherently people-oriented: it relates to the activities of people' (Brown & Duguid, 2000). As Schultze & Boland (2000) note, the term
practice is deliberately used to capture the essence of 'what people actually do', as underpinned by their knowledge, skills and experience, and evidenced through their behaviour (see Figure 1).
• In a specific domain: Given the study's explicit focus on benefits realization, we were only interested in those practices that might directly contribute to managing the realization of benefits from systems development projects.
• To achieve a defined - and generally measurable - outcome: All practices should have a clear and specified benefits-oriented aim. As Carlile (2002) notes, practices are typically defined in terms of their means' and most importantly their ends', which allow the success of the practice to be demonstrated.
• Creates the basis for responding appropriately to individual circumstances: A practice is not a set of highly formalized rules that prescribe in great detail the way an activity should be undertaken. As Schultze & Boland (2000, p. 204) note, it is not a mechanical reaction to rules, norms or models, but a strategic, regulated improvisation responding to the situation'.
Based upon the above discourse around capabilities, competences and practices, it is possible to posit a clear relationship between these three constructs, and envisage how they might be configured in the context of benefits realization through IT. From this analysis, the benefits realization capability will be enacted through a coherent
Figure 1 The relationship between capabilities, competences and practices.
set of benefits realization competences. As demonstrated in Figure 1, each benefits realization competence will be underpinned by a closely related suite of benefits realization practices, which in their totality help to define the competence.
The benefits realization competences framework
In our search for competences that explicitly contribute to the realization of benefits from IT projects, we sought to identify clusters of knowledge, skills and routines, each of which would be complementary, yet would make a distinct contribution to the overall benefits realization capability. As well as being distinct, it was also important that each identified competence should be applicable across a wide variety of IT development projects and organizations, irrespective of role or context.
Although the literature is rather limited, with respect to benefits realization competences, two pieces of work were particularly influential in shaping our model: the ' information competences' framework (Peppard et al., 2000; Peppard & Ward, 2004) helped in structuring our framework, while the process model for benefits management (Ward et al., 1996, 2007;Ward & Daniel, 2006; Peppard et al., 2007) influenced its content. Based upon the literature review, four distinct competences were ultimately identified which should enhance an organization's ability to realize value from their IT investments, on a more consistent basis:
1. Benefits Planning: Benefits do not simply emerge, as if by magic, from the introduction of a new technology. Their realization needs to be carefully planned and managed (Lin & Pervan, 2003;Markus, 2004). We define the benefits planning competence as the ability to effectively identify and enumerate the planned outcomes of an IS development project and explicitly stipulate the means by which they will be achieved'. The benefits planning competence should be capable of being applied to every individual IT project. When planning benefits it is important that a degree of realism is applied: long-term value is far more likely to be realized in situations where compromises are made, to ensure that the planned benefits are perceived to satisfy both the organization and all its stakeholders (Jurison, 1996).
2. Benefits Delivery: As Strassman (1990, p. 519) notes computers add value only if surrounded by appropriate policy, strategy, methods for monitoring results, talented and committed people, sound relationships and well designed information systems'. Indeed, it can be argued that benefits primarily arise from the organizational change that accompanies an IT implementation, rather than directly from the technology (Peppard & Ward, 2005;Hughes & Scott Morton, 2006). Consequently, organizations will only be able to deliver value from their IT investments if they can develop a benefits delivery competence, which we define as: the ability to design and execute the
programme of organizational change necessary to realize all of the benefits specified in the benefits realization plan'. It is important to note that the benefits delivery competence is not simply enacted at a single point in the development life-cycle, but is applied from the point at which the benefits plan is approved through to the system's implementation beyond go-live: as the technical solution gradually takes shape, so does the organizational re-design.
3. Benefits Review: The benefits from IT investments will only be realized if they are measured and managed in a systematic way' (Jurison, 1996, p. 272). Organizations must, therefore, be able to effectively monitor and evaluate the results of their IT projects, on an on-going basis (Tallon et al., 2000), to ensure that its ability to deliver business value is incrementally improved (Remenyi & Sherwood-Smith, 1999). Consequently, the benefits review competence can be defined as the the organization's ability to effectively assess the success of a project in terms of the potential benefits, the delivered benefits, and the identification of the ways and means by which further benefits might be realized'. Benefits review is therefore conceived as being an ideal opportunity for organizational learning (Ward & Daniel, 2006), to ensure that its capacity to successfully realize benefits from IT projects can over a period of time be enhanced.
4. Benefits Exploitation: Ward & Daniel (2006) argue that the quest to leverage benefits from a piece of business software should not cease as soon as it has been implemented. Indeed, it is often the case that the full potential of a particular application does not become apparent until it is fully operational and the stakeholder community has become experienced in its use (Eason, 1988;Markus, 2004). In this respect, the ability to work with information is particularly crucial (Marchand et al., 2000;Davenport et al., 2001). It is therefore necessary for organizations to develop and apply a benefits exploitation competence, which can be defined as the adoption of the portfolio of practices required to realize the potential benefits from information, applications and IT services, over their operational life'. More specifically, to ensure the long-term delivery of benefits it is necessary to conduct an on-going programme of organizational change and software enhancements, both of which are determined by, and directed through, the application of the benefits exploitation competence.
As can be seen for the schematic representation of the benefits realization capability (presented in Figure 2), the relationship between the competences can unfold in a number of different ways. The first and most obvious route - as represented by the thicker, unbroken arrows -moves seamlessly from benefits planning, through the benefits delivery phase to a major review of benefits, and finally to on-going benefits exploitation, once the system is fully operational. The second, and probably
Figure 2 The benefits realization capability model.
more realistic, approach adopts the same primary relationship between competences, but views benefits review as an on-going activity: plans are reviewed and adjusted, delivered benefits are reviewed and modified, and the on-going exploitation also requires on-going review.
The four competences were each decomposed into a number of distinct, yet complementary practices, each of which was identified from the literature. Given the relative immaturity of the IT benefits realization literature, it was necessary to conduct a broader review of the IS literature, to identify potentially relevant benefits realization practices. In particular, contributions from the socio-technical (e.g. Clegg, 2000;Doherty & King, 2005), IT-enabled change (e.g. Markus & Benjamin, 1997; Markus, 2004;Hughes & Scott Morton, 2006) and IT evaluation (e.g. Farbey etal., 1993;Remenyi & Sherwood-Smith, 1999) literatures were found to be very useful, as they all have a strong focus on stakeholder involvement, project outcomes and organizational change. These literatures presented insights into the various approaches, techniques or behaviours that might help to facilitate benefits realization. However, none was explicitly positioned as a benefits realization practice. Consequently, the literature was initially used to identify any activities that might support a benefits realization initiative, which could be classed as 'candidate' practices. Each of these was then critically reviewed, and if it conformed to our definition of a practice, it was used to derive a benefits realization practice. These practices are listed and described in Table 1 - columns 1-5.
Research approach for empirical data collection
The research was designed as an exploratory study to provide new insights into benefits realization from IT
investments, from both theoretical and empirical perspectives, and in so doing to help set the direction for future research. To this end, a case study approach (Silverman, 2000) was adopted, to ensure that the benefits realization capability could be explored from a number of different organizational perspectives. More specifically, our aim was to critically review the conduct of IS development projects, in a sample of case organizations, to explore the extent to which the approaches and methods they utilized mapped onto our framework of competences and practices. In so doing, we anticipated extending our initial framework through the identification of new practices, as well as exploring the extent to which existing practices were deployed.
As a whole, the research philosophy adopted for this study can best be described as 'pluralist', as it incorporated both interpretive and positivist elements, as recommended by Lee (1991) and Mingers (2004). At first glance, it may appear reasonable to view an approach based upon the derivation and empirical testing of a research model as being wholly positivistic, in terms of its philosophical orientation. However, in executing the empirical element we attempted to adopt more of an ' interpretive' style, as our overarching aim was to gain ' knowledge of reality' through the study of social constructions, as manifested in language and documents (Klein & Myers, 1999). In particular, when reviewing the project documentation and the interview responses, we were very sensitive to the fact that it was highly unlikely that our candidate practices would manifest themselves in a consistent and clearly sign-posted manner. Moreover, we recognized that, based upon their socially constructed manner, there were likely to be many additional benefits-oriented practices that have not as yet been addressed in the literature.
Table 1 A framework of benefits realization practices
Code* Practice
Description
Output
Literature
Incidence
BP1 Identify strategic drivers
BP2 Analyse
stakeholder expectations
BP3 Identify and define benefits
BP4 Establish benefit/ process interactions
BP5 Establish benefit/ stakeholder interactions
BP6 Establish
organization/ benefits interactions BP7 Establish
technology/
benefits
interactions
Design a framework for business change governance
'Top down' activity to clarify the strategic/business drivers for the project and its contribution to the achievement of business strategy.
Conduct a structured, 'bottom-up' analysis of the stakeholders' requirements, in terms of delivered benefits. Review of strategic drivers and the stakeholder requirements, to identify/agree the target benefits.
Relate the benefits to business processes to identify where changes will take place and help identify relevant measures. Assess the variability and uncertainty in the process and consider the implications for benefits realization.
Identify stakeholder groups affected by the technology, and changes required to realize the benefits. Identify business change issues and actions required including communication and engagement with the stakeholders, and the re-design of job specifications. Explore the interaction between the benefits and a full range of
Strategic
drivers
analysis
Analysis of expectations by stakeholder
Benefits analysis including: agreed measures, targets and benefit owners
Process/benefit map
Stakeholder
impact
assessment
Organizational impact
perspectives on the organization. assessment
Conceptual architecture overview
Plan benefits realization
Establish a design for an IS solution that takes account of the capabilities of the technology.
Develop an overall plan to show Benefits realization the business case (what the plan: defines the benefits are) and how they are benefits and the going to be realized. The plan actions required to relates to the type of project and realize them ensures the delivery of benefits is phased as relevant and that there is appropriate consideration of organizational factors.
Design a governance framework Governance
addressing the business change framework
project, including the enabling
IS/IT activities. Agree how to
bring together the sponsor,
benefits owners, project manager
and other stakeholders through
Ward & Elvin (1999) and Ward & Daniel (2006)
Edwards & Peppard (1997) and Neely et al. (2002)
Peppard & Ward (2005) and Peppard et al. (2007)
Peppard et al. (2007), Bohn (1994), Brooke (2000), Ward & Daniel (2006) and Bashein et al. (1994)
Eason (1988), Joshi (1991), Benjamin & Levinson (1993) and Doolin (2004)
Moderate - many projects used the language of driver analysis -but often at a high level and with a technical focus.
None - Users were sometimes involved with projects, but there was no evidence that this activity had a benefits' focus. Moderate. Most organizations articulated the expected benefits, but often in very vague, or technically oriented, terms. Few projects established measurable targets, and in no cases were benefit owners established. Very low. Some projects gave limited consideration to localized processes.
Doherty & King (2001) Peppard et al. (2007)
Eason (1988), Peppard et al. (2007)
Ward et al. (1996) and Clegg et al. (1997)
Clegg et al. (1997) and Avgerou (2001)
Low. Several projects identified different stakeholders and particularly different groups of users. The analysis was not followed through to addressing business change issues related to each stakeholder (group) or to ensure the participation of the groups.
Very low. Not tackled in a structured way.
Very low. But many projects took advantage of the technology capabilities - this was typically requirement rather than benefit driven.
Very low. However, one project to set up a new business operation involved a solution based on establishing a range of business competences. In this scenario the plan was equivalent to a benefits realization plan.
Very low. Projects had a business sponsor but this was not an active role and there was limited involvement of other project stakeholders in project governance. As a result the actual focus was largely on technical change.
Table 1 Continued
Code Practice
Description
Output
Literature
Incidence
BP10 Benefits-driven risk assessment BD1 Establish an adaptive project life-cycle
Actively lead the business change
Ensure continuing active involvement of stakeholders
Specify changes to work and organizational design
Make benefits-driven trade-offs
Ensure benefits-driven risk management Implement organizational changes
Benefits-driven training and education
appropriate meetings, workshops and other forms of communication. Take a proactive approach to risk that focuses on business change and benefits realization. Establish a project life-cycle enabling change during the project in response to learning/ uncertainty - based on iterative, incremental delivery and a small number of major phases controlled by phase end milestone reviews. The adaptive life-cycle continues into benefits ramp up and evolution deployment. Design, build and lead the project team and governance framework with a focus on realizing benefits. In particular, address responsibility for benefits for the organization/sponsor, benefits for the end user and the effectiveness of the team. Ensure there is communication and involvement with all stakeholders (based on the stakeholder analysis) to gain insight, ownership and support for changes.
The project focuses on the design and delivery of a business solution. This will typically require consideration of: business processes, working practices, structures, roles, management framework, performance measures and culture. Trade-off decisions (features, cost and schedule) are driven from a benefits perspective.
Take a proactive approach to risk that focuses on business change and benefits realization. Implement new and revised business processes, working practices, structures, roles, management framework and performance measures. Take action as required to encourage cultural changes. Ensure education and training are focused on the realization of benefits.
Risk assessment and action plan
Project approach -including definition of phases, deliverables and milestones
Role descriptions
Participation and
communication plan
Business solution design
Gibson (2003) Also found in PRINCE2
Low. Generally focused on solution delivery.
Eason (1988, p. 48) and Very low. Many projects took this Boehm & Turner (2004) approach but the focus seen was on solution delivery.
Ward & Daniel (2006), Markus (2004) and Serafeimidis & Smithson (2000)
Eason (1988), Clegg et al. (1997) and Benjamin & Levinson (1993)
Eason (1988) and Clegg et al. (1997)
Low. Several examples involving the development of new products/ services for consumers had active leadership from Marketing (a Product Manager).
Change
log/decision
Updated risk assessment and action plan Changed
organization - this activity needs to be monitored to ensure that planned changes are actioned
Boehm & Turner (2004) Very low. All the projects adopted a clear strategy for trade-off decisions but with no explicit focus on benefits impossible for most projects. A small number did identify the need for a benefits focus. As above
Ward & Elvin (1999)
Eason (1988) and Clegg et al. (1997)
Very low
Eason (1988), Clegg et al. (1997), Marchand et al. (2000) and Davenport et al. (2001)
Table 1 Continued
Code* Practice Description Output Literature Incidence
BRI Establish Establish project evaluation Evaluation Ward & Peppard (2002) None
portfolio- criteria related to the application framework and and Farbey et al. (1999)
based portfolio - that is, using either criteria
evaluation different criteria for different
criteria areas of the portfolio or using
a basket of measures and
changing the weighting.
BR2 Benefits- Use agreed evaluation criteria Benefits Ward & Peppard (2002), Low - There is some evidence
driven to undertake a systematic assessment Farbey et al. (1999) and of benefits being evaluated, but
project assessment of benefits. report Gwillim et al. (2005) not in a comprehensive and
appraisal systematic manner.
BR3 Identify Where planned benefits have not Benefits Ward & Peppard (2002) None
actions to been achieved, or opportunities action and Farbey et al. (1999)
realize further for new benefits have been plan
benefits identified, a benefits' action
plan needs to be established.
BR4 Facilitate Carry out lessons learned reviews Lessons learned Tippins & Sohi (2003) Moderate. Carried out as part
lessons at key stages in the project and report and Included in PRINCE2 of a post implementation
learned on project completion. action plan review.
reviews
BR5 Complete Carry out a review on completion Updated Earl & Khan (2001) None
architectural of a project/to consider the architecture
roadmap contribution to the overall IS/IT roadmap
review architecture. Also consider the
strategic alignment of a
programme and implications
for future projects/releases.
BEI Ensure Establish a clear business role for Agreed/active Ward & Peppard (2002), Very low. One organization saw the
ownership of ongoing ownership of realizing benefits owner Weill & Woodham output from a project as a number
continued benefits. (2003) and Goh & of services and established owners
benefits Kauffman (2005) for ensuring use and realization
exploitation of value from these services.
BE2 Maintain Training is focused around Up to date training/ Clegg et al. (1997) None
benefits- benefits realization and education resources.
driven establishing new ways of Ongoing training plan
training working. and provision
BE3 Evolve Continue to evolve working Revised working Brown & Duguid (2000, Very low. A small number of
working practices post deployment practices Chapter 4) projects identified the need for
practices to realize further benefits. ongoing, gradual learning
and change.
* Key to Practice 'Code': BP: benefits planning practices, BE: benefits exploitation practices, BD: benefits delivery practices, BR: benefits review practices. Key to 'Incidence' of practices: Very Low: found only in 1-3 cases, Moderate: found in 10-15 cases, Low: found in up to 10 cases, High: found in more than 15 cases.
Data collection
Having established a framework of practices, linked to the four distinct competences for benefits realization (see Table 1), it was important to establish a systematic and thorough method for acquiring evidence, from a variety of recent IT development projects, relating to the adoption and significance of each of these practices. It was, therefore, necessary to identify a suitable sample of projects, and an appropriate mechanism for collecting data relating to each project. Both the sample and the primary research data were located at the same, rather innovative source, namely a large global IT consultancy's knowledge-base. This data source was chosen as it
contains comprehensive knowledge and information about the conduct and outcomes of a range of IS development projects. The knowledge base contains detailed records of the vast majority of the projects carried out by the consultancy, and holds electronic copies of all critical project documentation, such as: vision/scoping documents, project plans, risk assessments, functional designs and post-implementation reviews.
In selecting projects for inclusion in our study, the primary aim was to choose only those projects that were judged successful, in order to have an opportunity to focus specifically on benefits realization. Moreover, only
Table 2 Summary of reviewed projects
Case Type of organization Project
1 Logistics Providing up to date information on the status/location of packages to customers
2 Media Web publishing and news solution
3 Oil Upgrade desktop and communications infrastructure
4 Government Portal site for access to government services
5 Retail E-commerce solution
6 Financial services Customer sales and service solution
7 Manufacturer Communications (email, etc.) infrastructure upgrade
8 Manufacturer Content management system across information and e-commerce websites
9 Leisure Communications infrastructure consolidation and upgrade
10 Food manufacturer Website migration to new technology including workflow and content management
11 Oil Development of POS system for retail stores
12 Government Transaction portal providing services to business and citizens
13 Financial services Provide access to customer on the status of a mortgage application
14 Retail E-commerce site
15 Manufacturer Enterprise architecture and new centralized infrastructure including directory services
16 Telecommunications Unified directory service to link employee information
17 Recruitment/HR Employee purchasing system
18 Armed forces Internal collaboration and communication
19 Government Integrate criminal justice systems
20 Government Web portal to manage/publish educational materials to schools
21 Beverages Sales force automation
22 IT services Knowledge management solution
23 Retailer E-commerce solution
24 Government Speech enabled access to web portal to provide access to education information and services
25 Leisure Provide access to sales information to vendors/partners
projects that were highly ranked by the consultants were considered, based on their assessment of the value, completeness and reusability of the project documentation. Ultimately, 25 projects were selected that provided a broad coverage in terms of organizational types and industry sectors represented. A summarized description of all 25 projects and their host organization are presented in Table 2.
To collect evidence from the knowledge base in a robust and consistent manner, a pro forma data collection instrument was designed and tested. The aim of this instrument was to ensure that we adopted a common approach in recording information. For each practice, the following issues were addressed:
1. Was there evidence that a specific benefits-oriented practice was adopted? The answer to this question was recorded as: 'Yes', 'No' or 'Yes - but'.1
2. If the researcher recorded an answer of 'yes', he was prompted to record the specific evidence that could be presented, to substantiate this claim: evidence was typically found in the form of a quote from a specific
1In many cases there was evidence that a practice was adopted, but not exactly in the way described in Table 1. For example, practices relating to risk assessment and user expectations' analysis were often conducted, but not with any clear or significant focus on benefits.
project document that gave credence to the explicit adoption of that practice.
3. If a verdict of 'yes - but' was recorded, the researcher was asked to record the specific evidence that could be presented, to substantiate this claim, particularly focussing upon the caveats with regard to how it was adopted.
4. Even if the researcher found no' evidence that a specific practice was adopted, he would still seek insights that might suggest why this practice was not adopted.
Although the pro forma was a fairly straightforward document, before it was used on all the cases, it was tested by two different members of the team independently reviewing the same five cases, to ensure there was a common understanding of its purpose and execution. Once data collection was underway, it became clear that although the knowledge base was a very rich source of project related information, in many cases it was difficult to determine the extent to which an explicit benefits-oriented approach had been adopted. Moreover, in some cases it was difficult to discern whether a specific benefits-related practice had definitely not taken place, or whether it might have taken place, but no mention of it had been recorded in the knowledge bases. Consequently, a follow-up exercise was initiated to provide richer insights into the adoption of benefits realization practices within each project. To this end, the project managers for all 25 of the projects in the sample were
contacted and asked a series of questions with regard to the success and conduct of their projects, from a benefits realization perspective. All project managers were asked to address the following five issues:
1. Did the customer see your project as successful in IT terms (i.e. on time and budget and delivering key functional requirements)?
2. Did the customer assess the project in terms of business benefits/value delivered?
3. If the delivered value of the project was assessed, how was it assessed?
4. Was the project successful in terms of business benefits/value delivered?
5. What roles/processes were instituted to manage the realization of benefits/delivery of value (over and above those used to deliver the core technology solution)?
Of the 25 project managers contacted, 18 replied, and of these 15 presented sufficiently detailed responses, to include them in our analysis. The initial contacts and the replies were handled via email, which proved to be a very rewarding medium, as it allowed the respondents to provide measured responses, and it let the researchers pose follow-up questions, in situations where clarification was needed.
Mode of analysis
To help make sense of the data, key chunks of text from the documents from the knowledge-base and the email-based interview responses were highlighted, and annotated with ' in-vivo' codes - that is codes derived from phrases used repeatedly by informants (Strauss & Corbin, 1990). Following the within case analysis of all the source documents, a variable-oriented cross-case analysis (Miles & Huberman, 1994) was conducted to identify key themes and patterns across the sample. To ensure that a rich and valid interpretation of the data was achieved, the within-case and cross-case analysis was not conducted in a single iteration. Indeed, the researchers sought to ' understand the whole' by continually revising it in ' view of the reinterpretation of the parts' (Myers, 1994, p. 56). Consequently, the researchers continued to re-visit the documentary evidence and the interview responses to help integrate the individual pieces of evidence into a coherent whole (Butler, 1998).
Research findings
The framework of practices for benefits realization (see Table 1; columns 1-5) was used as an explicit mechanism for exploring the extent to which each practice -identified from the literature - was being adopted in our sample of systems development projects. The results of this exercise have also been summarized in Table 1 (column 6), and are reviewed more fully in the following discussion. In carrying out the analysis we kept in mind that we would expect different organizations to adapt practices based on their experience and culture. So in identifying evidence of a practice we focused particularly
on its intended outcome and did not expect to find precisely the same form of the practice across the different organizations in the sample. In the following discussion, practices are referred to within square brackets as follows: [establish benefits dependency network]. Illustrative quotes, taken directly from project documents, are presented in italics, while the document in which the practice was cited and the case in which a particular practice was observed are highlighted as follows {Post-implementation review: P1}, where 'P' refers to project number. It is not the intention of this review of the findings - which has been organized around the four high-level competences (see Figure 2) - to discuss in detail each and every practice, but rather to focus specifically on the general trends emerging from the data.
Benefits planning
As all IS projects should be primarily driven by the host organization's strategic imperatives (Earl, 1993), it was reassuring to find that one of the most commonly occurring benefits planning practices was to review the project's drivers [BP2: identify strategic drivers], to ensure that the project would contribute positively to corporate strategy. However, there was a tendency for these drivers to be expressed in very high level, and often vague, terms such as:
• ' to develop a platform upon which to build new and support existing revenue' {Vision and Scope: P2};
• ' to be a showcase for the use of information technology in government bodies' {Vision and Scope: P4};
• ' to reduce time to market {Vision and Scope: P8};
• ' to provide improved reporting to enhance strategic purchasing' {Vision and Scope: P17}.
• ' to create a new and stable Internet portal which helps the end user to obtain information quickly {Vision and Scope: P24};
Although this practice [BP2: identify strategic drivers] was widely adopted, it became apparent that it was not being done rigorously. It was as if the project teams knew it was required but did not have the knowledge or motivation required to enact the practice effectively. Although the project aims were typically articulated in strategic terms, there was no explicit discussion of how these strategic aims would be realized, nor any explicit links to corporate strategies to provide evidence of alignment.
Having established the strategic drivers, most organizations had broken these down into a number of lower level benefits [BP3: identify and define benefits]. For the most part these were also fairly ill-defined, such as:
• reduce the operational costs for maintaining the web-site' {Vision and Scope: P8};
• to provide searchable indexing for web-site' {Vision and Scope: P20};
• ' to make the work of representatives more effective' {Vision and Scope: P21};
• to provide users with easy-to-use online e-procurement for ordering office supplies' {Vision and Scope: P24}.
In a small number of cases, there were examples of benefits that were articulated in a more measurable, but not necessarily a business-oriented form, such as: 'generate 1 million visitors per month' {Vision and Scope: P14}. In another case {Vision and Scope: P17}, a portfolio of distinct 'business goals' had been established, each of which was supported by a detailed discussion of why it was important, but for the most part, these goals were expressed in terms of the system's functionality.
By and large, the need to articulate benefits, during a project's planning phase, had been recognized across projects, but all too often these benefits were either articulated in a very general business sense, or in terms of the system's functionality and features or its intended usage, rather than clearly measurable business terms. Moreover, there was absolutely no evidence of organizations explicitly identifying owners for these benefits, to help facilitate their ultimate realization. The difficulty of getting organizations to provide clear measures for benefits was highlighted by a project manager {P21} who lamented: 'At the start of the project we asked about success criteria and how they (the customer) would measure return on investment. All we could get out of them was that other players in the market already had similar technologies, and they wanted to eliminate all paper from their sales cycle'.
Even where business benefits are clearly identified, this is not sufficient to facilitate their realization, as the delivery of business value is dependent upon the redesign of business processes, organizational structures and user working practices, as well as the provision of new technical functionality. However, there was very little evidence from our study that any of the case organizations explicitly addressed these issues in the planning phase of their projects. In a small number of cases, there was a recognition that the realization of benefits was predicated upon the changing of business processes. As one report noted: ' one of the biggest mistakes of this project would be to introduce new technologies without changing the processes' {Vision and Scope: P4}. However, there was no evidence of any attempts to explicitly establish the relationship between the re-design of specific business processes with the realization of benefits [BP4: establish benefit-process interactions]. The linking of the delivery of business benefits to changes in stakeholder behaviour was perhaps a little more positive [BP5: establish stakeholder-benefit interactions]. Indeed, in one case an entire section of the functional specification was devoted to detailing: the types of people who would be affected by the release, and the manner in which they will interact with the system' {P14}. However, in the vast majority of cases, these analyses focused on the manner in which stakeholders would interact with the system, rather than explicitly detailing how their roles and responsibilities should be modified to facilitate the realization of benefits.
Overall, from our data, the adoption of practices in support of benefits planning is very limited and sporadic.
All too often practices are ignored, or where they have been adopted, typically have a focus on the delivery of features and technical functionality, rather than the realization of benefits. For example, one particularly glaring hole in the case organizations' adoption of practices was the absence of any explicit attempt to formulate a benefits realization plan [BP8: plan benefits realization]. Indeed, the main rationale for identifying benefits, in the planning stages of our case organizations, was to facilitate the projects' approval, rather than as a driver for how it is managed. Project teams still strongly prioritize, and focus upon, planning for the delivery of an IS/IT solution, rather than engaging in any systematic attempt to understand the linkage between delivered functionality, complementary organizational change and the ultimate realization of business benefits. One possible explanation for this technological orientation was offered by an interviewee {P6} who noted: ' too many techies were involved in the logical design - there should have been more input from business analysts'. This view was supported by another project team, where one of the key lessons learned was that: ' to make a successful delivery we need to emphasise the business-driven principle, for the requirements' collection, analysis and prioritization' {post-implementation review;P14}.
Benefits delivery
The empirical data provided little evidence that any of the practices related to the benefits delivery competence were being routinely or uniformly applied. For example, the majority of the projects adopted an ' agile' approach to IS development [BD1: adopt an adaptive project life-cycle], such as that advocated by Boehm & Turner (2004), which might have been expected to facilitate the phased delivery of benefits. However, in practice, it was focused upon features and functions, rather than benefits: the release of the solution will be divided into multiple releases culminating in a fully functional, feature-rich solution' {Vision Scope: P11}. With the adoption of agile and adaptive development approaches comes the opportunity to make trade-off decisions between different development alternatives, based upon the strength of their relationship to benefits realization [BD5: make benefits-driven trade-offs]. For example, in the case of a wide-ranging e-Government project, an overriding aim was to: find quick win and win-win possibilities and see how the solution can be implemented in the most efficient and quickest way' {Vision and Scope: P4}. However, while such approaches were the exception, rather than the rule, at least one other organization realized their importance, albeit in retrospect; as one interviewee noted: there should have been more emphasis on the trade-off triangle' {P6}.
One area where there was a little more evidence of practice adoption was with regard to the appointment of a business manager to lead the business change and to facilitate communication with the stakeholder communities [BD2: Actively lead the business change]. For
example, many case organizations appointed product managers or project sponsors, who had a range of responsibilities, including making the final decisions regarding scope, cost and project resources' {Vision and Scope: P2}, or ' defining project objectives and success criteria to ensure that the project remains focussed on successfully fulfilling its defined vision' {Vision and Scope: P20}. In another case, a technology committee' had been established, which had responsibility for making: business-based IT decisions' {P6}. However, in some cases it was evident that the act of appointing business owners or committees had not been translated into any benefits-oriented activity, as was made clear in one postimplementation review: the product owner has not been involved in this project' {P8}. Moreover, in many organizations it was clear that the project sponsor had failed in one of their primary roles, namely facilitating communication. As one interviewee {P6} noted: communications did not always filter down to the teams in a timely manner'.
All the projects reviewed followed a tried and tested model that proved to be very effective in facilitating the timely delivery of IS/IT solutions, with a small team of fairly technically oriented staff. Unfortunately, there was virtually no evidence to suggest that the project teams had actively engaged in the critical element of benefits realization, namely the enactment of changes to the design of the host organization, nor the working practices of project stakeholders [BD7: implement business changes]. Indeed, a review of all the project plans confirmed that no time or resources had been explicitly reserved to enact a programme of organizational change, either before or after systems' implementation. This view was supported by the largely negative responses from project managers when questioned about the roles and processes they had in place to manage the delivery of value. Typical responses included: from a business point of view, I don't know {P13}';' there was no formal role to manage value delivery' {P21};' honestly very few' {P24};and ' not many' {P25}.
Although evidence for the adoption of a wide range of benefits delivery practices, from the framework, was very sparse, there was one important area in which a new candidate practice emerged, namely, the facilitation of knowledge transfer'. Having identified the planned benefits one organization {P8} recognized the need to stimulate knowledge sharing throughout the project, in support of benefits delivery. More specifically, this organization introduced: 'Regularly scheduled, informal briefing sessions, to allow interaction of project personnel and serve as a communication technique for members of the project teams, to provide an effective method of knowledge transfer between individuals and projects' {Vision and Scope: P8}.
Benefits review
From our review of the literature, it was relatively easy to establish a strong case for organizations to develop a competence in benefits review, but there was very little
evidence that any specific practices in support of this were being adopted in any of our case studies. In particular, it was rather disappointing that little evidence could be found to suggest that case organizations were either identifying a set of criteria upon which the success of their projects could be judged [BR1: establish evaluation criteria] or formally reviewing the benefits realized from their IT investments [BR2: benefits-driven project appraisal]. In most cases, the project managers had a clear view as to whether, and which ways, the project delivered value, but admitted that no concrete evidence had been collected to support these perceptions. Typical responses included: no hard value numbers were collected' {P3}; it was successful, but I don't think they tried to quantify it' {P6};and ' there was no assessment in terms of business impact' {P26}. By contrast, in a small number of the cases, specific benefit measures had been collated, using measures such as: reduction in unhappy calls to their call centre' {P5} or ' reductions in the time it took for suppliers to receive feedback on their product sales' {P25}. However, these tended to be very targeted assessments of one or two key benefits, rather than systematic and comprehensive reviews of all benefits. In only one case had a project team attempted to establish a clear link between the original project goals, and the extent to which each had been successfully achieved {post-implementation review, P8}.
It was also interesting to note that while all 25 of the projects were considered to be technically successfully, they were not always viewed as being successful in terms of benefits realized or value delivered. In one notable case {P14}, a website for online sales was delivered to a client on-time, to budget and to specification. However, within a year the website had been withdrawn as it was failing to attract customers, and deliver any meaningful benefits. As the Project Manager {P14} noted: ' during the dot.com frenzy, value metrics were often overlooked'. This finding is important as it underlines the point that the successful delivery of an effective IT solution does not guarantee that the resultant system will deliver meaningful business benefits.
Although limited evidence of benefits review practices could be found, there was some recognition that this was a major deficiency, and therefore something that should be changed in future projects. For example one team highlighted the need for: better tracking of the complete investment and projected return, in terms of product sales, increased customer satisfaction, service and support' {postimplementation review: P8}. In a similar vein, a project manager {P24} noted: 'Return on Investment (ROI) is used to justify projects, but during execution we loose focus on value, and monitoring value'. If nothing else, this provides some evidence that organizations were reflecting upon how the performance of the projects could be improved [BR4: conduct review of lessons learned]. Overall, however, the general situation was that projects ended at or very soon after the 'go-live' date for the new software, with project success judged by the on-time, on-budget delivery of a
technology solution, rather than through the realization of the benefits.
Benefits exploitation
Because project teams tended to be disbanded very soon after the go-live date, there was very little evidence to suggest that on-going benefits exploitation was explicitly practiced in any of the case organizations. However, in two cases managers were appointed to have responsibility for the long-term management and performance of the operational software [BE1: ensure on-going ownership of benefits]. For example one project manager {P6} noted that after a long battle, we managed to get them to name a person with responsibility for running the complete system -software, people and processes'. In a similar vein, another organization had explicitly planned to appoint a manager whose responsibilities would include: process improvement' and relationships with top managers in various business units and with stakeholders' {project plan: P15}. While in neither case were these individuals explicitly tasked with benefits exploitation, their focus upon the on-going management of people and processes, as well as technology, put them in an ideal position to do so.
Discussion
An urgent problem facing both the IS academic and the IT practitioner communities is how the billions of dollars that organizations collectively invest in IT can be more effectively and consistently translated into meaningful business benefits. There is growing recognition that the adoption of more explicit and proactive approaches to realizing any expected benefits might be one effective way of facing this challenge (Ward & Elvin, 1999;Lin & Pervan, 2003;Ward & Daniel, 2006). However, the literature in this area is relatively immature, and there have been few studies that explicitly address the practice of benefits realization from IT investments. We lack well-defined methods, techniques and tools for benefits realization. Benefits realization should be conceived of as an enterprise-wide capability, which is then operation-alized through an integrated framework of competences and ' socially defined practices' (Wenger et al., 2002). Although the explicit literature on benefits realization is fairly thin, our study shows that there is much relevant knowledge, embedded within the wider IS literature, which can be used to derive a benefits realization framework.
By using a conceptual framework as the lens for exploring the extent to which a sample of organizations are already adopting benefits realization practices, our analysis of these cases shows that there is a very substantial gap between what we know from the literature about the impact of adopting a strong benefits focus (Ward et al., 1996), when managing IS projects, and what happens in practice. Indeed, the vast majority of the projects investigated for this study focused on the design and delivery of an IS/IT (technical) solution with only a very limited focus on the wider issues of work re-design,
process re-engineering, organizational change management and benefits realization. We found no evidence -across the cases - of the adoption of a well integrated portfolio of benefits realization practices, which could be seen to demonstrate a ' benefits realization capability'. Moreover, although we discovered many individual instances of specific practices being adopted, overall, these practices were not in widespread use, either within or across organizations.
Despite this rather sporadic adoption of benefits realization practices, it was possible to discern a number of important patterns, across the case organizations. For example, it was evident that the focus upon business benefits was most acute at the project's outset: most organization's attempted to identify the strategic drivers for their projects, and then establish the benefits that were sought. However, the rationale for adopting these practices owed more to getting the project authorized and funding approved than it did to acting as a point of departure for the proactive management of benefits. Consequently, following their initial identification, business benefits tended to disappear from the project teams' agendas until the software was implemented, at which point the benefits might possibly be evaluated, but rarely in a comprehensive or systematic fashion. The only other significant juncture at which business benefits were explicitly considered was during the post-implementation review, at which stage a number of project teams made clear recommendations that more specific benefits-related practices should be adopted in future projects.
Given that we found very little evidence to suggest that benefits-oriented practices are being adopted in any comprehensive or systematic way, it is important to question why this situation might have arisen. There are at least three plausible explanations to this question. Firstly, as IS development projects are enacted by a wide variety of human actors interacting in a multitude of ways with a complex technical artefact (De Sanctis & Poole, 1994;Rose & Jones, 2005), there is a high probability that their outcomes and impacts will be exceptionally difficult to predict in advance: there are always likely to be planned outcomes that are not realized (Clegg et al., 1997), as well as a wide variety of unintended consequences (Schultze & Orlikowski, 2004; McAulay, 2007). Consequently, it can be argued that organizations would be wasting their money in trying to proactively manage benefits. An alternative, or perhaps complementary, explanation might be found in the composition of the sample. It can be argued that consultancy organizations would be very focused on discussing benefits with their clients at the outset of a project, but would then want to be left alone to get on with what they perceive to be the job: delivering a technical solution on time and to budget. Under this scenario, the consultancy would see its job as delivering a solution that has the potential to deliver benefits, but responsibility for managing the conversion of potential into realized value (Davern & Kauffman, 2000) would lie
with the client organization, and how they adapt and appropriate their delivered system. However, it is highly unlikely that benefits will ultimately be delivered if the client has not been actively involved in the identification and realization of benefits during the application development phase. The third and perhaps most positive interpretation of our findings is that organizational development teams have limited awareness of the importance of proactively managing benefits, or the availability of benefits-oriented approaches. If this final interpretation should prove to be at least a partial explanation to this apparent lack of interest in managing benefits, then hopefully this paper should make a useful contribution in this respect.
Our findings arose because we adopted a practice lens for studying IS projects, rather than opting for the more common, and rather discredited, focus upon development methodologies (Nandhakumar & Avison, 1999). Practices relate to how people actually work and may provide an effective way to share knowledge and enable organizations to establish the competences required to realize the potential benefits of IS/IT. We have found the concept of a practice', a useful way to compare how people actually approach realizing benefits from investments in IS/IT, across a wide range of organizations. However, it would be naive to suggest that the framework of practices could be applied, in an undifferentiated form, in all situations and circumstances. Practices are socially constructed ways of working (Newell et al., 2004), and groups of stakeholders, operating in a particular organizational context, will need to adopt and adapt them to suit their requirements.
Although we have sought to adopt systematic and rigorous research approaches, there is only so much ground that a single study can cover. Consequently, there are important opportunities for follow-up studies, which are explicitly designed to build upon and extend our reported findings. In so doing, a more complete picture of the practices and competences required to realize benefits from IS/IT should start to emerge. For example, because we focussed primarily on the review of project documentation, it may well be that additional practices become evident through studies based upon observation and more comprehensive interviews. The focus on projects undertaken by a single consultancy might have also helped to shape our findings, so follow-up studies that target a variety of different organizational contexts will also be important. Of particular interest will be detailed case-based studies in organizations that have already accepted the need for a strong benefits' orientation, so that their adoption of socially constructed practices can be explored at close quarters.
Lessons learned
From the study's findings, it is possible to distil a number of important lessons that have implications for managers both within, and outside, the IT organization. For those managers who find the arguments for adopting a more
explicit benefits realization approach compelling, our framework of competences and practices (see Table 1) provides some very interesting insights into how such a programme might be organized and managed. However, it is important to provide some qualifications on how this framework might best be utilized, as we do not want it to become simply another necessary fiction' (Nandhakumar & Avison, 1999), with respect to the management of IT projects. In particular, the empirical study has demonstrated that different organizations have enacted a common practice in their own distinct ways. Moreover, it is unlikely that every identified practice will be needed in all circumstances. Consequently, we would suggest that the framework of practices should be viewed as a reference guide and point of departure for organizations to develop their own benefits realization capability, which is tailored to their own ways of working and specific organizational requirements. Other important lessons include:
• Focus on the benefits, not the technology: For the IT manager the key message must be that the delivery of a successful technical solution may be a necessary, but certainly not a sufficient, condition for the realization of a range of significant business benefits. Consequently, the outcomes of future IS projects must be defined in terms of specific benefits to be realized, rather than the functionality of the technology that is to be delivered.
• Effective benefits realization requires on-going commitment: Having identified the benefits to be delivered, project managers will need to initiate a proactive and on-going benefits realization programme that ensures that benefits remain the primary focal point for all decisions with regard to the development of the IS.
• Benefits realization is a shared responsibility: Because of its strong focus on corporate strategies and organizational change, benefits realization is not an undertaking that can be solely accomplished by IT professionals. Indeed, the primary responsibility for benefits realization should probably reside with managers from the host department, team or business unit that will ultimately own the system. However, IT professionals will need to work in close collaboration with business stakeholders to ensure that the resultant system's functionality and performance is well aligned with the users' needs.
• The management of consultancy projects: At a time when the proportion of IT projects being outsourced to consultancies is high and rising (Sauer & Cuthbertson, 2003), there is a need for the customer to be prepared to take on responsibility for the benefits realization activity, as the consultancy may not see this as being within their terms of reference.
Because of the novelty of this research, both in terms of its focus and approach, it has also been possible to identify a number of important implications for the researcher. In particular, the framework of benefits
realization competences and practices provides an important set of new constructs that can be further explored and validated through future studies. Moreover, the adoption of an innovative source of research data -namely the consultancy knowledge-base - might also be an approach that other academics, working in this domain, might care to adopt.
Conclusions
The cost of failed IS/IT projects, in the U.K. alone, is many billions of pounds annually (British Computer Society, 2004). The cost of missed opportunities is probably a great deal more. Consequently, in seeking to explore how more systems development projects might result in the delivery of benefits, rather than end in failure, this study has tackled a major issue. The research has resulted in the development of conceptual model of a benefits realization capability, enacted through competences and underpinned by practices that explicitly support the effective management of benefits. As a practice has been defined in terms of socially defined ways of doing things in an organization', this stakeholder-oriented model provides a novel alternative to the more common ways of viewing systems development projects, in terms of formal tools, techniques and methods. Unfortunately, the empirical element of the study suggests that very few of these benefits-oriented practices have been adopted in devel-
opment projects, largely because IT professionals still tend to focus primarily on the delivery of a technical solution, on time, on budget and to specification.
Despite the absence of any clear evidence that it is already being utilized, there are strong grounds to believe that organizations should be looking to establish a benefits realization capability, rather than continuing to pin their hopes solely on the use of traditional systems development methodologies. It has long been recognized that the real benefits of IS are typically realized once users begin to appropriate the technology and adapt it to their own requirements and working contexts (Boiney, 1998; Majchrzak et al., 2000). Moreover, IT should not be viewed and managed as an island, but rather seen as an integral part of organizational life. Consequently, the establishment of an enterprise-wide, benefits realization capability, based upon socially constructed ways of working, may have an important role to play in organizations wanting to rise to the challenge of generating value from their IT investments.
Acknowledgements
We thank the anonymous referees for the many helpful suggestions made with respect to initial drafts of this paper, as well as the Associate Editor and Editor-in-Chief, for their many constructive proposals, which played an important role in helping to shape the final version of this paper.
About the authors
Colin Ashurst is the FME Senior Teaching Fellow in IS and Business Transformation in the Business School at the University of Durham. His research interests include IS strategy (identifying opportunities and priorities for IS investments);realizing benefits from IS investments; succeeding with projects and programmes to deliver organizational change, particularly where business innovation and change are significant; improving the productivity of knowledge work;and building the organizational capability to succeed in realizing value from the IS investment portfolio. Colin has broad commercial experience from a range of IT management and consulting roles.
Neil Doherty currently holds the Chair in Information Management in the Business School at Loughborough University. In addition to benefits management, his research interests include the interaction between organizational issues and technical factors in IS development, understanding the reasons for failures of IS projects, strategic IS planning and e-commerce. Neil has had papers published in a range of academic journals, including European Journal of Information Systems, Journal of Information Technology, Journal of Strategic Information Systems, Information Resources Management Journal, IEEE Transactions in Engineering Management, Journal of Business
Research, European Journal of Marketing, Journal of End User Computing, Information Technology & People, Behaviour & IT and Information & Management. Neil is currently an associate editor for Information Technology and People and the International Journal of Electronic Business Research. Joe Peppard holds the Chair in IS and is also Director of the Information Systems Research Centre, the IT Leadership Forum and has recently launched the innovative Cranfield IT Leadership Programme. He is also Adjunct Professor at the University of South Australia. Over the years he has held academic appointments at Loughbor-ough University, Trinity College Dublin, Groningen University and the University of Sydney. The focus of Professor Peppard's research and teaching is in the area of IS and technology strategy and management. He has published widely in academic and general business and management journals. His most recent books include Strategic Planning for Information Systems (Wiley) and Customer Relationship Management: Perspectives from the Marketplace (Butterworth-Heinemann). He is currently working on a book Creating a Value-Adding IS Capability: Delivering High Performance, Liberating Business Value. He is European Editor of the Journal of Information Technology and an Editorial Board Member of European Management Journal.
References
Ahn J and Skudlark A (1997) Resolving conflict of interests in the process of an information system implementation for advanced telecommunication services. Journal of Information Technology 12, 3-13.
Amit R and Schoemaker PJH (1993) Strategic assets and organizational rent. Strategic Management Journal 14, 33-46.
Avgerou C (2001) The significance of context in information systems and organizational change. Information Systems Journal 11(1), 43-63.
Barney JB (1991) Firm resources and sustained competitive advantage. Journal of Management 17, 99-120.
Bashein BJ, Markus ML and Riley P (1994) Preconditions for BPR success, and how to prevent failures. Information Systems Management 11(2), 7-13.
Benbasat I and Zmud RW (1999) Empirical research in information systems: the practice of relevance. MIS Quarterly 23(1), 3-16.
Benjamin RI and Levinson E (1993) A framework for managing IT-enabled change. Sloan Management Review, 34(4), 23-33.
Black JA and Boal KB (1994) Strategic resources: traits, configurations, and paths to sustainable competitive advantage. Strategic Management Journal 15, 131-148.
Boehm B and Turner R (2004) Balancing Agility and Discipline: A Guide for the Perplexed. Addison-Wesley, Boston.
Bohn RE (1994) Measuring and managing technological knowledge. MIT Sloan Management Review, 36(1), 61-73.
Boiney LG (1998) Reaping the benefits of information technology in organizations. The Journal of Applied Behavioral Science 34(3), 327-346.
Bowman C and Ambrosini V (2000) Value creation versus value capture: towards a coherent definition of value in strategy. British Journal of Management 11, 1-15.
Breu K and Peppard J (2003) Useful knowledge for information systems practice: the contribution of the participatory paradigm. Journal of Information Technology 18, 177-193.
British Computer Society (2004) The Challenge of Complex IT Projects. BCS, London.
Brooke C (2000) A framework for evaluating organizational choice and process redesign issues. Journal of Information Technology 15(1), 17-28.
Brown JS and Duguid P (2000) The Social Life of Information. Harvard Business School Press, Boston.
Butler T (1998) Towards a hermeneutic method for interpretive research in information systems. Journal of Information Technology 13, 285-300.
Carlile P (2002) A pragmatic view of knowledge and boundaries: boundary objects in new product development. Organization Science 13(4), 442-455.
Clegg C, Axtell C, Damodaran L, Farbey B, Hull R, Lloyd-Jones R, Nicholls J, Sell R and Tomlinson C (1997) Information technology: a study of performance and the role of human and organizational factors. Ergonomics 40(9), 851-871.
Clegg CW (2000) Socio-technical principles for system design. Applied Ergonomics 31, 463-477.
Davenport TH, Harris G, Delong D and Jacobson AL (2001) Data to knowledge to results: building an analytical capability. California Management Review 43(2), 117-138.
Davern MJ and Kauffman RJ (2000) Discovering potential and realizing value from information technology investments. Journal of Management Information Systems 16(4), 121-143.
Davidson E and Chiasson M (2005) Contextual influences on technology use mediation: a comparative analysis of electronic medical records system. European Journal of Information Systems 14(1), 6-18.
Desanctis G and Poole MS (1994) Capturing the complexity in advanced technology use: adaptive structuration theory. Organization 5(2), 121-147.
Doherty NF, Coombs CR and Loan Clarke J (2006) A re-conceptualization of the interpretive flexibility of information technologies: redressing the balance between the social and the technical. European Journal of Information Systems 15(6), 569-582.
Doherty NF and King M (1998) The consideration of organizational issues during the systems development process: an empirical analysis. Behaviour & Information Technology 17(1), 41-51.
Doherty NF and King M (2001) An investigation of the factors affecting the successful treatment of organizational issues in systems
development projects. European Journal of Information Systems 10, 147-160.
Doherty NF and King M (2005) From technical to socio-technical change: tackling the human and organizational aspects of systems development projects. European Journal of Information Systems 14(1), 1-5.
Doherty NF, King M and Al-Mushayt O (2003) The impact of inadequacies in the treatment of organizational issues on information systems development projects. Information and Management 41(1),
49-62.
Doolin B (2004) Power and resistance in the implementation of a medical management information system. Information Systems Journal 14(4), 343-362.
Earl MJ (1993) Experiences in strategic information systems planning. MIS Quarterly 17(1), 1-24.
Earl MJ and Khan B (2001) E-Commerce is changing the face of IT. MIT Sloan Management Review, 43(1).
Eason K (1988) Information Technology and Organizational Change. Taylor & Francis, London.
Eason K (2001) Changing perspectives on the organisational consequences of information technology. Behaviour & Information Technology 20(5), 323-328.
Edwards C and Peppard J (1997) Operationalizing strategy through process. Long Range Planning 30(5), 753-767.
Ewusi-Mensah K and Przasnyski Z (1994) Factors contributing to the abandonment of information systems development projects. Journal of Information Technology 9, 185-201.
Farbey B, Land F and Target D (1993) How to Assess Your IT Investment. Butterworth-Heinemann, Oxford.
Farbey B, Land F and Targett D (1999) Evaluating investments in IT: findings and a framework. in Beyond the IT Productivity Paradox (Willcocks L and Lester S, Eds), pp 183-215, Wiley, Chichester.
Gibson C (2003) IT-enabled business change: an approach to understanding and managing risk. MISQ Executive 2(2), 104-115.
Goh KH and Kauffman RJ (2005) Towards a theory of value latency for IT investments. Paper presented at 38th Hawaii International Conference on Systems Science, January, p. 231a.
Grant RM (1996a) Toward a knowledge-based theory of the firm. Strategic Management Journal 17, 109-122.
Grant RM (1996b) Prospering in dynamically competitive environments: organizational capability as knowledge integration. Organization Science 7, 375-387.
Grint K and Woolgar S (1997) The Machine at Work. Polity Press, Cambridge.
Gwillim D, Dovey K and Wieder B (2005) The politics of postimplementation reviews. Information Systems Journal 15(4), 307-319.
Helfat CE and Peteraf MA (2003) The dynamic resource-based view: capability lifecycles. Strategic Management Journal 24(10), 997-1010.
Hochstrasser B and Griffiths C (1991) Controlling IT Investment. Chapman Hall, London.
Hughes A and Scott Morton MS (2006) The transforming power of complementary assets. MIT Sloan Management Review, 47(4),
50-58.
Joshi K (1991) A model of users' perspective on change: the case of information systems technology implementation. MIS Quarterly 15(2), 229-242.
Jurison J (1996) Towards more effective management of information technology benefits. Journal of Strategic Information systems 5(4), 263-274.
Kangas K (1999) Competency and capabilities based competition and the role of information technology: the case of trading by a Finland-based firm to Russia. Journal of Information Technology Cases and Applications 1(2), 4-22.
Keen P (1991) Keynote address: relevance and rigor in information systems research. in Information Systems Research: Contemporary Approaches and Emergent Traditions (Nissen HE, Klein HK and Hirschheim R, Eds), pp 27-49, Elsevier Publishers, Amsterdam.
Klein HK and Myers MD (1999) A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly 23(1), 67-94.
Lederer AL and Nath R (1991) Managing organizational issues in systems development. journal of Systems Management 24(11), 23-27.
Lee A (1991) Integrating positivist and interpretive approaches to organizational research. Organization Science 2(4), 342-365.
Lin C and Pervan G (2003) The practice of IS/IT benefits management in large Australian organizations. Information & Management 41(1), 31-44.
Lyytinen K and Hirschheim R (1987) Information systems failures: a survey and classification of the empirical literature. Oxford Surveys in Information Technology 4, 257-309.
Majchrzak A, Rice RE, Malhotra A, King N and Ba S (2000) Technology adaptation: the case of a computer-supported inter-organizational virtual team. MIS Quarterly 24(4), 569-600.
Marchand DA, Kettinger W and Rollins JD (2000) Information orientation: people, technology and bottom line. Sloan Management Review, 41 (4), 69-80.
Markus ML (2004) Technochange management: using IT to drive organizational change. journal of Information Technology 19(1), 4-20.
Markus ML and Benjamin RI (1997) The magic bullet theory of IT-enabled transformation. Sloan Management Review 38(2), 55-68.
Markus ML and Robey D (1988) Information technology and organisational change: causal structure in theory and research. Management Science 34(5), 583-595.
Martinsons MG and Chong PK (1999) The influence of human factors and specialist involvement on information systems success. Human Relations 52(1), 123-152.
McAulay L (2007) Unintended consequences of computer mediated communications. Behaviour & Information Technology 26(5), 385-398.
McGrath RG, MacMillan IC and Venkatraman S (1995) Defining and developing competence: a strategic process paradigm. Strategic Management journal 16, 251-275.
Miles MB and Huberman AM (1994) Qualitative Data Analysis. Sage Publications, Thousand Oaks, CA.
Miller D and Shamsie S (1996) The resource-based view of the firm in two environments: the Hollywood film studios from 1936-1965. Academy of Management journal 39(3), 519-543.
Mingers J (2004) Realizing information systems: critical realism as an underpinning philosophy for information systems. Information and Organization 14, 87-103.
Moingeon B, Ramanantsoa B, M étais E and Orton JD (1998) Another look at strategy - structure relationships: the resource-based view. European Management journal 16(3), 298-304.
Myers MD (1994) Dialectical hermeneutics: a theoretical framework for the implementation of information systems. Information Systems journal 5(1), 51-70.
Nandhakumar J and Avison D (1999) The fiction of methodological development: a field study of information systems. Information Technology and People 12(2), 176-191.
National Audit Office (2006) Delivering successful IT-enabled business change. Report by the Comptroller and Auditor General, HC 33-1, Session 2006-2007, November.
Neely A, Adams C and Kennerley M (2002) The Performance Prism: The Scorecard for Measuring and Managing Business Success. FT Prentice-Hall, London.
Newell S, Tansley C and Huang J (2004) Social capital and knowledge integration in an ERP project team: the importance of bridging and bonding. British journal of Management 15, S43-S57.
Orlikowski WJ (1992) The duality of technology: rethinking the concept of technology in organizations. Organization Science 3(2), 398-427.
Orlikowski WJ and Hoffman D (1997) An improvisational model for change management: the case of groupware technologies. Sloan Management Review, 38(2), 11-21.
Peppard J, Lambert R and Edwards C (2000) Whose job is it anyway? Organizational information competencies for value creation. Information Systems journal 10(4), 291-322.
Peppard J and Ward J (2004) Beyond strategic information systems: towards an IS capability. journal of Strategic Information Systems 13(2), 167-194.
Peppard J and Ward J (2005) Unlocking sustained business value from IT investments. California Management Review, 48(1), 52-69.
Peppard J, Ward J and Daniel E (2007) Managing the realization of business benefits from IT investments. MIS Quarterly Executive 6(1), 1-11.
Pfeffer J and Sutton R (2000) The Knowing-Doing Cap: How Smart Companies Turn Knowledge Into Action. Harvard Business School Press, Boston.
Prahalad CK and Hamel G (1990) The core competencies of the corporation. Harvard Business Review 68(3), 79-91.
Remenyi D and Sherwood-Smith M (1999) Maximise information systems value by continuous participative evaluation. Logistics Information Management 12(1/2), 145-156.
Remenyi D, Sherwood-Smith M and White T (1997) Achieving Maximum Value from Information Systems: A Process Approach. John Wiley & Sons, Chichester.
Robey D and Boudreau MC (1999) Accounting for contradictory organizational consequences of IT. Information Systems Research 10(2), 167-185.
Robey D and Markus L (1998) Beyond rigor and relevance: producing consumable research about information systems. Information Resources Management Journal 11(1), 7-15.
Rose J and Jones M (2005) The double dance of agency: a socio-theoretic account of how machines and humans interact. Systems, Signs and Actions 1(1), 19-37.
Sahay S and Robey D (1996) Organisational context, social interpretation, and the implementation and consequences of geographic information systems. Accounting, Management and Information Technology 6(4), 255-282.
Sauer C and Cuthbertson C (2003) The State of IT Project Management in the UK 2002-2003. Templeton College, Oxford.
Schultze U and Boland R (2000) Knowledge management technology and the reproduction of knowledge work practices. Journal of Strategic Information Systems 9(2-3), 193-212.
Schultze U and Orlikowski W (2004) A practice perspective on technology-mediated network relations: the use of internet-based self serve technologies. Information Systems Research 15, 487-523.
Serafeimidis V and Smithson S (2000) Information systems evaluation in practice: a case study of organizational change. Journal of Information Technology 15(2), 93-105.
Silverman D (2000) Doing Qualitative Research. Sage Publications, Thousand Oaks, CA.
Stalk G, Evans P and Shulman LE (1992) Competing on capabilities: the new rules of corporate strategy. Harvard Business Review, 70(2), 57-69.
Strassman P (1990) The Business Value of Computers. The Information Economics Press, New Canaan, Connecticut.
Strauss AL and Corbin J (1990) Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Sage, Newbury Park, CA.
Tallon PP, Kraemer KL and Gurbaxani V (2000) Executives' perceptions of the business value of information technology: a process-oriented approach. Journal of Management Information Systems 16(4), 145-173.
Teece D and Pisano G (1994) The dynamic capabilities of firms: an introduction. Industrial and Corporate Change 3(3), 537-556.
Teece DJ, Pisano G and Shuen A (1997) Dynamic capabilities and strategic management. Strategic Management Journal 18, 509-533.
Tippins MJ and Sohi RS (2003) IT competency and firm performance: is organizational learning a missing link? Strategic Management Journal 24(8), 745-761.
Ward J and Daniel E (2006) Benefits Management. John Wiley & Sons, Chichester.
Ward J, De Hertog S and Viaene S (2007) Managing benefits from IS/IT investments: an empirical investigation into current practice. Proceedings of the Hawaii International Conference on Systems Science (HICSS), Hawaii.
Ward J and Elvin R (1999) A new framework for managing IT-enabled business change. Information Systems Journal 9(3), 197-222.
Ward J and Peppard J (2002) Strategic Planning for Information Systems. John Wiley & Sons, Chichester.
Ward J, Taylor P and Bond P (1996) Evaluation and the realization of IS/IT Benefits. European Journal of Information Systems 4, 214-225.
Weill P and Woodham R (2003) Don't just lead, Govern: implementing effective it governance. MIT Sloan School of Management Working Paper 4237-02, Boston.
Wenger W, McDermott R and Snyder WM (2002) Cultivating Communities of Practice. Harvard Business School Press, Boston. Wernerfelt B (1984) A resourced-based view of the firm. Strategic
Management Journal 5, 171-180. Zuboff S (1988) Dilemmas of Transformation in the Age of the Smart Machine. Basic Books, New York.
