Scholarly article on topic 'After a Successful Business Case of ERP – What Happens then?'

After a Successful Business Case of ERP – What Happens then? Academic research paper on "Economics and business"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia Computer Science
OECD Field of science
Keywords
{"Business Case" / "Cost-Benefit Analysis" / "ERP: Evaluation" / "ERP investments"}

Abstract of research paper on Economics and business, author of scientific article — Björn Johansson, Lucas Karlsson, Emil Laine, Viktor Wiksell

Abstract Enterprise Resource Planning (ERPs) systems are increasingly playing a central role in many organizations. However, before implementing ERPs organizations create business cases that are used when deciding on an ERP investment. The research questions are: How do organizations evaluate ERP investments and what models do they use to follow up the business case of an ERP investment? To say something about this we investigated the development and use of ERP business cases in five manufacturing organizations, by conducting semi-structured interviews with executives. The main conclusion from the research is that the realization of the business case in many cases is made to put together arguments to “sell” a specific investment to management and decision makers. When a decision then is made and there is consensus within the organization that the project is necessary, it is expressed that there is no longer any need to monitor returns from an investment in ERPs.

Academic research paper on topic "After a Successful Business Case of ERP – What Happens then?"

CrossMark

Available online at www.sciencedirect.com

ScienceDirect

Procedia Computer Science 100 (2016) 383 - 392

Conference on ENTERprise Information Systems / International Conference on Project MANagement / Conference on Health and Social Care Information Systems and Technologies, CENTERIS / ProjMAN / HCist 2016, October 5-7, 2016

After a Successful Business Case of ERP - What happens then?

Björn Johansson*, Lucas Karlsson, Emil Laine, Viktor Wiksell

Department of Informatics, School of Economics and Mangement, Lund University, Ole Römers väg 6 SE-22363 Lund, Sweden

Abstract

Enterprise Resource Planning (ERPs) systems are increasingly playing a central role in many organizations. However, before implementing ERPs organizations create business cases that are used when deciding on an ERP investment. The research questions are: How do organizations evaluate ERP investments and what models do they use to follow up the business case of an ERP investment? To say something about this we investigated the development and use of ERP business cases in five manufacturing organizations, by conducting semi-structured interviews with executives. The main conclusion from the research is that the realization of the business case in many cases is made to put together arguments to "sell" a specific investment to management and decision makers. When a decision then is made and there is consensus within the organization that the project is necessary, it is expressed that there is no longer any need to monitor returns from an investment in ERPs.

© 2016 The Authors. Publishedby ElsevierB.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the organizing committee of CENTERIS 2016 Keywords: Business Case; Cost-Benefit Analysis; ERP: Evaluation; ERP investments.

1. Introduction

As the role of Information Technology (IT) has steadily increased and being more central in companies and businesses so has also the budget for IT investments come to grow. Companies' spends more and more on IT and IT

* Corresponding author. Tel.: +46 462228021; fax: +46 462224528. E-mail address: bjorn.johansson@ics.lu.se

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the organizing committee of CENTERIS 2016 doi:10.1016/j.procs.2016.09.173

support and in many organizations the Enterprise Resource Planning (ERPs) system consists of the largest part of the IT cost [1]. Today's Enterprise systems has grown since the 60s, and since the term ERP (Enterprise Resource Planning) began to be used as a name for them in the early 90s, these systems has gone through a generational change and functionality has extended [2]. When ERPs comes to be used on a broad front around the turn of the millennium they were often understood as being a strategic investment to create competitive advantage and improve the efficiency of production, but today ERP systems has become commonplace for manufacturing companies. It has also been questioned whether the system delivers a competitive advantage or not [1]. There have even been opinions expressed that ERP implementations could possible lead to a loss of competitive advantage because of the standardization process that often occurs in conjunction with the introduction of the system.

The fundamental question of business value from IT investments is a subject that has been under intensive discussion for long. The requirement of return and a clear business case for IT investment has become increasingly critical [3, 4]. Relatively few companies believe that IT is trouble-free to implement satisfactory, and several studies show that the issue of measuring actual business value of IT investments is a concern in many organizations [3-5]. Measurement of IT value is a high priority for many businesses [5], and it is one of the most important parts of the work for IT managers today [3].

From the literature it can be claimed that evaluation of ERP investments is a complex task, and the approach on how to do it is fragmented. These investments are difficult to assess since there are a number of underlying factors that generate value. Based on this, we find that it is relevant to examine how organizations work with evaluation of ERP investments, and how this differs from the literary point of view. The research questions addressed in this research are: How do organizations evaluate ERP investments and what models do they use to follow up the business case of an ERP investment?

The purpose of this research is to examine how companies are working to assess the return on ERP investments, and if they take account of monetary as well as non-monetary values, how do they do it? The rest of the paper is organized as follows. The next section presents a literature review on business case and evaluation methods. Section 3 presents how the research was done, followed by Section 4 that presents and discusses empirical findings. In Section 5 we then discuss the results and finally in section 6 we present our conclusions as well as contributions and future research directions.

2. The Concept of ERP Business Case and How ERP investments is described as being evaluated

2.1. ERP Business Case yu76y

A business case is a structured proposal, often used for justifying investments in terms of expected costs and revenues [6]. It thereby becomes a mechanism to justify and manage projects, and determine what projects are most vital to invest in. To develop a business case before approving IT projects are commonplace among businesses today, especially regarding ERP investments. In a study by Ward, Daniel and Peppard [4] 96% of participating organizations said that some kind of business case was needed to get approval for an investment. Despite this, there is little research on the business case for ERP [6]. Traditionally, a business case has been developed in order to get funding for projects for change, but according to Ward, Daniel and Peppard [4] there is also four additional reasons why a business case is necessary: 1) Enable prioritization between different investments, 2) Identify how a mix of IT and business-related changes will support expected values, 3) Secure commitment from business leaders, and 4) Create a base for evaluation and follow-up once the investment is completed.

Ward, Daniel and Peppard [4] has developed a framework for building robust business cases for investments in IT systems. They have identified that many companies have trouble estimating the value of their IT investments, and they therefore also lacks an informed base for decisions. They also describe it as a problem when organizations focus too much on monetary values when making investment calculations. According to Ward, Daniel and Peppard [4] does their approach differs from other approaches by emphasizing on identifying different types of values and making specific links to specific changes, and that measurable values are identified in addition to identification of the origin for each value/benefit nominated [4]. Previous business case approaches often do not addresses aspects of how the ERP system is adapted to the goals and strategy of the organization, and key users are not always involved in the project to the extent that is desired [6], which indicate that business cases often tends to focus on technology

rather than people and processes. In this research a successful ERP business case is defined from the outcome that the investment is made, and as such the business case justifies the investment.

2.2. Evaluation of ERP investments

A common approach to evaluate investment is Return of Investment (ROI). Calculating of ROIs is done by dividing return on an investment with cost involved in the investment; the results are then displayed as a percentage. If resulting percentage has a positive or negative value indicates if the investment is profitable or not expressed in a ROI value. ROI calculation is often regarded as vague and incomplete, because ROI is strictly economic and only takes into account the revenue and costs for investments [7]. It gives a good overview of a project's returns and costs to a useful performance measure. However, this measure does not take into account time and risk which are two essential elements that should be part of a complete economic calculation [7]. According to Kumar and Keshan [8] calculating ROI of ERP is problematic since it has to measure both hard and soft values and explore the relationship between them and the ERP system.

Another approach for evaluation of investments is the Cost Benefit Analysis (CBA). CBA can be carried out when organizations or individuals in organizations considers between investment choices. It provides a holistic overview of Benefits (benefit) and Costs (expenses) related to the investment, the result of the analysis is a monetary valuation of the Net Benefits of the investment [9]. When considering different investment options, these tend to only take into account the benefits and costs related to themselves, but CBA aims at taking into account all costs and benefits [9].

According to Ward, Daniel and Peppard [4], it is problematic if organizations use CBA to calculate and evaluate IT investments, if cost first is estimated and then it easily becomes that profits are adjusted to cover costs incurred by either overestimate or underestimate. This phenomenon is rooted in that it is complex to assess an accurate value on the benefits, and that the person making the estimate can be colored by their own opinions about the project. To deal with this problem, Ward, Daniel and Peppard [4] therefore suggest a reverse analysis process for IT investments, where estimation of the values (Benefits) is carried out before estimation of costs (Costs) is made.

Another approach for discussing cost and value of IT investments is the well-established concept Total Cost of Ownership (TCO). TCO aims at discussing the cost and value of IT, and is supposed to cover all costs related to the investment, which includes direct, indirect and hidden costs [3]. Regarding IT investments such as ERPs the purchase cost consists of only a fraction of the TCO, the larger part consists of deployment and maintenance costs. According to Fryling [10] comprises purchase cost of ERP systems only 10% of TCO, while the remaining 90% include additional costs such as training, consultants, increased personnel, etc. Estimating TCO for IT projects is often a very difficult task when projects involve not just IT, but also affect business processes, the organization at large and the environment that the organization works in [10].

Evaluating ERP activities or performance can also be made by Key Performance Indicators (KPI). KPI is a collection of selected performance indicators to evaluate an activity or solution performance [11], and they are either detailed or in-depth analyzed information, but these rather give an overall picture of what is important. KPIs thus define a collection of target values which the organization can measure against, for example, financial, process based-, value based- or practice-based indicators [11]. In addition to improve processes can KPIs provide support for decision-making and thus help management in strategic decision-making [11].

The goal of the various tools and frameworks presented above is to increase knowledge about planned IT investments, and thereby create a better base for decision making. The economic analysis tools for investment (ROI, TCO and CBA) takes only hard, monetary values into consideration, and should therefore not be used as the sole evidence to justify an ERP investment. There is consensus in the literature about evaluation of ERPs and that estimation of its value is a difficult question. Silvius [3] also claims that estimated value also differs depending on the position of the one that makes the evaluation have. Hugoson, Johansson and Seigerroth [5] argues that there are three main reasons why the evaluation is difficult to implement: 1) Whether an investment is successful or not depends on the original expectations, 2) IT investments deliver both soft and hard values, and that both these categories must be considered during an evaluation, and 3) IT is essential for most businesses today, and thus woven into the organizations' business processes. This latter phenomenon is also known as The IT productivity paradox.

3. RESEARCH METHODS

To answer our research question - How do organizations evaluate ERP investments and what models do they use to follow up the business case of an ERP investment? - as comprehensive and accurate as possible, we must take into account the difficulties presented in section 2, but also investigate the whole evaluation process. Based on this, we have chosen to design our research method, the empirical investigation and analysis, into three parts. These three are: 1) Motivation for investment and business case, 2) Estimation and quantification of hard and soft values, and 3) Monitoring and evaluation of investments. Semi-structured interviews were conducted with executives in five manufacturing organizations, labeled as CarD, Milky, BrakeD, Loads and Mechanics. The interviews were transcribed and analyzed from the three parts described above.

4. EMPIRICAL RESULTS

In this part of the paper we present the empirical data and the result from our analysis, the presentation is organized after the three parts presented in the method section: 1) Motivation for investment and decision-making, 2) Quantification of soft and hard values, and 3) Monitoring and evaluation of investments.

4.1. Motivation for investment and decision-making

The organizations interviewed justified their ERP investments in various ways. At CarD, Mechanics, Loads and Milky an estimate of the value of the investment was required, but even if the requirements on that request differed, the fundamental principle was that the investment should deliver some kind of business-related benefit. This was in most cases compiled into a business case which then was presented to decision makers. At BrakeD the requested motivation behind the investment was less clear, and the requirement of delivering a return was not clear to the same extent. Resources spent on an ERP system was considered as an operating cost rather than an investment that should return a value. Although the aim basically was to streamline and improve processes at BrakeD, BrakeD did not follow the same basic principle as the other organizations.

When Loads was in the position of a decision regarding update or obtaining a new ERP they always built a business case. The business case aimed at comparing new functionality with current ones, in order to see what savings opportunities that exist. Key parameter of Loads's business case is savings in monetary values. The IT manager at Loads describes their business case as:"it is often very theoretical, but with little sense making of course".

At Milky there was a very clear requirement that investments would deliver 100% ROI and a payback period of less than five years. If taking longer than five years to deliver this value the controller at Milky stated that it was very difficult to get support for the investment. To figure out whether an investment can achieve this requirement, a CBA was carried out, in where it was demonstrated how the savings would occur. The central building block of Milky's CBA was how much time savings that can be achieved with a system replacement, the controller said: "It's so we put up our CBA analysis, how much time we can save by switching system?". The CBA Milky creates is made up of purely monetary terms and values. Soft values were only considered to be secondary.

BrakeD investments decisions is made by a group consisting of a combination of IT staff and super users from the business called CAB (Change Advisory Board), except for some major strategic decisions made at a higher hierarchical level. Super users play a very central role in these decisions as they are very familiar with the business processes. The system administrator describes these super users as: "People outside the IT department but who still cooperate much with us. These people can processes better than we can, for example in the production, HR, Finance, etc.". Investments agreed on in the group do not need to have direct return, but the group's work was rather to prioritize investments that would be enforced on the basis of a given annual budget. This group also handles, to some extent, issues related to the maintenance of the system, and these matters were covered by the same budget, which may explain why a strict ROI perspective was missing. The Development manager said: "From the management side it has been said that they accept changes within this range, and then we add these super users who get to decide what to do". Availability of internal and external resources was also considered in the decisions at BrakeD.

At CarD it is clearly seen that investments decisions are very personal dependent. The ERP Manager was the one that run the decision from start to finish, and was responsible for estimating both cost and potential return on the investment. The ERP manager freely decides on if to develop a business case for the investment, and then seeking approval from the CEO. No formal CBA or ROI analysis was needed, but to get approval for an ERP investment the ERP manager had to be able to demonstrate that the invested amount would be returned in some way. This was done by showing that the current KPIs could be improved, the ERP manager explained: "If I buy a new license to a purchaser, I want to see that they can handle more of their purchase orders in a week, compared to what they managed before the additional license was bought". Other examples of KPIs that could be part of a business case where the lead time internally and the number of cars produced per year. A translation into monetary terms was not necessary, and an investment could be made from the perspective of soft values, as long as the ERP manager and the CEO estimated that the investment is worth implementing.

At Mechanics they, the IT and logistics manager, usually build a business case for the systems they had selected in a comparative study of various potential systems. Functionality and economics perspective are used in selection phase, but the greatest emphasis is on functionality. They filled up the functional requirements supplemented with data about the total economic cost of the system. This system was later presented to the management as a business case which then took the final decision on the proposed investment and if it would be realized or not. When the business case is presented to management, repayment calculations are included to show how the investment will pay for itself. The IT and logistics manager describes it in the following way: "When we took the decision that it was profitable based on an ROI calculation it showed very positive numbers. But all the conditions were presented, and it was stated that it provides an equivalent or enhancing support. Had I not been able to do so the calculation had not played any role".

Table 1 Indicator results on Motivation for investment and decision-making

Indicator CarD Milky BrakeD Loads Mechanics

Clearly demands a business case Yes, but no clear requirements on it Yes No Yes Yes

The investment must return a positive financial value? No, the value may be non-monetary Yes No Yes No

Quantification of expected effects No Yes No Yes No

Models used to calculate payback - CBA, ROI - ROI ROI

Requirements on payback time? No Yes, 100% in 5 years No Yes No

Can the decision be based on soft values? Yes No Yes Yes, but only on risk parameters Yes, but only on risk parameters

The empirical data so far shows that there are, in some form, a business case that demonstrates that there are advantages gained from the investment before the actual decision is taken. Sometimes it takes into account both hard and soft values. Overall, it considers hard monetary values to a higher extent when organizations face a major investment. However, soft non-monetary values play a role in a business case as they are considered to complement the hard values. They make the case story more interesting for decision makers, which can lead to that soft values can be the base for investment decisions, despite being unable to demonstrate that these will lead to actual savings. The empirical data on motivation for investment and decision-making was summarized in table 1.

4.2. Estimation and quantification of hard and soft values

All participating organizations took into account hard respectively soft values in some way, and they were all aware of their existence. At CarD they stated some investment decisions is based on expected soft values and benefits that were expected to follow. There is a belief that soft values are important and could deliver actual cost savings in the future, although the timing and the amount was not defined. However, soft values were not quantified into purely monetary values and the ERP manager described the problems of implementing this in the following way: "For example. I bought a number of additional licenses a little while ago so the production would not have to

go to a new workstation to access the system. That change resulted in that the staff thought it was a pleasant working environment, but is difficult to measure in pure productivity or money for that matter ".

BrakeD just like CarD make certain investment decisions based solely on anticipated effects of soft values, and similar to CarD, they did not quantify these soft values, which is largely due to these investments did not have to show a positive financial return. When asked if they do not have any structured way to assess gains from soft values, the Development manager said: "Unfortunately not, and the focus is a lot around money or hours". Furthermore, the Development manager state that they work strictly from an IT budget, and that should not be exceeded.

At Loads they first and foremost try to find a potential economic savings in the business case. In the business case they also include soft values, but these are not quantified in economic terms. The respondents said that today's ERP systems will help employees work more efficiently in everyday work and often make employees happier, but this is difficult to translate into pure monetary values. "Soft values does not count, but there are arguments that you can build a case on, absolutely" stated the IT manager. The hard values are a requirement for obtaining financing of an investment, but soft values may act as a flavor on top. Furthermore, the interviewees at Loads say that they are not using a specific model to try to quantify soft values.

Milky behaves just like Loads, meaning that they discuss soft values when specifying expected core values generated by the investment, but there is still a requirement that the investment should deliver returns in hard values. Milky tried not to quantify or estimate soft values in monetary terms, instead they treats them as something of secondary importance, described by the Controller in the following way: "We do a pure numerical calculation of what to make of this. From this we can see how long it takes to reach break-even. On top of this we have a lot of soft parameters we can add, but these are hard to translate into numbers. But they are included in the CBA, it gets a little more life in the analysis".

The IT and Logistics manager at Mechanics said that soft values are important, but it is hard values making an investment gets approval, just like in Loads and Milky. The manager at Mechanics describes the work when the past system was purchased, as: "This work led to that a mass of soft values were intercepted and pointed to. But basically it was the calculation that made it kick-off, even though we believe that soft values are important". However, no systematic way exist to quantify soft values, they are appreciated, but never translated into monetary values.

The empirical data describes that none of the organizations are using a predefined template to capture soft values according to the interviewee. Milky was the organization that worked most structured with soft values, since they put together lists of what they considered to be all soft effects and included them in its CBA. This may seem strange when it was the company that had the most stringent requirements of monetary return and paid least attention to soft values when making investments decisions. However, this can be explained by the Milky is a relatively large company and the project, which was primarily discussed during the interview was a global ERP roll-out projects, which would mean that all elements of the project carried out in more detail than in many of the projects discussed at the other interviews.

Common to all the organizations is that they can use soft values together with the hard values to build a business case. The hard values has to be there, but the soft values can make a business case much more interesting and thus help lead to an investment decision. There was lack of attempts to quantify soft values, and translate them to monetary values. It is of course difficult to do this precisely, but in the literature there are techniques which aim to facilitate this. The general attitude among interviewees was that this is too hard to do and requires training for a long time before it would be considered worth it. A final result of such calculations was not considered reliable enough to justify the workload needed to do it.

Table 2 Indicator results on Estimation and Quantification of hard and soft values

Indicator CarD Milky BrakeD Loads Mechanics

Soft values as part of the business case Yes Yes Yes Yes Yes

Soft values as the sole base for investment decisions Yes No Yes No No

Quantification of soft values No No No No No

Predefined template or model to capture soft and No No No No No

hard values

4.3. Monitoring and evaluation of investments

At BrakeD only cost of the projects are followed up, and they do not look at all on generated value from the investment. The system administrator said: "This cost estimate that we got in the beginning, for instance that it would take 20 hours, then we follow up how many hours it actually became. But to investigate if we actually got business value for the money, we are not good in doing". The only time a discussion is on the effects is when something has gone wrong and complaints came in from the business. However, lack of evaluation is not only related to hard values, but also soft values. If a project is carried out with soft values as the base for the investment decision it is not checked whether these values are delivered or not, but the project is considered to be completed as soon as it is implemented technically.

CarD followed up their investments by looking at selected KPIs, but also in this part of the investment process it highly dependent on individuals. The follow-up is basically done by that the business manager and CEO sits down and compared expected outcome with actual outcome, "We measure this very simple, call it common sense, the department managed so many order proposals last week, now do you this many" said the ERP manager when he describes how they work with follow-up their business case. The ERP manager makes the evaluation when he considers the investment should have had an effect on the business, and from that an assessment of whether the investment in question was successful or not is done. The basic yardstick is that the investment should have generated an ROI of over 100% in order to be successful, but that estimate is very informal and soft values could be considered sufficient if there is a belief that these would generate monetary value in the future. The ERP manager expressed that it is largely a matter of common sense and a feeling, since the time to implement a more structured and detailed assessment is not there.

Loads saw little value in conducting evaluations of their investments in ERPs, and thereby they spent very little resources to implement evaluation. They felt that if we had made the right or wrong one knew this, and then try to quantify and prove that miscalculated investments is superfluous. Lack of resources for doing evaluation analysis was also described as a hindrance, and the IT manager: "When time for evaluation then we are already doing something else". The Purchasing manager described doubts about doing evaluations of investments and he said: "I put very little value to such post-investments evaluations, and the reason is that even if you can learn a lot about how to perform the analysis perfectly, it is almost always the case that those who do it are not objective".

Milky has recently initiated a global ERP roll-out project involving all production sites within the organization. The first implementation was not yet ready to be evaluated, so the interviewee instead described the plans for evaluation. The idea was that the project would be evaluated with a number of performance indicators that would provide a picture of how, among other things, internal efficiency and lead times were affected by the roll-out, while cost and time were monitored. The controller, however, stated that this is a standardized way of doing it and therefore there are some shortcomings, which he described in the following way: "Then one can question whether the KPIs used and our project is just right for it, it may be that they show that the project is not entirely successful, and vice versa. There may be things that make the metrics showing the wrong results. So there is an evaluation, but it is not a clear cut".

It was claimed that this evaluation should be done continuously, so that lessons learned from implementation at one production site could be taken to the next, thereby streamlining the actual implementation.

At Milky, just like the other organizations, the ambition to calculate an exact value and return was lacking, and although an assessment model based on selected KPIs were advised there were some criticism about accuracy of this. There was certainly a willingness to evaluate project outcomes, but KPIs were not giving the whole picture. There was, as mentioned earlier a very clear return, but also a strong belief that it was possible to manipulate the numbers until they yielded desired result. This was the basis for skepticism about the KPI-evaluation accuracy and from that the controller stated that: "the best basis for evaluation is a mixture of hard numbers and subjective feelings".

In Mechanics, there had been a change of system relatively recently (2011). Even this lacks a follow-up calculation of ROI. It is clear that the main emphasis is on ensuring that intended functionality is available from the system itself, and it is not a question if it has given a ROI or not. The IT and logistics manager at Mechanics describe the evaluation in the following way: "Not from an economic perspective, it has not. But when we did a project summary, we could determine the outcome was expected against the calculation that was made before, so that's

done. Perhaps more of a project summary. Since then we have our key figures of the company which is specified as previously - and we cannot see any real improvements or deteriorations in the numbers". In this particular case, Mechanics switched to a new ERP system that cost half of what the old system would cost to upgrade and get the same functionality from. The IT and logistics manager gives his explanation on why doing a business case as a function of selling the investment to management and get an approval: "What's the meaning of doing a follow-up a few years after? We can hardly rewind the tape anyway. Sure, it would surely be able to learn to the next acquisitions but as conditions change all the time it is more interesting to do the evaluation for a change instead of between two systems, just to find out if it's good or bad what we already done ".

Among the investigated organizations there were generally very small ambitions to carefully evaluate investments in retrospective and calculate generated value. All companies require some form of reasoning to justify the investment of the decision, and some had a clear obligation on the return. This was compiled mostly in the form of a business case. But there were no clear guidelines on how and when evaluation should be done to follow up what is promised in the business case.

Loads, Milky and Mechanics base their investment decisions on the risk involved in not implementing the investment. All the companies are talking about there is no opportunity to not have any ERP system, instead it is an investment that must be made. This makes it difficult to evaluate the value that may be generated by the investment, because you cannot compare to the alternative of not doing an ERP investment.

Overall, there was a strong doubt about doing post evaluations of ERP investments, and the expressed reason is that it is hard to really believe in the evaluation since it is so dependent on who do the evaluation. Much depends on the person carrying out the analysis, if he was in favor of the project from the beginning it would probably be evaluated very positively and vice versa.

Table 3 Indicator results on Monitoring and Evaluation of investments

Indicator CarD Milky BrakeD Loads Mechanics

Follow-up costs Yes Yes Yes Yes Yes

Follow-up new functionality Yes Yes Yes Yes Yes

Follow-up expected effects Yes Yes No No Yes

Calculating return on investment Yes, informally No No No No

5. Discussion

Regarding the motivation for investment and decision-making it was found among investigated organizations that there were generally clear requirements on the business case used to justify investments. This is in line with what Silvius [3] and Ward, Daniel and Peppard [4] state about usage of business case as a basis for an investment. How detailed and comprehensive these were varied between the organizations, however, this could be explained from the fact that investments that the respondents referred to were in various sizes.

When developing the business case, there was often a belief that the investment would return any form of value. It was also stated that investments in ERP can return both soft and hard values, and that both should be considered in the decision-making process, also this is in line with what Hugoson, Johansson and Seigerroth [5] as well as Kumar and Keshan [8] suggest.

In some organizations the investment was expected to have a specific payback within a certain period of time, there was therefore a clear need for monetary returns. We see a clear pattern to the organizations that focused on hard values (numbers) as a basis in the business case, and that these were the main arguments why one would implement an investment. The soft values was used as a "topping" on top of hard values to shape a more interesting business case, often used to increase the chance of obtaining support for the investment. It is a fact that the usage of business cases plays a crucial role to whether ERP investments receive funding or not. From this we can conclude that the business case can be seen as a tool to sell an investment to decision makers.

Other motives behind purely economic ones were found at Loads and Mechanics, and one motive was expressed as the risk of not implementing a particular investment, implying that the ERP investment was seen as a necessity to

do. They felt that their existing support system was unsustainable, and the consequences of delaying new investment could be considerably more expensive in the long run. Lack of reliability and service at risk in the long run threaten the stability of the system, and although the risk was not immediate, both Loads and Mechanics explicitly stated that the change was done as a preventive measure. Based on this we can see that the requirement of a positive ROI is not as central as risk parameters are as a driving factor for an ERP investment decision.

In relation to estimation and quantification of hard versus soft values, it is a fact that all investigated organizations considered both hard and soft values, and there was a desire to identify all the values related to the investment. However, there was a lack of structured approach to capture soft values. Thus, no model was used to identify these values. Overall, the organizations used sense making a lot when they identified positive effects and postulated what could be the result of the investment when suggesting arguments for an investment. This approach was described as very person dependent, and the identification of soft values was very subjective and therefore suggested soft values were not obvious to everybody.

Ambitions to quantify soft values were generally very small. It was described as that soft values have a value, but it is difficult to prove what they are actually worth. The type of estimation of values that Ward, Daniel and Peppard [4] suggest did not occur. As previously mentioned, estimated values plays an important role in decision-making on investments, and, overall, hard values is described as more important than soft values. Soft values were used on top of the hard to create a more attractive business case. However, in some cases could soft values be the trigger, for instance BrakeD and CarD justified investments using soft values, especially when it came to smaller sums. Also in Loads and Mechanics were soft values, especially risks, a driving factor for decisions. But, in all cases no quantification or monetary value was added to these values.

Regarding monitoring and evaluation of investments the empirical results show that there is a structured follow-up of the costs and technical features related to the investment, but it does not focus on the return value. As mentioned earlier in the discussion, overall there were clear demands on a business case, which promised a return value of the investment. In general, these are not followed up and there is no attempt to find out what value you actually got out from the investment. The decision makers who decided on the investment did therefore not get any financial feedback on the actual outcome of the investment.

At a general level all organizations considered evaluation and aimed at follow up investments, but they stated that it was difficult to perform with precision, which is consistent with what Hugoson, Johansson and Seigerroth [5]l. (2009), Silvius [3] and Ward, Daniel and Peppard [4] also state about difficulties of estimating real business value. We felt that organizations in general had a hard time and found it to be difficult to put any monetary value to soft values, and if doing so they felt strongly that these values becomes were subjective. It was clearly spelt out that it heavily depends on one's attitude to investment from the beginning; if you wanted it to be implemented from the beginning, your evaluation probably show that it was a good decision. It was also found that an evaluation of a made investment often is heavily impacted by the original expectations, which impedes that an objective evaluation is carried out.

6. Conclusion

The research question posed was: How do organizations evaluate ERP investments and what models do they use to follow up the business case of an ERP investment?

The study shows that the evaluation of ERP investments is flawed, especially when it comes to determining the economic outcome. None of the investigated organizations were particularly interested to figure out the exact return on their investment. All organizations based their investments decision on business cases and calculated effects until the moment the investment decision was taken. When investments were made little importance was added to follow up actual outcome and compare with expectations in the business case, in reality only one organization focused on doing regularly evaluations.

Theoretical models for evaluation were not used at all among the organizations, nor were framework for interception or quantification of softer values. Hard values were evaluated to some extent, and compared with costs, but in case of soft values evaluation these was handled separately and we saw no attempt to translate these into monetary terms.

If evaluation was made it was mainly based on a number of varying KPIs. These were supplemented largely by subjective perceptions and assessments, and regardless of size of the company or project. There was a slightly bigger dependency on a specific person in smaller ERP investments, which was quite expected, but it differed surprisingly little about how hard and soft values in overall were assessed. KPIs used were developed with the purpose to provide a picture of how an investment affected the company, but there was skepticism of relying on these results. KPIs were considered to give a part of the truth, but they were not able to take into account the reality of the complexity, and therefore personal assessments was of relative importance.

There were a number of reasons why organizations chose not to evaluate ROI. In general terms, the organizations stated that they know whether the projects implemented were successful or not, and they found themselves not having any need to produce an evaluation of outcomes. In addition, respondents said that it is difficult to provide an exact and correct figure, and that the reliability of such a figure would be low, as it was considered to be a high risk that the figure is colored by opinions of those who made the evaluation. In short, an evaluation was not considered so valuable that it justified the workload required.

The main conclusion from the research is that the realization of the business case in many cases is made to put together arguments to "sell" a specific investment to management and decision makers. When a decision is then made and there is consensus within the organization that the project is necessary, it is expressed that there is no longer any need to monitor returns. This lack of follow-up and the low interest for doing so could definitely be questioned, and we see this attitude to be very strange as the decision-makers then not receive any confirmation of what their decisions actually led to and economic consequences good or bad are not considered at all.

In light of the conclusions presented a number of issues that could act as the basis for future research in this area, particularly from the consequences of a lack of financial monitoring. The potential future research issues we see are: 1) What leads to the lack of follow-up of business cases and the lack of following up monetary return? 2) Is there a risk that IT managers and other stakeholders exaggerate the positive effects of the business case to get the investment approved?, and 3) Does this in such cases means that organizations risk losing control of their IT budget? It may also be that the future of ERP systems neither can nor should be evaluated in monetary terms because they can be absolutely essential for business survival, and that the IT budget thus be adapted to existing costs. If this is the case, we believe that the traditional decision-making process of the business case and ROI requirements are no longer relevant, and that a more holistic decision making with greater focus on soft factors are needed.

References

1 Wu, F., Liu, C., Li, H., Gao, K., and Tian, J.: 'The benefits evaluation of ERP project investment based on real options'. Proc. Systems, Man and Cybernetics, 2006. SMC'06. IEEE International Conference on2006 pp. Pages

2 Klaus, H., Rosemann, M., and Gable, G.G.: 'What is ERP?', Information systems frontiers, 2000, 2, (2), pp. 141 -162

3 Silvius, A.: 'Does ROI matter? Insights into the true business value of IT', The Electronic Journal Information Systems Evaluation, 2006, 9, (2), pp. 93-104

4 Ward, J., Daniel, E., and Peppard, J.: 'Building Better Business Cases for IT Investments', MIS Quartely executive, 2008, 7, (1), pp. 1-15

5 Hugoson, M.-Â., Johansson, B., and Seigerroth, U.: 'The Total Picture-A Framework for Control of IT Investments': 'Information Systems Development' (Springer, 2009), pp. 441-449

6 Al-Twairesh, N., and Al-Mudimigh, A.S.: 'Business Cases for ERP Implementations', Journal of Theoretical and Applied Information Technology, 2011, 25, (1), pp. 43-49

7 Erdogmus, H., Favaro, J., and Strigel, W.: 'Return On Investment', IEEE Computer Society, 2010

8 Kumar, S., and Keshan, A.: 'ERP implementation in Tata steel: focus on benefits and ROI', Journal of Information Technology Case and Application Research, 2009, 11, (3), pp. 68-103

9 Boardman, A.E., Greenberg, D.H., Vining, A.R., and Weimer, D.L.: 'Cost-benefit analysis: concepts and practice', 2006

10 Fryling, M.: 'Estimating the impact of enterprise resource planning project management decisions on post-implementation maintenance costs: a case study using simulation modelling', Enterprise Information Systems, 2010, 4, (4), pp. 391 -421

11 Selmeci, A., Orosz, I., Györök, G., and Orosz, T.: 'Key Performance Indicators used in ERP performance measurement applications', in Editor (Ed.)A(Eds.): 'Book Key Performance Indicators used in ERP performance measurement applications' (IEEE, 2012, edn.), pp. 43-48