Scholarly article on topic 'Barriers to innovation through public procurement: A supplier perspective'

Barriers to innovation through public procurement: A supplier perspective Academic research paper on "Economics and business"

Share paper
Academic journal
OECD Field of science
{Innovation / "Public procurement" / "Public sector" / "Innovation policy" / R&D}

Abstract of research paper on Economics and business, author of scientific article — Elvira Uyarra, Jakob Edler, Javier Garcia-Estevez, Luke Georghiou, Jillian Yeow

Abstract Public procurement is increasingly viewed as having important potential to drive innovation. Despite this interest, numerous barriers prevent the public sector from acting as an intelligent and informed customer. This paper seeks to understand how barriers related to processes, competences, procedures and relationships in public procurement influence suppliers׳ ability to innovate and to reap the benefits of innovation. We address this by exploiting a dedicated survey of public sector suppliers in the UK, using a probit model to investigate the influence of structural, market and innovation determinants on suppliers׳ perception of these barriers. The main barriers reported by suppliers refer to the lack of interaction with procuring organisations, the use of over-specified tenders as opposed to outcome based specifications, low competences of procurers and a poor management of risk during the procurement process. Such barriers are perceived most strongly by R&D intensive organisations. Our results also indicate that certain organisations, particularly smaller firms and not-forprofit organisations, encounter greater difficulties with innovation arising from the procurement process, for instance in relation to contract size, lack of useful feedback and communication of opportunities. Government procurement policies are queried in light of the findings.

Academic research paper on topic "Barriers to innovation through public procurement: A supplier perspective"

Technovation I (I


Contents lists available at ScienceDirect


journal homepage:

Barriers to innovation through public procurement: A supplier perspective

Elvira Uyarraa'*, Jakob Edlera, Javier Garcia-Estevez b, Luke Georghioua, Jillian Yeowa

a Manchester Institute of Innovation Research, Manchester Business School, University of Manchester, United Kingdom b United Nations Development Programme (UNDP), Colombia



Keywords: Innovation Public procurement Public sector Innovation policy R&D

Public procurement is increasingly viewed as having important potential to drive innovation. Despite this interest, numerous barriers prevent the public sector from acting as an intelligent and informed customer. This paper seeks to understand how barriers related to processes, competences, procedures and relationships in public procurement influence suppliers' ability to innovate and to reap the benefits of innovation. We address this by exploiting a dedicated survey of public sector suppliers in the UK, using a probit model to investigate the influence of structural, market and innovation determinants on suppliers' perception of these barriers.

The main barriers reported by suppliers refer to the lack of interaction with procuring organisations, the use of over-specified tenders as opposed to outcome based specifications, low competences of procurers and a poor management of risk during the procurement process. Such barriers are perceived most strongly by R&D intensive organisations. Our results also indicate that certain organisations, particularly smaller firms and not-forprofit organisations, encounter greater difficulties with innovation arising from the procurement process, for instance in relation to contract size, lack of useful feedback and communication of opportunities. Government procurement policies are queried in light of the findings. Crown Copyright © 2014 Published by Elsevier Ltd. This is an open access article under the CC BY license


1. Introduction

The use of public procurement to stimulate private sector innovation has been a much-debated issue of late, particularly in the context of so-called 'demand-side' innovation policies (Edler et al., 2006; OECD, 20H). While this debate is not new, it has seen a renewed impetus in policy discussions in the European Union (Kok, 2004; Aho et al., 2006) and across the OECD (Myoken, 2010; OECD, 20H). For instance, the OECD (2011) examined a range of experiences in the use of targeted demand-side innovation policies, including public procurement, regulation, standards, user-led innovation initiatives, and 'lead market' policies (see also European Commission, 2007). Further, the Europe 2020 strategy includes public procurement as one of the key market-based policy instruments for smart, sustainable and inclusive growth (European Commission, 2010).

It has been argued that public procurement can help counteract market and systemic failures hindering innovation (Edler and Georghiou, 2007; Edquist and Zabala-Iturriagagoitia, 2012). Put simply, the public sector can overcome market failures by enlarging the market for certain goods and services, thus ensuring sufficient critical mass to encourage R&D investment. The public sector also influences standards

* Corresponding author at: Manchester Institute of Innovation Research, Manchester Business School, The University of Manchester, Oxford Road, Manchester M13 9PL, UK. Tel.: + 44 (0) 161 275 7055; fax: + 44 (0) 161 275 0923.

through procurement, thus facilitating diffusion (Blind, 2013). Public procurement can also help offset systemic failures by enabling interaction between users and potential suppliers, and by articulating and signalling unmet needs to the market. While the flexibility for interaction during a procurement process is regulated in all countries that are part of the Government Procurement Agreement in the WTO and EU Directives, there is still is ample room for general engagement to signal market needs upstream, and a range of legal procedures such as Competitive Dialogue allow for controlled interaction even during the procurement phase (Arrowsmith and Treumer, 2012; Treumer and Uyarra, 2013). Finally, the use of public procurement has been associated with the emergence of so-called 'lead markets' (see e.g. Geroski, 1990, Georghiou, 2007; Edler and Georghiou, 2007).

Innovation scholars have provided empirical evidence of the impact on innovation of public procurement vis-à-vis traditional innovation policy instruments. For instance, Rothwell and Zegveld (1981) found that procurement was more likely to generate innovations than R&D subsidies. Geroski (1990) reviewed a series of innovations emerging from public procurement and concluded that, provided certain conditions were met, procurement was an effective means to stimulate innovation. Aschhoff and Sofka (2009) contrasted the effects of procurement vis-à-vis other instruments (regulation, R&D subsidies and university research) on the innovation activities of German enterprises and found that both public procurement and the provision of knowledge infrastructure in universities had positive

0166-4972/Crown Copyright © 2014 Published by Elsevier Ltd. This is an open access article under the CC BY license (

effects on innovation success. Using Innobarometer data for 2006-2008, Guerzoni and Raiteri (2012) also observed that public procurement had a greater influence in stimulating firms' innovation behaviour than R&D subsidies, but noted that the strongest effects on innovation occurred when both instruments were combined.

Despite this body of work, few studies have investigated in detail the specific conditions or mechanisms within public procurement that actually lead to, or hinder, innovation (Geroski, 1990; Rothwell and Zegveld, 1981). Given the multitude of practices and procedures involved in the procurement process, what matters in order to inform policy design and implementation is not so much whether it can influence innovation but how and under what conditions that impact takes (or could potentially take) place.

This is an important issue because, despite the perception of procurement as something of a policy panacea and repeated efforts to put procurement budgets to work to drive innovation, efforts have been met with limited success (NESTA, 2012). Barriers to effective implementation, including organisational, regulatory, skills and the inherent risk aversion of the public sector, have been documented in the literature (see e.g. Wilkinson et al., 2005; Rolfstam, 2013). However this evidence tends to be anecdotal, case-study based and at times not consistent. This article provides a contribution in that it uses a dedicated and original survey to capture the perceptions and experiences of a broad range of suppliers, including small firms and third sector organisations (not-for-profit and non-governmental organisations), across different sectors and areas of the public sector. This paper thus underpins, qualifies and amends existing evidence by shedding a quantitative light on these questions: What are the main barriers that prevent the government from capturing innovation through public procurement? How do different types of suppliers experience these barriers? More specifically, the paper investigates the influence of structural, market and innovation determinants on suppliers' perceptions of these barriers.

We analyse this issue by looking at the UK and in particular we draw from a dedicated survey of 800 suppliers to the UK public sector. The UK is an interesting case to assess the development of these policies because it is considered a 'first mover' (Edler and Uyarra, 2013) in the promotion of policies and initiatives seeking to stimulate innovation through public procurement, as well as addressing the modernisation of public procurement more generally. In addition, and given the extent of private and third sector involvement in the delivery of public services, its 'public services industry' is generally considered to be one of the largest and the most developed in the world (Julius, 2008). Therefore the UK experience offers interesting lessons for other economies with similar agendas of privatisation and public sector reform.

This paper is structured as follows. In Section 2, the paper reviews the literature that addresses the key conditions or mechanisms in the procurement process that are seen to influence the effectiveness of procurement in stimulating innovation. The paper seeks to investigate how government suppliers perceive these aspects. We address this by means of a survey of suppliers and we use a probit model to ascertain the influence of structural, market and innovation variables on their perception of those barriers. Section 3 explains the data, the variables, and the approach for our econometric analysis. In Section 4 we present the results and the final section discusses the findings and draws some conclusions and implications for policy.

2. Theoretical framework and key assumptions

2.1. Policy effort to boost public procurement of innovation - and their limits

Public procurement refers to the acquisition of goods and services by government or public sector organisations. Public

procurement is first and foremost a vehicle allowing public sector organisations to perform their functions and deliver key services effectively. This notwithstanding, a growing body of scholars and policy makers throughout the OECD have recognised that by purchasing innovative products and solutions the public sector can not only deliver services more effectively and efficiently but also influence the innovation activities of private firms (The Economist, 2010; OECD, 2011). Public procurement of innovation has been associated with instances where public agencies act to purchase, or place an order for, a product-service, good or system that does not exist at the time but which could be developed within a reasonable period; that is, that requires innovative work to fulfil the demands of the buyer (Edquist and Hommen, 2000; Edler et al., 2006). This contrasts with 'regular procurement', where governments place orders for 'off-the-shelf' products.

A renewed interest in the use of public procurement to drive innovation, and in so called demand-side innovation policies more generally (Edler, 2010), has emerged as a result of a perception of a relative failure of traditional, mainly supply-side, policies, to improve innovation performance (OECD, 2011). In addition, the rationales exercised for the use of public procurement have broadened (including the pursuit of further societal outcomes such as local employment or sustainability) (McCrudden, 2007), in parallel with a change in the understanding of innovation (from an R&D-centred, linear view to systemic approaches to innovation) and a realisation of the potential of the public sector as an innovator in its own right (Flanagan et al., 2011).

Initiatives to advance this policy agenda have proliferated in recent years both in OECD countries and emerging economies (Georghiou et al., 2010; Li, 2011; OECD, 2011; Uyarra, 2013; Lember et al., 2013; Vecchiato and Roveda, 2014). Such initiatives vary strongly in their rationales and implementation modes, a reflection of national differences in the governance and structure of public procurement, different objectives associated with this policy across countries and policy sectors but also the inherent complexity of procurement processes (Dimitri et al., 2006). Georghiou et al. (2013) elaborate a policy framework and taxonomy of such interventions based on the functions supporting the procurement of innovation and the deficiencies they seek to remedy. Interventions may for instance address framework conditions for procurement (legislation, governance frameworks), organisational arrangements and capabilities for innovation procurement, the identification, specification and signalling of needs, and the provision of incentives for suppliers to take up innovative solutions.

The UK has been particularly active in this area in the last decade, introducing a host of initiatives and reforms in order to mobilise the use of procurement to support competitiveness and innovation (for a review see Uyarra et al., 2013). However, the implementation of such strategies and initiatives has been reported as being slow and fraught with difficulties. For instance, the Office for Government Commerce (OGC, 2004) noted that the public sector was failing to fully 'capture innovation' through procurement. In his review on creativity in business for HM Treasury, George Cox argued that, despite much progress in shifting the policy agenda, changing procurement practices remained an important challenge, a difficulty compounded by the fragmented nature of procurement in the UK (Cox, 2005). The 2008 'Innovation Nation' White Paper similarly concluded that "procuring innovative solutions has tended to be a low priority" (DIUS, 2008: p.23; see also Heseltine, 2012).

Indeed, despite a generalised optimism regarding the potential of procurement to stimulate commercial innovation, the challenges are considerable. This paper thus aims to better understand what prevents suppliers from proposing innovative solutions. A better acknowledgement of such barriers should facilitate an

E. Uyarra et al. / Technovation ■ (■■■■) ■■■-■■■ 3

adapted and more differentiated policy response to make the procurement of innovation more effective when and where it is required.

2.2. Conditions for and barriers to capturing innovation through public procurement

Metcalfe and Georghiou (1997) pinpoint four dimensions underpinning innovation in firms: the awareness of the set of innovation opportunities, the capabilities to seize these opportunities, the set of resources they can command to innovate, and the incentive structures rewarding innovation. Procurement can influence these dimensions and thus encourage or hamper innovation in different ways, for instance through aggregating contracts, setting standards, dealing with intellectual property rights, engaging with suppliers to inform specifications, or enabling a level playing field through tendering procedures (OFT, 2002; Uyarra and Flanagan, 2010). A number of barriers can in turn prevent the public sector from 'capturing innovation'. These include inadequate early warning, lack of engagement between procurers and suppliers, overly prescriptive and burdensome procurement processes, risk aversion, and procurement capability shortfalls (OGC, 2004; House of Lords, 2011). This section discusses these barriers in detail and formulates some assumptions as to their perception by different types of suppliers. These assumptions are the yardstick against which we analyse our data.

2.2.1. Procurement capabilities

In comparison with the procurement of off-the-shelf goods for the lowest possible price, the procurement of innovation requires a greater degree of in-house competence (Rothwell and Zegveld, 1981). A shortage of commercial skills among procurers has been commonly found to limit engagement with the marketplace and the development of closer supply relations. In a survey of public procurers in small countries in Europe, Georghiou et al. (2010) identified a lack of sufficient procurement expertise for complex purchases involving innovation as well as an absence of formal training for procurers. In the UK, a review for the Cabinet Office by Green (2010) noted that commercial skills were very inconsistent across central government. Considerable corporate capability constraints have been identified for English local authorities (Byatt, 2001). Skill constraints in turn significantly hinder the adequate use of potentially more 'innovation-friendly' procurement procedures such as competitive dialogue (HM Treasury, 2010a). Indeed, as the HM Treasury (2010:p.7) notes: "the outcome of a procurement will be influenced as much by the capacity and capability of those party to the process as by the nature of the contract to be delivered".

Particularly at lower levels of governance and in procurement systems that are decentralised, there may be a shortage of professional procurers and therefore the lack of skills for innovative purchasing becomes an important challenge (Uyarra, 2010; OECD, 2011). Our assumption is that suppliers to areas of the public sector that are more fragmented, such as the National Health Service (NHS) and local government, would be particularly concerned about the capacity of procurers. Further, innovative firms and firms with a stronger commitment to R&D would be more sensitive to a perceived lack of expertise on the side of procurers and see it as a strong barrier to supplying innovative goods or services. We would also expect this barrier to be particularly acute in the procurement of complex goods and services such as construction related projects.

2.2.2. Management of risks associated with procuring innovations

Risk aversion of public agencies has also been identified as a

barrier to the procurement of innovation (HM Treasury, 2005;

DIUS, 2008). Decision making in the public sector is affected by strong expectations regarding transparency and accountability (Tsipouri et al., 2010). Accordingly, risk management in the public sector entails "having in place a corporate and systematic process for evaluating and addressing the impact of risks in a cost effective way and having staff with the appropriate skills to identify and assess the potential for risks to arise" (NAO, 2000: p.2). The importance of risk management increases when the R&D itself is part of the procurement (Tsipouri et al., 2010). Georghiou et al. (2013) describe how some OECD countries have introduced measures to reduce uncertainty or offset the perceived risks of purchasing innovations, for instance through the provision of financial incentives, insurance guarantees and the use of quality certificates.

In line with Tsipouri et al. (2010), our assumption is that how risk is managed is a particularly important consideration for suppliers that invest more in R&D. Procurers may be risk averse and unwilling to take up new solutions and suppliers may be reluctant to invest heavily in R&D and innovation activities if they fear that they won't get the necessary return.

2.2.3. Buyer-supplier interaction

OGC (2004) suggests the need for early interaction in procurement in order to better 'capture innovation'. Systems of innovation approaches emphasise the interactive nature of innovation and in particular the influence of users and user-producer interaction in the production of innovations. Interaction in procurement can create an environment of trust that reduces opportunism, the need for costly monitoring and general transaction costs associated with exchange in instances where there is information asymmetry (Erridge and Nondi, 1994). Close and early engagement with suppliers can also allow access to industry knowledge not available in-house that can be used to draw better tender specifications (Uyarra, 2010). However, in procurement, interaction and communication between public buyers and potential suppliers can be constrained by lack of skills, risk aversion or too rigid an application of procurement procedures and practices (Erridge and Greer, 2002).

As mentioned in the introduction, public procurement has the potential to bridge end users and potential suppliers, particularly in those instances where procurement is conducted on behalf of end users other than the procuring public agency or authority (e.g. in services such as health or social care). However, interaction with final users is often hindered by lack of communication within organisations between the procurement function and operational or service areas that are closer to end users' needs (Uyarra, 2010). Centralised procurement functions within organisations and/or a lack of cross-functional team working in procurement can thus lead to a structural disconnect between potential suppliers, users and buyers.

The need for communication is the more important the greater the number of parties involved and the greater the need for user engagement (Caldwell et al., 2009). We would therefore assume that suppliers in the construction sector or health, or in personal services such as social care, are more likely to perceive insufficient interaction as a barrier. It is also expected that a lack of interaction would affect suppliers with different innovation profiles differently. We would thus expect it to be a concern for R&D intensive organisations, and for organisations introducing service innovations that rely on greater user interaction.

2.2.4. Public demand for innovation

The above discussion relates to the potential key role of the public sector as 'intelligent' or 'demanding' client or customer. Suppliers will adapt to the signals of public demand and respond

with innovative solutions if they see the public sector as a demanding customer (Uyarra and Flanagan, 2010). As Geroski (1990: p.189) noted, "procurement policy is [...] successful in stimulating innovation when [...] it leads to the expression of a clear, consistent set of needs towards which innovative efforts can be directed". Lack of market demand has indeed been identified as a key obstacle for innovation generally (Gallup, 2011), public demand being especially important in sectors such as construction, transport or healthcare or in those instances where the public sector is a 'first user' of the innovation (Dalpe et al., 1992; Rothwell and Zegveld, 1981). We therefore assume that the lack of public sector demand for innovation is a stronger barrier for suppliers in certain sectors and for suppliers with a stronger commitment to R&D and product innovation, since early demand encourages investment to enable such outcomes.

2.2.5. Tender specifications

The influence of procurement on innovation is also shaped by the nature of tender specifications. Specifications phrased in terms of outcomes or performance are considered to be better at allowing industry to propose innovative solutions (Rothwell and Zegveld, 1981; Geroski, 1990). Conversely, innovations may be less likely when specifications are made too rigid and narrow. Such rigidity prevents suppliers from proposing innovative ways of delivering outcomes. While there is increasing awareness of the importance of outcome specifications (and procurers are increasingly incorporating them in their tenders), there is a range of reasons why they are not used more comprehensively (see e.g. the Byatt, 2001), such as general inertia and transaction cost of change, and the additional requirements for tender evaluations, including development of defendable evaluation criteria and methodologies. As Edler et al. (2005) have shown, the extent to which innovation can be supported through public procurement depends on the formulation of functional specifications and the availability of a variety of skills to evaluate tenders. Procurers often fail to review or market test these specifications, not questioning whether they may be stifling innovation and raising costs. This inertia constrains both innovation and opportunities to achieve savings, as different and over-specified requirements may co-exist for identical services and purposes (Uyarra, 2010). In terms of differentiation by supplier types, we would expect organisations that are more innovation active to be especially sensible to such rigid specifications as they would perceive that the public sector is unwilling to consider different, potentially more innovative, solutions.

2.2.6. Incentives for the supply of innovative solutions

Even if the public sector demands innovation, incentives may not be in place for that innovation to be realized. As Cabral et al. (2006: p.485) note "the buyer must ensure that the suppliers have enough incentives to invest in the innovative knowledge that will eventually be incorporated in the goods it needs". In order to provide greater incentives to innovation, procurers may decide to pool resources to provide bigger (or longer) contracts, harmonise and standardise requirements, or aggregate demand to exploit their purchasing power. By enlarging the market for certain goods, the public sector can reduce uncertainty by guaranteeing a certain return of investment associated with the development of new products and processes. Contrariwise, fragmentation of public demand can limit potential scale effects and thus discourage investment in innovation (Edquist and Hommen, 2000). This is particularly relevant in those industries characterised by heavy R&D requirements and substantial economies of scale in production (Porter, 1990). We would expect that contracts that are perceived not to be substantial enough would be a

disincentive and therefore a barrier to innovation for those firms that invest in R&D, for larger and more innovative firms, and in sectors such as construction.

2.2.7. Management of IPR

In developed OECD economies, public procurement is embedded within a clear set of IPR rules. However, IPR rules leave some flexibility as to how IP is allocated in different procurement procedures. The way this is done influences the incentive structures of suppliers within public procurement procedures. In many instances, standard terms and conditions mandated by the government do not contain any provisions covering intellectual property ownership or licensing conditions (Wilkinson et al., 2005). In other cases, the purchasing agency takes on full IPR ownership. The European procurement directives allow the transfer of intellectual property to the supplier, however this tends to be the exception (Wilkinson et al., 2005). This notwithstanding, we assume that a poor management of IPR would be perceived as a strong barrier by certain suppliers, particularly those more committed to R&D and innovation.

2.2.8. Access to tenders and other process related constraints

As it has become evident so far, the intentional use of public procurement to promote innovation faces a number of challenges. Additional difficulties associated with taking this agenda forward include the ability of SMEs to access public sector contracts and thus their potential to deliver innovative solutions (Glover, 2008). Securing SME participation is particularly important when the number of bidders is low and more competition needs to be stimulated (Yukins and Piga, 2012). Constraints include regulatory complexity, potential conflicts between policy objectives, a perceived lack of a level playing field, high entry costs or complex procurement processes.

The structure of public tendering (particularly the use of large bundled contracts, or selection criteria privileging size or experience) may reinforce incumbents' advantages and lock SMEs out of the market (Cabral et al., 2006). Breaking down tenders, either in smaller contracts or geographically, can increase SME participation (GHK, 2010). Other practices enabling access include the provision of information, through e.g. web portals and other forms of notification, improving dialogue with SMEs and simplifying tendering procedures (GHK, 2010).

An additional concern relates to the provision of good quality feedback to unsuccessful businesses which could help improve SME capacity (Glover, 2008). Suppliers, especially SMEs, may also be negatively affected by fragmentation in the way procurement is conducted and by a lack of consistency in how public sector needs are transmitted to the market. Lack of consensus over priorities, inconsistent definition of needs, or even frequent changes in policy would have the effect of increasing uncertainty and decreasing the likelihood of innovation (Rothwell and Zegveld, 1981; Uyarra, 2010).

Associated concerns include a lack of openness on the part of the public sector to receive and take on board unsolicited ideas from suppliers and a lack of appreciation by procurers of suppliers past performance, particularly in relation to the successful delivery of innovative goods and services to the private sector. Public sector procurers may fail to appreciate previous innovations serving private markets, which could be incorporated and/or adapted into the public sector realm. This lack of spillovers between the private and private sector may diminish the potential catalytic effect of procurement on innovation.

E. Uyarra et al. / Technovation I (I

3. Methodology and data sources

This paper draws from a dedicated survey of suppliers to public sector organisations in the UK. The survey was conducted using computer-assisted telephone interview (CATI) by the UK survey company Harris Interactive during May and July 2011 and addressed general managers or heads of public sector contracts and addressed general company managers or heads of public sector contracts. The 'order effect' on responses is minimised by allowing randomisation of answer choices.

The focus of the survey was to understand the elements that act as barriers and drivers to stimulating innovation in the procurement process. The survey asked for information on a wide range of issues related to the innovation activities of supplier firms, the types of procurement they are engaged in, as well as general perceptions on the main practices and competences of procuring organisations, including perceived barriers to innovation.

By public sector organisations, we mean UK central government departments, local government and NHS in England, which constitute the lion's share (around 90 per cent) of public procurement spending in the UK (HM Treasury, 2010b). In order to identify a sampling frame of organisations that had supplied to the public sector, we used publicly available information on government contracts during 2010. This included suppliers to the NHS in 5 of the 9 English NHS regions, 93 Local authorities in England, and 97 department entities belonging to all of the 25 central government departments in the UK. Only 'core suppliers', or organisations whose aggregated annual contracting with the public sector was above a set threshold of £50,000 in the 2010 financial year were considered. Procurement (particularly at the local level) is characterised by a concentration of a high proportion of spend by relatively few firms, while there is a fairly long "tail" in the distribution of businesses that have quite small contracts by value (Peck and Cabras, 2011).

Names of suppliers were then matched with commercial databases (FAME and Companies House) to obtain the details of 8214 organisations. A first round of contacts led to the exclusion of 1619 invalid entries from the sample. Of the remaining organisations, 2724 were not accessible, leaving an effective sample of 4343 organisations. By July 2011 800 full interviews had been conducted, which represents approximately 10 per cent of the original sample, or 20 per cent when considering the effective sample. Our respondents were either general managers or heads of public sector contracts.

Since there is no commonly agreed definition of the public services industry in the UK (Julius, 2008), we are unable to test how representative our sample is of the total population of public sector suppliers. However we found no statistically significant differences in the number of responses and in the number of refusals by sector of activity, or by areas of government. This said, the survey cannot claim to represent the overall population of organisations supplying to the UK public sector.

The organisations that provide goods and services to the public sector make up a heterogeneous sector, comprising commercial, third sector and publicly owned bodies (Julius, 2008). Furthermore, it is important to note that organisations supplying goods and services to the public sector often serve both government and private sector markets, so the public sector may be a relatively large or a relatively small influence on their innovation activities. Naturally this would vary across sectors and supply markets. Reflecting this heterogeneity, the sample contains organisations with different degrees of contracting with the public vis-à-vis private sector, private firms and voluntary organisations, large and small organisations, and suppliers in manufacturing as well as in service sector industries (see Table 2).

Given the way the sampling frame has been constructed, one potential outcome is the likely under-surveying of the smallest organisations (although micro enterprises are still 10 per cent of the sample). Under representation of micro sized firms is a common concern in studies of firm level innovation processes, leading to a potential over-estimate of population levels of innovation, and investment in R&D (Freel and Robson, 2004). This however should not compromise the usefulness of our findings.

3.1. Variables description and model

3.1.1. Dependent variables

We are interested in procurement related factors that supplier organisations perceive as barriers to innovation. Our first set of dependent variable(s) are extracted from a question where suppliers were asked, from a list of procurement related issues, if they experienced any of them as barriers to innovation and how intensely. The dependent variables were translated into dichot-omous values to indicate the importance attached by the supplying organisation to each type of barrier to innovation. Each variable takes a value of one if that particular feature is perceived as a barrier (either very significantly or moderately significantly) and zero otherwise. Our second set of dependent variables stem from a question where respondents were asked to agree to a number of statements related to the procurement process and take a value of 1 if they agree and 0 if not. The original question and the responses are shown in Figs. 1 and 2 below.

Each set of dependent variables are analysed separately. We use techniques for binary outcomes, that is, probit regression models. In the first step, we analyse univariate probit models, separately for all dependent variables. In the second step, to capture the possible interdependence of the barriers we employ a multivariate probit model. The multivariate probit model generalises the bivariate probit model, which is a natural extension of the probit model that allows more than one equation with correlated disturbances. Then, disturbances across equations are allowed to

- Very significant Moderately significant Not at all significant

Poor management of risk Inadequate management of IPR Low capabilities of procurers Lack of interaction with procuring organisations Contracts not long enough Contracts not big enough Specifications too prescriptive Lack of demand for innovation

33% 39% 29%

14% 30% 56%

35% 36% 29%

46 b 33% 21%

27% 33% 40%

21% 30% 49%

38% 35% 28%

27% 41% 33%

Fig. 1. Question: "Have you experienced any of the following as barriers to innovation?"

E. Uyarra et al. / Technovation I (l

- Strongly disagree 2

Pre-qualiflcation procedures and/or conditions discoura e us from participating in tenders Public bodies sufficiently recognise private sector delivery history when assessing bids The public sector is generally open to unsolicited ideas from the


When unsuccessful in a bid, we tend to receive useful feedback

There is sufficient information available about tender opportunities:

Larger contracts make it difficult for us to participate in tenders

Different parts of the public sector procure similar products and

services consistently

3 - 4 Strongly agree

?q% 23% 18%

-L9% - 12%

14% 26% _43%

1fiYn 8%

10% 18%

29% 28%

23% , im,7%

_ 21% 21%

-L32L— 17%

Fig. 2. Question: "Do you agree with any of the following statements?".

be freely correlated. Each individual equation is a standard probit model.

3.1.2. Independent variables

Given the sectoral variation in terms of technological sources and opportunities which impinge upon innovation performance (Pavitt, 1984), analysis of innovation activities must, in principle, control for sectors. Standard industrial classification categories are however less appropriate to capture sectoral variety in public procurement, given the specific nature of the government service market and the greater weight of certain service sectors (Atkinson, 2005). The government services market is generally characterised according to activities such as professional services, facilities management, construction services, IT services and products, and professional services (DCLG/PwC, 2006; Julius, 2008). We follow a similar approach in this study and differentiate between facilities management, professional services, social services, suppliers of IT & office equipment, healthcare supplies and services, and works. In order to understand sectoral variation, dummies for the different (self-reported) supply markets are therefore created.

Variables are also included to account for the size of organisations and innovation characteristics. Within the latter we introduce dummy variables to distinguish whether suppliers have introduced a product innovation that is 'new to the market', a service innovation that is 'new to the market' and whether the organisation has invested in R&D in the last three years. As innovation measures, we choose 'new to the market' products and services to better reflect our working definition of public procurement of innovation as the 'purchase of a product-service, good or system that does not exist'.

Variables are also included in order to account for different areas of the public sector. Procurement of social services is mainly concentrated in local government, whilst procurement of healthcare equipment, supplies and services is mostly responsibility of the NHS. Central government departments procure a greater diversity of goods and services. We have therefore created a dummy to indicate whether the main customer is central government.

Third sector organisations are increasingly encouraged to take a greater role in public sector delivery, yet they face specific barriers when tendering for public contracts, including the short-term nature of contracts and competitive tendering and decision-making procedures (Home Office and OGC, 2004). Most third sector organisations are also suppliers of social services, so we have included a dummy variable for the former but not for the latter to prevent multicollinearity.

Table 1 presents information on the variables used in the analysis.

The descriptive statistics are presented in Table 2 and the correlation matrixes are presented in Tables A3 and A4 in the

Appendix. Correlations are generally low to moderate, which indicates that there is a low risk of collinearity.

In order to better understand and contextualise these barriers, additional information has been extracted from an open ended question included at the end of the survey, which gave the respondents an opportunity to add further considerations, clarifications and/or recommendations. These responses are used to corroborate, contextualise or provide a new angle to the analysis. These responses, though helpful to contextualise results, need to be taken with some caution as these may be biased in the light of the questions that precede them.

4. Discussion

The main barriers reported by firms refer to the lack of interaction with procuring organisations, the use of over-specified tenders as opposed to outcome based specifications, low competences of procurers and a poor management of risk. Additional key concerns include (lack of) feedback from unsuccessful bids, a low appreciation of unsolicited ideas and previous private sector delivery history, and the difficulties for participation posed by pre-qualification procedures and conditions.

Tables 3 and 4 show the results of the univariate probit models. Table 3 contains the eight equations estimating the eight barriers to innovation. Table 4 shows the results for the probit models estimating the additional seven process-related barriers. Tables Al and A2 in the Appendix in turn show the results of the multi-variate probit models that we conduct to improve the robustness of the results and test for complementarities between the dependent variables. The results of both sets of models are largely consistent.

As elaborated below, we find evidence of the relative relevance of certain firm, sector and innovation related characteristics on the perception of these barriers.

4.1. Firm characteristics

As expected, firms of different sizes have different perceptions of barriers in relation to the influence that size of contracts have on innovation. Larger firms are particularly sensitive to contract size. Indeed, and as noted in the literature, too small or too short public contracts may act as a strong disincentive to innovation. Demand may in this case not be sufficiently large to pull innovation (let alone encourage them to participate in the tender). How risk is managed is also perceived strongly as a barrier by larger firms.

By contrast, the smaller the firms the greater the difficulty large contracts pose for their participation in tenders. Size is also related to whether information about tender opportunities is perceived to be sufficient. Smaller firms are also particularly concerned about the variable quality of feedback or lack thereof. As one respondent

E. Uyarra et al. / Technovation I (I

Table 1

Variables' description.



Dependent variables - Innovation barriers

Lack of demand for

innovation Specifications too

prescriptive Contracts not big enough Contracts not long enough Lack of interaction

Binary 1

Binary 1

Binary 1

Binary 1

Binary 1

Binary 1

Binary 1

Binary 1

if lack of demand for innovation is considered a very significant or a moderately significant barrier, and 0 otherwise

if specifications being too prescriptive is considered a very significant or a moderately significant barrier, and 0 otherwise

if not large enough contracts are considered a very significant or a moderately significant barrier, and 0 otherwise if not long enough contracts are considered a very significant or a moderately significant barrier, and 0 otherwise if lack of interaction with procuring organisations is considered a very significant or a moderately significant barrier, and otherwise

if procurers capabilities are considered a very significant or a moderately significant barrier, and 0 otherwise if inadequate management of IPR is considered very significant or a moderately significant barrier, and 0 otherwise

if poor risk management is considered a very significant or a moderately significant barrier, and 0 otherwise

Dependent variables - Procurement process-related barriers

Based on the statement "Different parts of the public sector procure similar products and services consistently" 1 if the respondents strongly or moderately agrees, 0 otherwise

Based on the statement "Larger contracts make it difficult for us to participate in tenders" 1 if the respondents strongly or moderately agrees, 0 otherwise

Based on the statement "There is sufficient information available about tender opportunities" 1 if the respondents strongly or moderately agrees, 0 otherwise

Based on the statement "When unsuccessful in a bid, we tend to receive useful feedback" 1 if the respondents strongly or moderately agrees, 0 otherwise

Based on the statement "The public sector is generally open to unsolicited ideas from the market" 1 if the respondents strongly or moderately agrees, 0 otherwise

Based on the statement "Public bodies sufficiently recognise private sector delivery history when assessing bids" 1 if the respondents strongly or moderately agrees, 0 otherwise

Based on the statement "Pre-qualification conditions discourage us from participating in tenders" 1 if the respondents strongly or moderately agrees, 0 otherwise

Inadequate management of IPR

Poor management of risk

Consistent procurement Binary Large contracts participation Binary Binary

Sufficient information

tenders Useful feedback

Unsolicited ideas

Private sector delivery

history PQ conditions

Binary Binary Binary Binary

Explanatory variables

Size Categorical

Social enterprise Binary

Facilities mgmt. services Binary

Healthcare Binary

IT & office equipment Binary

Works Binary

Professional services Binary

Central government Binary

R&D Binary New to the market product Binary


New to the market service Binary innovation

1 if the organisation has between 1 and 10 employees; 2 if it has between 11 and 50 employees; 3 if it has between 51 and 250 employees; 4 if it has more than 250 employees 1 if the organisation is a non-for-profit organisation, 0 otherwise

1, if the main category of goods and services supplied are 'Facilities management services', 0 otherwise

1, if the main category of goods and services supplied are 'healthcare', 0 otherwise

1, if the main category of goods and services supplied are 'IT & office equipment', 0 otherwise

1, if the main category of goods and services supplied are 'works', 0 otherwise

1, if the main category of goods and services supplied are 'professional services', 0 otherwise

1, if the main public sector client is central government, 0 otherwise

1, if the firm had any R&D activities between 2008 and 2010

1, if the firm had introduced any new to the market product innovation between 2008 and 2010 1, if the firm had any R&D activities between 2008 and 2010

put it: "Some [feedback is] really good, some very bad. Doesn't help for next bid."

As expected, smaller firms feel that pre-qualification procedures and/or conditions discourage them from participating in tenders (see Table 4). As one supplier noted: "We have to provide the same information in a different format - this is a waste of time at the pre-qualification stage". Another supplier shared the same sentiment by stating: "The whole process is time consuming, completely ridiculous and puts pressure on small businesses."

Social enterprises have specific concerns. Prior studies have noted the uncertainty faced by third sector suppliers when competing in the procurement market (Cloutier-Fisher and Skinner, 2006). For instance one supplier noted: "We always had experience in the tradition of 'Grant Aid'. So making the transition to the structure of full public sector bidding is a huge culture change and shock which requires a lot of changes and learning on our part".

In particular, our results show that too short contracts are a very significant barrier for these suppliers, while at the same time large bundled contracts are a barrier to entry (see Table 4). It is important to note however that short contracts are in these cases not just a barrier to innovation but more generally a threat to the survival of these organisations. The following quote illustrates the

precarious financial position facing some charitable organisations: "If the contracts were awarded for 3 years a lot of us (voluntary sector) wouldn't have to sit on reserves to have 6 months running costs in case of contracts ending."

Regression results also suggest that, controlling for other things, suppliers that belong to the voluntary sector are also affected by rigid specifications. One supplier for instance noted: "The bidding process should reflect the outcomes relating to people rather than price". Some innovative social enterprises lament that the public sector does not acknowledge or appreciate the "added value of the voluntary sector" and, consequently, their potential to innovate and take risks. Another organisation posited that "our quality of service must stay high because of our ethos".

The public sector is hardly a coherent whole. Rather, as one respondent put it, "knowledge and appetite for innovation differs vastly across different [government] sectors". Social enterprises in particular are concerned about the lack of consistency in procurement across the public sector (as one of them put it, "what is good in one sector (e.g. NHS) is poor in others and vice versa") and consider that "public sector clients need to share knowledge and experience better". Organisations that mainly serve central government departments complain about the lack of demand for innovation, the small size of contracts and the way risk and IPR are

Table 2

Descriptive statistics.

Categories Frequency

Size (No. of employees)

Less than 10 10.3

Between 10 and 49 37.1

Between 50 and 250 28.2

More than 250 23.8

Type of organisation

Private 81

Social enterprise 17

Main categories of goods and services supplied

Facilities mgmt. services 11.4

Healthcare equipment, supplies and services (inc. dental & 14.5 optical)

IT & office equipment 7.6

Professional services 19.9

Social community care, supplies & services 16.6

Works 18.1

Main client

Local government 23.9

NHS 52.6

Central government 17.1


R&D 65.4

New2market product innov 45.25

New2market service innov 53.38

managed. Our assumption that suppliers in more fragmented local government and NHS markets are more sensitive to innovation related barriers is therefore not confirmed.

4.2. Market characteristics

In relation to market category, we do not find strong differences in the perception of barriers across sectors except for a few supply markets. In particular, organisations supplying goods and services related to works (construction and other services related to the built environment) show a range of concerns. As expected given the relative complexity of the goods and services provided, insufficient demand size acts a strong disincentive to innovation in the sector. Too rigid specifications and, albeit less significantly, the quality of public demand, poor procurement capabilities and risk management practices are also perceived to be barriers to innovation.

Perceptions also differ across sectors in relation to the coherence and consistency of procurement across the public sector. Only suppliers of facilities management services tend to agree that "different parts of the public sector procure similar products and services consistently". Equally, suppliers in construction admit that the public sector recognises private sector delivery history and that feedback tends to be useful. This reflects the dual market (both private and public) nature of such services and a greater professionalisation of procurement processes in construction in the UK (as reflected in sector- specific tendering portals, frameworks and simplified pre-qualification procedures), although bureaucratic purchasing processes may also be associated with perceived barriers such as low demand for innovation and rigid specifications as mentioned above.

4.3. Innovation characteristics

The literature has stressed the influence of certain procurement related practices, particularly around contract size, management of IPR and capability shortages, on R&D and innovation. These are corroborated by our findings. We find support for our assumptions that suppliers that undertake R&D activities are more likely to perceive issues such as lack of demand for innovation, size of contracts, rigid specifications, lack of capacities and lack of risk management as barriers to innovation.

In relation to a lack of demand for innovation, a number of R&D intensive suppliers complained that, despite the widespread rhetoric on innovation and procurement, these intentions are not reflected in processes and procedures. This is illustrated by comments such as "the public sector wants to hear about innovation but it won't take on innovative practices", or "for all they ask about innovation they don't want it", or simply "there is not the desire for innovation in public sector procurement".

Related barriers perceived by suppliers conducting R&D include rigid specifications and lack of competences. For instance one respondent noted: "we often find ourselves in situations where the procuring body may be open to innovations but there isn't time or opportunity to secure a departure from that specification." The use of rigid specifications is closely linked with a perceived lack of competences. A supplier thus highlighted that "the majority [of procurers] are 'old school' who are either afraid of, or incapable of understanding the benefits of commercial innovation."

According to suppliers, procedures that are too rigid are linked to a one-size-fits-all approach, whereby procurers "apply the standard generic procurement criteria and process to all procurement projects regardless of the procurement type whether that be a paper clip or the procurement of large specialist fleet."

Lack of skills is related to the likelihood and quality of buyer-supplier interaction, also perceived as a barrier by organisations carrying out R&D. One of them emphasised that "as a bidder, you need this dialogue to innovate effectively." This is in turn related to the strong perception among these suppliers that the public sector is not open to unsolicited ideas from the market (see Table 4). Management of IPR is also a significant barrier for firms conducting R&D, indeed one supplier questioned the government requiring the ownership of the IPR: "why not give the IPR to the party which will exploit them?"

We find mixed support for our assumption about the influence of certain barriers on organisations that are innovation active. Organisations that have introduced service innovations (new to the market services) are strongly associated with barriers related to contract size, lack of interaction, poor procurement capabilities and management of intellectual property rights in procurement. One such supplier for instance lamented that "there is no direct communication between the bidder and the 'user', all is via the restrictive interface that is the procurer. This results in poor specification and ineffective procurement. If you are going to drive innovation you need that 'face to face' dialogue to get under the skin of the user". This reflects the structural disconnect mentioned in Section 2 between potential providers, users and buyers. In the case of organisations that have introduced product innovations the results are quite different. In this case, suppliers that have not introduced any new to the market products are more likely to report demand, interaction and capability constraints as barriers for innovation.

These results may be explained by the different nature of the markets these suppliers are serving. Meeting public sector demand for certain services is likely to require innovation in the way these are delivered. Social services or professional services are inherently more customised or user oriented, so innovation may

E. Uyarra et al. / Technovation I (l

Table 3

Univariate probit. Barriers to innovation.


Lack of demand for Specifications too innovation (1) prescriptive (2)

Contracts not big Contracts not enough (3) long enough (4)

Lack of


Low capabilities of procurers (6)

Inadequate mgmt of IPR (7)

Poor mgmt of risk (8)

Facilities mgmt. services


IT & office equipment

Professional services

New2market product innov

New2market service innov

Central government

Social enterprise


Log likelihood

McKelvey and Zavoina's R2

0.053 (0.051) 0.613***

(0.177) 0.150 (0.157) -0.022

(0.195) 0.323** (0.152) 0.055



(0.105) - 0.203 **

(0.098) 0.031

(0.096) 0.266**

(0.130) - 0.126

(0.136) - 0.156 (0.173) 507.24 0.08

0.045 (0.052) 0.073

(0.172) 0.081 (0.161) -0.066

(0.198) 0.430** (0.157) 0.280*

(0.148) 0.321** (0.106) - 0.118

(0.101) 0.046

(0.097) 0.150

(0.135) 0.276*

(0.145) - 0.062 (0.177) 483.13 0.06


(0.050) 0.189

(0.169) 0.069 (0.158)

- 0.165

(0.199) 0.377** (0.153) 0.157




- 0.204**

(0.098) 0.171*

(0.095) 0.249**

(0.126) 0.027

(0.137) -1.011*** (0.178) 515.60 0.11



- 0.033


- 0.011 (0.160)

- 0.108

(0.197) 0.004 (0.152) 0.006

(0.143) 0.033 (0.104) - 0.190*

(0.098) 0.202**

(0.096) 0.163



(0.147) - 0.427** (0.173) 511.69 0.12

0.098* (0.054) 0.201

(0.185) -0.096 (0.169)

- 0.135

(0.204) 0.162 (0.165) 0.174


0.345*** (0.110)

- 0.201 *

(0.106) 0.222**

(0.103) - 0.108

(0.137) 0.087

(0.151) 0.107 (0.181) 432.68 0.07

0.031 (0.051) 0.193

(0.171) 0.121 (0.159) -0.033

(0.198) 0.287* (0.156) 0.108

(0.145) 0.349** (0.105) - 0.190*

(0.099) 0.220**

(0.096) 0.221*

(0.131) 0.035

(0.141) - 0.133 (0.174) 500.37 0.06

0.009 (0.051) 0.330*

(0.172) 0.191 (0.165) 0.423**

(0.200) 0.215 (0.157) 0.314**

(0.145) 0.275** (0.108) -0.030

(0.099) 0.223**

(0.098) 0.320**

(0.125) -0.066

(0.140) - 1.001 *** (0.182) 482.77 0.07


(0.052) 0.125

(0.174) - 0.122 (0.161)

- 0.173

(0.201) 0.372** (0.163) -0.057


0.283*** (0.106)

- 0.128

(0.102) 0.119

(0.097) 0.350**

(0.138) 0.154


- 0.461 ***

(0.178) 483.89 0.12

Observations: 787. Robust standard errors in parentheses. *** p < 0.01. ** p < 0.05. * p < 0.10.

still take place despite the perceived barriers. In other markets however, suppliers may be able to respond to public sector demand with standard products without the need for innovation. Thus results may be strongly influenced by the nature of the supply markets in terms of the relative importance of public demand and their associated service vis-à-vis product innovations, besides the use or not of 'innovation-friendly' public procurement practices.

If suppliers are supplying standard, 'off-the-shelf goods rather than new products, this may indicate, at best, that the public sector may be responsive to existing innovations but less likely to trigger innovations. At worst, it may indicate a dominance of cost related versus quality related considerations when awarding tenders. Other questions of the survey may provide some additional clues. Organisations that innovate in products are less likely to be influenced by the public sector in their innovation activities when compared with their service innovation counterparts. Indeed, around 62 per cent of new to the market product innovators recognised an influence of public procurement on their innovations whereas for service innovators this was the case for 75 per cent. Product innovators are also more likely to be larger firms serving global as well as domestic markets. As stated by one large product innovating firm "Our group's market is global, so innovation is driven on global scale as opposed to domestic UK."

Although these results merit more investigation, we could venture to suggest that the public sector may be failing to capture innovation from organisations that are innovative and that could either increase their R&D investment, their rate of innovations or supply more of their innovations to the public sector. Equally, it is likely that a lot of potentially innovative suppliers are driven away from the public sector market either by a perceived lack of competence or lack of demand for innovation, or because they are unable to access it given cost related and other process related barriers.

5. Conclusion and policy implications

The aim of this paper was to shed light on the barriers related to processes, competences, procedures and relationships in public procurement that influence suppliers' ability to innovate and to reap the benefits of innovation. Contrary to previous studies that sought to assess the innovation impact of public procurement vis-à-vis other policy instruments, we were able to look at several aspects in public procurement that are thought to constrain innovation.

We used a multivariate probit model to take into account the complementarities between the different barriers and concerns, as firms may simultaneously experience them. Our analysis tested

E. Uyarra et al. / Technovation I (I

Table 4

Univariate probit. Procurement process-related barriers.


Consistent procurement (1)

Large contracts hinder participation (2)

Sufficient info tenders (3)

Useful feedback (4)

Unsolicited ideas(5)

Private sector delivery history (6)

conditions (7)

Facilities mgmt. services


IT & office equipment

Professional services

Central government

New2market product innov

New2market service innov

Social enterprise


Log likelihood McKelvey and Zavoina's R2

- 0.011 (0.052) 0.400**

(0.168) 0.077 (0.162) 0.044


- 0.197 (0.157)


(0.145) 0.046 (0.110) 0.120 (0.127)

- 0.018


- 0.107


- 0.318** (0.144)

-0.289 (0.178)

478.75 0.05

- 0.324***


- 0.175


- 0.113 (0.169)

- 0.087

(0.205) 0.272* (0.155) 0.090


- 0.148 (0.106)

- 0.118 (0.134)

- 0.034

(0.102) 0.142



(0.139) 0.425** (0.178)

470.30 0.15


(0.051) 0.106

(0.166) 0.031

(0.159) -0.121

(0.201) 0.147

(0.149) -0.006

(0.141) - 0.088

(0.105) 0.151

(0.126) -0.053

(0.097) 0.166*


- 0.072


- 0.878***


521.03 0.07

0.135** (0.059)

- 0.011

(0.205) 0.153 (0.193) 0.325

(0.223) 0.383** (0.176) 0.228

(0.166) 0.172 (0.123) 0.032 (0.135) 0.103


- 0.134

(0.108) 0.111 (0.160)

- 1.490*** (0.216)

377.59 0.05

0.071 (0.066) 0.246

(0.231) 0.371* (0.221) 0.351

(0.274) 0.410** (0.204) 0.444**


-0.260* (0.145)

- 0.170 (0.179) 0.083

(0.135) 0.260**

(0.132) 0.218 (0.193)

- 1.327***


272.54 0.05

0.089* (0.054) 0.093

(0.187) 0.048 (0.179) 0.092

(0.221) 0.388** (0.164) 0.133

(0.156) 0.026 (0.116) -0.144 (0.137) 0.034

(0.108) 0.044

(0.104) 0.259* (0.150) - 1.081 *** (0.192)

415.87 0.03

- 0.240***


- 0.181


- 0.179 (0.172)

- 0.018

(0.210) 0.078 (0.156) 0.081

(0.148) - 0.188* (0.106) - 0.106 (0.135) 0.023

(0.102) - 0.118

(0.099) - 0.127 (0.148) 0.339* (0.181)

461.51 0.09

Observations: 782. Robust standard errors in parentheses. *** p o 0.01. ** p o 0.05. * p o 0.10.

the influence of different types of variables on what the literature associates with key challenges for the procurement of innovation. To the best of our knowledge, this has not been done before. In particular, we tested the impact of different firm, market and innovation characteristics of suppliers on the dependent variables.

We also considered other organisational and governance aspects influencing procurement in general and participation of SMEs in particular and, in order to contextualise the results, we complemented these results with specific insights from supplier organisations.

The main barriers reported by firms were a lack of interaction with procuring organisations, the use of rigid as opposed to outcome-based specifications, low competences of procurers and a poor management of risk. Additional key concerns expressed by suppliers included poor feedback, a low appreciation of unsolicited ideas and previous private sector delivery history, and cumbersome pre-qualification procedures and conditions.

We find confirmation that skills and capacity shortages, and related aspect of risk management are seen as problematic by suppliers of construction related services, R&D intensive organisations and larger suppliers. We also find evidence that small contract size is a disincentive to innovation in large and in R&D intensive organisations. The latter are also affected by a perceived lack of demand for innovation, something that suppliers of facilities management and construction related services also experience. Finally, user engagement is an issue for R&D intensive and for service innovating suppliers.

Thus one key finding is that, while being pervasive, many barriers are specific to different product markets. Different suppliers in different sectors and in different parts of the public sector have different perceptions as to whether the public sector is a demanding and intelligent customer.

Our results also suggest that certain organisations encounter greater difficulties arising from the procurement process. Indeed, particular types of suppliers appear to be at a disadvantage by virtue of their size or non-profit orientation. SMEs, and more specifically micro and small firms, perceive certain procurement practices as disadvantageous, particularly the size of contracts, the lack of useful feedback and the pre-qualification requirements. Policy measures trying to ease SME access to public tenders, simplifying procedures, using lots, and promoting better information on contracts thus appear justified, for small firms clearly compete on unequal terms. Equally, suppliers in the voluntary sector feel at a disadvantage by contracts that are perceived to be too large and at the same time not long enough for them to innovate. They are also sensitive to inconsistent requirements and conditions across the public sector and specifications that are often too rigid to unleash innovation in this sector. There is a perception among the surveyed voluntary organisations that their particular idiosyncrasies and their innovation potential (see Nugroho, 2011) are not being sufficiently recognised.

The main and most consistent result of the analysis concerns the perception of barriers among suppliers undertaking R&D. We find support for our assumptions that firms with stronger

E. Uyarra et al. / Technovation I (llll) lll-lll 11

commitment to R&D, controlling for other factors, are more likely to perceive issues such as lack of demand for innovation, size of contracts, lack of capacities, rigid specifications and lack of risk management as important barriers to innovation in the context of public procurement. Since we lack data on relative R&D efforts of suppliers we are unable provide more refined results.

Moreover, our results reveal a different perception between product and service innovation. This may be related to an over-representation of service sector activities in the sample and reflect the nature of innovation dynamics underpinning different public sector markets. It may also suggest that the public sector is less likely to trigger new product innovations and instead tends to adopt existing products in a responsive way.

In the light of these findings it may plausibly be concluded that the public sector is missing out on fully capturing innovation through procurement. The identified barriers may be preventing firms from increasing their commitment to R&D and their rate of (particularly product) innovations. Equally, it could be argued that innovative companies do not see the public sector as an intelligent customer and thus fail to exploit their full innovation potential. At the same time there is a danger that small firms are not able to get into contracts that would help them innovate.

These results are of high policy relevance. Recent initiatives in the UK directed at improving capacity, simplifying procedures, channelling feedback and concerns of SMEs (via e.g. the Mystery Shopper Scheme), and improving consistency in public tendering, can be assessed favourably in the light of these findings. Initiatives directed at centralising and streamlining procurement, while positive from the point of view of avoiding duplication and increasing efficiency, will need to be carefully designed not to further drive away small innovative suppliers and not to further disconnect suppliers from final users. These considerations notwithstanding, it is nevertheless surprising that the government have sought to mandate many of these changes only in central government and not in other parts of the public sector. As noted by the House of Lords (2011): 42, this "laissez faire approach to the dissemination of best practice in procurement from central to local government appears to be overly optimistic". Furthermore, and given the anxieties expressed by suppliers in relation to the lack of demand for innovation, concerns can also be raised in terms of what appears to be a diminished role for innovation vis-a-vis efficiency related considerations in government procurement in the UK (House of Lords, 2011; Uyarra, 2013).

These considerations are particularly pertinent in the current economic crisis. As a consequence of the economic downturn, government efforts to 'capture innovation' through procurement are likely to diminish even further. For instance a supplier argued: "I think they've gone backwards. Best value when first came out was a good driver for innovation and has been forgotten about in the last couple of years with economic downturn". If true, it is short-sighted for government to pursue lowest price off-the-shelf solutions only to pay more not only over the life cycle of the purchase but also to cut itself off from the potential to transform cost and service quality through innovation.

Nevertheless, this study has some limitations. Firstly, because of the sampling strategy, the survey may not be an accurate reflection of the whole UK public sector, both in terms of functional areas and geographically. Secondly, it does not consider the view of organisations trying to access public sector markets and not succeeding. We suspect there is a pool of innovative organisations unable or unwilling to bid for public sector contracts because of the aforementioned barriers, however their views are not reflected here. Finally, our analysis is focused only on the UK which limits the extent to which findings and lessons can be generalised. It would be interesting to extend our empirical results to other countries also seeking to orient procurement budgets towards innovation.


This work was funded through the UK Economic and Social Research Council Grant RES-598-25-0037 with contributed support from the Department of Business Innovation and Skills, NESTA and the Technology Strategy Board. The authors would like to acknowledge their generous support. Beyond normal peer review input they have no influence on the content of this paper or the decision to publish. Finally, we wish to thank Colin Cram, Kieron Flanagan and two anonymous referees for their constructive advice and feedback.

Appendix A

See Tables A1-A4.

. l 'o e

era oj

O a . r

o r .t oo .i

0 n oo

era 3" P C CT

Table A1

Multivariate probit. Barriers to innovation.

Variables Lack of demand for innovation (1) Specifications too prescriptive (2) Contracts not big enough (3) Contracts not long enough (4) Lack of interaction (5) Low capabilities of procurers (6) Inadequate mgmt of IPRPoor mgmt of risk (7) (8)

Size 0.0465 0.0454 0.2081*** 0.1398*** 0.1021* 0.0313 0.0112 0.1934***

(0.0503) (0.0512) (0.0502) (0.0503) (0.0529) (0.0506) (0.0510) 0.0511

Facilities mgmt. 0.5990*** 0.0819 0.2065 -0.0062 0.1932 0.1870 0.3413* 0.1205


(0.1723) (0.1674) (0.1693) (0.1728) (0.1801) (0.1677) (0.1697) 0.1711

Healthcare 0.1472 0.0908 0.0792 0.0361 - 0.0329 0.1669 0.2333 - 0.0818

(0.1524) (0.1582) (0.1573) (0.1618) (0.1694) (0.1591) (0.1684) 0.1607

IT & office - 0.0026 - 0.0431 -0.2232 - 0.1363 - 0.1018 - 0.0251 0.4009** - 0.1663


(0.2057) (0.2019) (0.2123) (0.2100) (0.2145) (0.2067) (0.2115) 0.2097

Works 0.3145** 0.4554*** 0.3975** 0.0621 0.2004 0.3072* 0.2854 0.4102**

(0.1498) (0.1549) (0.1552) (0.1510) (0.1622) (0.1549) (0.1591) 0.1622

Professional services 0.0414 0.2961* 0.1748 0.0589 0.2534 0.1355 0.3256** - 0.0295

(0.1440) (0.1462) (0.1418) (0.1419) (0.1541) (0.1495) (0.1479) 0.1478

R&D 0.3174*** 0.2867*** 0.3080*** 0.0253 0.2983*** 0.3280*** 0.3066** 0.2698***

(0.1057) (0.1054) (0.1036) (0.1045) (0.1095) (0.1069) (0.1096) 0.1062

New2market - 0.1955** - 0.1111 - 0.2073 ** - 0.1872* - 0.1995* - 0.1932* - 0.0493 - 0.1176


innov (0.0965) (0.1004) (0.0980) (0.0989) (0.1039) (0.0998) (0.1006) 0.1020

New2market service 0.0179 0.0387 0.1641* 0.2162** 0.2024** 0.1983** 0.2446** 0.1034

innov (0.0959) (0.0964) (0.0952) (0.0970) (0.1020) (0.0964) (0.0992) 0.0974

Central government 0.2601** 0.1439 0.2613** 0.1533 - 0.1408 0.2037* 0.3105** 0.3561**

(0.1289) (0.1351) (0.1249) (0.1287) (0.1344) (0.1308) (0.1273) 0.1344

Social enterprise - 0.1164 0.2687* 0.0448 0.6866*** 0.0959 0.0579 0.0037 0.1810

(0.1319) (0.1350) (0.1322) (0.1427) (0.1433) (0.1381) (0.1386) 0.1392

_cons - 0.1446 -0.0568 -1.0314*** - 0.4374*** 0.1026 - 0.1318 - 1.0586*** - 0.4545***

(0.1722) (0.1806) (0.1826) (0.1776) (0.1861) (0.1805) (0.1854) 0.1831

Rho1 Rho2 Rho3 Rho4 Rho5 Rho6 Rho7

Rho/2 0.5016*** (0.0615)

Rho/3 0.2912*** (0.0576) 0.4042*** (0.0579)

Rho/4 0.3652*** (0.0568) 0.4333*** (0.0635) 0.6464*** (0.0598)

Rho/5 0.5798*** (0.0709) 0.5912*** (0.0673) 0.4736*** (0.0646) 0.4243*** (0.0624)

Rho/6 0.5369*** (0.0610) 0.5309*** (0.0620) 0.4152*** (0.0601) 0.4712*** (0.0611) 0.5545*** (0.0673)

Rho/7 0.3910*** (0.0640) 0.3483*** (0.0582) 0.3883*** (0.0578) 0.3832*** (0.0593) 0.4314*** (0.0615) 0.4610*** (0.0598)

Rho/8 0.4180*** (0.0659) 0.4817*** (0.0603) 0.4418*** (0.0579) 0.5045*** (0.0599) 0.5144*** (0.061l) 0.5596*** (0.0638) 0.4898*** (0.0625)

Log pseudolikelihood = -3454.72. Wald X 2(88)=232.38. Observations: 787. *** p < 0.01. ** p < 0.05. * p < 0.10.

// i ds

era oj

- . vi

'TT ^ .t

n 3" 3 o

vati o

Table A2

Multivariate probit. Procurement process-related barriers.

VARIABLES Consistent procurement (1) Large contracts participation (2) Sufficient info tenders (3) Useful feedback (4) Unsolicited ideas (5) Private sector delivery history (6) PQ conditions (7)

Size - 0.0233 - 0.2692*** 0.2348*** 0.1219* 0.0608 0.0322 - 0.2311 ***

(0.0638) (0.0657) (0.0638) (0.0725) (0.0703) (0.0659) (0.0679)

Facilities mgmt. services 0.2694 - 0.0437 0.2862 0.0248 0.2605 0.1858 -0.0710

(0.2028) (0.2081) (0.2033) (0.2398) (0.2429) (0.2209) (0.2264)

Healthcare 0.0019 - 0.2918 - 0.1348 0.1234 0.4011* 0.2038 -0.0360

(0.2029) (0.2142) (0.2075) (0.2406) (0.2400) (0.2125) (0.2205)

IT & office equipment -0.0221 - 0.0997 - 0.2317 0.2994 0.4640 0.1668 0.1906

(0.2467) (0.2535) (0.2534) (0.2729) (0.2847) (0.2673) (0.2659)

Works - 0.1438 0.2480 0.1186 0.4424* 0.4541** 0.5742*** 0.0230

(0.1864) (0.1868) (0.1818) (0.2073) (0.2131) (0.1927) (0.1959) E.

Professional services - 0.1816 - 0.0096 0.0980 0.2421 0.4775** 0.0880 0.0686 Uya >

(0.1832) (0.1839) (0.1778) (0.2029) (0.2106) (0.1954) (0.1900) O ra

R&D 0.0249 - 0.0092 - 0.0342 0.2781* - 0.2662* 0.0709 - 0.1232 et H

(0.1362) (0.1343) (0.1342) (0.1542) (0.1537) (0.1394) (0.1367) l o Im

Central government 0.2001 - 0.1768 0.0814 -0.0882 -0.0876 - 0.1642 0.0241 /

(0.1569) (0.1673) (0.1561) (0.1684) (0.1811) (0.1708) (0.1663) ec h

New2market product innov 0.0042 - 0.1052 - 0.0418 0.1019 0.0826 0.0221 -0.0884 3

(0.1280) (0.1287) (0.1244) (0.1359) (0.1396) (0.1313) (0.1291) o < a z

New2market service innov - 0.0732 0.0552 0.1632 -0.0359 0.2427* 0.0478 - 0.1735 o

(0.1211) (0.1230) (0.1204) (0.1332) (0.1398) (0.1238) (0.1243) 3 Tl m C/3 03

Social enterprise - 0.4818 ** 0.5498*** 0.0587 0.2907 0.2575 0.2930 - 0.1686 l(

(0.1887) (0.1782) (0.1754) (0.1958) (0.2020) (0.1839) (0.2020)

_cons - 0.1943 0.2976 - 0.8490*** - 1.5429*** - 1.3428*** - 0.9950*** 0.2390

(0.2202) (0.2217) (0.2202) (0.2577) (0.2599) (0.2334) (0.2328)

Rho1 Rho2 Rho3 Rho4 Rho5 Rho6

Rho/2 0.0144 (0.0730)

Rho/3 0.0654 (0.0705) - 0.0630 (0.0735)

Rho/4 0.0853 (0.0800) 0.0217 (0.0787) 0.2129** (0.0767)

Rho/5 0.2199** (0.0791) - 0.0295 (0.0766) 0.2577** (0.0790) 0.0553 (0.0891)

Rho/6 0.0508 (0.0743) - 0.0156 (0.0774) 0.1131 (0.0732) 0.1184 (0.0784) 0.1560 (0.0799)

Rho/7 0.1385 (0.0709) 0.4334*** (0.0796) - 0.0912 (0.0762) - 0.1093 (0.0760) - 0.0162 (0.0884) - 0.0478 (0.0770)

Log pseudolikelihood = -

*** p < 0.01. ** p < 0.05. * p < 0.10.

1996.18. Wald X2(77) = 152.15. Observations: 500.

t •a

d t . El

era oj

2 3' 0 r

'0 o .

era ET

Table A3

Barriers to innovation. Descriptive statistics and correlation matrix.

% of 1 1

Dependent variables

1 Lack of demand for innovation 61 1.00

2 Specifications too prescriptive 66.88 0.32

3 Contracts not big enough 46.13 0.23

4 Contracts not long enough 55.38 0.23

5 Lack of interaction 73.88 0.36

6 Low capabilities of procurers 63.38 0.37

7 Inadequate manage of IPR 33.25 0.26

8 Poor management of risk 64.25 0.30

0.27 1.00

0.31 0.42 1.00

0.39 0.29 0.26 1.00

0.37 0.26 0.29 0.39 1.00

0.26 0.24 0.21 0.25 0.29 1.00

0.33 0.30 0.33 0.33 0.36 0.28

Explanatory variables

9 Size 0.07 0.06 0.18 0.11 0.09 0.06 0.04 0.18 1.00

10 Facilities mgmt. services 11.38 0.11 - 0.03 0.01 -0.03 0.03 0.01 0.03 0.02 0.01 1.00

11 Healthcare 14.5 - 0.01 - 0.03 - 0.03 -0.04 -0.05 - 0.02 - 0.03 - 0.07 0.02 - 0.15 1.00

12 IT & office equipment 7.63 - 0.03 - 0.05 - 0.06 -0.06 -0.05 - 0.03 0.07 - 0.04 0.01 - 0.10 - 0.12 1.00

13 Works 18.13 0.05 0.06 0.08 -0.03 0.01 0.03 - 0.03 0.09 0.10 - 0.17 - 0.19 - 0.14 1.00

14 Professional services 19.88 - 0.01 0.06 0.04 0.01 0.05 0.03 0.08 - 0.02 - 0.10 - 0.18 - 0.20 - 0.14 -0.24 1.00

15 R&D 65.4 0.10 0.11 0.12 0.05 0.12 0.13 0.13 0.12 0.21 0.00 0.05 0.11 -0.20 0.08 1.00

16 Central government main client 17.13 - 0.03 - 0.04 - 0.04 -0.08 -0.05 - 0.04 0.04 - 0.02 0.10 0.09 0.08 0.19 -0.10 -0.09 0.22 1.00

17 New2market product innov 45.25 0.01 0.04 0.08 0.11 0.10 0.10 0.09 0.07 0.08 -0.02 - 0.13 0.00 - 0.09 0.07 0.17 0.05 1.00

18 New2market service innov 53.38 0.08 0.06 0.09 0.06 0.00 0.07 0.12 0.11 0.08 0.00 - 0.16 0.05 - 0.08 0.22 0.12 0.06 0.04

19 Social enterprise 17.38 - 0.05 0.06 0.01 0.21 0.04 0.02 - 0.03 0.05 -0.03 - 0.10 - 0.09 - 0.13 - 0.21 -0.04 0.09 - 0.10 0.15

1.00 0.03

Table A4

Procurement process-related barriers. Descriptive statistics and correlation matrix.

% of 1 1

Dependent variables

1 Consistent procurement 35.25 1.00

2 Large contracts participation 36.25 - 0.04 1.00

3 Sufficient information tenders 43.63 0.05 - 0.05 1.00

4 Useful feedback 20.2 0.05 0.01 0.15 1.00

5 Unsolicited ideas 20.25 0.12 0.00 0.15 0.05 1.00

6 Private sector delivery history 26.2 0.03 0.01 0.06 0.09 0.09 1.00

7 PQ conditions 30.48 0.08 0.27 - 0.08 - 0.10 0.03 - 0.06 1.00

Explanatory variables

8 Size - 0.01 - 0.19 0.18 0.10 0.03 0.05 - 0.17 1.00

9 Facilities mgmt. services 11.38 0.09 - 0.05 0.08 - 0.05 0.00 - 0.02 -0.04 0.04 1.00

10 Healthcare 14.5 0.00 - 0.09 - 0.07 - 0.01 0.00 0.01 - 0.01 0.00 - 0.15 1.00

11 IT & office equipment 7.63 0.02 - 0.05 - 0.07 0.04 0.02 - 0.02 0.03 0.02 - 0.11 - 0.12 1.00

12 Works 18.13 - 0.02 0.06 0.03 0.06 0.04 0.12 0.02 0.08 - 0.19 -0.20 - 0.15 1.00

13 Professional services 19.88 - 0.05 0.01 0.02 0.02 0.05 - 0.05 0.03 -0.07 - 0.18 -0.20 - 0.14 -0.25 1.00

14 R&D 65.4 0.00 - 0.06 0.02 0.09 - 0.07 0.01 - 0.10 0.23 -0.03 0.08 0.14 -0.22 0.07 1.00

15 Central government main client 17.13 0.04 - 0.04 0.04 0.00 - 0.02 - 0.06 - 0.01 0.04 -0.00 - 0.18 0.02 - 0.10 0.23 0.10 1.00

16 New2market product innov 45.25 0.03 - 0.09 - 0.01 0.04 0.01 0.01 -0.04 0.11 0.10 0.09 0.22 - 0.10 - 0.13 0.24 - 0.15 1.00

17 New2market service innov 53.38 - 0.04 0.02 0.09 0.02 0.08 0.02 -0.09 0.08 0.05 - 0.15 0.02 -0.09 0.06 0.19 0.06 0.06

18 Social enterprise 17.38 - 0.12 0.16 0.02 0.04 0.01 0.03 -0.05 -0.04 - 0.11 -0.07 - 0.13 -0.20 0.02 0.10 0.14 - 0.13

E. Uyarra et al. / Technovation l (llll) lll-lll 15


Aho, E., Cornu, J., Georghiou, L., Subira, A., 2006. Creating an Innovative Europe. Report of the Independent Expert Group on R&D and Innovation Appointed Following the Hampton Court Summit. European Commission, Brussels.

Arrowsmith, S., Treumer, S., 2012. Competitive Dialogue in EU Procurement. Cambridge University Press, Cambridge.

Aschhoff, B., Sofka, W., 2009. Innovation on demand—can public procurement drive market success of innovations? Res. Policy 38 (8), 1235-1247.

Atkinson, T., 2005. Measurement of Government Output and Productivity for the National Accounts. Final Report. HMSO, London.

Blind, K., 2013. "The Impact of Standardization and Standards on Innovation" Nesta Working Paper 13/15. Nesta, London.

Byatt, I., 2001. Delivering Better Services for Citizens - A Review of Local Government Procurement in England. Department for Transport, Local Government and the Regions and the Local Government Association, London.

Cabral, L., Cozzi, G., Denicolo, V., Spagnolo, G., Zanza, M., 2006. Procuring innovations. In: Dimitri, N., Piga, G., Spagnolo, G. (Eds.), Handbook of Procurement. Cambridge University Press, Cambridge, pp. 483-530.

Caldwell, N.D., Roehrich, J.K., Davies, A.C., 2009. Procuring complex performance in construction: London Heathrow terminal 5 and a private finance initiative hospital. J. Purch. Supply Manag. 15 (3), 178-186.

Cloutier-Fisher, D., Skinner, M.W., 2006. Levelling the playing field? exploring the implications of managed competition for voluntary sector providers of long-term care in small Town Ontario. Health Place 12 (1), 97-109.

Cox, G., 2005. Creativity in Business: Building on the UK's Strengths. HM, London.

Dalpe, R., DeBresson, C., Xiaoping, H., 1992. The public sector as first user of innovations. Research Policy 21 (3), 251-263.

DCLG/PwC, 2006. Developing the Local Government Services Market to Support the Long Term Strategy for Local Government. Department of Communities and Local Government, London.

Dimitri, N., Piga, G., Spagnolo, G., 2006. Introduction. In: Dimitri, N., Piga, G., Spagnolo., G. (Eds.), Handbook of Procurement. Cambridge University Press, Cambridge, pp. 3-13.

DIUS, 2008. Innovation Nation. Department for Innovation, Universities and Skills, London.

Edler, J., 2010. Demand oriented innovation policy. In: Smits, R., Kuhlmann, S., Shapira, P. (Eds.), The Theory and Practice of Innovation Policy an International Research Handbook. Edward Elgar, Cheltenham, pp. 177-208.

Edler, J., Edquist, C., Georghiou, L., Hommen, L., Hafner, S., Papadakou, M., Rigby, J., Rolfstam, M., Ruhland, S., Tsipouri, L., 2006. Innovation and Public Procurement: Review of Issues at Stake. Office for Official Publications of the European Communities, Luxembourg.

Edler, J., Georghiou, L., 2007. Public procurement and innovation - resurrecting the demand side. Res. Policy 36 (7), 949-963.

Edler, J., Uyarra, E., 2013. Public procurement of innovation. In: Brown, L., Osborne, S. (Eds.), The Handbook of Innovation and Change in Public Sector Services. Edward Elgar, Cheltenham, pp. 224-237.

Edquist, C., Hommen, L., 2000. Public Technology Procurement and Innovation Theory. In: Edquist, C., Hommen, L., Tsipouri, L.J. (Eds.), Public Technology Procurement and Innovation. Kluwer Academic Pub., Dordrecht, pp. 5-70.

Edquist, C., Zabala-Iturriagagoitia, J.M., 2012. Public procurement for innovation as mission-oriented innovation policy. Res. Policy 41 (10), 1757-1769.

Erridge, A., Greer, J., 2002. Partnerships and public procurement: building social capital through supply relations. Public Adm. 80 (3), 503-522.

Erridge, A., Nondi, R., 1994. Public procurement, competition and partnership. Eur. J. Purch. Supply Manag. 1 (3), 169-179.

European Commission. 2007. A Lead Market Initiative for Europe. COM (2007) 860 final. Brussels.

European Commission. 2010. Europe 2020 Flagship Initiative: Innovation Union SEC (2010) 1161. Brussels.

Flanagan, K., Uyarra, E., Laranja, M., 2011. Reconceptualising the 'policy mix' for Innovation. Res. Policy 40 (5), 702-713.

Freel, M., Robson, P., 2004. Small firm innovation, growth and performance evidence from Scotland and Northern England. Int. Small Bus. J. 22 (6), 561-575.

Gallup. 2011. Innobarometer 2010: Analytical Report - Innovation in Public Administrations, Flash Eurobarometer 305.

Georghiou, L., 2007. Demanding Innovation Lead Markets, Public Procurement and Innovation. NESTA, London.

Georghiou, L., Edler, J., Uyarra, E., Yeow, J., 2013. Public procurement as an innovation policy tool: choice, design and assessment. Technol. Assess. Soc. Chang. (Available online 24.10.13).

Georghiou, L., Y. Li, E. Uyarra, J. Edler. 2010. Public procurement for innovation in small European countries. A Report from the ERA-PRISM: (Policies for Research and Innovation in Small Member States to Advance the European Research Area) OMC-Net Project, Brussels.

Geroski, P., 1990. Procurement policy as a tool of industrial policy. Int. Rev. Appl. Econ. 4 (2), 182-198.

GHK. 2010. Evaluation of SMEs' Access to Public Procurement Markets in the EU, DG Enterprise and Industry, Brussels.

Glover, A., 2008. Accelerating the SME Economic Engine: Through Transparent, Simple and Strategic Procurement. HM Treasury, London.

Green, P., 2010. Efficiency Review: Key Findings and Recommendations. Cabinet Office, London.

Guerzoni, M., E. Raiteri. 2012. Innovative Procurement and R&D Subsidies: Compounding Effects and New Empirical Evidence on Technological Policies in a Quasi-experimental Setting. Working paper No. 18/2012, Department of Economics "Cognetti de Martiis", University of Turin.

Heseltine, M., 2012. No Stone Unturned in Pursuit of Growth. BIS, London.

HM Treasury 2005. Cox Review of Creativity in Business: Building on the UK's Strengths. London.

HM Treasury, 2010a. Review of Competitive Dialogue. HM Treasury, London.

HM Treasury, 2010b. Public Expenditure Outturn Update. HM Treasury, London.

Home Office, and OGC. 2004. Think Smart, Think Voluntary sector-Good Practice Guidance on Procurement of Services from the Voluntary and Community Sector. London: the Office of Government Commerce and the Home Office.

House of Lords. 2011. Public Procurement as a Tool to Stimulate Innovation. Science and Technology Committee - First Report. London.

Julius, D., 2008. "Understanding the Public Services Industry: How Big, How Good, Where Next?". Department for Business Enterprise & Regulatory Reform (BERR), London.

Kok, W., 2004. Facing The Challenge. The Lisbon Strategy for Growth and Employment. Office for Official Publications of the European Communities, Luxembourg.

Lember, V., Kalvet, T., Kattel, R., 2013. Public Procurement Policy for Innovation: International Perspectives. Springer, New York.

Li, Y. 2011. Public Procurement as a Demand-side Innovation Policy Tool in China - A National Level Case Study. Paper presented at the 2001 DRUID Conference, Copenhagen, June15-17, 2011.

McCrudden, C., 2007. Buying Social Justice: Equality, Government Procurement, & Legal Change. Oxford University Press, Oxford.

Metcalfe, S., Georghiou, L., 1997. Equilibrium and Evolutionary Foundations of Technology Policy. CRIC Discussion Paper, No. 3. Centre for Research on Innovation and Competition, University of Manchester, Manchester.

Myoken, Y., 2010. Demand-orientated policy on leading-edge industry and technology: public procurement for innovation. Int. J. Technol. Manag. 49 (1), 196-219.

NAO, 2000. Supporting Innovation: Managing Risk in Government Departments. Report by the Comptroller and Auditor General. National Audit Office, London.

NESTA, 2012. Plan I: The Case for Innovation-led Growth. NESTA, London.

Nugroho, Yanuar, 2011. Opening the black box: the adoption of innovations in the voluntary sector—the case of indonesian civil society organisations. Res. Policy 40 (5), 761-777 (June).

OECD, 2011. Demand Side Innovation Policy. OECD, Paris.

OFT, 2002. Innovation and Competition Policy Part I - Conceptual Issues. Office of Fair Trading, London.

OGC. 2004. Capturing Innovation: Nurturing. Suppliers' Ideas in the Public Sector.

Pavitt, K., 1984. Sectoral patterns of technical change: towards a taxonomy and a theory. Res. Policy 13 (6), 343-373.

Peck, F., Cabras, I., 2011. The impact of local authority procurement on local economies the case of Cumbria, North West England. Public Policy Adm. 26 (3), 307-331 (July 1).

Porter, M.E., 1990. The Competitive Advantage of Nations. Free Press, New York.

Rolfstam, M., 2013. Public Procurement and Innovation. Edward Elgar Publishing, Cheltenham.

Rothwell, R., Zegveld, W., 1981. Government regulations and innovation - industrial innovation and public policy. In: Rothwell, R., Zegveld, W. (Eds.), Industrial Innovation and Public Policy. Frances Pinter, London, pp. 116-147.

The Economist, 2010. Promoting Innovation: Growth on the Cheap. May 27th 2010 (accessed 20.10.13).

Treumer, S., Uyarra, E., 2013. Competitive dialogue and contractual design fostering innovation and need analysis. In: Piga, G., Treumer, S. (Eds.), The Applied Law and Economics of Public Procurement. Routledge, London, pp. 13-27.

Tsipouri, L., Edler, J., Rolfstam, M., Uyarra, E., 2010. Risk Management in the Procurement of Innovation. Concepts and Empirical Evidence in the European Union. European Commission, Brussels.

Uyarra, E., 2010. Opportunities for Innovation Through Local Government Procurement: A Case Study of Greater Manchester. Research Report. NESTA, London.

Uyarra, E. 2013. Review of Measures in Support of Public Procurement of Innovation. Nesta Working Paper No. 13/17. London: Nesta.

Uyarra, E., Edler, J., Gee, S., Georghiou, L., Yeow, J., 2013. Public procurement for innovation: the UK case. In: Lember, L., Kalvet, T., Kattel, R. (Eds.), Public Procurement Policy for Innovation: International Perspectives. New York, Springer, pp. 233-257.

Uyarra, E., Flanagan, K., 2010. Understanding the Innovation Impacts of Public Procurement. Eur. Plan. Stud. 18 (1), 123-143.

Vecchiato, R., Roveda, C., 2014. Foresight for public procurement and regional innovation policy: the case of Lombardy. Res. Policy 43 (2), 438-450.

Wilkinson, R., Georghiou, L., Cave, J., et al., 2005. Public Procurement for Research and Innovation. Office for Official Publications of the European Communities, Luxembourg.

Yukins, C., Piga, G., 2012. Dialogue: set-asides for small and medium firms, as in the USA system with the small business act that reserves shares of tenders to SMEs only. In: Piga, G., Treumer, S. (Eds.), The Applied Law and Economics of Public Procurement. Routledge, London, pp. 215-230.