Scholarly article on topic 'Assessing economic impact of research and innovation originating from public research institutions and universities—case of Singapore PRIs'

Assessing economic impact of research and innovation originating from public research institutions and universities—case of Singapore PRIs Academic research paper on "Economics and business"

0
0
Share paper
Academic journal
Triple Helix
OECD Field of science
Keywords
{""}

Academic research paper on topic "Assessing economic impact of research and innovation originating from public research institutions and universities—case of Singapore PRIs"

Cheah and Yu Triple Helix (2016) 3:6 DOI 10.1186/s40604-016-0037-6

Triple Helix

RESEARCH

Open Access

Assessing economic impact of research and ® innovation originating from public research institutions and universities—case of Singapore PRIs

CrossMark

Sarah Cheah1* and Christopher Yu2

* Correspondence: sarahcheah@ hotmail.com

1NationalUniversity of Singapore Business School, Biz 1 8-46, 15 Kent Ridge Drive, Singapore 119245, Singapore

Fulllist of author information is available at the end of the article

Abstract

Research and innovation activities from universities and public research institutes (PRIs) contribute to economic growth in significant ways. Given the hefty investment in R&D in both public and private sectors, it is essential for policy makers to have meaningful, relevant and practical metrics for measuring the impact of public-funded research and innovation. However, the definition of impact and its measurements seem to differ across the literature, making it difficult for scholars and policy makers to inform policy and direct future research. This paper aims to consolidate all parts of the proverbial elephants by reviewing the extant literature for university/PRI impact studies, analysing and synthesising the erstwhile disparate studies, discussing the current metrics and their issues and identifying gaps among the different research strands. Building on the endogenous growth theory and the Triple Helix, we contribute to the body of knowledge by offering a multi-level Impact Assessment Framework for assessing the economic impact of public research. In the context of the framework, we applied the transaction cost economics theory and net present value concept to propose a new indicator, realised imputed commercial value (RICV). We empirically establish that RICV is a better predictor than the traditional licensing revenue metric used by universities/PRIs of firms' propensity to repeat licensing agreement with them, thereby indicating academic innovation impact.

Keywords: Innovation impact, Technology transfer, Commercialisation, Public R&D, Net present value

Springer Open

© 2016 Cheah and Yu. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Resumen

Las actividades de investigación e innovación de las universidades e institutos de investigación públicos (PRI) contribuyen al crecimiento económico de forma significativa. Dada la fuerte inversión en I&D en los sectores público y privado, es esencial para legisladores contar con medidas significativas, relevantes y prácticas para medir el impacto de la investigación y la innovación con fondos públicos. Sin embargo, la definición de impacto y sus medidas parecen diferir a través de la literatura. Esto dificulta la toma de decisiones óptimas sobre el financiamiento y la dirección de la investigación pública. En este trabajo deseamos consolidar todas las partes del elefante proverbial mediante la revisión de la literatura de impacto, el análisis y la síntesis de estudios existentes, la discusión de fortalezas y debilidades de medidas cuantitativas en uso, y la identificación de brechas entre las diferentes líneas de investigación. Basándonos en la teoría del crecimiento endógeno y la Triple Hélice, contribuimos al cuerpo de conocimiento, ofreciendo un marco de evaluación multi-nivel del impacto económico de la investigación pública. Combinando este marco, con la teoría de costos de transacción y el concepto del valor actual neto proponemos un nuevo indicador: Valor comercial imputado realizado (VCIR). Establecemos empíricamente que VCIR es un mejor predictor de la propensión de las empresas para repetir negocios con con universidades/PRI que el indicador tradicional de ingresos por licencias.

Résumé

Les activités de recherche et d'innovation des universités et instituts de recherche publics contribuent à la croissance économique de façon significative. Vu le lourd investissement dans la R&D à la fois dans les secteurs public et privé, il est essentiel que les décideurs politiques disposent d'indicateurs significatifs, pertinents et pratiques pour mesurer l'impact de la recherche et de l'innovation financée sur des fonds publics. Cependant, la définition de l'impact et ses mesures semble varier à travers la littérature, rendant difficile l'élaboration de la politique et l'orientation de la recherche future par les universitaires et les décideurs politiques. Cet article a pour objectif de lever les différents non-dits sous tous leurs aspects en passant en revue la littérature existante sur l'impact des universités et instituts de recherche publics, en analysant et en synthétisant les études autrefois disparates, en discutant les mesures actuelles et leurs insuffisances, et en identifiant les lacunes des différents aspects de la recherche. En nous basant sur la théorie de la croissance endogène et la Triple Hélice, nous contribuons au corpus de la connaissance en mettant à disposition un cadre d'évaluation à plusieurs niveaux de l'impact économique de la recherche publique. Dans ce cadre, nous avons appliqué la théorie économique du coût de transaction et le concept de la valeur actuelle nette pour proposer un nouvel indicateur, la Valeur commerciale réalisée imputée (VCRI). Nous établissons empiriquement, que la VCRI est un meilleur prédicteur que la mesure traditionnelle des revenus de licence utilisée par les universités et les instituts de recherche publics de la propension des entreprises à renouveler avec eux un accord de licence, ce qui indique l'impact de l'innovation universitaire.

да щ

штшттшшш&шттттшт* Атт^жш^ш^ш^щлтштттттжттда ттштт* ф^йж&нш^^^^пшЙшШЖ^Я»

M(RICV)o ЙЖ^/PRIsfflíЖШШШ

Аннотация

Исследовательская и инновационная деятельность в университетах и публичных исследовательских институтах (Public Research Institution, далее - PRIs) оказывает влияние на экономический рост различными путями. Осуществляя разумные инвестиции в исследования и разработки как в публичном, так и частном секторе, представителям власти необходимо иметь результативные, актуальные и практические инструменты для измерения воздействия на исследования и инновации, финансируемые публично. Однако, методики определения влияния и его изменения могут различаться в различных литературных источниках, усложняя оценку для представителей правительства и ученых при определении достигнутых результатов и координации дальнейших исследований. Целью настоящей статьи является объединение накопленных знаний в данной сфере путем обзора релевантной литературы по тематике управления PRIs, последующего анализа и синтеза нестандартных теорий, обсуждения известных методологий и их оценки, а также идентификации недостатков в различных исследовательских подходах.

Формируя эндогенную теорию роста в рамках Тройной спирали, мы осуществили вклад в базу знаний, предложив многоуровневую модель оценки влияния публичных исследований на экономику. В рамках данной модели мы применили теорию трансакционных издержек и концепцию оценки чистой приведенной стоимости, предложив новый показатель - Реализованная Вмененная Коммерческая Ценность (Realised Imputed Commercial Value, далее - RICV)). Эмпирическим путем нами было показано, что RICV представляет собой лучший индикатор, чем традиционная методика определения выручки, используемая создаваемыми при университетах фирмами для продления лицензионных соглашений с ними, тем самым характеризуя влияние академических инноваций.

Resumo

As atividades de pesquisa e inovagao em universidades e institutos de pesquisa públicos (IPPs) contribuem de maneira significativa para o crescimento económico. Dado os fortes investimentos em P&D em ambos os setores público e privado, é essencial para os formuladores de políticas terem métricas úteis, práticas e relevantes para mensurarem o impacto do financiamento público em pesquisa e inovagao. No entanto, a definigao de impacto e suas mensuragoes parecem divergir ao longo da literatura, tornando difícil para os pesquisadores e formuladores de políticas pública retornar informagoes para a política e direcionar estudos futuros. Este artigo visa consolidar todos estes aspectos através de uma revisao da literatura existente sobre estudos de impacto para Universidade e IPPs, analisando e sintetizando estudos díspares de outrora, discutindo as métricas atuais e as suas questoes, e identificando as lacunas entre as diferentes vertentes de pesquisa. Com base na teoria do crescimento endógeno e na Hélice Tríplice, nós contribuímos para o conhecimento teórico através de um quadro de avaliagao de impacto multi-nível, para inferir o impacto económico da pesquisa pública. No contexto desta estrutura, nós utilizamos a teoria económica dos custos de transagao e o conceito do valor presente para propor um novo indicador, o Valor Comercial Teórico Realizado (VCTR ou RICV- Realised Imputed Commercial Value ). Nós estabelecemos empiricamente que o VCTR é um indicador melhor do que as métricas de receita de licenciamento tradicionais utilizadas por Universidades e IPPs para medir a propensao de empresas em repetir os acordos de licenciamento, indicando, assim, o impacto da inovagao académica.

Multilingual abstract

Please see Additional file 1 for translation of the abstract into Arabic. Introduction

The role of public-funded research in the creation of new knowledge in the national innovation system is fast growing in importance (Etzkowitz et al. 2005). Research and research-related commercialisation activities from universities, public research institutes (PRIs) and health institutions contribute to economic growth in significant ways. The development of human capital in the academic and PRIs and their subsequent knowledge transfer through publications, consulting or migration to the industry could have profound social and economic impact (Cohen et al. 2002).

Although it is widely acknowledged that public-funded research has contributed to the intellectual capital, human capital and industrial capital to fuel economic development and advance the society, the assessment of such contributions has presented challenges to the government funding agencies that pursue national innovation system as part of their economic growth strategy. These issues are pertinent to the policy makers not only in North America (Vincett 2010; Roessner et al. 2013) but also in Europe and Asia Pacific. In the USA, for example, over USD37 billion had been spent annually on R&D by the National Science Foundation and the National Institutes of Health (NSF 2015; NIH 2015). Based on a recent survey of public research institutes in OECD countries, the absolute real R&D expenditure in the public sector increased from USD69.1 billion in 1998 to USD 84.3 billion in 2008, with the strongest being in Spain and Turkey at annual growth of over 9 and 16 %, respectively (OECD 2011). For a few OECD countries that reduced their government's R&D intensity on PRIs during the

period, a corresponding increase was found in the university's R&D expenditure, thereby indicating the significance of PRIs and universities as two main actors of the public research systems.

Given the significant expenditures on the public research centres, it is essential for scholars and public policy makers to have meaningful, relevant and practical metrics for measuring the impact of public-funded research and innovation. These indicators serve as important tools to inform policy and decision-making, shape research agenda and optimise resource allocation. The public research commercialisation metrics currently used by many research commercialisation offices include the number of intellectual property (IP) licenses, IP licensing revenues and spin-out formation rate. While these primary metrics are important, relevant and easy to capture (AUTM 1991-2012), variation in methodological approaches across the studies does not facilitate benchmarking with similar organisations.

According to Metrics for Knowledge Transfer from Public Research Organisations in Europe published by European Commission (2009), differences in target populations and expected response rates, as well as the normalisation approach for variation in research expenditures and research resource input pose challenges for policy makers in comparing inter-university/PRI performance in their contribution to their regional economy. In addition, there is concern that the current indicators do not adequately reflect the value of the technological innovation that public research organisations have brought to the firm and industry (Roessner et al. 2013). Large-scale investments in academic R&D are generally rationalised by a broad set of public principles or priorities. However, the evaluations are typically made in strictly commercial terms. Although some R&D projects that might have been shelved or aborted did not yield any economic return, prior studies have shown that failed R&D projects could still contribute to the overall innovation process (Elmquist and Le Masson 2009). The first global attempt to measure the socio-economic impact of public-funded research comprehensively was made for the Australian Research Quality Framework. Case study approach was used and indicators verified and proposed. However, the framework was never implemented due to a change in government in 2007 (Penfield et al. 2014).

It is evident that the current public research impact literature has several issues that require attention. First, most studies assumed linear knowledge flow from universities/ PRIs to the industry to achieve commercial spill-overs and deliver economic impact (Smith et al. 2011). Second, the lack of clear and consistent definition about the term "impact" and its host of related indicators may be seen in the diverse metrics used in the studies: R&D intensity (Nelson 1986), invention disclosures (Carlsson and Fridh

2002), publications (Nelson 2009), patents (Fischer and Varga 2003), technology licenses with companies (Bach and Llerena 2007), academic spin-offs (Colombo et al. 2010), innovative sales from licensees (Vincett 2010), as well as employment and income (Armstrong 1993). Third, the theoretical frameworks adopted varied widely from resource-based theory (Colombo et al. 2010) and CDM (Barajas et al. 2012) to input-output model (Roessner et al. 2013) and total factor productivity (Fischer and Varga

2003), while the methodologies ranged from individual case study (Brownrigg 1973) to econometric analysis (Felsenstein 1996). The use of different theoretical and methodological approaches has certainly enriched the literature with multi-disciplinary perspectives. However, these have also fragmented the literature, making it difficult for scholars and policy

makers to perform meaningful international benchmarking of their innovation impact to inform policy and direct future research (Roessner et al. 2013, Penfield et al. 2014).

This paper aims to consolidate all parts of the proverbial elephants by reviewing the extant literature for university/PRI impact studies, analysing and synthesising the erstwhile disparate studies, discussing the current metrics and their issues, and identifying gaps among the different research strands. Building on the endogenous growth theory (Romer 1990) which advocates knowledge-based economic growth strategy and the Triple Helix model1 (Etzkowitz et al. 2005), we contribute to the body of knowledge by offering a multi-level framework for assessing the economic impact of public research through structuring the indicators into four groups of stakeholders: (a) inventive output of universities/PRIs, (b) commercialisation output of universities/PRIs by their technology transfer organisations (TTOs), (c) innovation of industry/firms and (d) community/ economy.

According to the Transaction Cost Economics (TCE) theory, market transactions are not costless and firms come into existence due to market failures. Buyers may have to bear various types of transaction costs ranging from costs of coordination to hazard of hold-up2 arising from asset specificity or IP appropriability3 (Williamson 1979). Before firms decide to procure external intellectual property (IP), such as in-license technologies from universities/PRIs, they will typically consider the "make internal" versus "buy external" options and weigh the relative costs and benefits of each option. Prior studies have shown that firms with satisfactory prior licensing experience and perception of high licensing benefits relative to costs have a higher propensity for in-licensing technology (Atuahene-Gima 1993).

Adopting the 'objectivist' approach (Klaes 2000) of objectively measuring the value of IP transaction to firms, we used the net present value (NPV) methodology (Fisher 1930; Magni 2009) that is regarded robust in theory and prevalent in practice for financial investment decisions to quantify the benefit of in-licensing PRIs' IPs to the firms. This is done by totalling the sum of projected or extrapolated commercial revenue streams from the royalty-bearing IP licensing transactions - also known as Realised Imputed Commercial Value (RICV).

RICV could thus be objectively computed for every one-time or royalty-bearing IP licensing deal to estimate the sales revenue that firms generate from IP commercialisation. Unlike the resource-intensive methods that could estimate project benefits upon project completion (e.g. RVM4, BETA5 or NIST6), the RICV7 method uses annual revenue streams received by the TTO/PRIs as units of observations by relaxing the assumption that the R&D project benefits should or could be attributed to the percentage of PRI's IP contribution. RICV also has the other advantages of being aggregated for firms at industry and country levels for micro-, meso- and macro-perspectives.

Building on the TCE theory and earlier works that established the positive relationship among licensing experience, benefits and propensity, we postulated in this study that firms' benefits from in-licensing of university/PRIs' IPs, represented by RICV, would be positively associated with their propensity to repeat in-licensing transactions with PRIs.

To examine empirically the relative significance of the concept of RICV, we posed the following research question: Is RICV a suitable predictor of firms' propensity to repeat in-licensing agreements with PRIs, compared to the traditional licensing revenue

indicator? Two hypotheses were formulated and binary logistic regression was used to test the model. The first hypothesis (H1) postulates that firms that incur higher IP licensing fees (costs) with universities/PRIs have lower propensity to repeat their IP licensing transactions, thereby reducing academic innovation impact on firms. The second hypothesis (H2) hence posits that firms that attain higher levels of RICV with universities/PRIs (benefits) have higher propensity to repeat their license transaction, indicating an increase in academic innovation impact on firms. With results confirming our hypotheses, we contribute to the literature of impact studies by proposing RICV as new secondary indicators to estimate the economic impact of public-funded technological innovation at the firm, industry and national levels.

We conclude the paper with discussion of the results and their implications.

Methodology

This study comprises two parts: (a) development of an Impact Assessment Framework and (b) the empirical investigation of the RICV concept as part of the framework. An exploration process was used for the first part while regression analysis of archived data of Singapore PRIs was performed for the second.

To develop a comprehensive framework for impact studies on public-funded research, we applied an exploration process comprising three stages: planning, execution and reporting (Tranfield et al. 2003). In the first stage of planning, we identified the key data source as the ISI Web of Knowledge's Social Sciences Citation Index (SSCI) as it provides peer-reviewed journals which are regarded as validated knowledge of high impact (Podsakoff et al. 2005). During the second stage of execution, we determined initial selection criteria in the form of keywords and various combinations8. With these search keys, we refined our search of SSCI database to articles in English language under the research categories as business economics, educational research, public administration, urban studies, social sciences and science and technology for the period from 1973 to 2013. This results in an initial list of 299 papers. After reviewing their abstracts, we identified 192 relevant papers. To avoid reference bias, we searched Google Scholar9 with similar keyword combination queries to arrive at an initial list of 108 peer-reviewed journal papers, of which 89 were found relevant. After checking against duplicates, we compiled our consideration set10 of 275 papers for analysis and classification by the types of the impact indicators used and synthesis of conceptual framework. The framework will be presented in the next section.

For the empirical investigation of the RICV concept as part of the framework, we used licensing agreement transactions as the units of analysis in non-contrived settings. To control for inter-nation and inter-PRI variations, this study focused on PRIs governed within the same IP, science and technology and organisational framework of the national research agency of Singapore. Operating under the same agency, the PRIs are assumed to have the same propensity to out-license their technologies to enterprises. This study used the IP commercialisation database the agency maintains on organisations that had signed licensing agreements with them to deploy technologies developed by their PRIs for in-house use or commercial purposes for a stipulated period of time. The dataset comprised 835 licensing agreement transactions that took place between 2006 and 2012, innovation outcome of the firms in terms of innovative product count and RICV, as well as their status as repeat licensees.

In H1, the independent variable is the annual IP licensing fees incurred by a firm for in-licensing IP developed by the PRI, payable from the firm to the TTO representing the PRI, as executed in the TTO firm licensing agreement. The dependent variable is the firm's propensity to repeat licensing, which is used as a proxy indicator of the impact that prior PRIs' IP licensing and knowledge transfer has on the firm's innovation output. This variable is binary with a value of 1 if the firm has repeated licensing agreements with the TTO or 0 if it has not.

In H2, the dependent variable is the same as that for H1, and the independent variable is annual RICV generated by the firm for executing an IP licensing agreement with the TTO. The annual RICV is made up of projected or extrapolated commercial revenue streams from royalty-bearing IP licensing deals for a particular year. Please refer to the Appendix for computation details and examples.

To control for variations contributed by other factors, control variables such as licensee organisation type and technological orientation of IP licensed are included in the study.

To validate H1, positing that firms that incur higher IP licensing fees with universities/ PRIs have lower propensity to repeat licensing transactions, binary logistic regression is performed, as the dependent variable, firm's propensity to repeat licensing is a binary variable, to examine how the independent variable, IP licensing fees, affect the dichotomous outcome. This approach of binary logistic regression had been used in prior study by Ambos et al. (2008) on the factors affecting the commercial output of academic research, also coded as a binary variable. In H2, the same binary-dependent variable used in H1, firm's propensity to repeat licensing, is regressed using binary logistic regression on the independent variable, RICV.

Results

Public research impact studies

This section presents a descriptive analysis of our consideration set of 275 articles and reviews the dimensions of public research impact assessment posited in these studies. Figure 1 shows that 275 articles were published in a total of 50 journals.

Given that impact studies may be undertaken at various levels of analysis, ranging from individual through institutional to national, the range of journals publishing such studies is correspondingly wide, from the fields of business economics to public administration. However, a closer examination of the journal titles shows that the distribution of journal titles is rather skewed, with three journals accounting for more than half of the journal population: Research Policy (35 %), Journal of Technology Transfer (12 %) and Technovation (5 %). While Research Policy has consistently topped the chart, Journal of Technology Transfer saw an explosion of publications in this area in the recent decade. There may be several explanations for this phenomenon. Using the lens of life cycle model to the development of new inquiry topic, the topic of technology transfer was in the nascent stage of development. It is noteworthy that following the US Bayh-Dole Act of 1980 and similar developments in other innovating economies, which encouraged universities and PRIs to patent their inventions with government funds, a rise in technology transfer from public research to the industry spear-headed a concomitant growth in the literature of technology transfer, particularly in the USA and Europe. As new topics of inquiry gain attention, the editorial teams of more prestigious journals that might be initially sceptical of their

research policy journal oftechnology transfer tech no/ati on r&d management higher education economic development quarterly science and public policy technological forecasting and social change industry and inno/ation urban studies research in higher education journal ofthe association of un iversity technology...

journal of higher education management science rand journal of economics international journal of industrial organization the american economic review higher education research & development technology analysis & strategic management research evaluation studies in higher education w the review of economics and statistics

journal of regional science the economicsof innovation and newtechnology tl journal of businessventuring

2 industrial and corporate change

^ annals of region al science

o strategic management journal

~' regional studies

evaluation and program planning journal of policy analysis and management scottish journal of polítical economy journal of evolutionary economics sci entom etr ics innovation-management policy & practice review of regional studies journal of economic geography local economy education economics journal of urban economics journal of education finance academy of management journal international journal of production economics journal of product innovation management technology analysis and strategic management environ ment and planning c: government and policy applied economics journal of management studies the quarterlyjournal of economics journal of management andgovernance

0 20 40 60 80 100 Count of publication

Fig. 1 Public research impact articles published per academic journal 1973-2013

validity became more willing to publish them or new journals such as the Journal of Technology Transfer being developed to carry the scholastic conversations (Kuhn 1962). During the period of 1973-2013, we observe an increasing volume of impact studies, as presented in Fig. 2.

Our review of public research and innovation impact studies shows that interest in this topic started back in the 1970s, when the US tertiary institutions were recommended to assess the direct and indirect consequences of their investment and expenditures on their regional economy by the American Council on Education (Goldstein 1989).

In the 1980s, the national innovation system (NIS) concept gained attention with the works of Freeman (1987) in Japan and Nelson (1993) in the USA in the area of public-funded research. Their works highlighted the significant roles that universities, PRIs and government agencies play in pursuing technological progress and contributing to economic growth. The public policy makers' rising expectations of academic innovation impact on the industry and society may be seen in the proliferation of studies on this topic as politicians, economists and technology management scholars questioned about the diverse aspects of public research impact ranging from the rationale and feasibility of measuring such impact to the conceptual framework and methodology of monitoring and managing it.

Brownrigg's (1973) case study of UK University of Stirling found that its impact on the local economy in terms of income and employment might be marginal due to the significance of other factors such as local industrial structure and its ability to promote trade and investment flow. However, Brownrigg highlighted the limitations of traditional input-out modelling used by the economists in such impact studies of local economy. A similar case study approach coupled with university panel data was adopted by Keane and Allison (1999) in their study of Australian University of Sunshine Coast, Queensland, to establish the strength of university-industry connection and the degree of its "embeddedness" in the region.

In the USA, Bozeman and his colleagues developed the research value mapping (RVM) approach to assess the socio-technical impact of applied R&D projects with the Department of Energy (Bozeman and Kingsley 1997; Bozeman and Klein 1999). Using a combination of qualitative (case studies) and quantitative techniques (data developed from case studies), Bozeman and his team spent a considerable amount of effort to track the knowledge flow of R&D projects and model the possible project outcomes such as technologies patented or licensed, new companies created, new or improved products or processes developed or enhanced human capital. The RVM was found to work well for summative evaluation; however, its adoption beyond the USA has been limited due to the extensive resources required to trace and perform value mapping. According to Hicks et al. (2002) report submission to the Japanese National Institute of Science and Technology Policy, the RVM's identification of projects with significant milestone achievement to be units of analysis could provide administrative convenience from the funding perspective but did not reflect how the researchers view their work and capture other non-economic payoffs such as the development of human capital (Georghiou and Roessner 2000).

Jaffe (1989, p. 957), on the other hand, used US state-level time-series data to present the significance of "geographically mediated spill-overs from university research to commercial innovation" in the form of corporate patents, particularly in the drugs, chemicals and electronics industrial sectors. These findings were corroborated by Fischer and Varga's (2003) work on 72 Austrian districts using knowledge production function framework, as well as Beugelsdijk and Cornet's (2002) study of 1510 Dutch firms, who found that their proximity to technical universities had significant spill-over effects on

the firms in the form of new products. Mansfield's (1991) study of 76 major US firms attributed 11 % of their new products and 9 % of new processes to academic research.

Harhoff's (1999) econometric analysis of German engineering industry sectors yielded similar evidence of regional spill-over effects from academic institutions on the sectors in the form of new business formation. However, Lôôf and Brostrôm's (2008) study in Sweden found significant academic impact only on large manufacturing firms in the form of innovative sales, while the average service firms did not seem to benefit from university collaboration.

More recent studies are also found in Spain by Barajas et al. (2012) who used the CDM (Crépon et al. 1998) model to measure academic spill-over effects in the form of sales per employee; and Italy by Colombo et al. (2010) who applied the resource-based theory (Wernerfelt 1984) to assess innovation impact in terms of spin-offs.

In France, Bach and Llerena (2007), in their study with BETA, urged to look beyond traditional technology innovation indicators, such as R&D expenditures and licensing revenue, to capture the interaction dynamics among the institutes. It is noteworthy that BETA of France and NIST of the USA had introduced their respective methods in the 1990s to assess the impacts of large-scale R&D projects such as the European Space Programme (Bach and Lambert 1992) and Measurement and Standards Laboratory Programme (Tassey 1999). These methods were subsequently adopted by other countries such as Japan in their study of techno-economic impact of large-scale R&D projects (Kondo 2012). While these methods have been useful for estimating the impact of R&D projects, their limitation on the timing and size of the impact studies has been highlighted as the projects selected for studies must have either completed or achieved some significant benefits. As the time elapsed between project start-up and project benefit realisation might range two to seven years, based on past studies (Tassey 1999), BETA and NIST used these methods only for large-scale projects, while employing other forms of intermediate indicators, such as new companies started and product or process innovation for periodic and more pragmatic assessment of their innovation impact.

To build a comprehensive multi-dimensional framework, we applied the R&D-based endogenous growth theory (Romer 1990) and Triple Helix (Etzkowitz et al. 2005) to model the flow of knowledge among the major stakeholders of the industrialised economies: universities and PRIs, firms and industry, as well as the community and society. The endogenous growth theory (Romer 1990) postulates that knowledge and technology, unlike physical objects, are characterised by increasing returns, thereby driving economic growth. The theory forms the basis of knowledge-based economy and emphasises the critical role that the economic processes of knowledge creation and diffusion play in shaping growth of firms, industries, communities and nations. The Triple Helix, on the other hand, underscores the central role of universities and its relations with the industry and government in advancing the knowledge-based society (Etzkowitz et al. 2005).

To synthesise the data gathered into the framework, we categorised the dimensions of impact assessment which emerged from our literature review by the different types of knowledge generated and associated output indicators from the various stakeholders: (1) inventive output of universities/PRIs, (2) commercialisation output of universities/PRIs by their TTOs, (3) innovation of industry/firms and (4) community/economy. Figure 3 shows the distribution of impact study papers by categories 1 to 4.

Fig. 3 Distribution of papers by output category

While the majority of papers (84 %) fall into only one of the four categories, some articles belong to more than one category. For example, some studies (e.g. Derrick and Bryant 2013) used both the inventive output of the universities (publications—category 1) and commercialisation output of universities/PRIs (licenses generated—category 2) to measure the impact of research incentives on Australian medical research organisations, thereby falling into in category (1/2) as illustrated in Fig. 3.

Figure 4 presents the conceptual framework depicting the processes and associated output captured by studies among these four stakeholder output categories.

The categories are represented as blocks and the knowledge flows among the blocks depicted by bilateral arrows. These bilateral flows among the blocks are better

Fig. 4 Impact Assessment Framework

representation of knowledge flow directions than the traditional linear knowledge flows from universities/PRIs through the firms/industry to the economy as illustrated by most studies in the current literature (Gulbrandsen and Rasmussen 2012). In practice, there is much knowledge exchange between universities/PRIs and the industry, as the inventive output from the former could be inspired by the problems encountered by the latter through research collaboration. In a similar vein, the commercialisation output of the former could be shaped and refined by the requirements of the latter through the open innovation approach adopted by firms (Chesbrough 2007). In addition, universities/PRIs could have direct impact on the economy through their direct expenditures and employment. This direct interaction between universities/PRIs and economic development is represented in our conceptual framework by enveloping universities/PRIs/TTOs (block 1 and block 2) and industry/firms (block 3) with the larger community/economy (block 4). The knowledge flows are therefore indicative of the four phases of the RIE value chain.

It is important to note that this framework is distinct from the chain-linked model postulated by Kline and Rosenberg (1986), which focuses on the relationships among research, invention, innovation and production within the boundaries of an organisation. The Impact Assessment Framework, in contrast, looks beyond the organisational boundaries to access complementary assets and open innovation. The categorisation of block 1 and block 2 is supported by Gulbrandsen and Rasmussen (2012), who made similar distinction between inventive output of universities/PRIs and their commercialisation output, articulating that the former measures the potential rather than the actual commercial value of public science. Block 3 is consistent with the findings of Crépon et al. (1998), who distinguished between innovation input (R&D) and innovation outcome (product and process innovation). Block 4 is corroborated by the work of Goldstein and Renault (2004, p. 735), who advocated that universities could have potential impacts including "direct and indirect spending impacts".

Case of Singapore PRIs

In the case of Singapore PRIs, all the indicators highlighted in blocks 1 and 2 were used to measure their inventive and commercialisation output for tracking progress and benchmarking with local and international counterparts, as indicated in Fig. 5.

To assess the commercialisation impact of PRIs' research at firm, industry and economy levels, the PRIs actively tracked since 2006 the RICV of the intellectual property (IPs) generated from the licensees' commercialisation of the IPs and aggregated them at the various levels. RICV was captured for two purposes: (a) obtain the actual commercial value generated by the IP and (b) validate their pre-deal assumptions of projected cash flows as feedback mechanism.

According to the descriptive statistics in Table 1, more than 50 % of the licensing agreement transactions were with small and medium enterprises (SMEs), 40 % with multi-national corporations (MNCs) and large enterprises and the remaining with start-ups. About one third of the licensing agreement transactions came from the biomedical sciences sector, one third from info-communication sector and the rest from engineering, manufacturing, electronics, materials, chemicals and other sectors.

■ indicators to be used in this study

Fig. 5 Impact Assessment Framework used by Singapore PRIs

Our regression results for H1 are presented in models 1 and 2 of Table 2. Model 1 is a baseline model that examines the variances contributed by the control variables. Model 2 describes how IP licensing fees affects a firm's propensity to repeat licensing pertaining to H1. Model 2 does not appear to be an improvement over model 1, as evident from the similar pseudo R square values, Cox and Snell R square (0.08) and Nagelkerke R square (0.13), as well as the percentage of correct classification (78 %). However, with a coefficient (b = -0.23, p < 0.1), IP licensing fees is found to be a significant but negative predictor of the firm's propensity to repeat licensing, indicating support for H1 that firms that incur higher IP licensing costs have lower propensity

Table 1 Descriptive statistics of organisations participating in licensing transactions

Mean Standard deviation

Start-ups 0.12 0.32

Smalland medium enterprises (SME) 0.52 0.50

Multi-nationalcorporations (MNC) and large enterprises 0.40 0.49 Sector

Biomedicalsciences 0.37 0.48

Info-comm 0.30 0.46

Engineering and manufacturing 0.21 0.41

Electronics, materials, chemical and other 0.12 0.33

Table 2 Regression results

Binary logistic regression (H1, H2)

Variables DV: firm's propensity to repeat licensing (N = 835)

Model 1 Model 2 Model 3 Model 4

Constant -0.44(0.37) 0.47(0.61) 0.29(0.22) 0.49(0.65)

Start-up 0.13(0.31) 0.20(0.32) —0.66***(0.35) —0.66***(0.35)

SME 1.79**(0.30) 1.74**(0.30) 0.72*(0.32) 0.67*(0.33)

MNC and large enterprises 1.85**(0.30) 1.82**(0.30) 0.91**(0.32) 0.88*(0.32)

Biomedicalsciences sector 0.61***(0.31) 0.42(0.32) —0.01(0.35) 0.01(0.35)

Info-comm sector 0.49(0.31) 0.47(0.31) 0.24(0.33) 0.24(0.33)

Engineering and manufacturing sectors —0.65*(0.30) —0.64*(0.3) —0.79*(0.33) —0.82*(0.33)

Log_IP licensing costs —0.23***(0.12) —1.79**(0.23) —1.85**(0.24)

Log_RICV 1.67**(0.21) 1.70**(0.21)

Log_RICV x Log_IP licensing costs —0.10(0.11)

Cox and SnellR square 0.08 0.08 0.17 0.17

Nagelkerke R square 0.13 0.13 0.27 0.27

Modelchi-square 69.76 73.42 157.73 158.57

Significance 0 0 0 0

Classification correct 78 % 78 % 82 % 83 %

*p < 0.05; **p <0.01; ***p <0.10; robust standard errors are in parentheses

to repeat their IP licensing transactions, thereby reducing academia innovation impact on firms.

The regression results for H2 are presented in model 3 of Table 2. Model 3 is an improvement over the baseline model 1, as seen from the increase in the Cox and Snell R square (from 0.08 to 0.17), Nagelkerke R square (0.13 to 0.27) and the percentage of correct classification (78 to 82 %).

With a positive coefficient (b = 1.67, p < 0.05), RICV is established to be a significant and positive predictor of the firm's propensity to repeat licensing, thereby validating H2 that firms that attain higher levels of RICV with universities/PRIs have higher propensity to repeat their license transaction.

To explore possible interaction effects between IP licensing fees and RICV, model 4 is a full model demonstrating the main and interaction effects of all variables, including the interaction term between IP licensing fees and RICV. The interaction term is found to be not significant. In models 1 to 4, some control variables such as firm size and engineering and manufacturing sector seem to be significant. After controlling for them, there is no change in the significant contributions of the independent variables to the dependent variable, confirming support for H1 and H2. Combining the results of H1 and H2, we can establish that RICV is a better indicator than IP licensing fees in predicting a firm's propensity to repeat licensing and indicating academic innovation impact on firms. With more than 100 cases per predictor variable, these models satisfy Field's (2005) guidelines of minimum 10 cases per predictor for reliable regression results. With a variance inflation factor (VIF) value of 1.234 that is lower than the cut-off value of 10 recommended by Chatterjee and Price

(1991), the probability of multi-collinearity issues affecting the findings is relatively low.

Combining the results of H1 and H2, we establish that while IP licensing fees and RICV have opposite effects on firm's propensity to repeat licensing with PRIs and academic innovation impact on them. The findings are consistent with Atuahene-Gima's (1993) findings that firms' ITL propensity was determined by their perceived value, namely, their benefits relative to their costs.

Although this study involves PRIs, its results may be seamlessly extended to universities for several reasons. First, since the implementation of the US Bayh-Dole Act and its equivalents in other knowledge-based economies, PRIs and universities in most countries have been charged with similar missions to create socio-economic impact through promoting knowledge generation and transfer to the industry (Link et al. 2011). Second, although PRIs are expected to work on more applied research while universities on basic research, both share similar strategic roles in setting future R&D directions, especially in areas critical to their nations, e.g., energy or climate. As a result of their longer planning horizon, their technologies are at an earlier stage of development compared to the industrial R&D centres, making their time to commercialisation longer and more risky than that of their industrial counterparts (Link and Scott 2013). As both universities and PRIs share similar goals, roles, pressures and challenges in their technology transfer processes, many studies in the technology transfer literature have treated them as a homogeneous group and extend their findings across them (Park et al. 2010).

Discussion

In the field of public research impact, the definition of the term "impact" differed across various stakeholders. To academia, the term is usually associated with intellectual contribution to one's discipline within academia while socio-economic impact is regarded as beyond academia. From the government perspective, the definition is much broader as that adopted by the Australian RQF and the UK Research Excellence Framework (REF). According to the UK Research Excellence Framework (REF2014 2011), impact is an effect on change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.

How broad should the definition be? Its breadth really depends on what and how it is to be assessed. To countries that focus on science, technology and innovation (STI) to fuel economic growth and generate wealth (e.g. Singapore), impact is typically defined and assessed with economic measures and STI indicators (Donovan 2011)

We made several contributions in this paper. First, we consolidated a large body of knowledge on impact studies with a theoretically grounded, multi-level framework for assessing public research and innovation impact, connecting the four stakeholders of the knowledge production process—PRIs/universities, TTOs, firms/industry, and community/economy.

Second, we identified literature gaps that create opportunities for future research. In this section, we will discuss several gaps, including the relationships among the various categories of stakeholder outputs, depicted as Impact Assessment Framework blocks 1 to 4 of Fig. 4.

Third, we established empirically the relative significance of RICV as a better indicator than IP licensing fees in predicting firms' propensity to repeat licensing from PRIs, thereby indicating some form of academic innovation impact on firms. We will elaborate in this section our findings of the RICV concept in the context of the integrated framework.

Inventive output of universities/PRI

In the first block of the framework in Fig. 4, the studies generally adopted the approach of using the inventive output of universities/PRIs as proxy indicators to estimate their social and economic impact. The notion of knowledge as key driver for the competitive advantage of firms and nations has its roots in the literature of economics building on Romer's endogenous growth theory (Romer 1990; Nonaka and Takeuchi 1995; Grant 1996). The capacity and capability to develop novel ideas and translate them into new products and processes for economic growth form the knowledge base of an economy (Huggins and Johnston 2009). These innovation development capabilities are increasingly being associated with universities and PRIs. As knowledge becomes publicly accessible either in the form of spill-overs or diffusion, it is widely regarded as a desirable outcome for the greater good of the society, spurring greater innovation and economic growth (Fischer and Varga 2003). Table 3 provides a summary of key studies using inventive output indicators.

As centres of knowledge production, universities/PRIs consistently engage in inventive process, where researchers develop ideas that could be novel and inventive. These inventions are then submitted as required by the Bayh-Dole Act or equivalent in the form of invention disclosures for evaluation by the TTOs for the appropriate intellectual property (IP) strategy to pursue. For invention disclosures that satisfy both internal and external patenting guidelines, the TTOs would proceed to file patent applications, while the researchers would submit their findings for publications. Some high-technology industries may find these publications or patents relevant and cite them or even participate in research collaboration with the universities/PRIs in the course of their R&D work.

Given the significant government R&D expenditures in universities and PRIs to develop the knowledge base of the economy, the studies of this category frequently built on the total factor productivity or knowledge production function frameworks to examine the various outputs in the inventive process as indicators to determine the economic benefits directly or indirectly generated (Jaffe 1989). Building on Cobb-Douglas production function, Pakes and Griliches (1984) pioneered the development and use of the full model of knowledge production function and the final output production function. In their study, they used R&D expenditures as knowledge creation investment and patent counts as economically valuable knowledge produced. Besides economic impact, many researchers have also adopted bibliometric approaches to assess the degree of international scientific influence using citation, mapping and network analysis of papers and patents (Tornquist and Hoenack 1996; Spencer 2001). Others have studied co-authored publications in their bibliometric examination of public-private research collaboration (Abramo et al. 2009a). While citation indicators may reflect productivity and impact, co-authorship indicators may illustrate informal network access, knowledge diffusion and skill transfer.

Disclosures Disclosures, patents

Knowledge infrastructure Non-resident tuition rate Patent citations

Patents

Table 3 Summary of studies using university/PRI inventive output indicators Inventive output indicators Authors

Chapple et al. 2005 Carlsson and Fridh 2002 Gulbrandsen and Rasmussen 2012 Park et al. 2010 Keane and Allison 1999 Baryla and Dotterweich 2001 Beaudry and Kananian. 2013 Jaffe et al. 1993 Meyer 2006

Owen-Smith and Powell2003 Spencer 2001 Wang et al. 2011 Agrawaland Henderson 2002 Bach and Llerena 2007 Baldini 2010

Bekkers and Freitas 2008 Bozeman and Kingsley 1997 Bozeman and Klein 1999 Fischer and Varga 2003 Georghiou and Roessner 2000 Huggins and Johnston 2009 Lissoni et al. 2013. Saez et al. 2002 Abramo et al. 2009b Tornquist and Hoenack 1996 Abramo et al. 2011 Abramo et al. 2009a Arundeland Geuna 2004 Cohen et al. 2002 Rogers et al. 1999 Dietz and Bozeman 2005 Nelson 2009 McMillan et al. 2000 Narin, Hamilton and Olivastro 1995 Narin et al. 1997 Nelson 1986 Hallet al. 2005 Villasana 2011

PRI-firm collaboration Publication citations

Publication co-authorship

Publications

Publications, budget Publications, patents

Publications, patents and citations

R&D intensity

R&D stocks, patents, citations University-industry research collaboration

Cohen et al. (2002) found that publications are important knowledge channels from the public sector to the manufacturing sector. This is supported by Agrawal and Henderson's (2002) interview of MIT faculty, who established that publications were two-and-one-half times more significant than patents as knowledge channels. Jaffe (1989) found that the proximity between universities/PRIs and firms could have

mediating effects on spill-overs from public-funded research to corporate patents. Subsequent studies suggested similar association between patent citation patterns (as indicator of "economic benefit") and geographical proximity in Austria (Fischer and Varga 2003) and in the USA (Jaffe et al. 1993).

Building on the RVM project, Dietz and Bozeman (2005) discovered significant differences between publication and patent productivity in their socio-technical human capital project. Career spent in industrial settings was found to be negatively related to publication productivity but positively related to patent productivity. In a similar vein, the source of industry support was identified to be unrelated to publication productivity but positively related to patent productivity. These findings suggest that governments that aim to improve the commercial relevance of their public-funded research should encourage "revolving door" policy between academia and industry in their development of science and technical human capital to enhance their university/PRI inventive output.

Pakes and Griliches (1984) established a strong association between R&D and patent counts at the firm and industry levels to demonstrate that patents are appropriate measures of inventive output. Although patent and patent citations have frequently been used as economic indicators, there is concern that patents do not provide the complete picture of economic contribution by universities/PRIs (Fischer and Varga 2003). First, patents do not capture all forms of knowledge, such as codified knowledge (trade secrets and copyrights) and tacit knowledge (know-how and experience), which is important for spill-over. Second, patent count is dependent on the intensity of patenting activity, which is in turn determined by the patenting policy of the institutions. Therefore, patent counts and citations are not necessarily reflective of the commercial value of underlying IP.

According to a study by Intellectual Ventures in 2008, although 60 % of the patents were developed by academia, PRIs and individual inventors, they contributed to less than 1 % of the licensing revenue. On the other hand, large enterprises which were the originators of 40 % of the patents received more than 99 % of the licensing revenue (Hagiu et al. 2011). The stark contrast in licensing revenue between these groups that is disproportionate to that in patent origination signals critical issues with patents as economic indicators.

This demonstrates the urgency for more commercialisation-related indicators that policy makers can implement if they are committed to encouraging more firms to license and commercialise public science (Cheah and Zalan 2013).

Commercialisation output of universities/PRIs by TTOs

The second block of the Impact Assessment Framework in Fig. 4 dwells on the output of commercialisation process that TTOs use in taking IPs from their universities/PRIs to the market. Since the 1990s, there has been an increasing awareness that technological innovation may not be confined within the boundaries of an organisation. It has been widely acknowledged that technology is an essential resource that can be converted into a capability and competitive advantage for firms and nations. Firms have been giving more attention to the sourcing of external knowledge to complement and enhance their existing R&D stock and the acquisition of external technology as part of

their R&D process. A summary of key studies using university/PRI commercialisation output indicators is listed in Table 4.

According to Hemmert (2004), external technology acquisition takes place by (a) licensing technology, (b) research collaboration with other organisations and (c) purchasing technology from other organisations. TTOs typically do not sell the technologies

Licenses, spin-offs

Table 4 Summary of studies using university/PRI commercialisation output indicators

Commercialisation output indicators Authors

Licensees that go IPO, royalties Powers and McDougall2005a

Licenses Bach and Llerena 2007

Bekkers and Freitas 2008 Carlsson and Fridh 2002 Heisey and Adelman 2011 Hewitt-Dundas 2012 Huggins and Johnston 2009 Lach and Schankerman 2008 Nelson 2009 Sine et al. 2003 Bozeman and Kingsley 1997 Bozeman and Klein 1999 Georghiou, and Roessner 2000 Gulbrandsen and Rasmussen 2012 Maia and Claro 2013 McAdam et al. 2009 Pries and Guild 2011 Rogers et al. 1999 Rogers et al. 2000 Zucker et al. 1998 Powers and McDougall 2005b Bania et al. 1993 Colombo et al. 2010 Di Gregorio and Shane 2003 Bach and Lambert 1992 Harhoff 1999 Jacobsson and Vico 2010 Lehrer et al. 2009 Lockett and Wright 2005 Marion et al. 2012 Miner et al. 2012 O'Shea et al. 2005 O'Shea et al. 2007 Probert et al. 2013 Rasmussen and Borch 2010 Tassey 1999 Coccia 2008.

New biotech enterprises No of spin-offs that go IPO Spin-off rate

Spin-offs

Technology transfer

originating from universities/PRIs, but prefer to assign non-exclusive licenses to firms if the inventions are broad in scope and wide in applications across industries, so as to encourage product development in more fields for maximum economic impact. Notwithstanding this, TTOs may consider exclusive licensing terms for inventions that require substantial investments for commercialisation. Nelson's (2009, p. 994) comparison of IP licenses, publications and patents found IP licenses to be the most reliable indicator of "downstream knowledge utilisation" as the "competing economic interests" between licensors and licensees ensure that the licensing agreement is a fair representation of industry demand. Besides licensing IPs to existing firms, TTOs may facilitate the formation of spin-offs by providing technology and business incubation services to their researchers.

As both developed and developing nations put in place the appropriate science, technology and education policies to facilitate technology commercialisation and academic entrepreneurship from universities/PRIs, there has been much scrutiny on their commercialisation output, namely, the number of IP licenses signed, the IP licensing revenues collected and the number spin-offs generated.

To date, more than 14 periodic surveys had been launched to capture such output indicators across different regions such as the USA (AUTM 1991-2012), Canada (AUTM 2000-2012), Europe (ProTon 1991-2012) and Singapore (A*STAR 2002-2011). The data collected from these surveys form the dataset for many studies in this literature strand. For example, AUTM data were used by Di Gregorio and Shane (2003) to show that the rate of university spin-offs depends on inventors' quality and equity stake in the spin-offs. Calls have been made to both AUTM and ASTP to standardise their variable definitions and methodological approaches to facilitate comparability of PRI indicators across the studies. However, there is a dearth of impact studies in the Asia.

One frequently used theoretical framework in these studies is the resource-based theory (Wernerfelt 1984), which postulates that a firm's performance is determined by its resources. Barney (1991, p. 117) defined resources as assets that are specific to the firm, "valuable, rare, imperfectly imitable, and non-substitutable". Building on the resource-based theory, O'Shea et al. (2005), Colombo et al. (2010) and Powers and McDougall (2005b) established that the quality of researcher and research has significant effect on spin-off formation and performance, which were commercialisation output indicators used to estimate public research economic impact.

While licensing number and revenue are useful and relevant indicators to measure commercialisation process efficiency, several issues have been raised about their adequacy in reflecting the economic benefits of public-funded research. First, as license data collected by TTOs only reflect commercialisation activities during the licensing period, which is capped between patent grant date and expiration date, they do not capture any further innovative activity that may continue to impact the economy beyond the licensing period. Second, it is widely recognised that the pathway of commercialisation from invention (that the firm has licensed from universities/PRIs) to innovation (that the firm can generate sales in the form of products and services) requires significant investment as it involves early-stage technology development. This pathway is often alluded to as the valley of death due to the high failure rate, and the role of the public sector in bridging this valley has been much debated upon in the UK and USA (Link and Scott 2013).

The current issues with commercialisation output indicators call for other indicators that would better reflect the downstream innovation process taking place in the firms to provide a more complete picture of public science impact on the industry.

Innovation of industry or firms

The third block of the Impact Assessment Framework focuses on the innovation of industry or firms. Table 5 summarises the studies using firm's innovation input and outcome indicators.

Some of the studies referred to the knowledge production function (Jaffe 1989). Acs et al. (1994) built on knowledge production function to ascertain that small firms could benefit (with increase in innovation count) from the knowledge of universities and the R&D centres of large firms. Varga (2000), on the other hand, applied the Griliches-Jaffe knowledge production function framework to establish that firms located in close proximity to the research university can benefit economically with an increase in their number of innovative products.

In 1998, Crépon et al. (1998) integrated both Cobb-Douglas production function and knowledge production function to develop the CDM model, which depicts two key relationships pursuant to a firm's decision to invest in innovation. The first relationship describes the link between innovation input (primarily R&D) and innovation output (product or process innovation) as accepted in the knowledge production function (Pakes and Griliches 1984). The second relationship established the connection between innovation output and productivity based on Cobb-Douglas production function. Since then, many studies have built on the CDM model using labour productivity (sales per employee) in Spain (Barajas et al. 2012) and in the Netherlands (Lôôf and Heshmati 2002) and turnover growth in Italy (Evangelista and Vezzani 2010).

Based on the CDM model, we have classified the following indicators as input to the innovation process of firms: (a) corporate R&D stocks, (b) innovation expenditures, (c) corporate patents, (d) investment in building and equipment for R&D and (e) induced investments. In particular, Pressman et al. (1995, p. 30) surveyed a sample of MIT licensees by collecting information on their induced investment defined as "Money spent developing new products and efficient ways to produce and market these products. It excludes the costs of producing (or investment required to produce) mature products". From the survey, they extrapolated their data to estimate an economic impact of US$922 million.

Using a similar approach, Kramer et al. (1997) estimated that University of Pennsylvania's licensees created an economic impact of US$4.6 billion in the form of induced investments. Although Pressman et al. (1995) and Kramer et al. (1997) have used induced investments to measure economic impact, these values are actually input to the firm's innovation process that should be distinguished from its innovation outcome. According to Eurostat (2012), innovation outcome may be in the form of product innovation (e.g. sales from innovative products) or process innovation (e.g. productivity gain from innovative process) (Goldstein and Drucker 2006). The distinction between innovation input and innovation outcome indicators is clearly depicted in block 3 of the framework in Fig. 4.

Table 5 Summary of studies using industry/firm innovation input/outcome indicators

Innovation indicators

Authors

Innovation input Corporate patent linked to university papers Corporate patents

Corporate patents, R&D expenditure Corporate R&D

Corporate R&D and innovation Corporate R&D stocks linked to universities Corporate patenting rate Induced investment

Industry-owned patents

Innovative expenditure

Investment in building and equipment

R&D intensity

Research collaboration

Agrawaland Cockburn 2003 Jaffe 1989

Autant-Bernard 2001 Boardman and Gray 2010 De Fuentes and Dutrenit 2012 Adams 2002 Hu 2008 Giesecke 2000 Kramer et al. 1997 Pressman et al. 1995 Lawson 2013a Lawson 2013b Beugelsdijk and Cornet 2002 Florax and Folmer 1992 Hauknes and Knell 2009 Martin 1998

Innovation outcome indicators Innovation sales

Innovative count Innovative output of firms Innovative products

New product innovation sales New products

New products and processes

No. of drugs entering each clinical R&D stage (from NIH grant expenditure) Product and process innovation

Sales and profitability Sales growth rate

Sales of new products, improved products and products with no or incremental changes

Sales of products and services of spin-offs Sales per employee

Beugelsdijk and Cornet 2002 Loofand Brostrom 2008 Anselin et al. 1997 Acs et al. 1994 Beise and Stahl1999 Varga 2000 Eom and Lee 2010 Mansfield 1991, 1995, 1998 Rosenberg and Nelson 1994 Blume-Kohout 2012 Bozeman and Kingsley 1997 Bozeman and Klein 1999 Coccia 2008 Concei^ao et al. 1998 Georghiou, and Roessner 2000 Kingsley et al. 1996 Klevorick et al. 1995 Bozeman 2000 Evangelista and Vezzani 2010 Faems et al. 2005

Vincett 2010 Barajas et al. 2012 Loof and Heshmati 2002

While many studies continue to use innovative product sales or proportion of new products as indicators of firms' innovative outcome, there are recommendations to refine innovative outcome modes and indicators. In a recent study, Evangelista and Vezzani (2010) expanded Eurostat's (2012) definition of innovation outcome from two to four modes to examine economic impact: (a) pure product innovation, (b) pure process innovation, (c) organisation innovation and (d) mix of product, process and organisational innovations. Penfield et al. (2014) further articulated the need to distinguish among output, outcome and impact.

Community/economy

The fourth block of the framework comprises studies that use economic indicators such as employment, income, output and GDP growth, as summarised in Table 6.

The impact studies in this category generally applied the input-output modelling or Keynesian multiplier approach. In UK, Harris' (1997) study of the University of Portsmouth with the input-output approach estimated that the direct expenditure by the university sector resulted in an output multiplier effect of 1.24 to 1.73. Glasson's (2003) input-output model found an output multiplier of between 0.70 and 1.12 for Sunderland University in Northeast England.

Using Keynesian multiplier approach, Brownrigg (1973) arrived at income multiplier between 1.45 and 1.80 based on a case study of the University of Stirling, while Huggins and Cooke (1997) found the corresponding multiplier at 1.46 to 1.52 at the Cardiff University. In the USA, Felsenstein (1996) applied econometric model on the input-output model to quantify the contribution of the Northwestern University to the Chicago metropolitan area at an output multiplier of 3.1 and employment multiplier of 1.55 in 1993. In a more recent study, Roessner et al. (2013) estimated the impact of university licensing on the whole US economy with a range of hypothetical licensing royalty rates at 2, 5 and 10 %, based on AUTM data (1996-2010) and product substitution effect adjustment.

Despite the popularity of the input-output and Keynesian multiplier approaches, these methodologies have raised some concerns. First, these techniques are restricted to estimating the backward linkages arising only from university expenditures and investment. The dynamic impact of university knowledge production and other forms of output on the business community are not captured (Martin 1998). Second, there is a lack of coun-terfactual hypotheses for comparison of impact with and without the university's presence, giving rise to wide variation in results (Goldstein and Drucker 2006, p. 24). Third, the input-output model assumes that product sales arising from university/PRI-licensed IPs are all contained within the local economy to contribute to GDP. This assumption becomes invalid when a significant proportion of IPs are commercialised overseas due to the small domestic market size. Fourth, the input-output model treats university/PRI licensing and research income as expenditures that can have direct and indirect effect on the output and employment of other industries within the local economy. However, in a small country with an open economy, it is unlikely that it has the capability and capacity to provide all the equipment, materials and talent required to conduct R&D.

Toward an integrated framework and the role of RICV

We understand that different indicators are useful and relevant for tracking the effects of intermediate output or outcome at the different stages of the long journey of

Table 6 Summary of studies related to economic impact

Economic impact indicators

Authors

Country competitiveness Earnings of non-farm worker

Economic growth Employment

Employment and income

Employment and output Employment growth

Employment, income and output

Employment, output and income Expenditures from staff and students GDP contributed by university R&D GDP growth and income growth Human capital

Income and output Jobs created

Physical capitalor labour

Private and socialrates of return on projects

Private and socialrates of return to public R&D

Property rent

Socialrate of return on academic research Value-add

van Hemert and Nijkamp 2010 Goldstein and Drucker 2006 Goldstein and Renault 2004 Bozeman 2000 Concei^äo et al. 1998 Kramer et al. 1997 Pressman et al. 1995 Riddeland Schwer. 2003 Swann and Prevezer 1996 Armstrong 1993 Brownrigg 1973 Lewis 1988 Glasson 2003

Henderson and Weiler 2010 Lendel2010 Felsenstein 1996 Harris 1997 Pastor et al. 2013, Roessner et al. 2013 Roessner et al. 2010 Steinacker 2005 Martin 1998

Baldwin and McCracken 2013

Huffman and Quigley 2002

Nagle 2007

Perkmann et al. 2011

Bleaney et al. 1992

Gulbrandsen and Rasmussen 2012

Miner et al. 2012

Wallmark 1997

Tassey 2005

Weiler 2000

Jamison and Jansen 2000 Salter and Martin 2001 Sivitanidou and Sivitanides 1995 Mansfield 1991, 1995, 1998 Huggins and Johnston 2009

research, innovation and enterprise (RIE) value-added activities involving multiple stakeholders from science produced by universities/PRIs through the innovation outcome generated by the industry/firms to economic development:

1. Inventive output of universities/PRIs, e.g. R&D intensity, invention or technology disclosures, publications and patents;

2. Commercialisation output of universities/PRIs, e.g. licenses executed, spin-offs;

3. Firm/industry's innovation input, e.g. corporate R&D stocks and its innovation outcome, e.g. innovative product sales and RICV;

4. Economic development in the form of employment, income and RICV at the economy level.

Given that RIE is a complex and iterative value chain with multiple feedback loops, it is apparent that no single indicator can furnish a complete picture of public-funded research and innovation impact. We therefore propose an integrated Impact Assessment Framework for universities/PRIs as a starting point to manage their value chain. The framework provides the policy makers and scholars the flexibility to define and refine indicators at different stages of the RIE journey while keeping a holistic view of the entire value chain, to facilitate periodic monitoring, analytics and adaptation at micro-, meso- and macro-levels. In particular, the framework includes a relatively new indicator, RICV, to estimate innovation outcome at firm or industry level, as well as economic impact at a regional or national level.

Implications and conclusion

In this paper, we addressed our research question whether RICV is a suitable predictor of firms' propensity to repeat their in-licensing agreements with PRIs, compared to the traditional licensing revenue indicator. Our results show that firms that incur higher IP licensing fees with universities/PRIs have lower propensity to repeat their IP licensing transactions, thereby reducing academic innovation impact on firms. On the other hand, firms that attain higher levels of RICV with universities/PRIs are found to have higher propensity to repeat their license transactions, indicating an increase in academic innovation impact on firms. With results confirming our hypotheses, we contribute to the literature of impact studies by proposing new secondary RICV indicators to estimate the economic impact of public-funded technological innovation at the firm, industry and national levels.

Our finding of RICV as a predictor of firms' propensity to repeat licensing engagement with universities/PRI has contributed to the TCE theory. In most studies of the current literature, the TCE is used to explain the propensity of firms to license technologies from other firms (pecuniary sources as traditional licensors). Given that these public institutions are non-pecuniary sources as they have socio-economic rather than pure financial objectives, the determinants of in-licensing propensity in firm-public institution engagement are likely to be different from that in firm-firm transaction. This study therefore adds a new dimension of RICV to the TCE theory.

In addition, our results show that the NPV theory from corporate finance, which has erstwhile been useful in evaluating investment decisions at the firm level, can be extended to the technology transfer discipline to estimate the impact of public research and innovation at the industry and economy level.

This study has made several managerial implications. As shown in Fig. 2, Singapore PRIs started using RICV since 2010 as part of their balanced scorecard strategy for performance management to measure and monitor the quality of the licenses signed and their impact at the firm level. Totalling the RICVs of all licensees for a particular year allows impact estimation at the economy level for the year. From 2010 to 2012, the

PRIs have seen a significant increase in total RICV by more than 80 % per annum, with corresponding increase in total licensing revenues. Second, by grouping the RICV of firms by industry, the PRIs could obtain a better grasp of research and innovation impact at the industry level. Trending analysis of RICV by technology has enabled the identification of promising and complementary technologies with rising RICVs to form innovation clusters to serve growing industry sectors. To date, three innovation clusters have been formed and increased their contribution from less than 5 % of total RICV in 2010 to more than one third in 2012.

The findings of this study indicate that RICV is a better indicator than the traditional licensing revenue metric used by universities/PRIs to measure and monitor innovation impact in terms of firm innovation output. Although this study was carried out with the public research organisations of Singapore, the findings are generalisable to those of other countries that pursue the strategy of innovation-driven economic development. Universities/PRIs of knowledge-based economies face similar challenges in measuring the intermediate input and output of their respective RIE value chain activities, as evident in their adoption of common traditional indicators for benchmarking despite their limitations (AUTM 2014). To address the challenges and limitations, the universities/ PRIs of the other innovation-driven economies are invited to use the RICV approach to inform strategic decision-making in shaping research agenda, allocating resources, influencing inventive outputs and determining the effectiveness of commercialisation outputs. The implementation of the RICV indicator in various universities/PRIs will build confidence in the indicator.

The policy makers of innovating economies may use the RICV and related indicators to benchmark the impact of research and innovation on enterprise and economy. Firms could also adopt and adapt the RICV approach to actively manage their IP and technology portfolio so as to increase the net present value of revenue streams generated by their products and services enabled by their IP and technology portfolio.

As an extension for this study, we propose development of a practical implementation guide for institutions seeking to use the framework and RICV indicator. The guide should comprise the development of tools, processes and training to enable the setup of a scalable and flexible system that can cater to the needs and objectives of the institutions.

Our study has identified a few areas for future research. First, while our paper focuses on the economic impact and STI indicators gleaned from our review of 275 articles, we acknowledge that there is much value in combining narratives with both qualitative and quantitative metrics. Although qualitative approaches such as field interviews and case study interrogations could provide rich and useful data, they could be very resource-intensive. The public institutions may therefore consider embarking on in-depth case studies (e.g. RVM) for completed large-scale R&D projects once every few years, not only to study about the performance of specific projects but also to use them to baseline and compare the relative significance of the current set of applicable indicators (as synthesised in our proposed Impact Assessment Framework), as well as analyse their relationships to evaluate their effectiveness.

Second, we are aware that RIE process from PRIs/universities' receipt of R&D grant award (to perform R&D) to their TTOs' successful commercialisation of IP generated (with licensing firms) may take an average of 3 to 8 years, involving multiple

organisations and departments. Although it is widely acknowledged that the process is non-linear, there is increasing interest among PRIs and governments (Smith et al. 2011) to trace the efficiency of the various stages of the process to evaluate its overall performance. Future studies may embark on this area to identify the determinants of the conversion efficiency among the intermediate outputs, for example, from invention disclosures to patents, from patents to licenses and from licenses to products.

Endnotes

1The Triple Helix model emphasises the central role of universities in research and innovation and the significance of university-industry-government relations in advancing knowledge-based society (Etzkowitz et al. 2005).

2Asset specificity refers to the extent to which assets can be transferred to other purposes. The TCE model postulates that the more specific the nature of the assets provided by the seller, the greater is the risk that the buyer faces in the transaction.

3IP appropriability refers to the risk of disclosing IP to expropriation.

4RVM is research value mapping approach developed by Bozeman and Kingsley (1997) in the US to assess the socio-technical impact of large applied R&D projects with the Department of Energy002E

5Bureau d'Economie Théorique et Appliquée (BETA), France

6National Institute of Standards and Technology (NIST), US-BETA and NIST developed methods in the 1990s to assess the impacts of large-scale R&D projects.

7RICV is treated as "imputed" for the following reasons: (a) the sales revenue is a projection or extrapolation that was mutually agreed between licensee and licensor; and more importantly, (b) the imputed commercial value is attributed in whole rather in part regardless of the contribution of the IP(s) licensed to the final product/services. In other words, it is assumed that all IPs play an integral part to the final innovation outcome to simplify computation for all practical intents and purposes. A similar approach was used by Roessner et al. (2013) in computing sales revenues from university IP royalties based on AUTM data in their estimation of economic impact using input-output model.

8Keyword combinations or variations include but not limited to "invention" or "publication" or "patent" or "commercialisation", "innovation" and "academic" or "university" or "public research" and "impact".

9For robustness, the keyword search constraints were relaxed using broad queries for a subset of journals (e.g., American Economic Review, Applied Economics, Economic Development Quarterly, Higher Education, Industry and Innovation, Journal of Technology Transfer, R&D Management, Research Policy, Review of Economics and Statistics, Science and Public Policy, Technovation and Urban Studies). We submit that any additional papers identified did not have a material influence on the model we inferred from our literature synthesis.

10Although newly published papers may not have time to accumulate citations, they could contribute to the literature by indicating research directions. We therefore did not adopt the citation-based method in our literature search to avoid discriminating against recent papers.

Appendix

RICV is computed as follows (Table 7):

Table 7 Computation of RICV

RICV = Actualrecurrent commercial value from royalty-bearing IP licensesa

+ Actuallicensing revenue from end-user licensing agreements (EULAs)b

+ Actuallicensing revenue from sale of equity in lieu of upfront payment in spin-offsc

+ Estimated commercial revenue from upfront and one-off licensing deals by licensee from product/ services incorporating the licensed IPd

aThis is easily derived from the revenue statement submitted by the licensees as part of the licensing agreements. It is the actual revenue derived from their sales of product/services incorporating the IP(s) licensed

bThis is computed based on the actual licensing revenue from EULAs, which are usually simple licenses such as software or productivity tools for improving process and work efficiency

cTo mitigate the cash flow demands in the initial years for spin-offs, it is typical to take equity in lieu of upfront, and/or even royalty payments. The quality of the spin-off is best reflected by its sales price. The portion allotted in lieu of upfront and/or royalty payments would be taken in as RICV in the year of transaction

dOften, we deal in (a) one-time payments or (b) upfront and royalty-bearing payments. The licensees offering one-time payments avoid paying royalties or licensing for defensive reasons, while the others making upfront and royalty-bearing payments hope to increase their upside as their sales are expected to be robust

All term sheet discussions are typically started on a royalty-bearing basis to compute an equitable stream for upfront and royalty payments. In exceptional cases where one-time payment is the agreed licensing agreement, the projected sales streams will be discounted to derive their net present value. To estimate the likely commercial revenue generated as if the IP(s) are taken to market, the RICV was derived by dividing the one-time or upfront payment collected by the expected royalty rate. Table 8 provides an illustration of the RICV imputed at $16.5 million.

Table 8 An illustration of RICV imputed

Components of RICV RICV

(a) Actualrecurrent commercial value from royalty-bearing IP licenses $1.0 mil

(b) Actuallicensing revenue from end-user licensing agreements (EULAs) $0.5 mil

(c) Actuallicensing revenue from sale of equity in lieu of upfront payment in spin-offs $5.0 mil

(Assume TTO/PRI holds 10 % equity stake of the spin-off, which has $50 milvaluation at the point of exit)

(d) Estimated commercial revenue from upfront and one-off licensing deals by licensee from $10.0 mil product/services incorporating licensed IP

(Assume upfront and one-off receipt of $1 milat expected royalty rate of 10 %)

TotalRICV imputed $16.5 mil

Additional file

Additional file 1: Translation of the abstract into Arabic. (PDF 124 kb)

Acknowledgements

We thank the Chief Executive of A*STAR ETPL, Mr Philip Lim, for his guidance and support of this paper. However, all errors of fact or interpretation in this paper remain ours.

Author details

1NationalUniversity of Singapore Business School, Biz 1 8-46, 15 Kent Ridge Drive, Singapore 119245, Singapore. 2Agency for Science, Technology and Research (A*STAR) ETPL, 30 Biopolis St, #09-02, Singapore 138671, Singapore.

Received: 12 July 2015 Accepted: 31 March 2016 Published online: 13 April 2016

References

A*STAR (2002-2011) National survey of R&D. Agency for Science, Technology and Research.

Abramo G, D'Angelo CA, Di Costa F (2011) University-industry research collaboration: a model to assess university

capability. Higher Educ 62(2):163-181 Abramo G, D'Angelo CA, Di Costa F, Solazzi M (2009a) University-industry collaboration in Italy: a bibliometric

examination. Technovation 29(6):498-507 Abramo G, D'Angelo CA, Di Costa F (2009b) Research collaboration and productivity: is there correlation? Higher Educ 57(2):155-171

Acs Z, Audretsch D, Feldman M (1994) R&D spillovers and recipient firm size. Rev Econ Stat 76:336-40 Adams J (2002) Comparative localization of academic and industrial spillovers. J Econ Geogr 2:253-78 Agrawal A, Cockburn I (2003) The anchor tenant hypothesis: exploring the role of large, local, R&D-intensive firms in

regional innovation systems. Int J Ind Organ 21:1227-53 Agrawal A, Henderson R (2002) Putting patents in context: exploring knowledge transfer from MIT. Manag Sci 48(1):44-60 AmbosTC, Makela K, Birkinshaw J, D'Este P (2008) When does university research get commercialized? Creating ambidexterity

in research institutions. J Manag Stud 45:1424-47 Anselin L, Varga A, Acs Z (1997) Local geographic spillovers between university research and high technology innovations. J Urban Econ 42:422-48

Armstrong H (1993) The local income and employment impact of Lancaster University. Urban Stud 30:1653-1668 Arundel A, Geuna A (2004) Proximity and the use of public science by innovative European firms. Econ Innov New Technol 13(6):559-580

Atuahene-Gima K (1993) Determinants of inward technology licensing intentions: an empirical analysis of Australian

engineering firms. J Prod Innov Manag 10:230-240 Autant-Bernard C (2001) Science and knowledge flows: evidence from the French case. Res Policy 30(7):1069-1078 AUTM (1991-2012) The AUTM Licensing Survey: Fiscal Year 1991-2012. Norwalk: Association of University Technology Managers

AUTM (1996-2010) The AUTM Licensing Survey: Fiscal Year 1996-2010. Norwalk: Association of University Technology Managers

AUTM (2000-2012) The AUTM Licensing Survey: Fiscal Year 2000-2012. Norwalk: Association of University Technology Managers

AUTM (2014) The AUTM Licensing Survey: Fiscal Year 2014. Norwalk: Association of University Technology Managers Bach L, Llerena P (2007) Indicators of higher-education institutes and public-research organizations technology transfer

activities: insights from France. Sci Public Policy 34(10):709-721 Baldini N (2010) Do royalties really foster university patenting activity? An answer from Italy. Technovation 30(2):109-116 Baldwin J, McCracken W III (2013) Justifying the Ivory Tower: higher education and state economic growth. J Educ Finance 38(3):181-209

Bania N, Eberts R, Fogarty M (1993) Universities and the startup of new companies: can we generalize from Route 128

and Silicon Valley? Rev Econ Stat 75:761-66 Barajas A, Huergo E, Moreno L (2012) Measuring the economic impact of research joint ventures supported by the EU

Framework Programme. J Technol Transf 37(6):917-942 Barney J (1991) Firm resources and sustained competitive advantage. J Manag 17:99-120 Baryla Jr, Dotterweich D (2001) Student migration: do significant factors vary by region? Educ Econ 9:269-80 Beaudry C, Kananian R (2013) Follow the (industry) money—the impact of science networks and industry-to-university

contracts on academic patenting in nanotechnology and biotechnology. Ind Innov 20(3):241-260 Beise M, Stahl H (1999) Public research and industrial innovations in Germany. Res Policy 28:397-422 Bekkers R, Freitas IMB (2008) Analysing knowledge transfer channels between universities and industry: to what degree

do sectors also matter? Res Policy 37(10):1837-1853 Beugelsdijk S, Cornet M (2002) A far friend is worth more than a good neighbour: proximity and innovation in a small

country. J Manag Gov 6:169-88 Bleaney M, Binks M, Greenaway D, Reed G, Whynes D (1992) What does a university add to its local economy. Appl Econ 24:305-311

Blume-Kohout ME (2012) Does targeted, disease-specific public research funding influence pharmaceutical innovation?

J Policy Anal Manage 31(3):641-660 Boardman C, Gray D (2010) The new science and engineering management: cooperative research centers as

government policies, industry strategies, and organizations. J Technol Transf 35(5):445-459 Bozeman B (2000) Technology transfer and public policy: a review of research and theory. Res Policy 29:627-655 Bozeman B, Kingsley G (1997) R&D value mapping: a new approach to case study-based evaluation. J Technol Transf 22(2):33-41

Bozeman B, Klein HK (1999) The case study as research heuristic: lessons from the R&D value mapping project. Eval

Program Plann 22(1):91-103 Brownrigg M (1973) The economic impact of a new university. Scott J Pol Econ 20:123-139 Carlsson B, Fridh AC (2002) Technology transfer in United States universities: a survey and statistical analysis. J Evol Econ 12:199-232

Chapple W, Lockett A, Siegel D, Wright M (2005) Assessing the relative performance of UK university technology

transfer offices: parametric and non-parametric evidence. Res Policy 34(3):369-384 Chatterjee S, Price B (1991) Regression analysis by example. Wiley, New York

Cheah S, Zalan T (2013) Why are some firms more successful than others in commercial adoption of technologies

licensed from public research institutes in Singapore? The Proceedings of XXIV ISPIM Conference, Helsinki Chesbrough H (2007) The market for innovation: implications for corporate strategy. Calif Manage Rev 49(3):45-66

Coccia M (2008) Spatial mobility of knowledge transfer and absorptive capacity: analysis and measurement of the

impact within the geoeconomic space. J Technol Transf 33(1):105-122 Cohen W, Nelson R, Walsh J (2002) Links and impacts: the influence of public research on industrial R&D. Manag Sci 48:1 -23 Colombo M, D'Adda D, Piva E (2010) An empirical analysis. J Technol Transf 35(1):113-140

Conceiçao P, Heitor MV, Oliveira P (1998) University-based technology licensing in the knowledge based economy.

Technovation 18(10):615-658 Crépon B, Duguet E, Mairesse J (1998) Research and development, innovation and productivity: an econometric

analysis at the firm level. Econ Innov New Technol 7(2):115-58 De Fuentes C, Dutrenit G (2012) Best channels of academia-industry interaction for long-term benefit. Res Policy 41 (9):1666-1682

Derrick GE, Bryant C (2013) The role of research incentives in medical research organisations. R&D Manag 43(1):75-86 Di Gregorio, Shane S (2003) Why do some universities generate more start-ups than others? Res Policy 32:209-227 Dietz JS, Bozeman B (2005) Academic careers, patents, and productivity: industry experience as scientific and technical

human capital. Res Policy 34(3):349-367 Donovan C (2011) State of the art in assessing research impact: introduction to a special issue. Res Eval 20(3):175-179 Elmquist M, Le Masson P (2009) The value of a 'failed' R&D project: an emerging evaluation framework for building

innovative capabilities. R&D Manag 3(2):136-152 Eom BY, Lee K (2010) Determinants of industry-academy linkages, and their impact on firm performance: the case of

Korea as a latecomer in knowledge industrialization. Res Policy 39(5):625-639 Etzkowitz H, Mello J, Almeida M (2005) Towards "meta-innovation" in Brazil: the evolution of the incubator and the

emergence of a triple helix. Res Policy 34(4):411 -424 European Commission (2009) Metrics for knowledge transfer from public research organisations in Europe. Report from

the European Commission's Expert Group on Knowledge Transfer Metrics, Brussels Eurostat (2012) viewed 20 Feb 2012. http://epp.eurostat.ec.europa.eu/

Evangelista R, Vezzani A (2010) The economic impact of technological and organizational innovations. a firm-level

analysis. Res Policy 39:1253-1263 Faems D, Van Looy B, Debackere K (2005) The role of interorganizational collaboration within innovation strategies:

towards a portfolio approach. J Prod Innov Manag 22(3):238-250 Felsenstein D (1996) The university in the metropolitan arena: impacts and public policy implications. Urban Stud 33:1565-80

Field A (2005) Effect sizes. Discovering Statistics Using SPSS. 2nd edit. Sage Publcations, London

Fischer MM, Varga A (2003) Spatial knowledge spillovers and university research: evidence from Austria. Ann Reg Sci 37:303-22

Fisher I (1930) The theory of interest. Macmillan, New York

Florax R, Folmer H (1992) Knowledge impacts of universities on industry: an aggregate simultaneous investment

model. J Reg Sci 32:437-66 Freeman C (1987) Technology policy and economic performance: lessons from Japan. Pinter, London Georghiou L, Roessner D (2000) Evaluating technology programs: tools and methods. Res Policy 29:657-678 Giesecke S (2000) The contrasting roles of government in the development of biotechnology industry in the US and

Germany. Res Policy 29(2):205-223 Glasson J (2003) The widening local and regional development impacts of the modern universities—a tale of two

cities (and North-South perspectives). Local Econ 18:21-37 Goldstein H (1989) Estimating the regional economic impact of universities: an application of input-output analysis. Plann Higher Educ 18:51-64

Goldstein H, Drucker J (2006) The economic development impacts of universities on regions: do size and distance matter? Econ Dev Q 20:22-43

Goldstein HA, Renault CS (2004) Contributions of universities to regional economic development: a quasi-experimental

approach. Reg Stud 38:733-46 Grant R (1996) Toward a knowledge-based theory of the firm. Strategic Management Journal 17: 109-122 Gulbrandsen M, Rasmussen E (2012) The use and development of indicators for the commercialisation of university

research in a national support programme. Technol Anal Strateg Manag 24(5):481 -495 Hagiu A, Yoffie D, Wagonfeld AB (2009) Intellectual Ventures. HBS Case: 710-423 Hall BH, Jaffe AB, Trajtenberg M (2005) Market value and patent citations. RAND J Econ 36(1):16-38. Harhoff D (1999) Firm formation and regional spillovers. Econ Innov New Technol 8:27-55 Harris R (1997) The impact of the University of Portsmouth on the local economy. Urban Stud 34:605-26 Hauknes J, Knell M (2009) Embodied knowledge and sectoral linkages: an input-output approach to the interaction of

high-and low-tech industries. Res Policy 38(3):459-469 Heisey PW, Adelman SW (2011) Research expenditures, technology transfer activity, and university licensing revenue. J Technol Transf 36(1):38-60

Hemmert M (2004) The influence of institutional factors on the technology acquisition performance of high-tech firms:

survey results from Germany and Japan. Res Policy 33:1019-1040 Henderson J, Weiler S (2010) Entrepreneurs and job growth: probing the boundaries of time and space. Econ Dev Q 24(1):23-32

Hewitt-Dundas N (2012) Research intensity and knowledge transfer activity in UK universities. Res Policy 41(2):262-275 Hicks D, Kroll P, Narin F, Thomas P, Ruegg R, Tomizawa H, Kobayashi S (2002) Quantitative Methods of Research Evaluation Used by the US Federal Government. Second Theory-Oriented Research Group, National Institute of Science and Technology Policy (NISTEP). Tokyo, Japan Hu MC (2008) Knowledge flows and innovation capability: the patenting trajectory of Taiwan's thin film transistor-liquid

crystal display industry. Technol Forecasting Soc Change 75(9):1423-1438 Huffman D, Quigley J (2002) The role of the university in attracting high tech entrepreneurship: a Silicon Valley tale. Ann Reg Sci 36:403-19

Huggins R, Cooke P (1997) The economic impact of Cardiff University: innovation, learning and job generation. GeoJournal 41(4): 325-337

Huggins R, Johnston A (2009) The economic and innovation contribution of universities. a regional perspective. Environ

Plann C Govern Policy 27(6):1088-1106 Jacobsson S, Vico EP (2010) Towards a systemic framework for capturing and explaining the effects of academic RD.

Technol Anal Strateg Manag 22(7):765-787 Jaffe A, Trajtenberg, Henderson R (1993) Geographic localization of knowledge spillovers as evidenced by patent

citations. Q J Econ 108:577-98 Jaffe AB (1989) Real effects of academic research. Am Econ Rev 79:957-70

Jamison D, Jansen C (2000) Technology transfer and economic growth. J Assoc Univ Technol Managers 12:23-45 Keane J, Allison J (1999) The intersection of the learning region and local and regional economic development:

analysing the role of higher education. Reg Stud 33:896-902 Kingsley G, Bozeman B, Coker K (1996) Technology transfer and absorption: an 'R&D value-mapping' approach to

evaluation. Res Policy 25(6):967-995 Klaes M (2000) The birth of the concept of transaction costs: issues and controversies. Ind Corp Change 9(4):567-593 Klevorick A, Levin R, Nelson R, Winter S (1995) On the sources and significance of inter-industry differences in

technological opportunities. Res Policy 24:185-205 Kline SJ, Rosenberg N (1986) An overview of innovation. In: Landau R, Rosenberg N (eds) The positive sum strategy:

Harnessing technology for economic growth. National Academy Press, Washington, DC, pp 275-304 Kondo M (2012, June) Techno-economic impacts of large-scale national R&D projects. In Management of Innovation and

Technology (ICMIT), 2012 IEEE International Conference: 88-93. IEEE Kramer PB, Scheibe SL, Reavis DY, Berneman LP (1997) Induced investments and jobs produced by exclusive patent

licenses: a confirmatory study. J Assoc Univ Technol Managers 9:79-100 Kuhn TS (1962) The structure of scientific revolutions. University of Chicago Press, Chicago, IL Lach S, Schankerman M (2008) Incentives and invention in universities. Rand J Econ 39(2):403-433 Lawson C (2013a) Academic inventions outside the university: investigating patent ownership in the UK. Ind Innov 20(5): 385-398

Lawson C (2013b) Academic patenting: the importance of industry support. J Technol Transf 38(4): 509-535

Lehrer M, Nell P, Garber L (2009) A national systems view of university entrepreneurial ISM: inferences from comparison

of the German and US experience. Res Policy 38(2):268-280 Lendel I (2010) The impact of research universities on regional economies: the concept of university products. Econ Dev Q 24(3):210-230

Lewis JA (1988) Assessing the effect of the polytechnic, Wolverhampton, on the local community. Urban Stud 25:53-61 Link AN, Scott J (2013) Bending the Arc of Innovation: Public Support of R&D in Small, Entrepreneurial Firms. Palgrave Macmillan

Link AN, Siegel DS, van Fleet DD (2011) Public science and public innovation: assessing the relationship between

patenting at US National Laboratories and the Bayh-Dole Act. Res Policy 40(8):1094-1099 Lissoni F, Pezzoni M, Poti B, Romagnosi S (2013) University autonomy, the professor privilege and academic patenting:

Italy, 1996-2007. Ind Innov 20(5):399-421 Lockett A, Wright M (2005) Resources, capabilities, risk capital and the creation of university spin-out companies. Res Policy 34(7):1043-1057

Lööf H, Broström A (2008) Does knowledge diffusion between university and industry increase innovativeness? J Technol Transf 33(1):73-90

Lööf H, Heshmati A (2002) Knowledge capital and performance heterogeneity: a firm-level innovation study. Int J Prod Econ 76(1):61 -85

Magni C (2009) Investment decisions, net present value and bounded rationality. Quant Finance 9(8):967-979 Maia C, Claro J (2013) The role of a proof of concept center in a university ecosystem: an exploratory study. J Technol Transf 38(5):641-650

Mansfield E (1991) Academic research and industrial innovation. Res Policy 20:1-12

Mansfield E (1995) Academic research underlying industrial innovations: sources, characteristics, and financing. Rev Econ Stat 77(1):55-65

Mansfield E (1998) Academic research and industrial innovation: an update of empirical findings. Res Policy 26:773-776 Marion TJ, Dunlap DR, Friar JH (2012) The university entrepreneur: a census and survey of attributes and outcomes. R&D Manag 42(5):401-419

Martin F (1998) The economic impact of Canadian university R&D. Res Policy 27:677-687

McAdam R, McAdam M, Brown V (2009) Proof of concept processes in UK university technology transfer: an absorptive

capacity perspective. R&D Manag 39(2):192-210 McMillan GS, Narin F, Deeds DL (2000) An analysis of the critical role of public science in innovation: the case of

biotechnology. Res Policy 29(1):1-8 Meyer M (2006) Are patenting scientists the better scholars? An exploratory comparison of inventor-authors with their

non-inventing peers in nano-science and technology. Res Policy 35(10):1646-1662 Miner AS, Gong Y, Ciuchta MP, Sadler A, Surdyk J (2012) Promoting university startups: international patterns, vicarious

learning and policy implications. J Technol Transf 37(2):213-233 Nagle M (2007) Canonical analysis of university presence and industrial comparative advantage. Econ Dev Q 21(4):325-338 Narin F, Hamilton KS, Olivastro, D (1995) Linkage between agency supported research and patented industrial technology. Research Evaluation 5(3): 183-187 Narin F, Hamilton KS, Olivastro D (1997) The increasing linkage between US technology and public science. Res Policy 26(3):317-330 Nelson AJ (2009) Measuring knowledge spillovers: what patents, licenses and publications reveal about innovation

diffusion. Res Policy 38:994-1005 Nelson RR (1986) Institutions supporting technical advance in industry. Am Econ Rev 76:186-189 Nelson RR (1993) National innovation systems: a comparative study. Oxford Univ. Press, New York

NIH (2015) About NIH: Budget. http://www.nih.gov/about-nih/what-we-do/budget. Accessed 26 Dec 2015 Nonaka I, Takeuchi A (1995) The knowledge-creating company. Oxford University Press, NY

NSF (2015) FY2014 Agency financial report. December 2014. http://www.nsf.gov/pubs/2015/nsf15002/pdf/nsf15002.pdf. Accessed 26 Dec 2015

O'Shea R, Allen T, Chevalier A, Roche F (2005) Entrepreneurial orientation, technology transfer and spinoff performance

of US universities. Res Policy 34(7):994-1009 OECD (2011) Public research institutions: mapping sector trends, OECD Publishing. http://dx.doi.org/10.1787/

9789264119505-en. Accessed 26 Dec 2015 O'Shea RP, Allen TJ, Morse KP, O'Gorman C, Roche F (2007) Delineating the anatomy of an entrepreneurial university:

the Massachusetts Institute of Technology experience. R&D Manag 37(1):1 —16 Owen-Smith J, Powell WW (2003) The expanding role of university patenting in the life sciences: assessing the

importance of experience and connectivity. Res Policy 32(9):1695—1711 Pakes A, Griliches Z (1984) Patents and R&D at the firm level: a first look. In: Griliches Z (ed) R&D, patents and

productivity. University of Chicago Press, Chicago, pp 55-71 Park JB, Ryu TK, Gibson DV (2010) Facilitating public-to-private technology transfer through consortia: initial evidence

from Korea. J Technol Transf 35(2):237-252 Pastor J, Pérez F, de Guevara J (2013) Measuring the local economic impact of universities: an approach that considers

uncertainty. Higher Educ 65(5):539-564 Penfield T, Baker M, Scoble R, Wykes (2014) Assessment, evaluations, and definitions of research impact: a review. Res Eval 23:21-32

Perkmann M, Neely A, Walsh K (2011) How should firms evaluate success in university-industry alliances? A performance

measurement system. R & D Manag 41(2):202-216 Podsakoff PM, MacKenzie SB, Bacharach DG, Podsakoff NP (2005) The influence of management journals in the 1980s

and 1990s. Strateg Manag J 26:473-88 Powers JB, McDougall P (2005a) Policy orientation effects on performance with licensing to start-ups and small

companies. Res Policy 34(7): 1028-1042 Powers JB, McDougall P (2005b) University start-up formation and technology licensing with firms that go public: a

resource-based view of academic entrepreneurship. J Bus Venturing 20(3): 291-311 Pressman L, Guterman S, Abrams I, Geist D, Nelsen L (1995) Pre-production investment and jobs induced by MIT exclusive patent licenses: a preliminary model to measure the economic impact of university licensing. J Assoc Univ Technol Managers 7:49-82 Pries F, Guild P (2011) Commercializing inventions resulting from university research: analyzing the impact of

technology characteristics on subsequent business models. Technovation 31(4):151 -160 Probert J, Connell D, Mina A (2013) R&D service firms: the hidden engine of the high-tech economy? Res Policy 42(6-7):1274-1285

ProTon (1991-2012) The ProTon Europe annual survey reports. Brussels: ProTon Europe

Rasmussen E, Borch OJ (2010) University capabilities in facilitating entrepreneurship: a longitudinal study of spin-off

ventures at mid-range universities. Res Policy 39(5):602-612 REF 2014 (2011) Decisions on assessing research impact, (Pubd online) http://www.ref.ac.uk/media/ref/content/pub/

assessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf accessed 19 Dec 2012 Riddel M, Schwer RK (2003) Regional innovative capacity with endogenous employment: empirical evidence from the U.S. Rev Reg Stud 33:73-84

Roessner D, Bond J, Okubo S, Planting M (2013) The economic impact of licensed commercialized inventions

originating in university research. Res Policy 42:23-34 Roessner D, Manrique L, Park J (2010) The economic impact of engineering research centers: preliminary results of a

pilot study. J Technol Transf 35(5):475-493 Rogers E, Hall B, Hashimoto M, Steffensen M, Speakman K, Timko M (1999) Technology transfer from university-based

research centers: the University of New Mexico experience. J Higher Educ 70:687-705 Rogers E, Yin J, Hoffmann J (2000) Assessing the effectiveness of technology transfer offices at U.S. research universities.

J Assoc Univ Technol Managers 12:47-80 Romer P (1990) Endogenous technological change. J Pol Econ 98:S71 -S102

Rosenberg N, Nelson R (1994) American universities and technical advance in industry. Res Policy 23:325-348

Sáez CB, Marco TG, Arribas EH (2002) Collaboration in R&D with universities and research centres: an empirical study of

Spanish firms. R&D Manag 32:321-41 Salter A, Martin B (2001) The economic benefits of publicly funded basic research: a critical review. Res Policy 30:509-532

Sine WD, Scott S, Di Gregorio D (2003) The halo effect and technology licensing: the influence of institutional prestige

on the licensing of university inventions. Manag Sci 49(4):478-496 Sivitanidou R, Sivitanides P (1995) The intrametropolitan distribution of R&D activities: theory and empirical evidence. J Reg Sci 35:391-415

Smith S, Ward V, House A (2011) 'Impact' in the proposals for the UK's Research Excellence Framework: shifting the

boundaries of academic autonomy. Res Policy 40(10):1369-1379 Spencer J (2001) How relevant is university-based scientific research to private high-technology firms? A United States-Japan

comparison. Acad Manage J 44:432-40 Steinacker A (2005) The economic effect of urban colleges on their surrounding communities. Urban Stud 42:1161-75 Swann P, Prevezer M (1996) A comparison of the dynamics of industrial clustering in computing and biotechnology. Res Policy 25:1139-57

Tassey G (1999) Lessons learned about the methodology of the economic impact studies: the NIST experience. Eval Program Plann 22:113-119

Tassey G (2005) The disaggregated technology production function: a new model of university and corporate research. Res Policy 34(3):287-303

Tornquist K, Hoenack S (1996) Firm utilization of university scientific research. Res Higher Educ 37:509-34 Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management

knowledge by means of systematic review. Br J Manag 14:207-22 van Hemert P, Nijkamp P (2010) Knowledge investments, business R&D and innovativeness of countries: a qualitative

meta-analytic comparison. Technol Forecasting Soc Change 77(3):369-384 Varga A (2000) Local academic knowledge transfers and the concentration of economic activity. J Reg Sci 40:289-309 Villasana M (2011) Fostering university-industry interactions under a triple helix model: the case of Nuevo Leon, Mexico. Sci Public Policy 38(1):43-53

Vincett P (2010) The economic impacts of academic spin-off companies, and their implications for public policy. Res Policy 39:736-747

Wallmark JT (1997) Inventions and patents at universities: the case of Chalmers University of Technology. Technovation 17(3):127-139

Wang X, Zhang X, Xu S (2011) Patent co-citation networks of Fortune 500 companies. Scientometrics 88(3):761 -770 Weiler S (2000) Information and market failure in local economic development: a new role for universities? Econ Dev Q 14:194-203

Wernerfelt B (1984) A resource-based view of the firm. Strateg Manag J 5:171-180

Williamson OE (1979) Transaction cost economics: the governance of contractual relations. J Law Econ 23(2):233-261 Zucker L, Darby M, Brewer M (1998) Intellectual human capital and the birth of the U.S. biotechnology enterprises. Am Econ Rev 88:290-306

Submit your manuscript to a SpringerOpen journal and benefit from:

7 Convenient online submission 7 Rigorous peer review 7 Immediate publication on acceptance 7 Open access: articles freely available online 7 High visibility within the field 7 Retaining the copyright to your article

Submit your next manuscript at 7 springeropen.com