Scholarly article on topic 'Standards and Interoperability: How Jisc's Work Supports Reporting, Communicating and Measuring Research in the UK'

Standards and Interoperability: How Jisc's Work Supports Reporting, Communicating and Measuring Research in the UK Academic research paper on "Educational sciences"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia Computer Science
OECD Field of science
Keywords
{Standards / "digital identifiers" / interoperability / "research information management" / "open access" / "research data"}

Abstract of research paper on Educational sciences, author of scientific article — Tamsin Burland, Catherine Grout

Abstract The UK research sector utilises a highly varied infrastructure of research information management platforms. To reduce the administrative burden of managing, reporting and evaluating research, it has become crucial to enable system-to-system communication. Jisc is the UK's not-for-profit organisation supporting digital services and solutions in the UK research sector. Here, we describe the role Jisc has played in: • Promoting interoperability tofacilitate reliable and transparent research reporting; • Supporting institutions to meet, manage and demonstrate compliance to funders’ open access policies; • Supporting researchers and managers to assess the impact of research using open citation and download statistics.

Academic research paper on topic "Standards and Interoperability: How Jisc's Work Supports Reporting, Communicating and Measuring Research in the UK"

(8)

CrossMark

Available online at www.sciencedirect.com

ScienceDirect

Procedía Computer Science 106 (2017) 276 - 282

13th International Conference on Current Research Information Systems, CRIS2016, 9-11 June

2016, Scotland, UK

Standards and interoperability: how Jisc's work supports reporting, communicating and measuring research in the UK

Tamsin Burland* and Catherine Grout

Jisc, Brettenham House (South Entrance), 5 Lancaster Place, London, WC2E 7EN, UK

Abstract

The UK research sector utilises a highly varied infrastructure of research information management platforms. To reduce the administrative burden of managing, reporting and evaluating research, it has become crucial to enable system-to-system communication. Jisc is the UK's not-for-profit organisation supporting digital services and solutions in the UK research sector. Here, we describe the role Jisc has played in:

• Promoting interoperability to facilitate reliable and transparent research reporting;

• Supporting institutions to meet, manage and demonstrate compliance to funders' open access policies;

• Supporting researchers and managers to assess the impact of research using open citation and download statistics.

©2017 The Authors.Publishedby ElsevierB.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-reviewunderresponsibilityofthe Organizing CommitteeofCRIS2016

Keywords: Standards; digital identifiers; interoperability; research information management; open access, research data

* Corresponding author. Tel.: +44 (0)20 3819 8293 E-mail address: tamsin.burland@jisc.ac.uk

1877-0509 © 2017 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the Organizing Committee of CRIS2016

doi:10.1016/j.procs.2017.03.026

1. Introduction

The UK research information management landscape is highly complex, and is populated by a large number of research organisations, research funders, learned societies and commercial publishers. More than 60 research institutions (including most research-intensive universities) have installed proprietary Current Research Information Systems (CRIS) supplied by one of three vendors: PURE (Elsevier), CONVERIS (Thomson Reuters), or Symplectic Elements. Others use open source solutions, while smaller and specialist institutions rely on a repository and simple database or spreadsheet for information storage. Other stakeholders, such as funders and publishers, employ various systems, both proprietary and locally built.

UK research is operating under an increasing number of reporting and compliance requirements. Additionally, there is intense focus on institutional and individual research performance. Both these factors necessitate effective management and frequent sharing of research information. However, the wide range of co-existing systems means that, nationally, the picture is a patchwork of highly varied services and infrastructure. There are multiple overlapping or competing workflows utilising the same entities (e.g., people, organisations, research outputs etc), but exchange of information between systems frequently necessitates manual data re-entry or bulk upload. To halt the ever increasing administrative burden of managing research information, it is crucial that system-to-system communication is enabled throughout the sector.

Jisc is a not-for-profit organisation that supports digital services and solutions in UK higher and further education establishments. It is an integral part of the UK research community, dedicated to the sector's individual and collective needs. Jisc is uniquely placed to lead and support work on standards and interoperability in the UK (and beyond) and works together with universities and research institutes, government bodies, funders, publishers and vendors on the practical and innovative application of digital technologies to research.

Using a standard workflow approach to the challenges faced by the sector, Jisc gathers intelligence, builds consensus, commissions work, trials approaches within the sector and publishes the outputs (e.g. application profiles, entries in CASRAI Data dictionary, plug-ins, guidelines or specifications for how services interact with standards). Here, we describe the role that Jisc has played in promoting standards, universal identifiers and interoperability between research institutions' research information management systems, their repositories and the funder and publisher systems.

2. Standards and identifiers

The Wilsdon Review of Metrics in Research1 called for improvement to the UK's research information management infrastructure to facilitate the appropriate use of research information in the governance, management and assessment of research. Pivotal to this is the role played by universal key standards and identifiers to ensure a 'common vocabulary' and promote interoperability between systems.

Alongside funders, universities, standards bodies and system vendors, Jisc is playing an important role in working towards common standards and identifiers. This work will realise a wide range of benefits within the research information management landscape, including the ability to track and share information between systems, easier compliance and reporting processes and more reliable indicators and, hence, business intelligence.

2.1. Working with CASRAI

The Consortia Advancing Standards in Research Administration Information (CASRAI) is a standards development organization which maintains a common data dictionary and advances best practices for research information exchange and reuse. Jisc worked with CASRAI to trial the 'CASRAI approach' to the UK research information management sector. Using a number of working groups, the trial investigated three topics -Organisational Identifiers, Data Management Plans and Open Access (OA) reporting. Following this work, a way forward has been proposed for Organisational Identifiers (see below). In addition, OA Article Processing Charge (APC) descriptions have been added to the CASRAI data dictionary and descriptions relating to Research Councils UK (RCUK), the UK's Research Excellence Framework (REF) and OpenAIRE are to be added in the near futur e. A

new CASRAI-UK National Chapter has now been established which will continue to work to identify topics where applying the CASRAI approach could be most useful.

2.2. CERIF

The Common European Research Information Format (CERIF) is a relational data model for all aspects of research information, such as people, organisations, projects, funding and publications. It is used widely across Europe in research information management systems, and is maintained and developed by euroCRIS, of which Jisc is a strategic member. The main proprietary CRIS systems in the UK are CERIF-compliant, and Jisc has been a leading force in investigating the use of CERIF to facilitate information exchange. The Jisc 'CERIF in Action' project created plug-ins for Pure, Symplectic and ePrints to facilitate exchange of research information using between institutions and major funders. Additionally, the use of CERIF in interoperability between institutions and major funders was also considered by several working groups during the Overview of Systems Interoperability Project (OSIP,see below).

2.3. Research information identifiers

Unique identifiers enable systems to manage, track and exchange records of specific entities. This aids the management, reporting and exchange of research information, and ensures that any information upon which reports and analyses are built is reliable. Jisc has been pivotal in promoting key identifiers in research information management.

2.3.1. Researcher identifiers

ORCID provides researchers with a unique, persistent identifier which can be used across multiple systems. In 2011, Jisc led a broad group of UK sector bodies in issuing a joint statement expressing support for the ORCID initiative. A Jisc pilot subsequently implemented ORCID in the systems of 8 UK research organisations, and delivered a cost-benefit analysis of this implementation. Following on from this successful pilot, Jisc now operates a UK national consortium arrangement with ORCID, allowing research organisations, funders and publishers within the UK to easily access and manage information about researchers and their outputs.

2.3.2. Organisational identifiers

The Jisc CASRAI-UK Pilot Organisational Identifiers Working Group was established to develop a sustainable process for maintaining authoritative lists of organisations in the CASRAI dictionary. The group concluded that no single candidate was found to fulfil all criteria or use cases, and it would be useful to separate the infrastructure element (the provision and maintenance of the identifier) and the service element (the services offered both to registrants and to end users of the identifier). Given this, the group recommended a hybrid approach, which uses the International Standard Name Identifier (ISNI) as an open and non-proprietary identifier, with specific registration agencies within the UK (commercial and non-commercial) providing an interface between organisations and ISNI.

ISNI takes an international approach to organisational identifiers. Related information held in different systems (including identifiers used by registration agencies) are linked under the ISNI identifier which acts as a universal, persistent bridging identifier, facilitating interoperability between different systems.

Many issues remain to be resolved. These include: questions over governance; dealing with complex organisations (where hierarchies of departments, divisions or colleges can be fluid); dealing with potentially very large numbers and high turn-over of organisations; identifying responsibilities for clean-up of existing information; interaction between ISNI and identifiers used in institutional finance systems; and, the issues around the adoption of ISNI, particularly as clean-up and maintenance of organisational identifier lists may be labour intensive. Future work by Jisc in the area of organizational identifiers may include:

• A pilot clean-up of a closed set of UK institution data;

• Investigating a bulk deal for UK organisations to use the services of commercial registration agencies;

• The feasibility of a UK-based registration agency facilitating bulk creation/checking of ISNIs for UK

organisations;

• A study of international use cases, such as where the ISNI API is already in use.

It is imperative that a sustainable and international approach is taken to this issue, and that other initiatives, such as the EU funded Technical and Human Infrastructure for Open Research project (THOR) are involved.

2.3.3. Digital object identifiers

DOIs are now commonly used for identifying journal articles and Jisc publically supported and welcomed the new Crossref guidance on early assignment of DOIs for journal articles2. Jisc is also involved in investigating the feasibility of DOIs for other research outputs. This includes promoting the adoption of DataCite DOIs for research data and supporting activity to seek common identifiers for research software outputs through the Jisc Research Data Spring programme.

3. Promoting interoperability and ensuring reliable and transparent research reporting

Within the UK, research performing organisations are required to submit to the REF evaluation every five to six years. While this evaluation is primarily based on identifying excellence through peer review, the REF also requires submission of information about the researchers, their outputs, students and research income. As a result, a REF submission is very 'information heavy' and during the last REF research institutions incurred a total of around £232M in preparation and submitting costs.

The burden of research reporting in the UK is not limited to the REF. Researchers and research managers report to both their institutions (for individual and departmental performance management purposes) and to a variety of funders (on research outcomes, impact and, increasingly often, OA compliance and expenditure).

While reporting and evaluation represent a significant burden to research institutions in terms of research information management, the need to collect and manage reliable information across the sector is critical. High quality information plays a vital role at an institution and national level, informing decision making, underpinning accountability, promoting public confidence in research integrity and influencing public policy.

3.1. The Overview of Systems Interoperability Project (OSIP)

OSIP was a joint project led by Jisc and RCUK, who are the distributors of the majority of project-specific government research funding within the UK. Currently, RCUK uses three systems to manage its research grants and awards; a submission system (Je-S), a research outcomes collection system (ResearchFish) and a system for making grant and outcome information available for re-use (Gateway to Research, GtR). The aim of OSIP was to enable systematic use and re-use of information held in these systems, and in doing so, minimise the reporting burden on researchers and research managers.

The project report3 emphasised the central role that universal identifiers and standards should play in promoting system interoperability and ensuring reliable and transparent research reporting. Crucially, RCUK committed to the introduction of both individual (ORCID) and organisational (ISNI) identifiers throughout their systems. RCUK joined the ORCID consortium in December 2015 and ORCID has been operational in Je-S since May 2016.

The use of ORCID in Je-S allows investigators to automatically import their research outputs into new grant applications. Looking forward, ORCID may also provide opportunities for the enhancement of function of ResearchFish and GtR. For ResearchFish, it could provide the opportunity to move away both from the current requirement for researchers to manually key their research output information, and from a future bulk upload model of data transfer, which has been recently piloted. For GtR, introduction of ORCID and ISNI identifiers could be a vital step in realising RCUK's vision of GtR as a single portal for accessing information about all RCUK funded research. In particular, these identifiers could facilitate sharing of information to other systems via APIs. GtR may also play a role in the development of resolvable digital identifiers for funded research projects.

3.2. RIOXX

Jisc has worked with RCUK and the Higher Education Funding Council for England (HEFCE) to define and standardise a set of key metadata to enable research organisations to manage their OA compliance reporting. The resulting 'RIOXX' profile4 defines the metadata that institutional repositories require in order to demonstrate compliance with the funder OA policies. RIOXX can also be mapped to OpenAIRE to fulfil EU OA requirements. Jisc has also recently launched the RIOXX-based REF Compliance Checker which can establish whether a deposited article will be eligible for the next REF exercise.

4. Supporting research institutions in implementing open access (OA)

The benefits of OA to scholarly literature and data in increasing visibility of research, accelerating research and enriching learning are now widely recognised. Within the UK, most major research funders require research articles originating from work they have funded to be made open access. Most notably, in March 2014, the UK National Funding Councils announced a new policy for open access for the next REF. This policy requires that, to be eligible for submission, journal articles must be available in OA form, either: uploaded to an institutional or subject repository (within a set time frame); or be published using the 'Gold' route via a payment of an APC.

An increasing number of funders are also introducing data management policies, with the aim that the research data that underpins a journal article is made available in an accessible and discoverable form.

The challenge for universities is to meet, manage and demonstrate compliance of funder and publisher OA policies within a complex research information environment. Through Jisc's extensive work with the UK research community, a suite of support tools and guidance for researchers and institutions in managing OA information throughout the entire research lifecycle have been developed.

4.1. Journal articles

4.1.1. Compliance to OA policies

Jisc's Sherpa services5 are online tools to access, interpret, and comply with funders' and publishers' OA policies. These services provide authoritative information on the OA policies of funders and publishers which allow authors to check prior to submission which journals will allow them to comply with their funder's policy. The most recent service is Sherpa REF, the beta version of which is now live. This service will help researchers and institutions identify whether a specific journal complies with the OA REF policy.

4.1.2. Automatic notification to institutions.

Part of the burden on institutions of OA compliance stems from identifying when and where research is being published, and submission of this information to the repository and/or CRIS. Jisc is developing a 'Publications Router' which automates the process of publishers (or other content providers) notifying institutions about accepted research articles by passing metadata and/or the full text article directly from publishers to the institutions' research information systems. Where applicable, the publishers' embargo end-dates are included in the metadata. The beta version of Publications Router is now testing with pilot institutions6.

4.1.3. Managing OA information

Institutions need to be able to track articles through the OA lifecycle in order to ensure compliance and monitor expenditure. This involves gathering information from a range of sources. As part of the Jisc Monitor suite, due for release in 2016, Montior Local has created a workflow support application for institutions to collate and manage information from multiple sources (automated where possible). This may include information on research funders, any block grants for OA, journals and publishers, APC payments and the compliance status of an article.

The sister service, Monitor UK will aggregate information from institutions (either via Monitor Local or other information sources) in order to evaluate cost and compliance at the national level. This will provide business intelligence for institutions to compare and benchmark Gold OA information (such as cost and volume of APCs) against other institutions or national averages.

4.1.4. Discovering and accessing an article

Funder OA policies require that articles within repositories are discoverable. As part of Jisc's Sherpa services, the Directory of Open Access Repositories (OpenDOAR7) provides support to repository managers to improve the quality of repository infrastructure. Once an institution's repository is registered with OpenDOAR, OA articles with the correct metadata will be discoverable worldwide via the Jisc/Open University service, CORE (COnnecting REpositories) 8. CORE is an aggregation of OA content from repositories and open access journals worldwide and provides unrestricted access to 24 million metadata records for OA outputs and 1.8 million full text outputs. The CORE API facilitates text mining, the linking of publications to research data and citation analyses of OA articles.

4.2. Research Data

Research data management and re-use is an emerging issue that universities need to address. Jisc has a major research and development stream of activity (Research at Risk) focused on this. The elements of this stream that are specifically relevant to open access are highlighted here.

4.2.1. Research data metadata and discovery

A system for the collection, managing and exposing of appropriate metadata is a crucial component for interoperability in research data systems and discoverability of research data9. Current approaches to metadata collection and management are varied, and may include use of existing or additional institutional repositories, or the use of a CRIS. As part of the Research Data Discovery project, Jisc is investigating the development of a discipline neutral schema for research data across the lifecycle which will facilitate interoperability across RDM systems and services, as well as aiding discovery. An alpha version of the UK Research Data Discovery Service is now live10. Jisc's involvement in the development of DOIs for data is discussed above in Section 2.3.3.

4.2.2. Journal research data policy registry

Jisc is helping to develop a website that curates a collection of the journal policies about how to submit, curate and allow access to the research data underpinning journal articles. However, this is proving extremely difficult due to the subjective and inconsistent nature of the available policies.

5. Assessing the impact of research outputs using open citation and download statistics.

The measurement and assessment of research, and its academic and societal impact, should use processes and measures that accurately reflect individual, discipline and institutional performance1. Open research practices are now providing opportunities to exploit download metrics and novel forms of citation data for use in establishing open research activity indicators.

5.1. Article Downloads

Jisc's usage statistics service IRUS-UK11 is a platform-neutral service that allows institutions to measure the usage of their journal articles. The service collects raw data from UK institutional repositories to provide COUNTER-compliant repository download metrics to more than two thirds of eligible UK repositories. Recently implemented technology enables downloads from CORE to be included in these reports.

5.2. Article Citations

Article metadata harvested by CORE, plus access to full text OA articles, is now providing a source of data for the development of article citation indicators based on the full text via text mining (Semontometrics12). Jisc has supported an experiment around this open, semantic citation dataset that aims to provide transparent article-level citation indicators13.

5.3. Research Data

Interest in reliable usage indicators for research data is rapidly growing. As in other areas of measuring and assessing research quality, well-designed indicators will be invaluable. As part of the Research at Risk project, Jisc is piloting a COUTNER-compliant research data download metrics service (IRUSdata-UK) which measures downloads at a file rather than an item level. This allows more accurate usage measures of large, multi-part datasets that can make up a single deposit. Jisc is also playing a role in a global effort to identify the best way to cite data and understand the demand for other forms of data usage metrics.

6. Conclusions

The UK research information management landscape is rapidly evolving, driven by both advances in technology and the increased burden of the regulatory and assessment framework within which institutions are operating. To operate more efficiently, further work in the following areas is imperative:

• Development of standardised terms, definitions and identifiers to underpin the research information management systems network;

• Development of open protocols to allow easy machine-readable access to, and exchange of, information currently held in the wide variety of existing systems.

• Development of open and reliable measures of research activity, which facilitate reliable reporting, assessment and analysis of research quality.

Acknowledgements

The authors wish to gratefully acknowledge the input of numerous external partners and the Jisc colleagues involved in the projects described here.

References

1. Wilsdon, J et al. The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management 2015;

DOI: 10.13140/RG.2.1.4929.1363

2. https://www.jisc.ac.uk/blog/new-publisher-led-initiatives-to-support-reporting-to-funders-21-mar-2016

3. Lyne I, Grout C, Notay B. Overview of Systems Interoperability Project. Report to the RCUK Research Funding Programme Board 2015;

http://www.rcuk.ac.uk/documents/publications/osipreport-pdf/

4. http://rioxx.net/

5. https://jisc.ac.uk/sherpa

6. https://pubrouter.jisc.ac.uk/

7. http://www.opendoar.org/

8. https://www.jisc.ac.uk/core

9. Brown S, Bruce R, Kernohan D. Directions for Research Data Management in UK Universities 2015;

http ://repository.jisc.ac.uk/5951/4/JR0034_RDM_report_200315_v5 .pdf

10. http://ckan.data.alpha.jisc.ac.uk/dataset

11. http://www.irus.mimas.ac.uk/

12. Knoth P, Herrmannova D. Towards Semantometrics: A New Semantic Similarity Based Measure for Assessing a Research Publication's

Contribution. D-Lib Magazine 2014; 20(11/12) http://www.dlib.org/dlib/november14/knoth/11knoth.html

13. Herrmannova D, Knoth P. Towards full-text based research metrics: Exploring semantometrics. Report of experiments 2016; http://repository.jisc.ac.uk/6376/1/Jisc-semantometrics-experiments-report-final.pdf