Last week I had the opportunity to attend the Jisc/CNI conference in Oxford, England, and share findings from our recent studies of academics in the US and in the UK. The conference focused on current issues and innovations in digital scholarship and allowed for international exchange on leading practices and policies. A number of themes emerged within and across the various sessions I attended.

Open data

Many speakers and attendees discussed the “openness” of research data, including discussion of what we mean when we refer to “open” data, and how open these data should be.

Professor Sir Nigel Shadbolt from Jesus College stressed a need for the development of common vocabulary for speaking about open data, and proposed language for the “data spectrum” ranging from closed to shared to open:

jisccni1

 

Shadbolt elaborated on a number of innovative UK-based projects in which open data has been used to improve healthcare services, discover cures for diseases, and improve ease of travel. In one such project, using open, non-personally identifiable information on patients’ prescriptions, it was determined that generic statins (a medication used to lower cholesterol) were just as effective as branded, more expensive statins. These findings provide an opportunity for physicians to make more informed decisions and could lead to enormous savings (and allow for investment in other healthcare innovations) if there was a major shift towards prescribing only generic forms of the medication.

Research incentives

As Shadbolt succinctly stated in his keynote address on the need to change behavior around making data more open: human behavior can so often be explained by examining incentives.

In a session on incentives for modern research, Veerle Van den Eynen from UK Data Service shared findings from a recent study of incentives and motivations from the perspective of the researcher. As Van den Eynen explained, in order to influence behavior when policies around data sharing are weak or not present, we must change norms, increase benefits associated with sharing, and strengthen leadership from funders and institutions.

In his talk on “doing science” in the digital age, Neil Chue Hong from the Software Sustainability Institute described a need for change within research culture and incentives. While researchers receive credit for publishing papers, getting grants, and perhaps societal impact, credit is often not assigned for releasing software, making software easy to use, supporting software for others to use, investing effort in learning new tools, and being helpful. While there is a fear of being “found out” for poor software practice, there currently isn’t much reward for publishing software and many organizations are unaware of how to support researchers developing and publishing software.

In our recent studies of academics in the US and in the UK, we found that traditional scholarly incentives still motivate research behavior. Respondents overwhelmingly agreed that more recognition should be awarded for traditional research publications, such as journal articles and academic books, as compared to many other types of research products – although there is a sizable share of respondents that believe that software and code merit the same amount of recognition (see Figure 1).

And while there is increased interest in the UK in reaching professionals outside of academia, undergraduate students, and the general public beyond the academic and associated professional community, with one’s research, we still see that academics most value reaching scholars within one’s own discipline or subdiscipline (see Figure 2).

Figure 1: “And, when you think about how your work is assessed, such as for appointment, promotion, or research funding, how much recognition should you receive for your research products compared to traditional research publications such as journal articles and academic books?” Percentage of UK respondents that indicated that they should receive less or more recognition for each of these research products.

jisccni2

 

Figure 2: How important is it to you that your research reaches each of the following possible audiences? Percentage of UK respondents who indicated that each of the following is a very important audience.

jisccni3

Research analytics

In his introductory message to attendees, Clifford Lynch, CNI Executive Director, emphasized the need to thoughtfully measure impact and assess outcomes in light of the increased use of analytics. As Lynch explained, and as other echoed throughout the conference, poor measures can lead to distorted behavior and create distorted systems.

In our recent study of UK academics, we have found a substantial increase in the share of respondents who shape their research to match the criteria they perceive for success in tenure and promotion processes; in 2015, 41% of respondents strongly agreed that they shape their research in this way as compared to 25% in our 2012 study. In the US, we found that 52% of respondents strongly agreed in 2015 – and that respondents that identified as female indicated substantially higher levels of agreement than those who identified as male (59% vs. 47%).

In light of the large share of respondents that shape their research publications and outputs in this manner, it is imperative that the measures used to evaluate this research are developed in a way that considers the kind of behavior that will be motivated.

Overall, the Jisc CNI conference provided a productive forum for understanding differing international perspectives on these topics and allowed for attendees to reflect on how collaborations across academics, institutions, and nations could best be facilitated to advance local and more wide-reaching practices and policies.

***

My slides from the conference are available at http://www.sr.ithaka.org/wp-content/uploads/2016/07/Equipping-the-researcher-patterns-in-the-UK-and-US_06072016.pdf .