Lessons for policy-related research are provided in their revisited blog post by Obasanjo Oyedele, Martin Atela and Ayo Ojebode at https://i2insights.org/2017/11/14/early-stakeholder-involvement/. Lesson 1: your research is your business, not the policy actors' priority, so be prepared to work to generate enthusiasm. Lesson 2: inertia or a cold shoulder does not mean a dead end, be prepared to be creative in generating interest. Perseverance, creativity, determination and communication are also required, eg try moving from a "partnership to an "at-your-service" mode of engagement.
===================================================
Professor Gabriele Bammer
National Centre for Epidemiology and Population Health
Research School of Population Health
ANU College of Health and Medicine
The Australian National University
62 Mills Road
Acton ACT 2601
+61 2 6125 0716
Gabriele.Bammer(a)anu.edu.au<mailto:Gabriele.Bammer@anu.edu.au>
@GabrieleBammer
http://i2s.anu.edu.au<http://www.anu.edu.au/iisn>
http://I2Insights.org
CRICOS Provider # 00120C
===================================================
Checklists for assessing uncertainty-awareness-ambiguity in decision making are provided by Fabio Boschietti at https://i2insights.org/2020/07/28/uncertainty-awareness-ambiguity/. Uncertainty-awareness-ambiguity provides a 3-dimensional space for mapping knowledge and there are simple questions for assessing them. Key questions are: What would change my mind? What evidence, novel insight or alternative framing would lead me to reconsider my conclusion? The blog post provides the checklists and your comments are welcome.
===================================================
Professor Gabriele Bammer
National Centre for Epidemiology and Population Health
Research School of Population Health
ANU College of Health and Medicine
The Australian National University
62 Mills Road
Acton ACT 2601
+61 2 6125 0716
Gabriele.Bammer(a)anu.edu.au<mailto:Gabriele.Bammer@anu.edu.au>
@GabrieleBammer
http://i2s.anu.edu.au<http://www.anu.edu.au/iisn>
http://I2Insights.org
CRICOS Provider # 00120C
===================================================
Today's #sosdci #POR comes from Florenta Teodoridis (University of Southern California) regarding the project: "Collaborative Research: The Impact of Research Costs on the Rate and Direction of Scientific Discovery".
Abstract at Time of Award
Understanding scientific and technical progress requires measures of both the rate and direction of resources directed to innovative efforts and the outputs produced. This project leverages advances in computational power to map the evolution of research fields based on researchers? project portfolios in ideas spaces. The research takes advantages of shocks, such as changes in policies and research costs, to use empirical research techniques to understand the factors that affect the directions into which science and technology evolve. This work has important implications for understanding the rate and direction of technological change. Specifically, the project uses techniques based on machine learning and natural language processing that measure the incidence and configuration of keywords in published research to quantify the similarity of groups of such articles to define idea space. And to subsequently measure the ways in which idea space evolve in response to shocks in research costs and public policies. The research applies these techniques to three contexts: (a) how changes in the costs of research materials affect research trajectories in motion-sensing technology, (b) how researchers in quantum computing change their project portfolios in response to a controversial approach that differs from an established research paradigm; and (c) how pharmaceutical firm research trajectories change in response to news about rivals? drug discovery projects
Publications Produced as a Result of this Research
Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from this site.
Jeffrey L. Furman and Florenta Teodoridis "Machine Learning Could Improve Innovation Policy" Nature Machine Intelligence, v.2, 2020, p.. doi:https://doi.org/10.1038/s42256-020-0155-8<https://www.research.gov/research-portal/exit.jsp?link=http%3A%2F%2Fdx.doi.…>
Florenta Teodoridis "Understanding Team Knowledge Production: The Interrelated Roles of Technology and Expertise" Management Science, v., 2017, p..
Florenta Teodoridis "Understanding Team Knowledge Production: The Interrelated Roles of Technology and Expertise" Journal, v.64, 2018, p.3469. doi:3970<https://www.research.gov/research-portal/exit.jsp?link=http%3A%2F%2Fdx.doi.…>
Jeffrey L. Furman and Florenta Teodoridis "Automation, Research Technology and Researchers' Trajectories: Evidence from Computer Science and Electrical Engineering" Journal, under review, v., 2020, p..
Project Outcomes Report
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
The major goals of this project include analyzing the impact of scientific and technical progress on the rate and direction of innovative efforts and outputs. Specifically, our goal is to leverage advances in computing power and empirical research techniques to map the evolution of research fields and researchers' project portfolios in "ideas space."
As part of this project, our research develops machine learning based techniques that we employed to evaluate the ways in which sets of ideas evolve in response to shocks in information, research costs, and public policies. Our findings include:
* The automation of research tasks that previously required substantial human labor presents opportunities for scientists to expand their research portfolios and for policymakers to accelerate the rate and broaden the direction of knowledge accumulation. Our research shows that, contrary to concerns that automating technology displaces individuals from work tasks, such an automation increased the production of ideas and induced researchers to pursue ideas more diverse than and distant from their original trajectories.
* Research technology costs influence cumulative innovation by altering the composition of expertise in teamwork. Sufficiently large reductions in the cost of research technology leads to greater collaborations across research domains.
* Risk aversion leads pharmaceutical firms to underinvest in radical innovation. Also, pharmaceutical firms are more likely to terminate clinical trials in response to learning about the failure of rival firms' drugs and to turn to the market to acquire drug projects from other firms, rather than developing internally.
Last Modified: 07/06/2020
Modified by: Florenta Teodoridis
Thanks for sharing, Carolyn. This is an important finding about what is happening when science responds to a global crisis. Finding that researchers seem to rely on familiarity during emergencies, shows that they are not really different from most others. More interesting are the findings about small teams, suggesting, also, a tie to research in other fields on teams and adaptability. Finally, most important are the results about elite universities. As you note, relying on reputation, rather than what might be more relevant expertise, or, in fact, greater talent, at non-elite universities, is another marker for future study. So this is an informative bibliometric analysis contributing to our understanding of science at a macro-level.
But, in your paper, you note that researchers lack a clear focus. How do you know that based upon analyzing a distribution of publications? Focus depends on the level of analyses. The teams, themselves, I’m sure, were quite focused in whatever it was they studied. Perhaps, at the level of the ‘field’, there was what appears to be a lack of focus. But, at the same time, one person’s lack of focus (or field's) is another’s exploration stage. With that said, in the paper, there are also speculations about coordination costs. How do we know these teams had high coordination costs? They were not queried about them. Similarly, the paper says that teams weighed an inherent tradeoff about their collaboration choice. Again, how do we know that is the case? Inferring psychological processes from citation analyses seems problematic - in the absence of asking the scientists, we remain too disconnected from those actually doing the work.
In short, although we have some sense of ‘what’ is occurring, what this type of analysis does not help us understand is any of the “why” these findings are occurring. Short of the exogenous trigger event, a global pandemic, we have no sense of the causal factors for any collaboration outcomes, let alone, processes. The paper speaks of team structure, but from the study of teams, we know there is more than simply size or international membership to team structure. Said another way, each one of the papers analyzed has a team behind them, and, associated with it, a rich set of interaction processes, a particular form of interdependency, a unique hierarchy, a composite of complementary cognition, shared and unshared attitudinal profiles, etc. etc. None of these can be understood by only looking at publications. Although bibliometrics and scientometrics studies, with their tens of thousands of data points, might be able to inform macroscopic patters of collaboration, they are but one lens onto this phenomenon. The Science of Team Science was specifically created to provide the theoretical and methodological foundation through which to understand not only the “what” of teamwork, but also the “why”. Behind every single one of the thousands of pre-prints and publication making up a bibliometric data set, there are hundreds of hours of individual work and teamwork. Although it is fine to speculate about the macro-phenomena bibliometrics and scientometrics can uncover, such analyses can only go so far. And any recommendations about policy decisions should be similarly tempered. These types of analyses are not appropriate for making inferences about the actual teamwork processes and emergent phenomena occurring at the level of the team, that is, the rich world of interaction ‘in’ the teams producing (or failing to produce) publications that are treated merely as a single data point for citation analyses in bibliometrics studies.
Best,
Steve
--------
Stephen M. Fiore, Ph.D.
Professor, Cognitive Sciences, Department of Philosophy
<http://philosophy.cah.ucf.edu/staff.php?id=134>
Director, Cognitive Sciences Laboratory, Institute for Simulation & Training (http://csl.ist.ucf.edu/)
<http://philosophy.cah.ucf.edu/staff.php?id=134>
<http://philosophy.cah.ucf.edu/staff.php?id=134>
University of Central Florida
sfiore(a)ist.ucf.edu
________________________________
From: A public forum for scientists. <scientists(a)sciencelistserv.org>
Sent: Thursday, July 23, 2020 1:04 PM
To: scientists(a)sciencelistserv.org <scientists(a)sciencelistserv.org>
Subject: Message: 9
Dear Friends,
Our paper on international collaboration on COVID-19 research is published now in PLOS One:
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0236307
Abstract:
This paper seeks to understand whether a catastrophic and urgent event, such as the first months of the COVID-19 pandemic, accelerates or reverses trends in international collaboration, especially in and between China and the United States. A review of research articles produced in the first months of the COVID-19 pandemic shows that COVID-19 research had smaller teams and involved fewer nations than pre-COVID-19 coronavirus research. The United States and China were, and continue to be in the pandemic era, at the center of the global network in coronavirus related research, while developing countries are relatively absent from early research activities in the COVID-19 period. Not only are China and the United States at the center of the global network of coronavirus research, but they strengthen their bilateral research relationship during COVID-19, producing more than 4.9% of all global articles together, in contrast to 3.6% before the pandemic. In addition, in the COVID-19 period, joined by the United Kingdom, China and the United States continued their roles as the largest contributors to, and home to the main funders of, coronavirus related research. These findings suggest that the global COVID-19 pandemic shifted the geographic loci of coronavirus research, as well as the structure of scientific teams, narrowing team membership and favoring elite structures. These findings raise further questions over the decisions that scientists face in the formation of teams to maximize a speed, skill trade-off. Policy implications are discussed.
Happy to have comments.
Caroline
Caroline S. Wagner
Milton & Roslyn Wolf Chair in International Affairs
John Glenn School of Public Affairs
The Ohio State University
Columbus, Ohio USA 43210
ORCID <div itemscope itemtype="https://schema.org/Person"><a itemprop="sameAs" content="https://orcid.org/0000-0002-1724-8489" href="https://orcid.org/0000-0002-1724-8489" target="orcid.widget" rel="me noopener noreferrer" style="vertical-align:top;"><img src="https://orcid.org/sites/default/files/images/orcid_16x16.png" style="width:1em;margin-right:.5em;" alt="ORCID iD icon">https://orcid.org/0000-0002-1724-8489</a></div>
Improving health & social care is discussed in two revisited blog posts 1) on Experience-Based Co-Design by Glenn Robert and Annette Boaz & 2) how to accelerate that process by Louise Locock at https://i2insights.org/2016/10/20/experience-based-co-design/ & https://i2insights.org/2017/10/24/measuring-versus-improving/. Key requirements for patients are respect & dignity, compassion, timeliness & convenience, information & communication, and meaningful involvement in decision-making.
===================================================
Professor Gabriele Bammer
National Centre for Epidemiology and Population Health
Research School of Population Health
ANU College of Health and Medicine
The Australian National University
62 Mills Road
Acton ACT 2601
+61 2 6125 0716
Gabriele.Bammer(a)anu.edu.au<mailto:Gabriele.Bammer@anu.edu.au>
@GabrieleBammer
http://i2s.anu.edu.au<http://www.anu.edu.au/iisn>
http://I2Insights.org
CRICOS Provider # 00120C
===================================================
Dear Friends,
Our paper on international collaboration on COVID-19 research is published now in PLOS One:
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0236307
Abstract:
This paper seeks to understand whether a catastrophic and urgent event, such as the first months of the COVID-19 pandemic, accelerates or reverses trends in international collaboration, especially in and between China and the United States. A review of research articles produced in the first months of the COVID-19 pandemic shows that COVID-19 research had smaller teams and involved fewer nations than pre-COVID-19 coronavirus research. The United States and China were, and continue to be in the pandemic era, at the center of the global network in coronavirus related research, while developing countries are relatively absent from early research activities in the COVID-19 period. Not only are China and the United States at the center of the global network of coronavirus research, but they strengthen their bilateral research relationship during COVID-19, producing more than 4.9% of all global articles together, in contrast to 3.6% before the pandemic. In addition, in the COVID-19 period, joined by the United Kingdom, China and the United States continued their roles as the largest contributors to, and home to the main funders of, coronavirus related research. These findings suggest that the global COVID-19 pandemic shifted the geographic loci of coronavirus research, as well as the structure of scientific teams, narrowing team membership and favoring elite structures. These findings raise further questions over the decisions that scientists face in the formation of teams to maximize a speed, skill trade-off. Policy implications are discussed.
Happy to have comments.
Caroline
Caroline S. Wagner
Milton & Roslyn Wolf Chair in International Affairs
John Glenn School of Public Affairs
The Ohio State University
Columbus, Ohio USA 43210
ORCID <div itemscope itemtype="https://schema.org/Person"><a itemprop="sameAs" content="https://orcid.org/0000-0002-1724-8489" href="https://orcid.org/0000-0002-1724-8489" target="orcid.widget" rel="me noopener noreferrer" style="vertical-align:top;"><img src="https://orcid.org/sites/default/files/images/orcid_16x16.png" style="width:1em;margin-right:.5em;" alt="ORCID iD icon">https://orcid.org/0000-0002-1724-8489</a></div>
Regarding the science of science policy, this piece about Brazil that applies no less to the present US environment as well may be interesting to many:
When evidence does not matter - What Brazil teaches us about the fragility of evidence based policymaking
https://blogs.lse.ac.uk/impactofsocialsciences/2020/07/22/when-evidence-doe…
Regards,
Holly
Holly J. Falk-Krzesinski, PhD (she/her/hers)
Vice President, Research Intelligence
Co-chair, Gender Working Group
Advisory Board Member, International Center for the Study of Research (ICSR)
Global Strategic Networks | ELSEVIER
Adjunct Senior Instructor, Philanthropy and Nonprofit Organizations
School of Professional Studies | Northwestern University
453 Cedar Court South, Buffalo Grove, IL 60089 USA
+1 847-848-2953 Mobile | h.falk-krzesinski(a)elsevier.com<mailto:h.falk-krzesinski@elsevier.com>
Executive Assistant: Lisa Gill, lisa.gill(a)elsevier.com<mailto:lisa.gill@elsevier.com>, +1 212-633-3933
Today's #sosdci publication comes from Debasis Mitra and Qiong Wang:
Management of intellectual asset production in industrial laboratories
IISE Transactions
Abstract
Industrial laboratories generate profit for their parent companies and in so doing benefit society through spillovers of novel technologies and solutions. However, research's share of corporate investment in R&D has been declining. To understand this trend from the operations perspective, we develop a model-based analysis of the management of intellectual asset production in industrial laboratories. The model consists of a linear network with multiple stages in which the first stage is the research division engaged in generating novel concepts and prototypes. It is followed by multiple development stages that transform research outputs into intellectual assets and marketable products. Management is responsible for strategic budget allocation to the stages, and tactical management of individual projects. Decisions are based on intrinsic return on investment in the laboratory, and option values of projects, both of which are endogenously determined. Our model and analyses have revealed several possible pathways that can lead to the management of the laboratories to reduce the share of research spending in their budgets, namely: (i) lower variability of project values; (ii) improved investment efficiency at development stages; and (iii) higher revenue realization from assets produced at early development stages.
Cassidy R. Sugimoto
Professor of Informatics
School of Informatics, Computing, and Engineering
Indiana University Bloomington
http://ella.slis.indiana.edu/~sugimoto/index.php
Evolution of the science of team science in the last decade is described by Ying Huang, Ruinan Li, Yashan Li and Lin Zhang in their blog post at
https://i2insights.org/2020/07/21/hot-topics-in-team-science/ (also in Chinese). Five key topics are: 1) measurement & evaluation of team science, 2) institutional support & professional development for teams, 3) characteristics & dynamics of teams, 4) team management & organization, 5) team structure & context. Over time there are more connection between topics & less isolated topics. What key trends in the evolution of the field have you experienced? What new emerging topics do you predict for the science of team science?
===================================================
Professor Gabriele Bammer
National Centre for Epidemiology and Population Health
Research School of Population Health
ANU College of Health and Medicine
The Australian National University
62 Mills Road
Acton ACT 2601
+61 2 6125 0716
Gabriele.Bammer(a)anu.edu.au<mailto:Gabriele.Bammer@anu.edu.au>
@GabrieleBammer
http://i2s.anu.edu.au<http://www.anu.edu.au/iisn>
http://I2Insights.org
CRICOS Provider # 00120C
===================================================
Call for papers
Dear colleagues,
You are invited to participate in the 1st Workshop on Scholarly Document
Processing (SDP 2020) to be held in conjunction with the 2020 Conference on
Empirical Methods in Natural Language Processing (EMNLP 2020) on November
19. The workshop will be held VIRTUALLY with EMNLP 2020.
Important update:
-
The final submission deadline for research papers (research track) is August
15, 2020.
About the workshop:
The SDP 2020 workshop will consist of a research track and three
summarization shared tasks.
The shared tasks include the 6th edition of the CL-SciSumm shared task and
two new summarization tasks -- CL-LaySumm and LongSumm -- geared towards
easier access to scientific methods and results. <
https://ornlcda.github.io/SDProc/sharedtasks.html>
Detailed call for papers:
** Introduction **
In addition to the long-standing challenge faced by scholars of keeping up
with the growing literature in their own and related fields, they must now
compete with malign pseudo-science and disinformation in informing public
policy and behavior. This has stimulated workshops and research focused on
enhancing search, retrieval, summarization, and analysis of scholarly
documents. However, the general research community on scholarly document
processing remains fragmented, and efforts towards natural language
understanding of scholarly text that is central to vastly improve all the
said downstream applications are not widespread.
To address these gaps, we propose the first Workshop on Scholarly Document
Processing.
We seek to reach to the broader NLP and AI/ML community to pool the
distributed efforts to improve scholarly document understanding and enable
intelligent access to the published research. The goal of SDP is two-fold:
to increase collaboration between communities interested in leveraging
knowledge stored in scholarly literature and data, and to establish SDP as
the single-focused primary venue for the field.
We seek to appeal to the mainstream NLP and ML community working on SDP
tasks – which are NLP tasks – to publish at SDP as we seek to establish SDP
as the integrated premier venue. We have established a steering committee <
https://ornlcda.github.io/SDProc/steeringcommittee.html> to help us turn
SDP into a conference in the forthcoming years.
** Topics of Interest **
We invite submissions from all communities interested in natural language
processing, information retrieval, and data mining problems in scholarly
documents; and in processing scholarly documents for easier access to
various audiences. The topics of interest include, but are not limited to:
-
Information extraction, text mining and parsing scholarly literature
-
Reproducibility and peer review
-
Lay summarization (i.e., summaries created for non-experts) of
individual and collections of scholarly documents
-
Discourse modeling and argument mining
-
Summarization and question-answering for scholarly documents
-
Semantic and network-based indexing, search and navigation in structured
text
-
Graph analysis/mining including citation and co-authorship networks
-
Analysing and mining of citation contexts for document understanding and
retrieval
-
New scholarly language resources and evaluation
-
Connecting and interlinking publications, data, tweets, blogs or their
parts
-
Disambiguation, metadata extraction, enrichment, and data quality
assurance for scholarly documents
-
Bibliometrics, scientometrics, and altmetrics approaches and applications
-
Other aspects of scholarly workflows including open access/science, and
research assessment
-
Infrastructures for accessing scholarly publications and/or research data
-
Results and research questions on the COVID-19 Open Research Dataset
(CORD-19)
** Submission Information **
Authors are invited to submit full and short papers with unpublished,
original work. Submissions will be subject to a double-blind peer review
process. Accepted papers will be presented by the authors at the workshop
either as a talk or a poster. All accepted papers will be published in the
workshop proceedings.
Submission Website: Submission is electronic, using the Softconf START
conference management system: https://www.softconf.com/emnlp2020/sdp2020/
The submissions should be in PDF format and anonymized for review. All
submissions must be written in English and follow the EMNLP 2020 formatting
requirements: https://2020.emnlp.org/call-for-papers.
Long paper submissions: up to 8 pages of content, plus unlimited references.
Short paper submissions: up to 4 pages of content, plus unlimited
references.
Final versions of accepted papers will be allowed 1 additional page of
content so that reviewer comments can be taken into account.
** Important Dates **
Research track:
Submission deadline – August 15, 2020
Notification of Acceptance – September 29, 2020
Camera-ready submission due – October 10, 2020
Workshop – November 19, 2020
** SDP 2020 Keynote Speakers **
SDP keynotes are invited by the organizing committee and will present in
the research track of the workshop.
1.
Kuansan Wang
<https://www.microsoft.com/en-us/research/people/kuansanw/>, Managing
Director, Microsoft Research Outreach Academic Services
2.
Steinn Sigurdsson <https://science.psu.edu/astro/people/sxs540>,
Scientific Director of arXiv and Professor at the Pennsylvania State
University
** Organizing Committee **
Muthu Kumar Chandrasekaran, Amazon, Seattle, USA
Anita de Waard, Elsevier, USA
Guy Feigenblat, IBM Research AI, Haifa Research Lab, Israel
Dayne Freitag, SRI International, San Diego, USA
Tirthankar Ghosal, Indian Institute of Technology Patna, India
Eduard Hovy, Research Professor, LTI, Carnegie Mellon University, USA
Petr Knoth, Open University, UK
David Konopnicki, IBM Research AI, Haifa Research Lab, Israel
Philipp Mayr, GESIS – Leibniz Institute for the Social Sciences, Germany
Robert M. Patton, Oak Ridge National Laboratory, USA
Michal Shmueli-Scheuer, IBM Research AI, Haifa Research Lab, Israel
Dominika Tkaczyk, Crossref, UK
** Steering Committee **
C. Lee Giles, David Reese Professor, College of Information Sciences and
Technology, Pennsylvania State University
Min-Yen Kan, Associate Professor, School of Computing, National University
of Singapore
Dragomir Radev, A. Bartlett Giamatti Professor of Computer Science, Yale
University
Jie Tang, Professor and Associate Chair of the Department of Computer
Science and Technology, Tsinghua University
Alex Wade, Group Technical Program Manager, Chan Zuckerberg Initiative
Kuansan Wang, Managing Director, Microsoft Research Outreach Academic
Services
Bonnie Webber, Professor, School of Informatics, University of Edinburgh
** Programme Committee **
Please visit our website for the complete list of PCs:
https://ornlcda.github.io/SDProc/programcommittee.html
More details available on the workshop website:
https://ornlcda.github.io/SDProc/
With kind regards,
SDP 2020 organizing committee
--
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Tirthankar Ghosal <https://tirthankarslg.wixsite.com/ainlpmldl>
Visvesvaraya Research Fellow <http://meity.gov.in/esdm/phd-scheme>
Elsevier Center of Excellence for Natural Language Processing
<http://www.iitp.ac.in/~ai-nlp-ml/collaboration.html>
Department of Computer Science and Engineering
Indian Institute of Technology Patna <http://www.iitp.ac.in/>
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++