Haustein, S. (2017). The evolution of scholarly communication and the reward system of science

The
evolution of
scholarly communication

and
the
reward system
of
science
Stefanie
Haustein
@stefhaustein

Outline
Scholarly
communication
From
the
16th century
to
Open
Science
Bibliometrics
From
library
management
to
research
evaluation
Altmetrics
Opportunities
and
challenges
Conclusions
and
Outlook

Invisible
Colleges
http://commons.wikimedia.org/wiki/File:Marin_mersenne.jpg#/media/File:Marin_mersenne.jpg
http://commons.wikimedia.org/wiki/File:Henry_Oldenburg.jpg
Père
Marin
Mersenne
(1588-‐1648)
Henry
Oldenburg
(1619-‐1677)

Scientific
Societies
http://upload.wikimedia.org/wikipedia/commons/thumb/6/61/Louis_XIV_und_Colbert_in_der_Akademie.jpg/640px-‐Louis_XIV_und_Colbert_in_der_Akademie.jpg
https://www.rcseng.ac.uk/museums/hunterian/images/lost-‐museums-‐2011/the-‐royal-‐society-‐repository/Image%201%20-‐%20GreshamCollege.jpg/image_preview
http://en.wikipedia.org/wiki/Nullius_in_verba#/media/File:Bookplate_of_the_Royal_Society_(Great_Britain).jpg
L’Académie
royale
des
sciences
22
December 1666
The
Royal
Society
28
November 1660

Scientific
Journals
http://de.wikipedia.org/wiki/Journal_des_s%C3%A7avans#/media/File:1665_journal_des_scavans_title.jpg
http://en.wikipedia.org/wiki/Philosophical_Transactions_of_the_Royal_Society#/media/File:Philosophical_Transactions_Volume_1_frontispiece.jpg
Le
journal
des
sçavans
5
January 1665
Philosophical Transactions
6
March
1665

Scientific
Articles
Harmon,
J.E.
&
Gross,
A.G.
(2007).
The
Scientific
Literature.
A
Guided
Tour.
Chicago:
University
of
Chicago
Press.
• Experiments
and
descriptions
of
the
natural
world
• Avoiding
“fine
speaking”
• Various
styles
of
arguing
• Qualitative
and
personal
judgements

Proportion
of
IMRaDadoption
in
medicaljournalsNumberof
references1900
to
2004
Scientific
Articles
Larivière,
V.,
Archambault,
É.
&
Gingras,
Y.
(2008).
Long-‐term
variations
in
the
aging
of
scientific
literature:
From
exponential
growth
to
steady-‐state
science
(1900-‐2004).
Journal
of
the

American
Society
for
Information
Science
and
Technology,
59(2),
288-‐296.
Sollaci,
L.B.
&
Pereira,
M.G.
(2004).
The
introduction,
methods,
results,
and
discussion
(IMRAD)
structure:
a
fifty-‐year
survey.
Journal
of
the
Medical
Library
Association,
92(3),
364-‐371
• Professionalized
and
highly
specialized
• Increased
focus
on
data,
graphs,
tables
and
theory
• Impersonal,
technical
and
codified
• Style
guides
and
gatekeeping
• Citations
• Introduction,
Methods,
Results
and
Discussion

Digital
Revolution
arXiv submission
statistics
from
http://arxiv.org/stats/monthly_submissions
Larivière,
V.,
Lozano,
G.A.
&
Gingras,
Y.
(2014).
Are
elite
journals
declining?
Journal
of
the
Association
for
Information
Science
and
Technology,
65(4),
649-‐655.
• Improved access
• Acceleration
• Collaboration
• Peer
review
• Distribution
of
preprints
• Decreasing importance
of

scientific journal
• Journal
functions
• Diversification
of

publication
venues
• Symbolic capital
of

journals unchanged
Submissionsto
arXiv
Share
of
top
1%
mostcitedpapers

Academic Publishing Market
Larivière,
V.,
Haustein,
S.,
&
Mongeon,
P.
(2015).
The
oligopoly
of
academic
publishers
in
the
digital
era.
PLoS ONE,
10(6),
e0127502.
doi:

10.1371/journal.pone.0127502
• Aggravation
of

serials
crisis
• Elsevier:
30%
increase

of
subscription
price

• Profit
margins
of

commercial
publishers

up
to
40%
• Decline
of

scientific
societies

as
publishers
• >50%
of
papers

owned
by
five
major

publishers

Open
Access
Archambault,
É.,
Amyot,
D.,
Deschamps,
P.,
Nicol,
A.,
Rebout,
L.
&
Roberge,
G.
(2013).
Proportion
of
Open
Access
Peer-‐Reviewed
Papers
at
the
European
and

World
Levels
2004-‐2011.
Report
for
the
European
Commission.
http://www.science-‐metrix.com/pdf/SM_EC_OA_Availability_2004-‐2011.pdf
Budapest
Open
Access
Initiative
“immediate,
free
availability
on
the
public
internet,
permitting

any
users
to
read,
download,
copy,
distribute,
print,
search
or

link
to
the
full
text
of
these
articles”
• Gold
and
Green
• Libre
and
Gratis
• Hybrid
• Elsevier:
$500
to
5,000
• Springer:
$3,000
• Wiley:
$3,000
Freelyavailablejournal
papers2004
to
2011
Budapest
Open
Access
Initiative
(2002)

Open
Science
Kraker,
P.,
Leony,
D.,
Reinhardt,
W.
&
Beham,
G.
(2011).
The
case
for
an
open
science
in
technology
enhanced
learning.
International
Journal
of
Technology

Enhanced
Learning,
3(6),
643-‐654.
“opening
up
the
research
process
by
making
all
of
its
outcomes,

and
the
way
in
which
these
outcomes
were
achieved,
publicly

available
on
the
World
Wide
Web”
• Open
Data
• Open
Source
• Open
Methodology
• Open
Access
• Open
Peer
Review
Krakeret

al.
(2011,
p.
645)

Bibliometrics
Gross,
P.L.K.
&
Gross,
E.M.
(1927).
College
libraries
and
chemical
education.
Science,
66(1713),
385-‐389.
Library
collection
management

Journalscitedin
the
Journal
of
the
American
Chemical
Society1926

Bibliometrics
Garfield,
E.
(1955).
Citation
indexes
for
science.
A
new
dimension
in
documentation
through
association
of
ideas.
Science,
122,
108-‐111.
Information
retrieval
•
“It
would
not
be
excessive
to
demand
that
the

thorough
scholar
check
all
papers
that
have

cited
or
criticized
such
papers,
if
they
could
be

located
quickly.
The
citation
index
makes
this

check
practicable.”
• Institute
for
Scientific
Information
• Science
Citation
Index
• Source
Author Index
• Citation
Index
Garfield
(1955,
p.
108)

Bibliometrics
Price,
D.
J.
d.
S.
(1961).
Science
Since
Babylon.
New
Haven
/
London:
Yale
University
Press,
Price,
D.
J.
d.
S.
(1963).
Little
Science,
Big
Science.
New
York:
Columbia
University
Press.
Sociology
of
science
Derek
J.
de
Solla Price
Science
since
Babylon
(1961)
Little
Science
– Big
Science
(1963)

Bibliometrics
Merton,
R.
K.
(1988).
The
Matthew
effect
in
science,
II:
Cumulative
advantage
and
the
symbolism
of
intellectual
property.
Isis,
79,
606–623.

Sociology
of
science
Robert
K.
Merton
• Social
norms of
science
• Communalism
• Universalism
• Disinterestedness
• Organized skepticism
• Matthew
effect
“symbolically,
[the
reference]
registers
in
the
enduring
archives

the
intellectual
property
of
the
acknowledged
source
by

providing
a
pellet
of
peer
recognition
of
the
knowledge
claim”
Merton
(1988,
p.
621)

Bibliometrics
Moed,
H.F.,
Burger,
W.J.M.,
Frankfort,
J.G,
van
Raan,
A.F.J.
(1985).
The
use
of
bibliometric
data
for
the
measurement
of
university
research
performance.

Research
Policy,
14(3),
131-‐149.

Research
evaluation
• Performance
measurement and
policy instrument
“When
used
properly,
bibliometric
indicators
can
provide
a

‘monitoring
device’
for
university
research-‐management
and

science
policy.
They
enable
research
policy-‐makers
to
ask
relevant

questions
of
researchers
on
their
scientific
performance,
in
order

to
find
explanations
of
the
bibliometric
results
in
terms
of
factors

relevant
to
policy.”
• Commercialization
Moedet
al.
(1985,
p.
131)

Bibliometrics
Research
evaluation
• Part
of
hiring,
promotion
and
funding decisions
• Dashboard
tools

Bibliometrics
Hvistendahl,
M.
(2013).
China’s
publication
bazaar.
Science,
342(6162),
1035-‐1039.
van
Noorden,
R.
(2013).
Brazilian
citation
scheme
outed:
Thomson
Reuters
suspends
journals
from
its
rankings
for
‘citation
stacking’,
Nature,
500(7464),
510-‐511.

Research
evaluation
• Oversimplification
• Publications
=
productivity
• Citations
=
impact
• Uninformed
use
and
misuse
• Impact
factor
• h-‐index
• Adverse
effects
• “Salami”
publishing
• Honorary
authorship
• Self-‐citations
• Citation
cartels

Scholarly metrics
Björneborn,
L.
&
Ingwersen,
P.
(2004),
Toward
a
basic
framework
for
webometrics.
Journal
of
the
American
Society
for
Information
Science
and
Technology,

55(14),
1216–1227.
Definitions
informetrics
scientometrics
bibliometrics
cybermetrics
webometrics
adaptedfrom:
Björneborn&
Ingwersen(2004,
p.
1217)

Scholarly metrics
Otlet,
P.
(1934).
Traité
de
documentation:
le
livre
sur
le
livre,
théorie
et
pratique.
Pritchard,
P.
(1927).
Statistical bibliography or
bibliometrics?
Journal
of
Documentation,
25,
348-‐349..
Bibliometrics
informetrics
scientometrics
bibliometrics
cybermetrics
webometrics
“La
«Bibliometrie»
sera

la
partie
définie
de
la

Bibliologie
qui
s'occupe

de
la
mesure
ou

quantité
appliquée
aux

livres.”

“the
application
of

mathematics
and

statistical
methods
to

books
and
other
media

of
communication”
Pritchard
(1969,
p.
348)
Otlet
(1934,
p.
14)

Scholarly metrics
Altmetrics
adaptedfrom:
Björneborn&
Ingwersen(2004,
p.
1217)
informetrics
scientometrics
bibliometrics
cybermetrics
webometrics altmetrics
Björneborn,
L.
&
Ingwersen,
P.
(2004),
Toward
a
basic
framework
for
webometrics.
Journal
of
the
American
Society
for
Information
Science
and
Technology,

55(14),
1216–1227.

Scholarly metrics
Priem,
J.
(2014).
Altmetrics.
In
B.
Cronin
&
C.
R.
Sugimoto (Eds.),
Beyond
bibliometrics:
harnessing multidimensional indicators of
performance
(pp.
263–287).

Cambridge,
MA:
MIT
Press.

Rousseau,
R.
&
Ye,
F.
(2013).
A
multi-‐metric approach for
research evaluation.
Chinese Science
Bulletin,
3288–3290.
doi:10.1007/s11434-‐013-‐5939-‐3
Altmetrics
informetrics
scientometrics
bibliometrics
cybermetrics
webometrics altmetrics
“study
and
use
of

scholarly
impact

measures
based
on

activity
in
online
tools

and
environments”
“a
good
idea
but
a

bad
name”
Rousseau
&
Ye(2013,
p.
3289)
Priem(2014,
p.
266)

Scholarly metrics
Definition
altmetrics
informetrics
scientometrics
bibliometrics
cybermetrics
webometrics
adaptedfrom:
Björneborn&
Ingwersen(2004,
p.
1217)
Scholarly
metrics
Björneborn,
L.
&
Ingwersen,
P.
(2004),
Toward
a
basic
framework
for
webometrics.
Journal
of
the
American
Society
for
Information
Science
and
Technology,

55(14),
1216–1227.

Scholarly metrics
Haustein,
S.,
(2016).
Grand
challenges
in
altmetrics:
heterogeneity,
data
quality
and
dependencies.
Scientometrics,
108(1),
413–423.

altmetrics
informetrics
scientometrics
bibliometrics
cybermetrics
webometrics
Scholarly
metrics
Acts: viewing, reading,
saving, diffusing,
mentioning, citing,
reusing, modifying, etc.
Scholarly documents:
papers, books, blog posts,
datasets, code, etc.
Scholarly agents:
researchers, universities,
funders, journals, etc.
“[S]cholarly metrics
are

thus
defined
as
indicators

based
on
recorded
events

of
acts
[…]
related
to

scholarly
documents
[…]

or
scholarly
agents
[…].”
Haustein
(2016,
p.
348)

Altmetrics
Priem,
J.,
Taraborelli,
D.,
Groth,
P.,
&
Neylon,
C.
(2010).
Alt-‐metrics:
a
manifesto.
October.
Retrieved from http://altmetrics.org/manifesto/

• Information
overload
“We
rely
on
filters
to
make
sense
of
the
scholarly
literature,
but

the
narrow,
traditional
filters
are
being
swamped.
However,
the

growth
of
new,
online
scholarly
tools
allows
us
to
make
new

filters;
these
altmetrics
reflect
the
broad,
rapid
impact
of

scholarship
in
this
burgeoning
ecosystem.”
• Criticism
against
current
form
of
research

evaluation
• Alternative
forms
of
research
output
• Alternative
use
and
visibility
of
publications
Priem et al. (2010)

Altmetrics
Coverage per
platform
Haustein,
S.,
Costas,
R.,
&
Larivière,
V.
(2015).
Characterizing
social
media
metrics
of
scholarly
papers:
The
effect
of
document
properties
and
collaboration

patterns.
PLoS ONE,
10(5),
e0127830.
doi:
10.1371/journal.pone.0120495
Zahedi,
Z.,
&
Haustein,
S.
(in
preparation).
Which
document
features
attract
users
in
Mendeley?
An
analysis
of
bibliographic
characteristics
of
Web
of
Science

publications
and
Mendeley
readership
counts.

Mathematics
&
Computer
Science
Natural
Sciences
&
Engineering
Life
&
Earth Sciences
Biomedical &
Health Sciences
Social
Sciences
&
Humanities
76,4
%
83,7
%
91,4
%
86,5
%
81,7
%
Mendeley
7,5
%
12,9
%
21,6
%
31,7
%
26,0
%
Twitter
Altmetrics
Coverage per
discipline
Haustein,
S.,
Costas,
R.,
&
Larivière,
V.
(2015).
Characterizing
social
media
metrics
of
scholarly
papers:
The
effect
of
document
properties
and
collaboration

patterns.
PLoS ONE,
10(5),
e0127830.
doi:
10.1371/journal.pone.0120495
Zahedi,
Z.,
&
Haustein,
S.
(in
preparation).
Which
document
features
attract
users
in
Mendeley?
An
analysis
of
bibliographic
characteristics
of
Web
of
Science

publications
and
Mendeley
readership
counts.

Altmetrics
Density and
intensity per
platform
Intensity
Haustein,
S.,
Costas,
R.,
&
Larivière,
V.
(2015).
Characterizing
social
media
metrics
of
scholarly
papers:
The
effect
of
document
properties
and
collaboration

patterns.
PLoS ONE,
10(5),
e0127830.
doi:
10.1371/journal.pone.0120495
Zahedi,
Z.,
&
Haustein,
S.
(in
preparation).
Which
document
features
attract
users
in
Mendeley?
An
analysis
of
bibliographic
characteristics
of
Web
of
Science

publications
and
Mendeley
readership
counts.

Altmetrics
Spearman
correlations with citations
Perfectnegativecorrelation
Perfectpositive
correlation
Haustein,
S.,
Larivière,
V.,
Thelwall,
M.,
Amyot,
D.,
&
Peters,
I.
(2014).
Tweets
vs.
Mendeley
readers:
How
do
these two social
media
metrics differ.

Information
Technology,
56(5),
207–215.
doi:
10.1515/itit-‐2014-‐1048
Haustein,
S.,
Costas,
R.,
&
Larivière,
V.
(2015).
Characterizing
social
media
metrics
of
scholarly
papers:
The
effect
of
document
properties
and
collaboration

patterns.
PLoS ONE,
10(5),
e0127830.
doi:
10.1371/journal.pone.0120495

Altmetrics
Spearman
correlations with citations
NSF
Subdiscipline General
Biomedical Research 2011
Size of data points represents number of Mendeley readers in Twitter graph (left) and number of tweets
in Mendeley graph (right).
Haustein,
S.,
Larivière,
V.,
Thelwall,
M.,
Amyot,
D.,
&
Peters,
I.
(2014).
Tweets
vs.
Mendeley
readers:
How
do
these two social
media
metrics differ.

Information
Technology,
56(5),
207–215.
doi:
10.1515/itit-‐2014-‐1048

Altmetrics
Document
types
Haustein,
S.,
Costas,
R.,
&
Larivière,
V.
(2015).
Characterizing
social
media
metrics
of
scholarly
papers:
The
effect
of
document
properties
and
collaboration

patterns.
PLoS ONE,
10(5),
e0127830.
doi:
10.1371/journal.pone.0120495

Altmetrics
Altmetrics
in
the
wild

Opportunities and
Challenges
• Heterogeneity
• Time
and
timing
• Audiences
and
user
groups
Altmetrics

Opportunities
• Different acts
• Diverse
motivations
Ø Diverse
impact
Challenges
• Understanding underlying processes
Ø Determining the
meaning of
metrics
Heterogeneity of
Altmetrics

Heterogeneity
Saving
to
Mendeley
Mentioning
in
News

Heterogeneity
Recommending
on
F1000
Tweeting

Heterogeneity
Bertin,
M.,
Atanassova,
I.,
Gingras,
Y.,
&
Larivière,
V.
(2015).
The
invariant
distribution
of
references
in
scientific
articles.
Journal
of
the
Association
for

Information
Science
and
Technology, 67(1),
164-‐177.
doi:
10.1002/asi.23367
Distribution
of
referencesalongthe
IMRaDstructure
Citing in
a
journal
article

Heterogeneity
Acts referring to
research objects
Haustein,
S.,
Bowman,
T.
D.,
&
Costas,
R.
(2016).
Interpreting “altmetrics”:
Viewing acts on
social
media
through the
lens of
citation
and
social
theories.
Dans

C.
R.
Sugimoto (dir.),
Theories of
Informetrics and
Scholarly Communication
(p.
372–405).
Berlin:
De
Gruyter Mouton.
doi:
10.1515/9783110308464-‐022
RESEARCH OBJECT

Time
and
Timing
Opportunities
• Detailed life
cycle
of
scientific output
Ø Fine-‐grained indicators and
adequate benchmarks
Challenges
• Versions
of
research output
• Publication
dates

Time
and
Timing
Journal
article
• Submitted manuscript
• Revised manuscript
• Accepted manuscript
• Version
of
Record
• Online
publication
• Journal
issue
• Online
date
• Issue
month
Ø Adjusting indicators
3
March
2014
15
July
2014
21
January2015
February2015

Time
and
Timing
Tweets
before publication?

Time
and
Timing
Weekday effects on
Twitter
weekday of
online publication:
based on:
8,765 Springer papers with
online publication date
19,010 tweets received within one year
of online publication date

Audiences
and
User
Groups
Opportunities
• Differentiating between types
of
users
• Measuring societal impact
Challenges
• Identifying users and
user
groups
• Determining engagement

Audiences
and
User
Groups
Alperin,
J.
P.
(2015).
Moving
beyond
counts:
A
method
for
surveying
Twitter
users.
In
altmetrics15:
5
years
in,
what
do
we
know?
Amsterdam,

The
Netherlands.
Retrieved
from:
http://altmetrics.org/wp-‐content/uploads/2015/09/altmetrics15_paper_3.pdf
Tsou,
A.,
Bowman,
T.
D.,
Ghazinejad,
A.,
&
Sugimoto,
C.
R.
(2015).
Who tweets
about
science?
In
Proceedings of
the
2015
International
Society
for

Scientometrics and
Informetrics (pp.
95–100).
Istanbul,
Turkey.
Identifying Twitter
users
• Altmetric.com
classification
• Among
a
random
sample
of
2,000
accounts
tweeting

papers,
34%
of
individuals
identified
as
having
PhD
• Of
286
users
linking
to
SciELO articles,
24%
employed
at

university,
23%
students,
36%
not
university
affiliated
*based on
Altmetric.com
data
06/2015
(Tsou, Bowman, Ghazinejad, & Sugimoto, 2010)
(Alperin, 2015)

1
2
3
Audiences
and
User
Groups
Haustein,
S.,
&
Costas,
R.,
(2015).
Identifying Twitter
audiences:
who is tweeting about
scientific papers?
Communication
présentée
au
SIG/MET
Workshop,

ASIS&T
2015
Annual Meeting,
7
novembre
2015,
Saint-‐Louis,
MO
(USA).
topics and
collectives
academic
personal
Node size
number of
accounts
associated with term
Node color
cluster
affiliation
Terms
in
Twitter
bio

Audiences
and
User
Groups
Engagement
with scientific papers on
Twitter
Haustein,
S.,
Bowman,
T.
D.,
Holmberg,
K.,
Tsou,
A.,
Sugimoto,
C.
R.,
&
Larivière,
V.
(2016).
Tweets
as
impact
indicators:
Examining
the
implications
of

automated
bot
accounts
on
Twitter.
Journal
of
the
Association
for
Information
Science
and
Technology,
67(1),
232–238.
doi:
10.1002/asi.23456

Audiences
and
User
Groups
Twitter
arXiv bots

Audiences
and
User
Groups
Journal
accounts

Conclusions
• Scholarly communication
and
the
reward system

of
science
are
changing
• Potential to
become more
transparent
and

diverse
• Open
Science
• Scholarly metrics
• Fundamental difference between posting on

social
media
and
academic publishing
• More
metrics =
more
complexity

Outlook
More
scientometric studies.
More
sociological research.
Avoid adverse
effects.

Haustein, S. (2017). The evolution of scholarly communication and the reward system of science

More Related Content

What's hot (20)

Viewers also liked (10)

Similar to Haustein, S. (2017). The evolution of scholarly communication and the reward system of science (20)

More from Stefanie Haustein (12)

Recently uploaded (20)

Haustein, S. (2017). The evolution of scholarly communication and the reward system of science