Research assessment, Questionable
Publishers, Whitelists and Improving the
Quality of Indian Journals
International Workshop on
"Information Management Tools for Academic and Research
Libraries"​, All India Shri Shivaji Memorial Society’s
College of Engineering, Pune – 1
18 to 22 December 2017 Lars Bjørnshauge
lars@doaj.org
Agenda
• Research Assessment and Reward systems –
an obstacle for the implementation of Open
Access
• Questionable publishers – and how to detect
them
• Improving the quality of journals published in
India
• Whitelists!?
Drivers for Open
• Researchers, who want to share there stuff to the
widest possible audience
• Technologies and communication tools
• Librarians, feeling that open is very near to the heart of
their profession
• Experimental and emerging business models
facilitating open access
• Emerging development of incentives (sticks and
carrots):
– Institutional and funder open access policies
– Institutional and funder mandates
Drivers for Open
• Researchers, who want to share there stuff to the widest
possible audience
• Technologies and communication tools
• Librarians, feeling that open is close to the heart of their
profession
• Experimental and emerging business models facilitating
open access
• Emerging development of incentives (sticks and carrots):
• Institutional and funder open access policies
• Institutional and funder mandates
• But it goes to slow!!
Obstacles for change
• Reward systems
• Academic Freedom
• Culture in the Academy
What pays off in the current system??
• As a Researcher:
• Publish in quality prestige journals – go for the High Impact Factor
journals and you will be rewarded (promotion, tenure and grants)
• Don´t bother to much about whether or not
• your results are actually accessible for the widest possible
audience
• your data are archived and open
• your software is documented and available
• your research is actually reproducable
• For your career it doesn´t really matter that much!
• As an Institution:
• Attract the researchers with the above behavior and the institution
will get higher rankings and receive more grants
Research Assessment
• Research assessment systems have to change
• Often based on the Journal Impact Factor (JIF)
– JIF and other journal level metrics are subject
to manipulation, gaming and fraud
– researchers are NOT primarily rewarded for
WHAT they publish, but WHERE they publish
The Culture of the Academy
• The Culture of the Academy needs to change too!
• The concept of Academic Freedom is often used as
an excuse for publishing in the “prestige” journals.
• It is the underlying logic of Green Access and Hybrid
Open Access.
• But Academic Freedom applies to what you are
researching, what you are investigating, the methods
you apply etc.
• Based on your agreement with your institution and
the grants you get, you will do your research.
The Culture of the Academy
• It is often argued that your decisions as to where
you publish, how you publish, the rights and
permissions you give to readers/users etc
belongs to my Academic Freedom.
• “It is my Academic Freedom to decide where to
publish”!
The Culture of the Academy
• It is often argued that your decisions as to where
you publish, how you publish, the rights and
permissions you give to readers/users etc
belongs to my Academic Freedom.
• “It is my Academic Freedom to decide where to
publish”!
• I disagree!
Academic Responsibility
• Applies to how you share your research, your
findings, your data, your software!!
• We need stronger mandates from research funders
and research institutions
• Research funders and research institutions should be
very specific as to how they expect researchers to
disseminate their findings!
• Responsible researcher conduct is to share results,
data and software in the open
• Librarians have an important role to play in educating
the future researchers!!
It should have been
open in the first place!
If your papers, your data and your
software are not in the open, it
should not count!
Who can change the system then?
• Not the Publishers – they are businesses, exploiting
the conditions offered to them.
• The research funders, university managements,
governments can change the system
– Changing the reward and incentive systems
– Require publishing in the open
– Setting the conditions for the publishers
– Changing the culture in the academy
What is needed is …
• More and much stronger funder and institutional
mandates
• Radical changes in the research evaluation system and
incentives for researchers to publish in the open!
– Today researchers are rewarded based on Where they
publish, i.e. in which journals they publish
– Not based on What they publish, the actual content
– and not based on How they publish, whether it is
open and reuseable or not
• A cultural change in academia is what needs to happen!
The scholarly system I want to see
• Research results are immediately accessible
to everyone.
• Research is verifiable and reproducable.
• Research is evaluated based on its actual
impact - not based on the wrapper (the
journal title)
• Research findings are evaluated in the open
after dissemination.
This means That:
• Research will be disseminated in the open
with generous reuse permissions.
• Research Data will be archived and made
accessible.
• Software associated with research will be
documented and available as well.
• Research Evaluation is transparent.
And this means that:
• Researchers are rewarded
– not only based on citations, but as well for
• the societal impact of their research,
• documenting their data and software and make
it open,
• contributing to peer review etc.
• In short:
• Researchers will be rewarded for all what they do
Questionable or unethical
publishers
Questionable publishers
• Predatory publishers – (Beall)
Definition
• Definition of predatory:
• inclined or intended to injure or exploit others for
personal gain or profit (Mirriam-Webster)
• A predatory publisher can then be described as
• a publisher who intends to injure or exploit
others for personal gain or profit.
•
• Consider this:
• “Does exploiting the divide between libraries (that typically pay for
subscriptions) and scholars (who typically use the subscriptions) in
order to make extraordinary high profits constitute predatory
conduct?”
• or this:
• “Does continuing to raise prices at several times the rate of
inflation, even as those increases cause direct injury to libraries by
robbing them of budget flexibility or even make it impossible for
them to continue to provide resources – does that constitute
predatory publishing?”
Beall´s list
• Investigation of Beall´s list (March 2014):
• 9219 journals (501 publishers and 320 independent journals)
• Probably thousands of the “journals” does not have an ISSN!
• 1142 journals (12,6%) are either hybrid journals, not OA or unreachable/unworkable
• 386 journals (4,2%) are dying or dormant
• 2836 “journals” (30,8%) haven´t published a single article in 2012, 2013 and the first months of
2014!
• 896 journals (9,7%) essentially empty
• 1832 journals (19,9%) has published less than 30 articles (in total) in 2012, 2013 and the first
months of 2014
• So far 77,4% of the journals on the list are
covered.
• The rest (23%):
• 784 journals (8,5%) should be regarded as
highly questionable
• 961 journals (10,4%) needs investigation
• 385 journals (4,2%) are apparently good
Questionable publishers – many names:
• Predatory publishers – (Beall)
• Illegitimate publishers – no law regulating academic
publishing
• Deceptive publishers –
• Unethical publishers
• In DOAJ we say Questionable publishers
My definition:
Questionable publishers is
publishers, who are not living up to
reasonable standards in terms of
content, services, transparency and
of business behavior.
Questionable publishing is not a
phenomenon that is specific to
Open Access publishing!
October 2013
February 2014
Lars Bjørnshauge
How we detect questionable journals
• Low publishing quality
• Journal name, website, fees, peer review, publisher,
ownership, volume of articles, advertisements, prominent
soliciting for editors, ambiguous company address, many
journals and few articles
• Low scientific quality
• focus, format, self-citations, plagiarism
• Malpractice
• false claims, hidden costs, spamming authors, wrong
information, fake impact factor
http://thinkchecksubmit.org/
Whitelists!?
Promoting OA journals
in National Whitelists
• Examples:
• The Science Europe Recommendations:
– DOAJ recognized in line with Web of Science and
Scopus
– The Nordic Research Councils collaborate on a
whitelist and supports DOAJ
– Indonesia
– Many universities have DOAJ listing as a criteria
for supporting APC payments for their researchers
THE NORDIC LIST
An international collaborative tool for publication
analysis with relevance for
open access
Collaboration with DOAJ
• In March 2017 a collaboration was started between DOAJ
and the Nordic List consortium
• The consortium would like to use DOAJ as a partner in
evaluating open access policies of publications channels
• This is an attempt to increase the effectivness of the Nordic
collaboration and also to be able to highlight good practice
in publishing
Proposals for India
• How to
– promote good Open Access journals in the UGC
list?
– move forward with the contacts regarding the
UGC-list
– get more Indian journals recognized as good OA-
journals
Your DOAJ Ambassadors
Proposal for India
• Step 1: Create a consortium of Indian
Academic Institutions – to provide resources
for DOAJ to handle applications, train journal
editors/publishers to produce good
applications
• Step 2: establish a working group to inform
about how the UGC list could be improved
Journal Applications from India handled by
DOAJ since March 2014
Issues
• A large proportion of questionable journals
come from India
• Gives India a bad reputation
• DOAJ are spenging a lot of resources and time
on poor quality applications
• DOAJ is not blacklisting, we want to help!!
Proposals for India
• Step 1: Create a consortium of Indian Academic
Institutions – to provide resources for DOAJ to
handle applications, train journal
editors/publishers to produce good applications
• Step 2: establish a working group to inform about
how the UGC list could be improved
• Step 3: DOAJ and your Ambassadors to conduct
workshops to train journals editors in best
publishing practice

Improving the Transparency and Credibility of Open Access Publishing

  • 1.
    Research assessment, Questionable Publishers,Whitelists and Improving the Quality of Indian Journals International Workshop on "Information Management Tools for Academic and Research Libraries"​, All India Shri Shivaji Memorial Society’s College of Engineering, Pune – 1 18 to 22 December 2017 Lars Bjørnshauge [email protected]
  • 2.
    Agenda • Research Assessmentand Reward systems – an obstacle for the implementation of Open Access • Questionable publishers – and how to detect them • Improving the quality of journals published in India • Whitelists!?
  • 3.
    Drivers for Open •Researchers, who want to share there stuff to the widest possible audience • Technologies and communication tools • Librarians, feeling that open is very near to the heart of their profession • Experimental and emerging business models facilitating open access • Emerging development of incentives (sticks and carrots): – Institutional and funder open access policies – Institutional and funder mandates
  • 4.
    Drivers for Open •Researchers, who want to share there stuff to the widest possible audience • Technologies and communication tools • Librarians, feeling that open is close to the heart of their profession • Experimental and emerging business models facilitating open access • Emerging development of incentives (sticks and carrots): • Institutional and funder open access policies • Institutional and funder mandates • But it goes to slow!!
  • 5.
    Obstacles for change •Reward systems • Academic Freedom • Culture in the Academy
  • 6.
    What pays offin the current system?? • As a Researcher: • Publish in quality prestige journals – go for the High Impact Factor journals and you will be rewarded (promotion, tenure and grants) • Don´t bother to much about whether or not • your results are actually accessible for the widest possible audience • your data are archived and open • your software is documented and available • your research is actually reproducable • For your career it doesn´t really matter that much! • As an Institution: • Attract the researchers with the above behavior and the institution will get higher rankings and receive more grants
  • 7.
    Research Assessment • Researchassessment systems have to change • Often based on the Journal Impact Factor (JIF) – JIF and other journal level metrics are subject to manipulation, gaming and fraud – researchers are NOT primarily rewarded for WHAT they publish, but WHERE they publish
  • 8.
    The Culture ofthe Academy • The Culture of the Academy needs to change too! • The concept of Academic Freedom is often used as an excuse for publishing in the “prestige” journals. • It is the underlying logic of Green Access and Hybrid Open Access. • But Academic Freedom applies to what you are researching, what you are investigating, the methods you apply etc. • Based on your agreement with your institution and the grants you get, you will do your research.
  • 9.
    The Culture ofthe Academy • It is often argued that your decisions as to where you publish, how you publish, the rights and permissions you give to readers/users etc belongs to my Academic Freedom. • “It is my Academic Freedom to decide where to publish”!
  • 10.
    The Culture ofthe Academy • It is often argued that your decisions as to where you publish, how you publish, the rights and permissions you give to readers/users etc belongs to my Academic Freedom. • “It is my Academic Freedom to decide where to publish”! • I disagree!
  • 11.
    Academic Responsibility • Appliesto how you share your research, your findings, your data, your software!! • We need stronger mandates from research funders and research institutions • Research funders and research institutions should be very specific as to how they expect researchers to disseminate their findings! • Responsible researcher conduct is to share results, data and software in the open • Librarians have an important role to play in educating the future researchers!!
  • 12.
    It should havebeen open in the first place! If your papers, your data and your software are not in the open, it should not count!
  • 13.
    Who can changethe system then? • Not the Publishers – they are businesses, exploiting the conditions offered to them. • The research funders, university managements, governments can change the system – Changing the reward and incentive systems – Require publishing in the open – Setting the conditions for the publishers – Changing the culture in the academy
  • 14.
    What is neededis … • More and much stronger funder and institutional mandates • Radical changes in the research evaluation system and incentives for researchers to publish in the open! – Today researchers are rewarded based on Where they publish, i.e. in which journals they publish – Not based on What they publish, the actual content – and not based on How they publish, whether it is open and reuseable or not • A cultural change in academia is what needs to happen!
  • 15.
    The scholarly systemI want to see • Research results are immediately accessible to everyone. • Research is verifiable and reproducable. • Research is evaluated based on its actual impact - not based on the wrapper (the journal title) • Research findings are evaluated in the open after dissemination.
  • 16.
    This means That: •Research will be disseminated in the open with generous reuse permissions. • Research Data will be archived and made accessible. • Software associated with research will be documented and available as well. • Research Evaluation is transparent.
  • 17.
    And this meansthat: • Researchers are rewarded – not only based on citations, but as well for • the societal impact of their research, • documenting their data and software and make it open, • contributing to peer review etc. • In short: • Researchers will be rewarded for all what they do
  • 18.
  • 19.
  • 20.
    Definition • Definition ofpredatory: • inclined or intended to injure or exploit others for personal gain or profit (Mirriam-Webster) • A predatory publisher can then be described as • a publisher who intends to injure or exploit others for personal gain or profit. •
  • 21.
    • Consider this: •“Does exploiting the divide between libraries (that typically pay for subscriptions) and scholars (who typically use the subscriptions) in order to make extraordinary high profits constitute predatory conduct?” • or this: • “Does continuing to raise prices at several times the rate of inflation, even as those increases cause direct injury to libraries by robbing them of budget flexibility or even make it impossible for them to continue to provide resources – does that constitute predatory publishing?”
  • 22.
    Beall´s list • Investigationof Beall´s list (March 2014): • 9219 journals (501 publishers and 320 independent journals) • Probably thousands of the “journals” does not have an ISSN! • 1142 journals (12,6%) are either hybrid journals, not OA or unreachable/unworkable • 386 journals (4,2%) are dying or dormant • 2836 “journals” (30,8%) haven´t published a single article in 2012, 2013 and the first months of 2014! • 896 journals (9,7%) essentially empty • 1832 journals (19,9%) has published less than 30 articles (in total) in 2012, 2013 and the first months of 2014
  • 23.
    • So far77,4% of the journals on the list are covered. • The rest (23%): • 784 journals (8,5%) should be regarded as highly questionable • 961 journals (10,4%) needs investigation • 385 journals (4,2%) are apparently good
  • 24.
    Questionable publishers –many names: • Predatory publishers – (Beall) • Illegitimate publishers – no law regulating academic publishing • Deceptive publishers – • Unethical publishers • In DOAJ we say Questionable publishers
  • 25.
    My definition: Questionable publishersis publishers, who are not living up to reasonable standards in terms of content, services, transparency and of business behavior.
  • 26.
    Questionable publishing isnot a phenomenon that is specific to Open Access publishing!
  • 27.
  • 28.
    How we detectquestionable journals • Low publishing quality • Journal name, website, fees, peer review, publisher, ownership, volume of articles, advertisements, prominent soliciting for editors, ambiguous company address, many journals and few articles • Low scientific quality • focus, format, self-citations, plagiarism • Malpractice • false claims, hidden costs, spamming authors, wrong information, fake impact factor
  • 30.
  • 31.
  • 32.
    Promoting OA journals inNational Whitelists • Examples: • The Science Europe Recommendations: – DOAJ recognized in line with Web of Science and Scopus – The Nordic Research Councils collaborate on a whitelist and supports DOAJ – Indonesia – Many universities have DOAJ listing as a criteria for supporting APC payments for their researchers
  • 33.
    THE NORDIC LIST Aninternational collaborative tool for publication analysis with relevance for open access
  • 34.
    Collaboration with DOAJ •In March 2017 a collaboration was started between DOAJ and the Nordic List consortium • The consortium would like to use DOAJ as a partner in evaluating open access policies of publications channels • This is an attempt to increase the effectivness of the Nordic collaboration and also to be able to highlight good practice in publishing
  • 36.
    Proposals for India •How to – promote good Open Access journals in the UGC list? – move forward with the contacts regarding the UGC-list – get more Indian journals recognized as good OA- journals
  • 37.
  • 38.
    Proposal for India •Step 1: Create a consortium of Indian Academic Institutions – to provide resources for DOAJ to handle applications, train journal editors/publishers to produce good applications • Step 2: establish a working group to inform about how the UGC list could be improved
  • 39.
    Journal Applications fromIndia handled by DOAJ since March 2014
  • 40.
    Issues • A largeproportion of questionable journals come from India • Gives India a bad reputation • DOAJ are spenging a lot of resources and time on poor quality applications • DOAJ is not blacklisting, we want to help!!
  • 41.
    Proposals for India •Step 1: Create a consortium of Indian Academic Institutions – to provide resources for DOAJ to handle applications, train journal editors/publishers to produce good applications • Step 2: establish a working group to inform about how the UGC list could be improved • Step 3: DOAJ and your Ambassadors to conduct workshops to train journals editors in best publishing practice