Ian Mulvany

December 10, 2024

Do we need a trust credit agency in scholarly publishing?

If you work in scholarly publishing, in any capacity really, I think you should go and read the recent report from STM on trust and identity - https://stm-assoc.org/new-stm-report-trusted-identity-in-academic-publishing/.

I have been thinking about the role of identity, and identity verification, in our industry for a few weeks now, and I’ve been struck by how odd it is that we don’t have strong identity verification practices. I’d been meaning to write up my thoughts on that with the proposal that we should move towards implementing what are called “know your customer” protocols, so this report is highly serendipitously timed from my point of view.

It’s very well written, and touches on many of the points that one should think about in relation to this question.

The reality is that the scale of publishing has now gotten so large that we can no longer rely on informal trust networks. At the same time our ability to manage research integrity issues using old techniques won’t work, and there are a lot more integrity issues to be solved.

My thought was surely proving identity would go a long way to solving this problem, but this report makes great points about the issues with equitably and access to publishing, as well as a strong case about layered trust.

I am still strongly of the opinion that this is something we must do, but I think that it should be implemented in a scaled way - where there are low trust signals enforce high barrier , where a person builds up a trust profile reduce those barriers.

As I was reading through this on the plane today it started to sound like a credit scoring system. That is probably what we need, even though approaching it would require the most careful implementation.

This approach would also not rule out all types of misconduct, for example one could have very high trust markers, and yet still pursue quite a biased research agenda (not fabricated, but ideologically biased). In an increasingly polarised world, that is something that is happening more frequently.

We should be moving to building infrastructure that in a way becomes an engine for context, to give every person interacting with the collective assembled knowledge of the world, the best possible tools to do what they need to do with that knowledge. Trusted identity becomes one part of the context that we need to be able to provide.

The paper ends with several questions, I want to give my first pass answer to these. My answers are thinking about what could happen, not what must happen, or what we might like to happen.

Several questions remain for further exploration

What alternative verification methods can be offered to researchers who cannot prove their affiliation through traditional means or point to verified previous work?

Get them to use know your customer protocols. The costs are low, a few dollars per instance.

Is it appropriate to use services that validate government documents,considering the potential effort, invasiveness, and cost?

Yes, I think it is, if we also provide a suite of routes for allowing the researcher to build trust. These systems can be made in a way where that data is not shared with the entity who is having the checks done on their behalf.

How can we offer a simple and consistent set of recommendations that balances the need for trust and accountability with the need for inclusiveness, low friction, and respect for privacy across various risk levels?

I think if we try to agree on this up front across the industry, we will fall far behind, so I would like to see some level of experimentation, but with an understanding that we should be working towards a systems where trust indicators can be shared, and interchanged - at some point.

I think there is a big opportunity for scholarly societies here (finally!!) as society membership could become another federated trust marker, if the society puts the effort into verifying their membership.

How can we design pilots to test a range of approaches and measure their effectiveness?

Observe cost, time to submit, retraction volumes, flow in paper submissions across publishers when one publisher implements something, quality metrics. Work with authors and editors on the design of the systems, and keep check on author satisfaction. Start with some hypothesis about what user behaviour you expect to change, and see if that change is happening.

Those are my answers.

I did get GPT to summarise the paper, so you can get a gist of it, but really, do go read it.

GPT Summary of the Report: Trusted Identity in Academic Publishing

This report explores the critical need for trusted identity systems in academic publishing to counteract growing challenges related to research integrity and fraud. Historically, academic publishing relied on implicit trust, requiring minimal user verification. However, the rise of identity manipulation, paper mills, and fraudulent peer-review practices has revealed vulnerabilities in this model.

The report identifies two dimensions of trust:

  1. Trust in individual identity: Confidence that the user is a real individual.
  2. Trust in claims: Evidence supporting the user’s affiliations and academic history.
Current methods, such as institutional email addresses, federated identity systems, and ORCID profiles, provide some level of identity verification. However, these methods are not universally accessible, creating barriers for researchers in less-resourced regions or those without institutional affiliations.

The report details the tactics fraudsters use to exploit systems, including fake identities, stolen credentials, and fabricated affiliations. To counter these threats, publishers are encouraged to combine institutional and individual identity verification with trust markers like ORCID. Manual and automated verification methods, such as direct contact and watch-lists of bad actors, are also proposed.

Key recommendations include collaborative efforts across publishers, proportional risk-based solutions, and balancing security with inclusivity. Future work involves piloting identity verification approaches and addressing privacy concerns.


20 Key Points from the Report

  1. Historical reliance on trust: Academic publishing traditionally assumed honesty from participants.
  2. Fraud’s rising impact: Mass retractions and paper mills threaten scholarly credibility.
  3. Identity dimensions: Verification should confirm both a user’s identity and their professional claims.
  4. Current challenges: Existing identity verification methods have limitations, especially for under-resourced researchers.
  5. Fake email misuse: Fraudsters use non-institutional or impersonated emails to exploit systems.
  6. Federated identity systems: Widely used but not globally accessible, limiting their effectiveness.
  7. ORCID trust markers: Offer scalable verification through validated affiliations and research history.
  8. Impersonation risks: Fake or stolen credentials undermine system integrity.
  9. Institutional trust networks: Linking researchers to trusted institutions enhances accountability.
  10. Verification costs: Robust methods like biometrics are effective but may exclude some users.
  11. Manual verification: Direct contact remains a last resort but is resource-intensive.
  12. Risk-based approaches: Systems should adjust verification stringency based on potential risks.
  13. Fraudulent tactics: Include fake reviews, false co-authors, and fabricated affiliations.
  14. Survey insights: Non-institutional emails and IDPs are most prone to abuse.
  15. Transparency in systems: Clear protocols for identity verification reduce misuse.
  16. AI-generated fraud: Growing sophistication in creating fake identities requires adaptive defenses.
  17. Inclusive policies: Ensuring accessibility for unaffiliated researchers is vital.
  18. Collaboration necessity: Unified approaches among publishers can strengthen defenses.
  19. Role of multi-factor authentication: Adds a layer of security against credential theft.
  20. Next steps: Pilot projects and inclusivity research are needed for robust, scalable solutions.

Table of Key Data Points


 | Data Point | Details
| Number of papers retracted (2023) | Over 10,000, marking a record high for retractions.
| Fraudulent scenarios ranked | Most common: fake non-institutional emails; Least common: compromised institutional admins.
| Fraudulent activities identified | Fake content submissions, fake reviews, false affiliations, and guest editor manipulations.
| Survey respondents | Input from 12 publishers regarding fraud tactics and their impacts.
| Email misuse prevalence | Non-institutional emails are the most abused method.
| Cost of manual verification | Resource-intensive for high-volume submission systems.
| Impact of fake peer reviews | High severity due to potential corruption of the scholarly record.
Linked Text:

Executive Summaries

Chinese

最近的STM报告探讨了学术出版中信任和身份验证的紧迫需求,以应对研究诚信和欺诈的挑战。学术出版历史上依赖于内在的信任,但身份操控和欺诈行为的增加暴露了这一模式的弱点。报告提出以合作方式来实施身份验证,并通过风险比例解决方案来平衡安全与包容性。建议包括试验验证方法,并解决隐私问题。

German

Der jüngste Bericht von STM behandelt die Notwendigkeit von vertrauenswürdigen Identitätssystemen im Bereich der wissenschaftlichen Publikationen, um den wachsenden Herausforderungen im Hinblick auf Forschungsintegrität und Betrug zu begegnen. Historisch basierte die wissenschaftliche Publikation auf implizitem Vertrauen, doch der Anstieg von Identitätsmanipulationen zeigt die Schwachstellen dieses Modells auf. Der Bericht empfiehlt eine Zusammenarbeit zwischen Verlagen und risikobasierte Lösungen, die Sicherheit und Inklusivität ausbalancieren. Zukünftige Arbeiten sollen Pilotversuche zur Identitätsverifizierung und die Berücksichtigung von Datenschutzproblemen umfassen.

Spanish

El informe reciente de STM explora la necesidad crítica de sistemas de identidad confiable en la publicación académica para contrarrestar los crecientes desafíos relacionados con la integridad de la investigación y el fraude. Históricamente, la publicación académica dependía de la confianza implícita, pero el aumento de la manipulación de identidades ha revelado vulnerabilidades. El informe recomienda esfuerzos colaborativos entre las editoriales, soluciones basadas en riesgos proporcionales y un equilibrio entre seguridad e inclusividad. El trabajo futuro implica probar enfoques de verificación de identidad y abordar preocupaciones de privacidad.

Tags from OpenAI:

academic publishing, research integrity, trust and identity, information verification, scholarly peer-review issues, misconduct in research, privacy concerns


About Ian Mulvany

Hi, I'm Ian - I work on academic publishing systems. You can find out more about me at mulvany.net. I'm always interested in engaging with folk on these topics, if you have made your way here don't hesitate to reach out if there is anything you want to share, discuss, or ask for help with!