GenAI just over a year on
I made some predictions in May of 2023 at the BMJ board offsite. Let's review where we are now compared to what I said just over a year ago.
Distribution
We said: LLMs could disrupt discovery services such as Google.
Since then: Google have accelerated introducing LLMs into their own search results, this remains a material risk, as Google is looking to choke off referrer traffic. We have not seen an impact yet, but need to keep an eye on this.
Status: Remains a risk to be monitored.
Tooling
We said: Tools are taking jobs, any text based activities are in target.
Since then:
- Software development with LLMs has advanced, has enhanced developer productivity, but not displaced it.
- Companies that provide copy editing services to STM have seen a collapse in some areas of revenue.
- Marketing and media teams in BMJ increasingly use these tools, displacing some work.
- We have not seen this impact front line customer service yet, but that has happened in other industries.
Status: Largely held true, and will continue to roll through into different roles.
Threats
We said: Could pose a brand threat due to hallucination, or a flood of fake papers.
Since then:
- STM has seen relatively few incidents of fake papers from these tools.
- Those have been identified quickly BMJ has seen none. Our governance process has worked well.
- Tools used for authors who have english as a second language has been a huge benefit.
Status:
- Overwhelmingly being used responsibly within BMJ.
- Indicates governance model could evolve away from risk and towards opportunities.
Local Models
We said: Models run locally will become significantly important.
Since then: Apple announced local models as a core part of their platform from 2025.
Status: Will emerge, and will continue to be a trend.
Limitations of models
We said: Models circa May 2023 had limitations, those limitations would gradually reduce.
Since then: At least three improved generations of limitation reducing models have been released.
Status: This trend is set to continue, with improvements expected every four to six months.
Language Generation
We said: This would accelerate the number of fake papers that we might receive.
Since then: We have not observed a significant increase, probably due to the high quality of our titles.
Status: Probably there are more opportunities for us to help our authors, than threats.
Workflow
We said: Opportunities to improve internal workflow, maybe including in peer review.
Since then: Has become a critical part of workflow for our learning products, we continue to believe in opportunities for peer review, but have been constrained.
Status: Worth continued focus.
Changing how we interact with information
We said: Long term, could disrupt how academics interact with our services.
Since then: High amount of interest in these tools in academic (see reports mentioned earlier) and high levels of investment by our completion, but not radical change has happened since the last year (nor would we expect it in that timeline)
Status: Worth continued observation.
Importance of Access to high quality data
We said: New markets for training data may emerge.
Since then: Other STM publishers have signed six and seven figure licensing deals, but access to payer remains hard.
Status: Has held true, but the market that is emerging is fragmented, and very early.