Our knowledge world is in the process of a shift in affordances. The development of large language models is creating tools and assistants that will change how we interact with information. I feel sometimes that I have a seat near the window, looking out at these changes happening before my eyes, but being separated from them, as I'm not involved in any way with their development. We hear about what ChatGPT can do, and we can see the kinds of images that StableDiffusion can generate, but what does it actually take to make these kinds of models? The following longish post https://www.semianalysis.com/p/nvidiaopenaitritonpytorch is a very nice write up of some of the shifts that are going on in the computing landscape to support the development of these kinds of models. I think for anyone interested in the impact that these technologies are having, this is a really interesting post to read through. 

One of the stories that emerges from this post is how the computing infrastructure is being driven by ease of use of the developer experience, and how those shifts are undercutting a closed source incumbent. 

One of the other interesting things in the article is the race for computational performance, and an understanding of where the limits are in the computational domain for generating large language models (memory bound and not compute bound). 

I have this vision of billions of dollars worth of silicon, straining to push data from one compute node to another, bargaining going on from vendors, multi-billion dollar contracts to build cathedrals to computation, occasional innovations that allow multiple runs to be pushed through the architecture at the same resource footprint that a single run used to use. 

I had a small amount of exposure to high performance computing when I was in university, and I am confident that the imagination of the developers, and the interest of the product folk, will mean that this infrastructure will not go unused for some time. On the other hand, hanging on my wall at home is a memory board from a machine that was mothballed in the 90s. That board cost well over £100K when it was built and initially paid for, and now it is an ornament.