1 LLMs predictably get more capable with increasing investment, even without targeted innovation. 
2 Many important LLM behaviours emerge unpredictably as a byproduct of increasing investment. 
3 LLMs often appear to learn and use representations of the outside world. 
4 There are no reliable techniques for steering the behaviour of LLMs. 
5 Experts are not yet able to interpret the inner workings of LLMs. 
6 Human performance on a task isn’t an upper bound on LLM performance. 
7 LLMs need not express the values of their creators nor the values encoded in web text. 
8 Brief interactions with LLMs are often misleading. 

recent methods can dramatically reduce explicit bias and toxicity in models’ output, largely by exploiting the fact that models can often recognize these bad behaviors when asked (Dinan et al., 2019; Bai et al., 2022b; Ganguli et al., 2023). While these mitigations are unlikely to be entirely robust, the prevalence and prominence of these bad behaviors will likely wane as these techniques are refined.

though, these encouraging signs do not mean that we can reliably control these models, and the issues noted in Section 4 still apply

Even though our AI models were trained by reading the whole internet, that training mostly enhances their reasoning abilities not how much they know. And so, the performance of today’s AI models is constrained by their lack of knowledge.

This is crucial to understand because it predicts that advances in the usefulness of AI will come from advances in its ability to access the right knowledge at the right time—not just from advances in its reasoning powers.

So, what does this mean for the future? I think there are at least two interesting conclusions:
1 Knowledge databases are as important to AI progress as foundational models
2 People who organize, store, and catalog their own thinking and reading will have a leg up in an AI-driven world. They can make those resources available to the model and use it to enhance the intelligence and relevance of its responses.  

We’re already seeing newer alternatives springing up that wrap some  [business logic](https://en.wikipedia.org/wiki/Business_logic)  around the database layer to make it easier for AI developers to do common tasks. Some of these are developer libraries like Langchain and LlamaIndex. And some seem to be more fully featured developer tools like  [Metal](https://www.getmetal.io/?utm_source=every)  and  [Baseplate](https://www.baseplate.ai/?utm_source=every) . Just like Pinecone, are also likely to raise a lot of money or already have! AI’s advancement is a raindance that calls forth capital from Patagonia vest wearing angels.

Using language models effectively is deceptively difficult So many of the challenges involving language models come down to this: they look much, much easier to use than they actually are. To get the most value out of them—and to avoid the many traps that they set for the unwary user—you need to spend time with them, and work to build an accurate mental model of how they work, what they are capable of and where they are most likely to go wrong.

I hope this “calculator for words” framing can help.

A flaw in this analogy: calculators are repeatable [Andy Baio](https://waxy.org/)  pointed out a flaw in this particular analogy: calculators always give you the same answer for a given input. Language models don’t—if you run the same prompt through a LLM several times you’ll get a slightly different reply every time. This is a very good point! You should definitely keep this in mind. All analogies are imperfect, but some are more imperfect that others.

Although humans are still better than GPT at a lot of things, there are many jobs where these capabilities are not used much. For example, many of the tasks done by a person in sales (digital or phone), service, or document handling (like payables, accounting, or insurance claim disputes) require decision-making but not the ability to learn continuously. Corporations have training programs for these activities and in most cases, they have a lot of examples of good and bad work. Humans are trained using these data sets, and soon these data sets will also be used to train the AIs that will empower people to do this work more efficiently.

Company-wide agents will empower employees in new ways. An agent that understands a particular company will be available for its employees to consult directly and should be part of every meeting so it can answer questions. It can be told to be passive or encouraged to speak up if it has some insight. It will need access to the sales, support, finance, product schedules, and text related to the company. It should read news related to the industry the company is in. I believe that the result will be that employees will become more productive

Global health and education are two areas where there’s great need and not enough workers to meet those needs. These are areas where AI can help reduce inequity if it is properly targeted. These should be a key focus of AI work, so I will turn to them now.

First, we should try to balance fears about the downsides of AI—which are understandable and valid—with its ability to improve people’s lives. To make the most of this remarkable new technology, we’ll need to both guard against the risks and spread the benefits to as many people as possible.

Second, market forces won’t naturally produce AI products and services that help the poorest. The opposite is more likely. With reliable funding and the right policies, governments and philanthropy can ensure that AIs are used to reduce inequity. Just as the world needs its brightest people focused on its biggest problems, we will need to focus the world’s best AIs on its biggest problems. 

Finally, we should keep in mind that we’re only at the beginning of what AI can accomplish. Whatever limitations it has today will be gone before we know it.

As a rule of thumb, around 95% of the code we need to build has already been written. Unfortunately, we often don’t have an effective way of finding that code that represents the same function as the code you were about to write. This is why IT shops endlessly build the same things over and over again.

I don’t see a long-term career in software anymore. Any dreams I had of earning decent money as a software engineer are slowly fading. Lex Fridman in his podcast said “if you’re anxious about GPT4 its probably because you’re a shitty programmer”. I mean, I’m not the smartest in the room but I have generated value with the software I’ve made. And was convinced that I’ll make decent money as long as put in the work. I’m just not that sure anymore.

In general, GPT-4 feels like a junior developer who is very fast at typing and knows about a lot of libraries, but is careless and easily confused.

Wilfrid Sellars, a philosopher of language, termed this world of human things the “manifest image” to contrast it against the world of atoms bumping around in the void (the “scientific image”), which is better understood.

Knowledge: a database of facts; language: a series of rules; meaning: logical propositions. All this is clumsy and excessive literalism to Skinner, without any scientific merit. In short, he thinks the bridge between the manifest image and the scientific image is not sturdy at all, and talk of mental software can only ever be a metaphor.

Indeed, anyone who has tried and failed to pin down something as simple as what a government or a fight or a game is in precise terms will know that things in the manifest image are strangely elusive, even those as seemingly concrete and exact as words or physical objects. 

It is able not just to respond to questions but to respond in the way you’d expect if it did indeed understand what was being asked. And, to take the viewpoint of Ryle, it genuinely does understand — perhaps not as adroitly as a person, but with exactly the kind of true intelligence we attribute to one.

But this behavior would be very unexpected in GPT, whose training doesn’t incentivize instrumental behavior that optimizes prediction accuracy! GPT does not generate rollouts during training. Its output is never sampled to yield “actions” whose consequences are evaluated, so there is no reason to expect that GPT will form preferences over the consequences of its output related to the text prediction objective._14_

What it does is more like animation than divination, executing the dynamical laws of its rendering engine to recreate the flows of history found in its training data

it was not optimized to be correct but rather realistic, and being realistic means predicting humans faithfully even when they are likely to be wrong.

GPT is behavior cloning. But it is the behavior of a universe that is cloned, not of a single demonstrator, and the result isn’t a static copy of the universe, but a compression of the universe into a generative rule. This resulting policy is capable of animating anything that evolves according to that rule: a far larger set than the sampled trajectories included in the training data, just as there are many more possible configurations that evolve according to our laws of physics than instantiated in our particular time and place and Everett branch.

The term “semiotic physics” here refers to the study of the fundamental forces and laws that govern the behavior of signs and symbols. Similar to how the study of physics helps us understand and make use of the laws that govern the physical universe, semiotic physics studies the fundamental forces that govern the symbolic universe of GPT, a universe that reflects and intersects with the universe of our own cognition. We transfer concepts from dynamical systems theory, such as attractors and basins of attraction, to the semiotic universe and spell out examples and implications of the proposed perspective.

small changes in the initial conditions do not lead to substantially different continuations.

A language model might create one particular trajectory rather than another because of the shape of the attractor landscape.