Ian Mulvany

January 2, 2024

Some great advice on working with LLMs

Cross posting this blog - https://towardsdatascience.com/classifying-source-code-using-llms-what-and-how-f04c7dbcba9b

It’s chock full of great advice on using LLMs. The task they worked on was determining if some given code was malicious. 

Some key advice: 

- running an LLM is expensive, check if another method might work for you. 
- LLMs can often pay more attention to instructions at the start and end of an input (they phase out on the content in the middle). 
- be consistent in the terms you use to prompt the LLM. 
- for classification tasks you can ask both for the classification, and the reason why the LLM applied the classification. Helpful for debugging. 
- you can ask the LLM to take on different kinds of personas and this gives you coverage of different points of view. These can be combined with some kind of consensus algorithm to get better results. 
- remember, another more powerful LLM is likely to come along at some point. 
The whole post is worth reading.  

About Ian Mulvany

Hi, I'm Ian - I work on academic publishing systems. You can find out more about me at mulvany.net. I'm always interested in engaging with folk on these topics, if you have made your way here don't hesitate to reach out if there is anything you want to share, discuss, or ask for help with!