Willison's llm

Simon Willison has created a marvelous tool for experimenting with LLMs at the command line: llm. It employs a plugin architecture and offers the ability to connect to hundreds of models, both remote and local, and supports direct prompts and embeddings. This tool makes it extremely easy to experiment with different LLMs.

It uses sqlite as the back-end data data store. While this is admirably simple, it is also somewhat limiting. I almost immediately produced embedding sets too large for sqlite to handle well. I hope to extend llm to support a PostgreSQL storage layer. This part of the program is not architected for extension by plugin, so the update is not so straightforward.

The llm program offers many features:

• Extension through plugins
• API access to underlying features
• Model aliases
• Conversations threading together multiple requests
• System prompts
• Image attachments
• Prompt templates with variable substitution
• Request logging
• Extensive documentation

There are also useful related tools:

• files-to-prompt: Structure files for prompt inclusion
• ttok: Count prompt tokens

If you want to experiment with LLMs and RAG models, I don’t know a better tool to learn with. Try it out!