What Are Language Models Good For?

Today we discuss the specific strengths of large language models and how they compare to classic symbolic AI. How can we explain what LLMS are good at?

First, LLMs are good at language modeling (duh). That's the most accurate but least relatable explanation. They do statistical prediction based on how language is used in their training set. This is useful because we use words in so many ways, and at scale deeper concepts are modeled. It takes some experience to understand what kind of work is a closer language modeling task.

Another take is that LLMs can convert information from one form into another. This could mean translating from one language to another, or ELI5 (explain like I'm five), many things that could call translation in a loose sense. As a result of that they can convert between structured and unstructured data. This makes a handy bridge between our world of human-written content and programs that process data at scale. For instance, we can tag metadata on existing documents, enabling better search and reporting.

Finally, since LLMs can make it cheap to generate plausible answers, it's easier to get value out of them when we have know how to check the results. This is part of why agents work well, they are an easy way of checking the results by giving the LLM contact with the world. This is also part of why code generation is a strong application for LLMs, we have ways of testing the results.

Concepts

• Large Language Models
• Foundation Models
• Natural Language Processing

• Knowledge Representation
• Data Modeling
• Agents

References

• Explanation of GPT: ⁠Understanding “Attention is All You Need” Using the 3-Pass Method⁠
• Another kind of language model: BERT 101
• What happened to the Semantic Web?

Hosted by Ray Myers and Liz Martinez. Find us online at ⁠⁠⁠thecodingagency.org⁠⁠⁠.