How AI language models work (just enough to use them well)

You do not need to understand the math behind AI to use it well, any more than you need to understand an engine to drive. But a few basic ideas will explain almost every surprising thing AI does, good and bad, and help you decide where it belongs in your business.

What a large language model is

The AI behind today's chat assistants and AI automation steps is a large language model (LLM). It was trained on an enormous amount of text, and from that it learned the patterns of language: grammar, facts that appear often, how arguments are structured, how a polite email sounds, how a contract is laid out.

When you give it a prompt, it generates a response one small piece at a time, each time choosing a likely next piece based on everything before it. That simple process, at huge scale, produces remarkably capable writing and reasoning.

What that means in practice

Because LLMs work with patterns in language, they are excellent at language tasks:

• Reading and sorting: is this email a complaint, a sales lead, or an invoice?
• Extracting: pull the name, date, and amount out of this document.
• Summarizing: turn a 40-minute call into five bullet points and next steps.
• Drafting and rewriting: write a first reply, make this friendlier, shorten this to 100 words.
• Answering from supplied material: answer this question using only our policy document.

And they have predictable weaknesses:

• They can be confidently wrong. If the answer is not in the prompt or well represented in training data, the model may produce something that sounds right but is not. This is often called a hallucination.
• They do not know your business. Your prices, policies, and customers are not in the model unless you include them.
• Their knowledge has a cutoff. Unless connected to a search or your systems, they do not know recent events.
• They are not calculators or databases. For exact math or record lookups, use regular software and let AI handle the language around it.

Tokens and context windows

Models read and write in tokens, chunks of text roughly three quarters of an English word. Two practical consequences:

1. Cost: AI providers charge per token, for both what you send and what comes back. The AI cost calculator shows how that adds up.
2. Context window: each model can only consider a limited amount of text at once, including your instructions, any documents, and its answer. Modern models handle long documents, but sending only what is relevant keeps results sharper and cheaper.

Models do not remember by default

Each request to a model through an API is independent. A chat assistant appears to remember the conversation because the app resends the earlier messages each time. In an automation, if the AI needs customer history or your policies, the workflow must include that information in the prompt, or retrieve it first. This is the idea behind retrieval-augmented generation (RAG): find the relevant facts in your own data, then give them to the model with the question.

Bigger is not always better

Providers offer several model sizes. Large models are better at nuanced writing and multi-step reasoning; small models are faster and much cheaper, and are often perfectly good at narrow tasks like classifying an email. A common pattern is to use a small model for high-volume sorting and a larger one for the occasional complex draft.

Next, you will learn how to write prompts that make the output reliable.