Read the map and sources

Draft · Current AI v.0.1.0

A documented draft of a text assistant that runs its model on servers. The browser sends the request to the provider, which processes it and returns the result to the same interface. Controls and tools depend on the product.

Request

What is the system being asked to do?

Example. “Write a short invitation to the neighbourhood book fair.”

The user decides what to ask for and writes a message. This map follows that message from submission in a web chat to the result displayed on screen.

Vercel · AI chat application

Connects to: Choose a model and send the request

Choose a model and send the request

What does the interface send to the provider?

Example. You choose a model from the chat selector and press Send. The browser transmits the model name and your request to draft the invitation.

The user writes in the chat and chooses a model from those offered by the interface. If the chat has no model selector, it uses the service’s default model. When the user presses Send, the interface transmits the message and, when applicable, the selected model’s identifier.

Vercel · AI chat application

Connects to: Request and selected model: Receive the request on the server

Receive the request on the server

What does the server do when it receives the request?

Example. The server receives “Write an invitation to the fair” along with the model you selected.

The application server, also called the backend, receives the browser’s message and the selected model identifier if one was supplied. This lets the provider route the request to the corresponding model.

Vercel · AI chat application

Connects to: Load or reuse the selected model

Load or reuse the selected model

Where is the selected model loaded?

Example. The provider already has the selected text model loaded on one of its servers. It routes the request about the fair there.

The provider identifies the engine that will serve the requested model. It can route the request to an engine that already has the model in memory or load it if needed. Gathering the messages can overlap with loading. The engine needs the model ready to run the computation.

Hugging Face · Text Generation Inference architecture

Hugging Face · Starting an inference endpoint on demand

Connects to: Open or restore the active chat

Open or restore the active chat

Where is this conversation’s work kept?

Example. The fair chat is the workspace for that project. You already noted “Admission is free” there. When you return to the chat, the application restores what was on the desk and adds your new message.

The application identifies the chat the request belongs to. If it is new, it creates a thread with the first message. If it already exists, it retrieves the messages and any retained work data. The chat can exist before the model loads. On each turn, the application starts from that thread and selects what to send to the model.

LangGraph · Persistence by thread and across threads

LangGraph · Manage short-term memory

Connects to: Gather the request context · Read and save: Cross-chat memory (optional)

Cross-chat memory (optional)

What can be retrieved or saved for other chats?

Example. You ask the service to save “I prefer short invitations.” If the feature is enabled, the service could retrieve that preference when drafting an invitation for another event in a different chat.

Cross-chat memory is a store outside the thread. If enabled, the application can look up relevant details for this turn and save specific information for reuse, such as a preference. Writes may occur during the turn or in the background. The service determines the scope and rules for saving.

LangGraph · Persistence by thread and across threads

LangChain · Memory overview

Connects to: Read and save: Open or restore the active chat

Model architecture

How is computation organised?

Example. Imagine a workshop with several workbenches. Its layout specifies the order work passes through them and how they share information.

Configuration specifies properties such as model type, layers, dimensions and available mechanisms. It defines the computational organisation that uses the weights. Changing request text does not redesign this architecture.

Hugging Face · Model configuration

Connects to: Trained model · Defines the computation: Training and parameter updates

Learned parameters

What persists between runs?

Example. Think of the settings on a mixing desk. It has the same knobs, but their positions change the result.

Weights are numerical values obtained during training and stored to load the model. They participate in the transformations producing output. They are not, by themselves, a faithful record of every training document.

Hugging Face · Loading pretrained models

Connects to: Trained model

Trained model

What is the model itself?

Example. The same trained model can draft the invitation and then suggest a title using the instructions it receives each time.

It is a computational structure with learned parameters. Using a pretrained model involves loading compatible configuration and weights. The active conversation, tools and permissions belong to the system running it.

Hugging Face · Loading pretrained models

Connects to: Load or reuse the selected model

Processors, memory and energy

What physical limits constrain execution?

Example. The processor does the calculations and RAM keeps the data at hand while it is in use.

Inference uses processors, memory and electricity. Memory holds the weights and temporary computation data. With longer sequences, the attention cache can grow and the engine typically does more work, which can increase energy use. The memory available on the server can limit the context size the service offers.

Hugging Face · Speed and memory

vLLM · Model context length

Hugging Face · Cache strategies

Fernandez et al. · Energy considerations of LLM inference

Connects to: Load or reuse the selected model

Style set by the service team

Who chooses the tone or format?

Example. The library team saves the instruction “Use a friendly tone.” It applies across different chat messages.

The people who develop or manage the service can choose how the assistant phrases its messages. They can save a shared style in the application configuration.

Google · Gemini system instructions

Connects to: Tone and format instruction

Tone and format instruction

When does the model receive the style instruction?

Example. One way to ask for the time is “State the start time.” In a friendlier tone, the same question could be “Could you tell me what time the fair starts, please?”

The server can add a writing instruction to the messages before asking the model to generate text. For example, “Use a friendly tone” can guide the wording of a question or answer from the first draft.

Google · Gemini system instructions

Google · Tone in system instructions

Connects to: With the messages: Gather the request context

Gather the request context

Who gathers the information sent to the model?

Example. The server retrieves “The fair is on Saturday” and “Admission is free” from earlier messages and combines them with your request.

Using the active chat, the server gathers what it needs for this turn. It includes the new message and can add earlier messages, tool results, service instructions or relevant memories. That selection forms the context the model receives.

Vercel · AI chat application

Hugging Face · Chat templates

Anthropic · Tool use

LangGraph · Manage short-term memory

Connects to: Fit the context window

Fit the context window

What determines the maximum width of the workspace?

Example. Imagine a workspace with room for 8,000 tokens. If the service leaves 1,000 for writing the answer, up to 7,000 remain for the request, instructions and fair notes. A summary can take the place of several pages of chat history.

In the analogy, the context window is the maximum width of the workspace for one turn. It is measured in tokens and includes the input and room to generate the answer. The limit starts from what the model supports and can be lower depending on the service configuration and available memory.Processing more tokens generally takes more computation. The attention cache can occupy more memory and electricity use can rise. The cost depends on the model, hardware and how inference is run.If the chat material exceeds the available space, the service can select or summarise messages. You can also start another chat and include a summary in the new request.

Anthropic · Context windows

vLLM · Model context length

Hugging Face · Cache strategies

Fernandez et al. · Energy considerations of LLM inference

LangGraph · Manage short-term memory

Connects to: Service input checks

Service input checks

Do configured checks allow processing to continue?

Example. The invitation request can pass this check. Another request might be blocked by the service’s rules.

The service may examine a request and block certain content categories. Thresholds and available protections depend on the product. This box represents a documented layer in some services, not a guarantee of universal detection.

Google · Gemini safety settings

Connects to: Send the messages to the engine · Blocked request: Return the response to the browser

Send the messages to the engine

How does the request reach the inference service?

Example. The server sends the engine the request about the fair, details from the conversation, and the instruction about tone.

If the request can proceed, the application server sends the gathered messages to the inference service. The software serving the model prepares those messages for computation.

Vercel · AI chat application

Hugging Face · Text Generation Inference architecture

Hugging Face · Chat templates

Connects to: Format messages and convert them to tokens

Format messages and convert them to tokens

Who converts the context into input for the selected model?

Example. The template marks “Write an invitation” as a user message. The tokenizer converts the markers and text into numbers the model can process.

The inference software applies a template compatible with the selected model, for example to mark who wrote each message. A compatible tokenizer converts that text into numeric identifiers. The model receives this sequence and processes it using its weights.

Hugging Face · Chat templates

Hugging Face · Tokenization algorithms

Connects to: Numerical representations

Numerical representations

How are input elements represented?

Example. It is like moving from a reference number to several coordinates. The model can work with those coordinates and the fragment’s position in the sentence.

Vector representations allow numerical operations on discrete elements. Language models use token representations and mechanisms accounting for position. These are computational objects; this box does not attribute sensory experience to AI.

Google · Embeddings

Connects to: Run layers and attention

Run layers and attention

How is contextual information related?

Example. When continuing “the fair will be on…”, attention can combine that fragment with “Saturday” from an earlier message.

Layers transform numerical representations. Attention combines information from different positions; causal generation restricts access to future positions. This operation is neither conscious observation nor a truth check.

Hugging Face · Attention mechanisms

Connects to: Next-token probabilities

Next-token probabilities

What could continue the sequence?

Example. After “The fair will take place on”, “Saturday” and “Sunday” are two possible continuations. The model gives them different scores.

The model calculates scores for possible continuations. Generation uses these scores to select tokens. A likely continuation can be wrong: textual probability and factual truth are different criteria.

Hugging Face · Text Generation Inference

Connects to: Write one token at a time

Write one token at a time

How do the wording and tone take shape?

Example. With a friendly-tone instruction, the model can start writing “Admission is free!” in its first draft.

The generation strategy selects a continuation from the scores. It may use deterministic selection or sampling with configurable parameters. Each token joins the sequence and influences the next one. If a tone instruction is present, it can shape the words and expressions generated from the start.

Hugging Face · Generation strategies

Hugging Face · Chat templates

Google · Tone in system instructions

Connects to: Has this generation finished?

Has this generation finished?

Is another token needed?

Example. When the invitation is complete, generation can produce an end signal and finish the turn.

Generation ends according to stopping conditions such as an end token or length limit. Otherwise the next step uses the extended sequence. Ending a generation does not necessarily complete the whole task.

Hugging Face · Text generation

Connects to: Yes: The model can identify what is missing · No: Reuse temporary states

Reuse temporary states

Which calculations can be reused?

Example. It is like keeping partial calculations on a notepad to reuse them in the next calculation.

The key-value cache keeps attention results for earlier tokens to avoid recomputing them. It speeds up generation and consumes memory. It is temporary execution state, distinct from changing learned parameters.

Hugging Face · How caching works

Connects to: Next token: Run layers and attention

The model can identify what is missing

How does this show up in the model’s output?

Example. The model receives the request to draft an invitation and the messages about the day and admission. It may notice the missing time and generate “What time does the fair start?”

The model interprets the request using the available information and what it learned during training. It may identify that it needs more information and ask a question or request a tool. It may also give an answer, overlook a missing detail or make an assumption.

Hugging Face · Text Generation Inference

Anthropic · When required parameters are missing

Anthropic · Tool use

Connects to: Does the model request a tool?

Does the model request a tool?

Which path should the application follow?

Example. If the model requests a search of the library calendar to find the time, the server checks whether it may use that tool.

The application reads the generated output. If the model issued a tool call, the service checks whether it can run it. Text output can be a clarifying question or an answer to the request.

Anthropic · Tool use

Connects to: No: Does the model ask for clarification? · Yes: Authorise the specific action

Authorise the specific action

Is this specific action permitted?

Example. The service checks whether the tool is allowed to read the library’s public calendar.

Permissions should be checked in the application and destination services. OWASP recommends least privilege and validating every tool request. Authorising an action should not depend solely on the model proposing it.

OWASP · Excessive Agency

Connects to: Execute or return a denial

Execute or return a denial

What happens to the requested action?

Example. The tool queries the calendar and returns “Saturday, 11 a.m.” If the query lacks permission, it returns a denial.

The system executes the authorised operation and receives data or an error. An unauthorised operation should return a denial. Inputs and outputs need validation: generated text should not become code or commands without appropriate handling.

OWASP · Improper Output Handling

Connects to: Add results to context

Add results to context

How is retrieved information used?

Example. The returned time is added to context with its source. The next draft can use the 11 a.m. time from the public calendar.

Returned information joins the context with its provenance. External documents or results may contain malicious instructions and should be treated as untrusted data. Including a result does not automatically make it true.

Anthropic · Mitigating jailbreaks and prompt injection

Connects to: Open or restore the active chat

Does the model ask for clarification?

What happens if the model needs a detail from the user?

Example. “What time does the fair start?” is a clarifying question. “Join us on Saturday at noon” can already be part of an invitation.

This branch distinguishes two kinds of generated text. If the model has asked a question to move forward, the service can deliver it. If it has drafted an answer, that answer proceeds to review. Both outputs go through the service’s delivery checks.

Anthropic · When required parameters are missing

Vercel · AI chat application

Connects to: No: Review content and tone (optional) · Yes: Question written by the model

Question written by the model

How does the user’s reply enter the context?

Example. When the question appears in the chat, the user replies “At noon.” The next run receives that message too.

The question generated by the model goes through review before appearing in the chat. When the user replies, that message starts another request through the same interface. The application includes the new detail and available conversation in the context the model processes next.

Anthropic · When required parameters are missing

Vercel · AI chat application

Connects to: Review content and tone (optional)

Review content and tone (optional)

When can a final edit happen?

Example. If the first draft says “There is no charge for admission,” the service can ask for a friendlier version. The model might return “Admission is free!”

If the service reviews a draft, it can verify facts against external sources or ask for a different tone while retaining the information. Rewriting with the model means generating another version from the draft and the instruction. When used, the new version goes through review and the output checks before delivery.

Madaan et al. · Self-Refine

Google Cloud · Professional rewrite

Connects to: Revise again?

Revise again?

Is another iteration justified with the available resources?

Example. If the address is missing and available, another draft is worthwhile. If the attempt limit has been reached, the service decides how to finish.

The model may flag a missing detail when reviewing a draft. An external check can also uncover a problem. The application uses that result and applies its rules, such as the retry limit, to decide whether to prepare another run.

Madaan et al. · Self-Refine

Connects to: No: Service output checks · Yes: Incorporate the review result

Incorporate the review result

What is sent to the model on the next attempt?

Example. Review finds that the draft says “in the afternoon,” even though the calendar lists noon. That feedback is added to the context and the model runs again.

The application adds the review feedback, correction or newly obtained detail to the context. The model can generate a different response from that input. Its weights stay fixed during the interaction. Training can change them for future runs.

Madaan et al. · Self-Refine

Google · Fine-tuning and prompt engineering

Connects to: Open or restore the active chat

Service output checks

Can the generated content be delivered?

Example. Before showing the invitation, the service can detect that the draft included a personal detail its rules protect.

Some services also filter generated responses and may block them under safety categories. Policies and thresholds depend on the service. Passing a filter does not establish the absence of every risk or error.

Google · Gemini safety settings

Connects to: Return the response to the browser

Return the response to the browser

What does the server return to the interface?

Example. The server sends the invitation text back to the browser.

The server prepares the output it can deliver. This may be an answer, a clarifying question or a message explaining a limit. It returns it to the browser that sent the request. In a streaming chat, the server sends the text in chunks.

Vercel · AI chat application

OWASP · Improper Output Handling

Connects to: Response from the provider: Display in the same interface

Display in the same interface

Where does the user see the response?

Example. The invitation appears in the same chat where you requested it.

The same chat interface that sent the request receives the server output and displays it. The browser handles presentation. In this flow the model’s computations take place entirely on the provider’s servers.

Vercel · AI chat application

Hugging Face · Text Generation Inference architecture

Connects to: Delivered result

Delivered result

What concludes this flow?

Example. You receive an invitation with a date, place and time. The next request can use the same model with different context.

The user receives the result in the chat. If they reply or send another message, the interface starts a new request that can include the earlier conversation. The model weights stay fixed during these interactions.

Vercel · AI chat application

Google · Fine-tuning and prompt engineering

Data for a separate training process

What information is selected to improve the model?

Example. To adapt an image generator to book fair posters, you could gather 500 varied posters and their descriptions. The number is illustrative.

Data is selected according to what the model should learn. For a text-to-image generator, images and captions help associate visual features with words. The quality and variety of the examples matter. A file attached in normal use serves as input if the product supports that format.

Hugging Face · Training datasets for image generation

Connects to: Training and parameter updates

Training and parameter updates

What does it mean to train a model?

Example. Fine-tuning an existing generator on those posters changes weights to favour useful patterns. You can then ask it for a new poster for a different fair.

Training adjusts weights using examples and a defined objective. The architecture determines how each prediction is calculated and which weights can be adjusted. Training measures the prediction error against the objective and updates the weights to reduce it. It repeats with more examples so the model can apply learned patterns to new cases. Adapting an already trained model to a task is called fine-tuning.

Hugging Face · Fine-tuning a pretrained model

Connects to: Evaluate a candidate version

Evaluate a candidate version

How are capabilities and limitations checked?

Example. Use prompts and images held out from fine-tuning to see whether the generator can make new posters in the intended style.

Evaluation uses tasks, datasets and metrics to measure behaviour. Results depend on what is assessed and do not prove universal reliability. A candidate version needs evaluation and deployment decisions outside this conversation.

Hugging Face · Evaluate

Connects to: Version available for future runs

Version available for future runs

How does an improvement reach later requests?

Example. If the adapted generator passes evaluation and is deployed, future poster requests can use that version.

When you use an assistant, requests run with the weights of the deployed version. A correction can improve the answer by adding context. Changing the weights used by future requests requires a separate training or fine-tuning process and deploying the resulting version.

Google · Fine-tuning and prompt engineering

Connects to: Weights for future runs: Learned parameters