Glossary

Term

Description

large language model (LLM)

A large language model (LLM) is a deep learning model for natural language processing trained on massive datasets (terabytes or more) with a vast number of parameters (billions or trillions). LLMs can understand, reason, and generate natural language text, and in some cases, audio, images, and video.

agent

An agent is a computational entity that perceives its environment, acts on it, and exhibits autonomy, reactivity, social ability, and proactivity. It uses a large language model as its core reasoning engine to perform a closed-loop workflow of Perception, Planning, and Action.

token

A token is the basic unit of text that a large language model (LLM) processes. Depending on the tokenizer used, a token can be a word, a subword, a punctuation mark, or a character. An LLM processes both input and output as sequences of tokens. The context length of a model, such as 8K or 32K, is the maximum number of tokens it supports.

memory

Memory is the mechanism for storing and managing conversation history, user preferences, task states, or external knowledge, which enables a model to maintain context across multiple turns or sessions. Memory can be divided into:

• short-term memory: The context of the current conversation (dialogue history), which is cleared when the session ends.

• long-term memory: User information persisted using vector databases, knowledge graphs, or structured storage, which can be retrieved and used in future interactions.

Tree of Thoughts (ToT)

Tree of Thoughts (ToT) extends Chain of Thought (CoT) by exploring multiple reasoning paths at each step of a task. It breaks a problem down into multiple thought steps and generates several ideas for each step, creating a tree-like structure. The search process can be a breadth-first search (BFS) or a depth-first search (DFS).

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is a technique that combines retrieval and generation models. It generates answers or content by referencing information from an external knowledge base. This approach offers strong interpretability and customization capabilities, making it suitable for various natural language processing tasks, such as question answering systems, document generation, and intelligent assistants. RAG models are versatile, support real-time knowledge updates, and can be evaluated end-to-end to provide more accurate and efficient results.

knowledge base

A knowledge base is a collection of structured or unstructured data that provides external factual information to a model. It is commonly used in Retrieval-Augmented Generation (RAG) architectures. The content of a knowledge base can come from various sources, such as enterprise documents, FAQs, databases, and web pages. When a user asks a question, the system first retrieves relevant snippets from the knowledge base. The system then passes this information to an LLM with the user's query to generate a more accurate and traceable answer.

query rewriting

Query rewriting is a technique used in information retrieval or RAG systems to semantically optimize or restructure a user's original query. The goal is to improve subsequent retrieval or generation. Common forms include query expansion, query refinement, and historical context fusion.

intent recognition

Intent recognition is a core task of natural language understanding (NLU) that determines the goal or purpose (the "intent") behind a user's input. In LLM applications, intent recognition can be used to:

• Route user requests to the appropriate business modules, such as skill-based routing in a customer service bot.

• Trigger specific tool calls or workflows.

• Improve the retrieval relevance of a RAG system by adjusting the retrieval strategy based on the intent.

observability

Observability is the ability to monitor, trace, and diagnose the internal runtime state of an LLM application. Due to the "black box" nature of LLMs, their outputs can be difficult to debug. Observability tools record data at key stages, including prompts, model outputs, retrieval results, tool calls, latency, token consumption, and cost.

LangChain

LangChain is a powerful framework for building end-to-end applications using language models. It provides a set of tools, components, and interfaces that simplify creating applications powered by large language models (LLMs) and chat models.

Langfuse

Langfuse is an open-source observability platform for LLM applications. It is used for tracing, evaluating, and debugging applications built on large language models, such as agents and RAG systems.

Dify

Dify is an open-source LLM application development platform that provides a visual interface and an API to rapidly build AI applications based on large language models. It integrates features such as prompt orchestration, knowledge base management, Retrieval-Augmented Generation (RAG), and agent orchestration, and supports a variety of mainstream large models.

RagFlow

RagFlow is an open-source framework or platform focused on enterprise-grade RAG applications. It emphasizes deep parsing, semantic chunking, and precise retrieval for complex documents such as PDFs, Word documents, and tables.