diff --git a/docs/async.md b/docs/async.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/batch.md b/docs/batch.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/communication.md b/docs/communication.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/flow.md b/docs/flow.md new file mode 100644 index 0000000..b1eebde --- /dev/null +++ b/docs/flow.md @@ -0,0 +1,7 @@ +--- +layout: default +title: "Flow" +--- + +# Flow + diff --git a/docs/index.md b/docs/index.md index 11303f7..8bf1275 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,11 +5,22 @@ title: "Home" # Mini LLM Flow -A **100-line minimalist LLM framework** for agents, task decomposition, retrieval-augmented generation, and more. +A 100-line minimalist LLM framework for agents, task decomposition, RAG, etc. + +![Alt text](/docs/assets/minillmflow.jpg) ## Core Abstraction -- [Node & Flow](./node_and_flow.md) +We model the LLM workflow as a **Nested Flow**: + +- Each **Node** handles a simple LLM task (e.g., text summarization, structure extraction, or question answering). +- Nodes are chained together to form a **Flow** for more complex tasks. One Node can be chained to multiple Nodes based on **Actions**, e.g., for agentic steps. +- A Flow can be treated as a Node for **Nested Flows**. +- Both Nodes and Flows can be **Batched** for data-intensive tasks (e.g., processing one file at a time in a directory of files). +- Nodes and Flows can be **Async**, e.g., for user feedback before proceeding. + +- [Node](./node.md) +- [Flow](./flow.md) - [Communication](./communication.md) - [Batch](./batch.md) - [Async](./async.md) @@ -20,6 +31,7 @@ A **100-line minimalist LLM framework** for agents, task decomposition, retrieva - Agent - Map Reduce - RAG +- Structured Output ## Example Use Cases diff --git a/docs/node.md b/docs/node.md new file mode 100644 index 0000000..7552496 --- /dev/null +++ b/docs/node.md @@ -0,0 +1,72 @@ +--- +layout: default +title: "Node" +--- + +# Node + +A **Node** is the smallest building block of Mini LLM Flow. Each Node has three lifecycle methods: + +1. **`prep(shared)`** + - Optionally preprocess data before calling your LLM or doing heavy computation. + - Often used for tasks like reading files, chunking text, or validation. + - Returns `prep_res`, which will be passed to both `exec()` and `post()`. + +2. **`exec(shared, prep_res)`** + - The main execution step, typically where you call your LLM or any external APIs. + - Returns `exec_res`, which is passed to `post()`. + +3. **`post(shared, prep_res, exec_res)`** + - Optionally perform post-processing, such as writing results back to the `shared` store or deciding the next action. + - Often used to finalize outputs, trigger next steps, or log results. + - Returns a **string** to specify the next action (`"default"` if nothing or `None` is returned). + + +## Fault Tolerance & Retries + +Nodes in Mini LLM Flow can **retry** execution if `exec()` raises an exception. You control this via a `max_retries` parameter when you create the Node. By default, `max_retries = 1` (meaning no retry). + +``` +# Example usage: +my_node = SummarizeFile(max_retries=3) +``` + +When an exception occurs in `exec()`, the Node automatically retries until: + +- It either succeeds, **or** +- The Node has retried `max_retries - 1` times already and fails on the last attempt. + +If you want to **gracefully handle** the error rather than raising it, you can override: + +`` +def process_after_fail(self, shared, prep_res, exc): + raise exc +`` + +By **default**, it just re-raises `exc`. But you can return a fallback result instead. That fallback result becomes the `exec_res` passed to `post()`. + +## Minimal Example + +`` +class SummarizeFile(Node): + def prep(self, shared): + filename = self.params["filename"] + return shared["data"][filename] + + def exec(self, shared, prep_res): + if not prep_res: + raise ValueError("Empty file content!") + prompt = f"Summarize this text in 10 words: {prep_res}" + summary = call_llm(prompt) # might fail + return summary + + def process_after_fail(self, shared, prep_res, exc): + # Provide a simple fallback instead of crashing + return "There was an error processing your request." + + def post(self, shared, prep_res, exec_res): + filename = self.params["filename"] + shared["summary"][filename] = exec_res + # Return "default" by not returning anything +`` +