diff --git a/.cursorrules b/.cursorrules
index 0aba9a3..f260627 100644
--- a/.cursorrules
+++ b/.cursorrules
@@ -1,110 +1,169 @@
================================================
-File: docs/agent.md
+File: docs/guide.md
================================================
---
layout: default
-title: "Agent"
-parent: "Design Pattern"
-nav_order: 6
+title: "Build your LLM App"
---
-# Agent
+# LLM Application Development Playbook
-Agent is a powerful design pattern, where node can take dynamic actions based on the context it receives.
-To express an agent, create a Node (the agent) with [branching](./flow.md) to other nodes (Actions).
+> If you are an AI assistant involved in building LLM Apps, read this guide **VERY, VERY** carefully! This is the most important chapter in the entire document. Throughout development, you should always (1) start with a small and simple solution, (2) design at a high level (`docs/design.md`) before implementation, and (3) frequently ask humans for feedback and clarification.
+{: .warning }
-> The core of build **performant** and **reliable** agents boils down to:
->
-> 1. **Context Management:** Provide *clear, relevant context* so agents can understand the problem.E.g., Rather than dumping an entire chat history or entire files, use a [Workflow](./decomp.md) that filters out and includes only the most relevant information.
->
-> 2. **Action Space:** Define *a well-structured, unambiguous, and easy-to-use* set of actions. For instance, avoid creating overlapping actions like `read_databases` and `read_csvs`. Instead, unify data sources (e.g., move CSVs into a database) and design a single action. The action can be parameterized (e.g., string for search) or programmable (e.g., SQL queries).
-{: .best-practice }
+## System Design Steps
-### Example: Search Agent
+These system designs should be a collaboration between humans and AI assistants:
-This agent:
-1. Decides whether to search or answer
-2. If searches, loops back to decide if more search needed
-3. Answers when enough context gathered
+| Stage | Human | AI | Comment |
+|:-----------------------|:----------:|:---------:|:------------------------------------------------------------------------|
+| 1. Project Requirements | ★★★ High | ★☆☆ Low | Humans understand the requirements and context best. |
+| 2. Utility Functions | ★★☆ Medium | ★★☆ Medium | The human is familiar with external APIs and integrations, and the AI assists with implementation. |
+| 3. Flow Design | ★★☆ Medium | ★★☆ Medium | The human identifies complex and ambiguous parts, and the AI helps with redesign. |
+| 4. Data Schema | ★☆☆ Low | ★★★ High | The AI assists in designing the data schema based on the flow. |
+| 5. Implementation | ★☆☆ Low | ★★★ High | The human identifies complex and ambiguous parts, and the AI helps with redesign. |
+| 6. Optimization | ★★☆ Medium | ★★☆ Medium | The human reviews the code and evaluates the results, while the AI helps optimize. |
+| 7. Reliability | ★☆☆ Low | ★★★ High | The AI helps write test cases and address corner cases. |
-```python
-class DecideAction(Node):
- def prep(self, shared):
- context = shared.get("context", "No previous search")
- query = shared["query"]
- return query, context
-
- def exec(self, inputs):
- query, context = inputs
- prompt = f"""
-Given input: {query}
-Previous search results: {context}
-Should I: 1) Search web for more info 2) Answer with current knowledge
-Output in yaml:
-```yaml
-action: search/answer
-reason: why this action
-search_term: search phrase if action is search
-```"""
- resp = call_llm(prompt)
- yaml_str = resp.split("```yaml")[1].split("```")[0].strip()
- result = yaml.safe_load(yaml_str)
-
- assert isinstance(result, dict)
- assert "action" in result
- assert "reason" in result
- assert result["action"] in ["search", "answer"]
- if result["action"] == "search":
- assert "search_term" in result
-
- return result
+1. **Project Requirements**: Clarify the requirements for your project, and evaluate whether an AI system is a good fit. An AI systems are:
+ - suitable for routine tasks that require common sense (e.g., filling out forms, replying to emails).
+ - suitable for creative tasks where all inputs are provided (e.g., building slides, writing SQL).
+ - **NOT** suitable for tasks that are highly ambiguous and require complex information (e.g., building a startup).
+ - > **If a human can’t solve it, an LLM can’t automate it!** Before building an LLM system, thoroughly understand the problem by manually solving example inputs to develop intuition.
+ {: .best-practice }
- def post(self, shared, prep_res, exec_res):
- if exec_res["action"] == "search":
- shared["search_term"] = exec_res["search_term"]
- return exec_res["action"]
+2. **Utility Functions**: AI system is the decision-maker and relies on *external utility functions* to:
-class SearchWeb(Node):
- def prep(self, shared):
- return shared["search_term"]
-
- def exec(self, search_term):
- return search_web(search_term)
-
- def post(self, shared, prep_res, exec_res):
- prev_searches = shared.get("context", [])
- shared["context"] = prev_searches + [
- {"term": shared["search_term"], "result": exec_res}
- ]
- return "decide"
-
-class DirectAnswer(Node):
- def prep(self, shared):
- return shared["query"], shared.get("context", "")
-
- def exec(self, inputs):
- query, context = inputs
- return call_llm(f"Context: {context}\nAnswer: {query}")
+
- def post(self, shared, prep_res, exec_res):
- print(f"Answer: {exec_res}")
- shared["answer"] = exec_res
+ - Read inputs (e.g., retrieving Slack messages, reading emails)
+ - Write outputs (e.g., generating reports, sending emails)
+ - Use external tools (e.g., calling LLMs, searching the web)
+ - In contrast, *LLM-based tasks* (e.g., summarizing text, analyzing sentiment) are **NOT** utility functions. Instead, they are *internal core functions* within the AI system—designed in step 3—and are built on top of the utility functions.
+ - > **Start small!** Only include the most important ones to begin with!
+ {: .best-practice }
-# Connect nodes
-decide = DecideAction()
-search = SearchWeb()
-answer = DirectAnswer()
+3. **Flow Design (Compute)**: Create a high-level outline for your application’s flow.
+ - Identify potential design patterns (e.g., Batch, Agent, RAG).
+ - For each node, specify:
+ - **Purpose**: The high-level compute logic
+ - **Type**: Regular node, Batch node, async node, or another type
+ - `exec`: The specific utility function to call (ideally, one function per node)
-decide - "search" >> search
-decide - "answer" >> answer
-search - "decide" >> decide # Loop back
+4. **Data Schema (Data)**: Plan how data will be stored and updated.
+ - For simple apps, use an in-memory dictionary.
+ - For more complex apps or when persistence is required, use a database.
+ - For each node, specify:
+ - `prep`: How the node reads data
+ - `post`: How the node writes data
-flow = Flow(start=decide)
-flow.run({"query": "Who won the Nobel Prize in Physics 2024?"})
+5. **Implementation**: Implement nodes and flows based on the design.
+ - Start with a simple, direct approach (avoid over-engineering and full-scale type checking or testing). Let it fail fast to identify weaknesses.
+ - Add logging throughout the code to facilitate debugging.
+
+6. **Optimization**:
+ - **Use Intuition**: For a quick initial evaluation, human intuition is often a good start.
+ - **Redesign Flow (Back to Step 3)**: Consider breaking down tasks further, introducing agentic decisions, or better managing input contexts.
+ - If your flow design is already solid, move on to micro-optimizations:
+ - **Prompt Engineering**: Use clear, specific instructions with examples to reduce ambiguity.
+ - **In-Context Learning**: Provide robust examples for tasks that are difficult to specify with instructions alone.
+
+ - > **You’ll likely iterate a lot!** Expect to repeat Steps 3–6 hundreds of times.
+ >
+ >
+ {: .best-practice }
+
+7. **Reliability**
+ - **Node Retries**: Add checks in the node `exec` to ensure outputs meet requirements, and consider increasing `max_retries` and `wait` times.
+ - **Logging and Visualization**: Maintain logs of all attempts and visualize node results for easier debugging.
+ - **Self-Evaluation**: Add a separate node (powered by an LLM) to review outputs when results are uncertain.
+
+## Example LLM Project File Structure
+
+```
+my_project/
+├── main.py
+├── flow.py
+├── utils/
+│ ├── __init__.py
+│ ├── call_llm.py
+│ └── search_web.py
+├── requirements.txt
+└── docs/
+ └── design.md
```
+- **`docs/design.md`**: Contains project documentation for each step above. This should be high-level and no-code.
+- **`utils/`**: Contains all utility functions.
+ - It’s recommended to dedicate one Python file to each API call, for example `call_llm.py` or `search_web.py`.
+ - Each file should also include a `main()` function to try that API call
+- **`flow.py`**: Implements the application’s flow, starting with node definitions followed by the overall structure.
+- **`main.py`**: Serves as the project’s entry point.
+
================================================
-File: docs/async.md
+File: docs/index.md
+================================================
+---
+layout: default
+title: "Home"
+nav_order: 1
+---
+
+# Pocket Flow
+
+A [100-line](https://github.com/the-pocket/PocketFlow/blob/main/pocketflow/__init__.py) minimalist LLM framework for *Agents, Task Decomposition, RAG, etc*.
+
+We model the LLM workflow as a **Graph + Shared Store**:
+- **Nodes** handle simple (LLM) tasks.
+- Nodes connect through **Actions** (labeled edges) for *Agents*.
+- **Flows** orchestrate a directed graph of Nodes for *Task Decomposition*.
+- Nodes within Flows communicate through a **Shared Store**.
+- A Flow can be used as a Node (for **Nesting**).
+- **Batch** Nodes/Flows for data-intensive tasks.
+- **Async** Nodes/Flows allow waits for asynchronous tasks.
+
+
+
+
-
+
-
-
