302 lines
14 KiB
Plaintext
302 lines
14 KiB
Plaintext
---
|
||
description: Guidelines for using PocketFlow, Agentic Coding
|
||
globs: **/*.py
|
||
alwaysApply: true
|
||
---
|
||
# DOCUMENTATION FIRST POLICY
|
||
|
||
**CRITICAL INSTRUCTION**: When implementing a Pocket Flow app:
|
||
|
||
1. **ALWAYS REQUEST MDC FILES FIRST** - Before writing any code, request and review all relevant MDC documentation files. This doc provides an explaination of the documents.
|
||
2. **UNDERSTAND THE FRAMEWORK** - Gain comprehensive understanding of the Pocket Flow framework from documentation
|
||
3. **AVOID ASSUMPTION-DRIVEN DEVELOPMENT** - Do not base your implementation on assumptions or guesswork. Even if the human didn't explicitly mention pocket flow in their request, if the code you are editing is using pocket flow, you should request relevant docs to help you understand best practice as well before editing.
|
||
|
||
**VERIFICATION**: Begin each implementation with a brief summary of the documentation you've reviewed to inform your approach.
|
||
|
||
# Agentic Coding: Humans Design, Agents code!
|
||
|
||
> If you are an AI agent involved in building LLM Systems, read this guide **VERY, VERY** carefully! This is the most important chapter in the entire document. Throughout development, you should always (1) start with a small and simple solution, (2) design at a high level (`docs/design.md`) before implementation, and (3) frequently ask humans for feedback and clarification.
|
||
{: .warning }
|
||
|
||
## Agentic Coding Steps
|
||
|
||
Agentic Coding should be a collaboration between Human System Design and Agent Implementation:
|
||
|
||
| Steps | Human | AI | Comment |
|
||
|:-----------------------|:----------:|:---------:|:------------------------------------------------------------------------|
|
||
| 1. Requirements | ★★★ High | ★☆☆ Low | Humans understand the requirements and context. |
|
||
| 2. Flow | ★★☆ Medium | ★★☆ Medium | Humans specify the high-level design, and the AI fills in the details. |
|
||
| 3. Utilities | ★★☆ Medium | ★★☆ Medium | Humans provide available external APIs and integrations, and the AI helps with implementation. |
|
||
| 4. Data | ★☆☆ Low | ★★★ High | AI designs the data schema, and humans verify. |
|
||
| 5. Node | ★☆☆ Low | ★★★ High | The AI helps design the node based on the flow. |
|
||
| 6. Implementation | ★☆☆ Low | ★★★ High | The AI implements the flow based on the design. |
|
||
| 7. Optimization | ★★☆ Medium | ★★☆ Medium | Humans evaluate the results, and the AI helps optimize. |
|
||
| 8. Reliability | ★☆☆ Low | ★★★ High | The AI writes test cases and addresses corner cases. |
|
||
|
||
1. **Requirements**: Clarify the requirements for your project, and evaluate whether an AI system is a good fit.
|
||
- Understand AI systems' strengths and limitations:
|
||
- **Good for**: Routine tasks requiring common sense (filling forms, replying to emails)
|
||
- **Good for**: Creative tasks with well-defined inputs (building slides, writing SQL)
|
||
- **Not good for**: Ambiguous problems requiring complex decision-making (business strategy, startup planning)
|
||
- **Keep It User-Centric:** Explain the "problem" from the user's perspective rather than just listing features.
|
||
- **Balance complexity vs. impact**: Aim to deliver the highest value features with minimal complexity early.
|
||
|
||
2. **Flow Design**: Outline at a high level, describe how your AI system orchestrates nodes.
|
||
- Identify applicable design patterns (e.g., [Map Reduce], [Agent], [RAG]).
|
||
- For each node in the flow, start with a high-level one-line description of what it does.
|
||
- If using **Map Reduce**, specify how to map (what to split) and how to reduce (how to combine).
|
||
- If using **Agent**, specify what are the inputs (context) and what are the possible actions.
|
||
- If using **RAG**, specify what to embed, noting that there's usually both offline (indexing) and online (retrieval) workflows.
|
||
- Outline the flow and draw it in a mermaid diagram. For example:
|
||
```mermaid
|
||
flowchart LR
|
||
start[Start] --> batch[Batch]
|
||
batch --> check[Check]
|
||
check -->|OK| process
|
||
check -->|Error| fix[Fix]
|
||
fix --> check
|
||
|
||
subgraph process[Process]
|
||
step1[Step 1] --> step2[Step 2]
|
||
end
|
||
|
||
process --> endNode[End]
|
||
```
|
||
- > **If Humans can't specify the flow, AI Agents can't automate it!** Before building an LLM system, thoroughly understand the problem and potential solution by manually solving example inputs to develop intuition.
|
||
{: .best-practice }
|
||
|
||
3. **Utilities**: Based on the Flow Design, identify and implement necessary utility functions.
|
||
- Think of your AI system as the brain. It needs a body—these *external utility functions*—to interact with the real world:
|
||
|
||
|
||
- Reading inputs (e.g., retrieving Slack messages, reading emails)
|
||
- Writing outputs (e.g., generating reports, sending emails)
|
||
- Using external tools (e.g., calling LLMs, searching the web)
|
||
- **NOTE**: *LLM-based tasks* (e.g., summarizing text, analyzing sentiment) are **NOT** utility functions; rather, they are *core functions* internal in the AI system.
|
||
- For each utility function, implement it and write a simple test.
|
||
- Document their input/output, as well as why they are necessary. For example:
|
||
- `name`: `get_embedding` (`utils/get_embedding.py`)
|
||
- `input`: `str`
|
||
- `output`: a vector of 3072 floats
|
||
- `necessity`: Used by the second node to embed text
|
||
- Example utility implementation:
|
||
```python
|
||
# utils/call_llm.py
|
||
from openai import OpenAI
|
||
|
||
def call_llm(prompt):
|
||
client = OpenAI(api_key="YOUR_API_KEY_HERE")
|
||
r = client.chat.completions.create(
|
||
model="gpt-4o",
|
||
messages=[{"role": "user", "content": prompt}]
|
||
)
|
||
return r.choices[0].message.content
|
||
|
||
if __name__ == "__main__":
|
||
prompt = "What is the meaning of life?"
|
||
print(call_llm(prompt))
|
||
```
|
||
- > **Sometimes, design Utilities before Flow:** For example, for an LLM project to automate a legacy system, the bottleneck will likely be the available interface to that system. Start by designing the hardest utilities for interfacing, and then build the flow around them.
|
||
{: .best-practice }
|
||
- > **Avoid Exception Handling in Utilities**: If a utility function is called from a Node's `exec()` method, avoid using `try...except` blocks within the utility. Let the Node's built-in retry mechanism handle failures.
|
||
{: .warning }
|
||
|
||
4. **Data Design**: Design the shared store that nodes will use to communicate.
|
||
- One core design principle for PocketFlow is to use a well-designed [shared store]—a data contract that all nodes agree upon to retrieve and store data.
|
||
- For simple systems, use an in-memory dictionary.
|
||
- For more complex systems or when persistence is required, use a database.
|
||
- **Don't Repeat Yourself**: Use in-memory references or foreign keys.
|
||
- Example shared store design:
|
||
```python
|
||
shared = {
|
||
"user": {
|
||
"id": "user123",
|
||
"context": { # Another nested dict
|
||
"weather": {"temp": 72, "condition": "sunny"},
|
||
"location": "San Francisco"
|
||
}
|
||
},
|
||
"results": {} # Empty dict to store outputs
|
||
}
|
||
```
|
||
|
||
5. **Node Design**: Plan how each node will read and write data, and use utility functions.
|
||
- For each [Node], describe its type, how it reads and writes data, and which utility function it uses. Keep it specific but high-level without codes. For example:
|
||
- `type`: Regular (or Batch, or Async)
|
||
- `prep`: Read "text" from the shared store
|
||
- `exec`: Call the embedding utility function. **Avoid exception handling here**; let the Node's retry mechanism manage failures.
|
||
- `post`: Write "embedding" to the shared store
|
||
|
||
6. **Implementation**: Implement the initial nodes and flows based on the design.
|
||
- 🎉 If you've reached this step, humans have finished the design. Now *Agentic Coding* begins!
|
||
- **"Keep it simple, stupid!"** Avoid complex features and full-scale type checking.
|
||
- **FAIL FAST**! Leverage the built-in [Node] retry and fallback mechanisms to handle failures gracefully. This helps you quickly identify weak points in the system.
|
||
- Add logging throughout the code to facilitate debugging.
|
||
|
||
7. **Optimization**:
|
||
- **Use Intuition**: For a quick initial evaluation, human intuition is often a good start.
|
||
- **Redesign Flow (Back to Step 3)**: Consider breaking down tasks further, introducing agentic decisions, or better managing input contexts.
|
||
- If your flow design is already solid, move on to micro-optimizations:
|
||
- **Prompt Engineering**: Use clear, specific instructions with examples to reduce ambiguity.
|
||
- **In-Context Learning**: Provide robust examples for tasks that are difficult to specify with instructions alone.
|
||
|
||
- > **You'll likely iterate a lot!** Expect to repeat Steps 3–6 hundreds of times.
|
||
>
|
||
>
|
||
{: .best-practice }
|
||
|
||
8. **Reliability**
|
||
- **Node Retries**: Add checks in the node `exec` to ensure outputs meet requirements, and consider increasing `max_retries` and `wait` times.
|
||
- **Logging and Visualization**: Maintain logs of all attempts and visualize node results for easier debugging.
|
||
- **Self-Evaluation**: Add a separate node (powered by an LLM) to review outputs when results are uncertain.
|
||
|
||
## Example LLM Project File Structure
|
||
|
||
```
|
||
my_project/
|
||
├── main.py
|
||
├── nodes.py
|
||
├── flow.py
|
||
├── utils/
|
||
│ ├── __init__.py
|
||
│ ├── call_llm.py
|
||
│ └── search_web.py
|
||
├── requirements.txt
|
||
└── docs/
|
||
└── design.md
|
||
```
|
||
|
||
- **`docs/design.md`**: Contains project documentation for each step above. This should be *high-level* and *no-code*.
|
||
- **`utils/`**: Contains all utility functions.
|
||
- It's recommended to dedicate one Python file to each API call, for example `call_llm.py` or `search_web.py`.
|
||
- Each file should also include a `main()` function to try that API call
|
||
- **`nodes.py`**: Contains all the node definitions.
|
||
```python
|
||
# nodes.py
|
||
from pocketflow import Node
|
||
from utils.call_llm import call_llm
|
||
|
||
class GetQuestionNode(Node):
|
||
def exec(self, _):
|
||
# Get question directly from user input
|
||
user_question = input("Enter your question: ")
|
||
return user_question
|
||
|
||
def post(self, shared, prep_res, exec_res):
|
||
# Store the user's question
|
||
shared["question"] = exec_res
|
||
return "default" # Go to the next node
|
||
|
||
class AnswerNode(Node):
|
||
def prep(self, shared):
|
||
# Read question from shared
|
||
return shared["question"]
|
||
|
||
def exec(self, question):
|
||
# Call LLM to get the answer
|
||
return call_llm(question)
|
||
|
||
def post(self, shared, prep_res, exec_res):
|
||
# Store the answer in shared
|
||
shared["answer"] = exec_res
|
||
```
|
||
- **`flow.py`**: Implements functions that create flows by importing node definitions and connecting them.
|
||
```python
|
||
# flow.py
|
||
from pocketflow import Flow
|
||
from nodes import GetQuestionNode, AnswerNode
|
||
|
||
def create_qa_flow():
|
||
"""Create and return a question-answering flow."""
|
||
# Create nodes
|
||
get_question_node = GetQuestionNode()
|
||
answer_node = AnswerNode()
|
||
|
||
# Connect nodes in sequence
|
||
get_question_node >> answer_node
|
||
|
||
# Create flow starting with input node
|
||
return Flow(start=get_question_node)
|
||
```
|
||
- **`main.py`**: Serves as the project's entry point.
|
||
```python
|
||
# main.py
|
||
from flow import create_qa_flow
|
||
|
||
# Example main function
|
||
# Please replace this with your own main function
|
||
def main():
|
||
shared = {
|
||
"question": None, # Will be populated by GetQuestionNode from user input
|
||
"answer": None # Will be populated by AnswerNode
|
||
}
|
||
|
||
# Create the flow and run it
|
||
qa_flow = create_qa_flow()
|
||
qa_flow.run(shared)
|
||
print(f"Question: {shared['question']}")
|
||
print(f"Answer: {shared['answer']}")
|
||
|
||
if __name__ == "__main__":
|
||
main()
|
||
```
|
||
|
||
|
||
# Pocket Flow
|
||
|
||
A [100-line](https://github.com/the-pocket/PocketFlow/blob/main/pocketflow/__init__.py) minimalist LLM framework for *Agents, Task Decomposition, RAG, etc*.
|
||
|
||
- **Lightweight**: Just the core graph abstraction in 100 lines. ZERO dependencies, and vendor lock-in.
|
||
- **Expressive**: Everything you love from larger frameworks—([Multi-])[Agents], [Workflow], [RAG], and more.
|
||
- **Agentic-Coding**: Intuitive enough for AI agents to help humans build complex LLM applications.
|
||
|
||
|
||
|
||
|
||
## Core Abstraction
|
||
|
||
We model the LLM workflow as a **Graph + Shared Store**:
|
||
|
||
- [Node] handles simple (LLM) tasks.
|
||
- [Flow] connects nodes through **Actions** (labeled edges).
|
||
- [Shared Store] enables communication between nodes within flows.
|
||
- [Batch] nodes/flows allow for data-intensive tasks.
|
||
- [Async] nodes/flows allow waiting for asynchronous tasks.
|
||
- [(Advanced) Parallel] nodes/flows handle I/O-bound tasks.
|
||
|
||
|
||
|
||
## Design Pattern
|
||
|
||
From there, it’s easy to implement popular design patterns:
|
||
|
||
- [Agent] autonomously makes decisions.
|
||
- [Workflow] chains multiple tasks into pipelines.
|
||
- [RAG] integrates data retrieval with generation.
|
||
- [Map Reduce] splits data tasks into Map and Reduce steps.
|
||
- [Structured Output] formats outputs consistently.
|
||
- [(Advanced) Multi-Agents] coordinate multiple agents.
|
||
|
||
|
||
|
||
## Utility Function
|
||
|
||
We **do not** provide built-in utilities. Instead, we offer *examples*—please *implement your own*:
|
||
|
||
- [LLM Wrapper]
|
||
- [Viz and Debug]
|
||
- [Web Search]
|
||
- [Chunking]
|
||
- [Embedding]
|
||
- [Vector Databases]
|
||
- [Text-to-Speech]
|
||
|
||
**Why not built-in?**: I believe it's a *bad practice* for vendor-specific APIs in a general framework:
|
||
- *API Volatility*: Frequent changes lead to heavy maintenance for hardcoded APIs.
|
||
- *Flexibility*: You may want to switch vendors, use fine-tuned models, or run them locally.
|
||
- *Optimizations*: Prompt caching, batching, and streaming are easier without vendor lock-in.
|
||
|
||
## Ready to build your Apps?
|
||
|
||
Check out [Agentic Coding Guidance], the fastest way to develop LLM projects with Pocket Flow!
|