update node doc
This commit is contained in:
parent
fa2250eaf2
commit
60e6a1ec98
59
docs/node.md
59
docs/node.md
|
|
@ -7,35 +7,38 @@ nav_order: 1
|
|||
|
||||
# Node
|
||||
|
||||
A **Node** is the smallest building block. Each Node has 3 steps:
|
||||
A **Node** is the smallest building block. Each Node has 3 steps `prep->exec->post`:
|
||||
|
||||
1. `prep(shared)`
|
||||
- A reliable step for preprocessing data from the `shared` store.
|
||||
- **Read and preprocess data** from `shared` store.
|
||||
- Examples: *query DB, read files, or serialize data into a string*.
|
||||
- Returns `prep_res`, which is used by `exec()` and `post()`.
|
||||
- Return `prep_res`, which is used by `exec()` and `post()`.
|
||||
|
||||
2. `exec(prep_res)`
|
||||
- The **main execution** step, with optional retries and error handling (below).
|
||||
- Examples: *primarily for LLMs, but can also for remote APIs*.
|
||||
- **Execute compute logic**, with optional retries and error handling (below).
|
||||
- Examples: *(mostly) LLM calls, remote APIs, tool use*.
|
||||
- ⚠️ This shall be only for compute and **NOT** access `shared`.
|
||||
- ⚠️ If retries enabled, ensure idempotent implementation.
|
||||
- ⚠️ This must **NOT** write to `shared`. If reads are necessary, extract them in `prep()` and pass them in `prep_res`.
|
||||
- Returns `exec_res`, which is passed to `post()`.
|
||||
- Return `exec_res`, which is passed to `post()`.
|
||||
|
||||
3. `post(shared, prep_res, exec_res)`
|
||||
- A reliable postprocessing step to write results back to the `shared` store and decide the next Action.
|
||||
- Examples: *update DB, change states, log results, decide next Action*.
|
||||
- Returns a **string** specifying the next Action (`"default"` if none).
|
||||
- **Postprocess and write data** back to `shared`.
|
||||
- Examples: *update DB, change states, log results*.
|
||||
- **Decide the next action** by returning a *string* (`action = "default"` if *None*).
|
||||
|
||||
> All 3 steps are optional. You could run only `prep` if you just need to prepare data without calling the LLM.
|
||||
> **Why 3 steps?** To enforce the principle of *separation of concerns*. The data model are operated separately from the business logic on them.
|
||||
>
|
||||
> All steps are *optional*. E.g., you can only implement `prep` and `post` if you just need to process data.
|
||||
{: .note }
|
||||
|
||||
|
||||
### Fault Tolerance & Retries
|
||||
|
||||
Nodes can **retry** execution if `exec()` raises an exception. You control this via two parameters when you create the Node:
|
||||
You can **retry** `exec()` if it raises an exception via two parameters when define the Node:
|
||||
|
||||
- `max_retries` (int): How many times to try running `exec()`. The default is `1`, which means **no** retry.
|
||||
- `wait` (int): The time to wait (in **seconds**) before each retry attempt. By default, `wait=0` (i.e., no waiting). Increasing this is helpful when you encounter rate-limits or quota errors from your LLM provider and need to back off.
|
||||
- `max_retries` (int): Max times to run `exec()`. The default is `1` (**no** retry).
|
||||
- `wait` (int): The time to wait (in **seconds**) before next retry. By default, `wait=0` (no waiting).
|
||||
`wait` is helpful when you encounter rate-limits or quota errors from your LLM provider and need to back off.
|
||||
|
||||
```python
|
||||
my_node = SummarizeFile(max_retries=3, wait=10)
|
||||
|
|
@ -46,28 +49,36 @@ When an exception occurs in `exec()`, the Node automatically retries until:
|
|||
- It either succeeds, or
|
||||
- The Node has retried `max_retries - 1` times already and fails on the last attempt.
|
||||
|
||||
You can get the current retry times (0-based) from `self.cur_retry`.
|
||||
|
||||
```python
|
||||
class RetryNode(Node):
|
||||
def exec(self, prep_res):
|
||||
print(f"Retry {self.cur_retry} times")
|
||||
raise Exception("Failed")
|
||||
```
|
||||
|
||||
### Graceful Fallback
|
||||
|
||||
If you want to **gracefully handle** the error rather than raising it, you can override:
|
||||
To **gracefully handle** the exception (after all retries) rather than raising it, override:
|
||||
|
||||
```python
|
||||
def exec_fallback(self, shared, prep_res, exc):
|
||||
raise exc
|
||||
```
|
||||
|
||||
By default, it just re-raises `exc`. But you can return a fallback result instead, which becomes the `exec_res` passed to `post()`.
|
||||
By default, it just re-raises exception. But you can return a fallback result instead, which becomes the `exec_res` passed to `post()`.
|
||||
|
||||
### Example: Summarize file
|
||||
|
||||
```python
|
||||
class SummarizeFile(Node):
|
||||
def prep(self, shared):
|
||||
filename = self.params["filename"]
|
||||
return shared["data"][filename]
|
||||
return shared["data"]
|
||||
|
||||
def exec(self, prep_res):
|
||||
if not prep_res:
|
||||
raise ValueError("Empty file content!")
|
||||
return "Empty file content"
|
||||
prompt = f"Summarize this text in 10 words: {prep_res}"
|
||||
summary = call_llm(prompt) # might fail
|
||||
return summary
|
||||
|
|
@ -77,18 +88,16 @@ class SummarizeFile(Node):
|
|||
return "There was an error processing your request."
|
||||
|
||||
def post(self, shared, prep_res, exec_res):
|
||||
filename = self.params["filename"]
|
||||
shared["summary"][filename] = exec_res
|
||||
# Return "default" by not returning anything
|
||||
shared["summary"] = exec_res
|
||||
# Return "default" by not returning
|
||||
|
||||
summarize_node = SummarizeFile(max_retries=3)
|
||||
summarize_node.set_params({"filename": "test_file.txt"})
|
||||
|
||||
# node.run() calls prep->exec->post
|
||||
# If exec() fails, it retries up to 3 times before calling exec_fallback()
|
||||
action_result = summarize_node.run(shared)
|
||||
|
||||
print("Action returned:", action_result) # Usually "default"
|
||||
print("Summary stored:", shared["summary"].get("test_file.txt"))
|
||||
print("Action returned:", action_result) # "default"
|
||||
print("Summary stored:", shared["summary"])
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue