Example for debugging
This commit is contained in:
parent
b7444fa1d3
commit
1264e71af4
|
|
@ -41,7 +41,7 @@ We model the LLM workflow as a **Nested Directed Graph**:
|
||||||
|
|
||||||
- [LLM Wrapper](./llm.md)
|
- [LLM Wrapper](./llm.md)
|
||||||
- [Tool](./tool.md)
|
- [Tool](./tool.md)
|
||||||
- [Visualization](./viz.md)
|
- [Visualization and Debugging](./viz.md)
|
||||||
- Chunking
|
- Chunking
|
||||||
|
|
||||||
> We do not provide built-in implementations.
|
> We do not provide built-in implementations.
|
||||||
|
|
|
||||||
93
docs/viz.md
93
docs/viz.md
|
|
@ -1,15 +1,15 @@
|
||||||
---
|
---
|
||||||
layout: default
|
layout: default
|
||||||
title: "Visualization"
|
title: "Visualization and Debugging"
|
||||||
parent: "Details"
|
parent: "Details"
|
||||||
nav_order: 3
|
nav_order: 3
|
||||||
---
|
---
|
||||||
|
|
||||||
# Visualization
|
# Visualization and Debugging
|
||||||
|
|
||||||
Visualizing the nested graph can help understanding. While we **don’t** include built-in visualization tools, we provide an example using **Mermaid**.
|
Similar to LLM wrappers, we **don't** provide built-in visualization and debugging. Here, we recommend some *minimal* (and incomplete) implementations These examples can serve as a starting point for your own tooling.
|
||||||
|
|
||||||
### Example: Visualization of Node with Mermaid
|
## 1. Visualization with Mermaid
|
||||||
|
|
||||||
This code recursively traverses the nested graph, assigns unique IDs to each node, and treats Flow nodes as subgraphs to generate Mermaid syntax for a hierarchical visualization.
|
This code recursively traverses the nested graph, assigns unique IDs to each node, and treats Flow nodes as subgraphs to generate Mermaid syntax for a hierarchical visualization.
|
||||||
|
|
||||||
|
|
@ -43,17 +43,18 @@ def build_mermaid(start):
|
||||||
```
|
```
|
||||||
{% endraw %}
|
{% endraw %}
|
||||||
|
|
||||||
### Usage Example
|
|
||||||
|
|
||||||
Here, we define some example Nodes and Flows:
|
For example, suppose we have a complex Flow for data science:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
class DataPrepBatchNode(BatchNode): pass
|
class DataPrepBatchNode(BatchNode):
|
||||||
|
def prep(self,shared): return []
|
||||||
class ValidateDataNode(Node): pass
|
class ValidateDataNode(Node): pass
|
||||||
class FeatureExtractionNode(Node): pass
|
class FeatureExtractionNode(Node): pass
|
||||||
class TrainModelNode(Node): pass
|
class TrainModelNode(Node): pass
|
||||||
class EvaluateModelNode(Node): pass
|
class EvaluateModelNode(Node): pass
|
||||||
class ModelFlow(Flow): pass
|
class ModelFlow(Flow): pass
|
||||||
|
class DataScienceFlow(Flow):pass
|
||||||
|
|
||||||
feature_node = FeatureExtractionNode()
|
feature_node = FeatureExtractionNode()
|
||||||
train_node = TrainModelNode()
|
train_node = TrainModelNode()
|
||||||
|
|
@ -63,23 +64,81 @@ model_flow = ModelFlow(start=feature_node)
|
||||||
data_prep_node = DataPrepBatchNode()
|
data_prep_node = DataPrepBatchNode()
|
||||||
validate_node = ValidateDataNode()
|
validate_node = ValidateDataNode()
|
||||||
data_prep_node >> validate_node >> model_flow
|
data_prep_node >> validate_node >> model_flow
|
||||||
build_mermaid(start=data_prep_node)
|
data_science_flow = DataScienceFlow(start=data_prep_node)
|
||||||
|
result = build_mermaid(start=data_science_flow)
|
||||||
```
|
```
|
||||||
|
|
||||||
The code generates a Mermaid diagram:
|
The code generates a Mermaid diagram:
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
graph LR
|
graph LR
|
||||||
N1["DataPrepBatchNode"]
|
subgraph sub_flow_N1[DataScienceFlow]
|
||||||
N2["ValidateDataNode"]
|
N2['DataPrepBatchNode']
|
||||||
N1 --> N2
|
N3['ValidateDataNode']
|
||||||
N2 --> N3
|
N2 --> N3
|
||||||
|
N3 --> N4
|
||||||
|
|
||||||
|
subgraph sub_flow_N5[ModelFlow]
|
||||||
|
N4['FeatureExtractionNode']
|
||||||
|
N6['TrainModelNode']
|
||||||
|
N4 --> N6
|
||||||
|
N7['EvaluateModelNode']
|
||||||
|
N6 --> N7
|
||||||
|
end
|
||||||
|
|
||||||
subgraph sub_flow_N4[ModelFlow]
|
|
||||||
N3["FeatureExtractionNode"]
|
|
||||||
N5["TrainModelNode"]
|
|
||||||
N3 --> N5
|
|
||||||
N6["EvaluateModelNode"]
|
|
||||||
N5 --> N6
|
|
||||||
end
|
end
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## 2. Call Stack Debugging
|
||||||
|
|
||||||
|
It would be useful to print the Node call stacks for debugging. This can be achieved by inspecting the runtime call stack:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import inspect
|
||||||
|
|
||||||
|
def get_node_call_stack():
|
||||||
|
stack = inspect.stack()
|
||||||
|
node_names = []
|
||||||
|
seen_ids = set()
|
||||||
|
for frame_info in stack[1:]:
|
||||||
|
local_vars = frame_info.frame.f_locals
|
||||||
|
if 'self' in local_vars:
|
||||||
|
caller_self = local_vars['self']
|
||||||
|
if isinstance(caller_self, BaseNode) and id(caller_self) not in seen_ids:
|
||||||
|
seen_ids.add(id(caller_self))
|
||||||
|
node_names.append(type(caller_self).__name__)
|
||||||
|
return node_names
|
||||||
|
```
|
||||||
|
|
||||||
|
For example, suppose we have a complex Flow for data science:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class DataPrepBatchNode(BatchNode):
|
||||||
|
def prep(self, shared): return []
|
||||||
|
class ValidateDataNode(Node): pass
|
||||||
|
class FeatureExtractionNode(Node): pass
|
||||||
|
class TrainModelNode(Node): pass
|
||||||
|
class EvaluateModelNode(Node):
|
||||||
|
def prep(self, shared):
|
||||||
|
stack = get_node_call_stack()
|
||||||
|
print("Call stack:", stack)
|
||||||
|
class ModelFlow(Flow): pass
|
||||||
|
class DataScienceFlow(Flow):pass
|
||||||
|
|
||||||
|
feature_node = FeatureExtractionNode()
|
||||||
|
train_node = TrainModelNode()
|
||||||
|
evaluate_node = EvaluateModelNode()
|
||||||
|
feature_node >> train_node >> evaluate_node
|
||||||
|
model_flow = ModelFlow(start=feature_node)
|
||||||
|
data_prep_node = DataPrepBatchNode()
|
||||||
|
validate_node = ValidateDataNode()
|
||||||
|
data_prep_node >> validate_node >> model_flow
|
||||||
|
data_science_flow = DataScienceFlow(start=data_prep_node)
|
||||||
|
data_science_flow.run({})
|
||||||
|
```
|
||||||
|
|
||||||
|
The output would be: `Call stack: ['EvaluateModelNode', 'ModelFlow', 'DataScienceFlow']`
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue