replace .cursorrules with the new cursor mdc files; create a script to generate/update the mdc files from the doc folder
This commit is contained in:
BO WEN
@@ -0,0 +1,52 @@
|
||||
---
|
||||
description: Guidelines for using PocketFlow, Utility Function, Text Chunking
|
||||
globs:
|
||||
alwaysApply: false
|
||||
---
|
||||
# Text Chunking
|
||||
|
||||
We recommend some implementations of commonly used text chunking approaches.
|
||||
|
||||
|
||||
> Text Chunking is more a micro optimization, compared to the Flow Design.
|
||||
>
|
||||
> It's recommended to start with the Naive Chunking and optimize later.
|
||||
{: .best-practice }
|
||||
|
||||
---
|
||||
|
||||
## Example Python Code Samples
|
||||
|
||||
### 1. Naive (Fixed-Size) Chunking
|
||||
Splits text by a fixed number of words, ignoring sentence or semantic boundaries.
|
||||
|
||||
```python
|
||||
def fixed_size_chunk(text, chunk_size=100):
|
||||
chunks = []
|
||||
for i in range(0, len(text), chunk_size):
|
||||
chunks.append(text[i : i + chunk_size])
|
||||
return chunks
|
||||
```
|
||||
|
||||
However, sentences are often cut awkwardly, losing coherence.
|
||||
|
||||
### 2. Sentence-Based Chunking
|
||||
|
||||
```python
|
||||
import nltk
|
||||
|
||||
def sentence_based_chunk(text, max_sentences=2):
|
||||
sentences = nltk.sent_tokenize(text)
|
||||
chunks = []
|
||||
for i in range(0, len(sentences), max_sentences):
|
||||
chunks.append(" ".join(sentences[i : i + max_sentences]))
|
||||
return chunks
|
||||
```
|
||||
|
||||
However, might not handle very long sentences or paragraphs well.
|
||||
|
||||
### 3. Other Chunking
|
||||
|
||||
- **Paragraph-Based**: Split text by paragraphs (e.g., newlines). Large paragraphs can create big chunks.
|
||||
- **Semantic**: Use embeddings or topic modeling to chunk by semantic boundaries.
|
||||
- **Agentic**: Use an LLM to decide chunk boundaries based on context or meaning.
|
||||
@@ -0,0 +1,115 @@
|
||||
---
|
||||
description: Guidelines for using PocketFlow, Utility Function, Embedding
|
||||
globs:
|
||||
alwaysApply: false
|
||||
---
|
||||
# Embedding
|
||||
|
||||
Below you will find an overview table of various text embedding APIs, along with example Python code.
|
||||
|
||||
> Embedding is more a micro optimization, compared to the Flow Design.
|
||||
>
|
||||
> It's recommended to start with the most convenient one and optimize later.
|
||||
{: .best-practice }
|
||||
|
||||
|
||||
| **API** | **Free Tier** | **Pricing Model** | **Docs** |
|
||||
| --- | --- | --- | --- |
|
||||
| **OpenAI** | ~$5 credit | ~$0.0001/1K tokens | [OpenAI Embeddings](https://platform.openai.com/docs/api-reference/embeddings) |
|
||||
| **Azure OpenAI** | $200 credit | Same as OpenAI (~$0.0001/1K tokens) | [Azure OpenAI Embeddings](https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?tabs=portal) |
|
||||
| **Google Vertex AI** | $300 credit | ~$0.025 / million chars | [Vertex AI Embeddings](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings) |
|
||||
| **AWS Bedrock** | No free tier, but AWS credits may apply | ~$0.00002/1K tokens (Titan V2) | [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/) |
|
||||
| **Cohere** | Limited free tier | ~$0.0001/1K tokens | [Cohere Embeddings](https://docs.cohere.com/docs/cohere-embed) |
|
||||
| **Hugging Face** | ~$0.10 free compute monthly | Pay per second of compute | [HF Inference API](https://huggingface.co/docs/api-inference) |
|
||||
| **Jina** | 1M tokens free | Pay per token after | [Jina Embeddings](https://jina.ai/embeddings/) |
|
||||
|
||||
## Example Python Code
|
||||
|
||||
### 1. OpenAI
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(api_key="YOUR_API_KEY")
|
||||
response = client.embeddings.create(
|
||||
model="text-embedding-ada-002",
|
||||
input=text
|
||||
)
|
||||
|
||||
# Extract the embedding vector from the response
|
||||
embedding = response.data[0].embedding
|
||||
embedding = np.array(embedding, dtype=np.float32)
|
||||
print(embedding)
|
||||
```
|
||||
|
||||
### 2. Azure OpenAI
|
||||
```python
|
||||
import openai
|
||||
|
||||
openai.api_type = "azure"
|
||||
openai.api_base = "https://YOUR_RESOURCE_NAME.openai.azure.com"
|
||||
openai.api_version = "2023-03-15-preview"
|
||||
openai.api_key = "YOUR_AZURE_API_KEY"
|
||||
|
||||
resp = openai.Embedding.create(engine="ada-embedding", input="Hello world")
|
||||
vec = resp["data"][0]["embedding"]
|
||||
print(vec)
|
||||
```
|
||||
|
||||
### 3. Google Vertex AI
|
||||
```python
|
||||
from vertexai.preview.language_models import TextEmbeddingModel
|
||||
import vertexai
|
||||
|
||||
vertexai.init(project="YOUR_GCP_PROJECT_ID", location="us-central1")
|
||||
model = TextEmbeddingModel.from_pretrained("textembedding-gecko@001")
|
||||
|
||||
emb = model.get_embeddings(["Hello world"])
|
||||
print(emb[0])
|
||||
```
|
||||
|
||||
### 4. AWS Bedrock
|
||||
```python
|
||||
import boto3, json
|
||||
|
||||
client = boto3.client("bedrock-runtime", region_name="us-east-1")
|
||||
body = {"inputText": "Hello world"}
|
||||
resp = client.invoke_model(modelId="amazon.titan-embed-text-v2:0", contentType="application/json", body=json.dumps(body))
|
||||
resp_body = json.loads(resp["body"].read())
|
||||
vec = resp_body["embedding"]
|
||||
print(vec)
|
||||
```
|
||||
|
||||
### 5. Cohere
|
||||
```python
|
||||
import cohere
|
||||
|
||||
co = cohere.Client("YOUR_API_KEY")
|
||||
resp = co.embed(texts=["Hello world"])
|
||||
vec = resp.embeddings[0]
|
||||
print(vec)
|
||||
```
|
||||
|
||||
### 6. Hugging Face
|
||||
```python
|
||||
import requests
|
||||
|
||||
API_URL = "https://api-inference.huggingface.co/models/sentence-transformers/all-MiniLM-L6-v2"
|
||||
HEADERS = {"Authorization": "Bearer YOUR_HF_TOKEN"}
|
||||
|
||||
res = requests.post(API_URL, headers=HEADERS, json={"inputs": "Hello world"})
|
||||
vec = res.json()[0]
|
||||
print(vec)
|
||||
```
|
||||
|
||||
### 7. Jina
|
||||
```python
|
||||
import requests
|
||||
|
||||
url = "https://api.jina.ai/v2/embed"
|
||||
headers = {"Authorization": "Bearer YOUR_JINA_TOKEN"}
|
||||
payload = {"data": ["Hello world"], "model": "jina-embeddings-v3"}
|
||||
res = requests.post(url, headers=headers, json=payload)
|
||||
vec = res.json()["data"][0]["embedding"]
|
||||
print(vec)
|
||||
```
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
---
|
||||
description: Guidelines for using PocketFlow, Utility Function, LLM Wrapper
|
||||
globs:
|
||||
alwaysApply: false
|
||||
---
|
||||
# LLM Wrappers
|
||||
|
||||
Check out libraries like [litellm](https://github.com/BerriAI/litellm).
|
||||
Here, we provide some minimal example implementations:
|
||||
|
||||
1. OpenAI
|
||||
```python
|
||||
def call_llm(prompt):
|
||||
from openai import OpenAI
|
||||
client = OpenAI(api_key="YOUR_API_KEY_HERE")
|
||||
r = client.chat.completions.create(
|
||||
model="gpt-4o",
|
||||
messages=[{"role": "user", "content": prompt}]
|
||||
)
|
||||
return r.choices[0].message.content
|
||||
|
||||
# Example usage
|
||||
call_llm("How are you?")
|
||||
```
|
||||
> Store the API key in an environment variable like OPENAI_API_KEY for security.
|
||||
{: .best-practice }
|
||||
|
||||
2. Claude (Anthropic)
|
||||
```python
|
||||
def call_llm(prompt):
|
||||
from anthropic import Anthropic
|
||||
client = Anthropic(api_key="YOUR_API_KEY_HERE")
|
||||
r = client.messages.create(
|
||||
model="claude-3-7-sonnet-20250219",
|
||||
max_tokens=3000,
|
||||
messages=[
|
||||
{"role": "user", "content": prompt}
|
||||
]
|
||||
)
|
||||
return r.content[0].text
|
||||
```
|
||||
|
||||
3. Google (Generative AI Studio / PaLM API)
|
||||
```python
|
||||
def call_llm(prompt):
|
||||
import google.generativeai as genai
|
||||
genai.configure(api_key="YOUR_API_KEY_HERE")
|
||||
r = genai.generate_text(
|
||||
model="models/text-bison-001",
|
||||
prompt=prompt
|
||||
)
|
||||
return r.result
|
||||
```
|
||||
|
||||
4. Azure (Azure OpenAI)
|
||||
```python
|
||||
def call_llm(prompt):
|
||||
from openai import AzureOpenAI
|
||||
client = AzureOpenAI(
|
||||
azure_endpoint="https://.openai.azure.com/",
|
||||
api_key="YOUR_API_KEY_HERE",
|
||||
api_version="2023-05-15"
|
||||
)
|
||||
r = client.chat.completions.create(
|
||||
model="",
|
||||
messages=[{"role": "user", "content": prompt}]
|
||||
)
|
||||
return r.choices[0].message.content
|
||||
```
|
||||
|
||||
5. Ollama (Local LLM)
|
||||
```python
|
||||
def call_llm(prompt):
|
||||
from ollama import chat
|
||||
response = chat(
|
||||
model="llama2",
|
||||
messages=[{"role": "user", "content": prompt}]
|
||||
)
|
||||
return response.message.content
|
||||
```
|
||||
|
||||
6. DeepSeek
|
||||
```python
|
||||
def call_llm(prompt):
|
||||
from openai import OpenAI
|
||||
client = OpenAI(api_key="YOUR_DEEPSEEK_API_KEY", base_url="https://api.deepseek.com")
|
||||
r = client.chat.completions.create(
|
||||
model="deepseek-chat",
|
||||
messages=[{"role": "user", "content": prompt}]
|
||||
)
|
||||
return r.choices[0].message.content
|
||||
```
|
||||
|
||||
|
||||
## Improvements
|
||||
Feel free to enhance your `call_llm` function as needed. Here are examples:
|
||||
|
||||
- Handle chat history:
|
||||
|
||||
```python
|
||||
def call_llm(messages):
|
||||
from openai import OpenAI
|
||||
client = OpenAI(api_key="YOUR_API_KEY_HERE")
|
||||
r = client.chat.completions.create(
|
||||
model="gpt-4o",
|
||||
messages=messages
|
||||
)
|
||||
return r.choices[0].message.content
|
||||
```
|
||||
|
||||
- Add in-memory caching
|
||||
|
||||
```python
|
||||
from functools import lru_cache
|
||||
|
||||
@lru_cache(maxsize=1000)
|
||||
def call_llm(prompt):
|
||||
# Your implementation here
|
||||
pass
|
||||
```
|
||||
|
||||
> ⚠️ Caching conflicts with Node retries, as retries yield the same result.
|
||||
>
|
||||
> To address this, you could use cached results only if not retried.
|
||||
{: .warning }
|
||||
|
||||
|
||||
```python
|
||||
from functools import lru_cache
|
||||
|
||||
@lru_cache(maxsize=1000)
|
||||
def cached_call(prompt):
|
||||
pass
|
||||
|
||||
def call_llm(prompt, use_cache):
|
||||
if use_cache:
|
||||
return cached_call(prompt)
|
||||
# Call the underlying function directly
|
||||
return cached_call.__wrapped__(prompt)
|
||||
|
||||
class SummarizeNode(Node):
|
||||
def exec(self, text):
|
||||
return call_llm(f"Summarize: {text}", self.cur_retry==0)
|
||||
```
|
||||
|
||||
- Enable logging:
|
||||
|
||||
```python
|
||||
def call_llm(prompt):
|
||||
import logging
|
||||
logging.info(f"Prompt: {prompt}")
|
||||
response = ... # Your implementation here
|
||||
logging.info(f"Response: {response}")
|
||||
return response
|
||||
```
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
---
|
||||
description: Guidelines for using PocketFlow, Utility Function, Text-to-Speech
|
||||
globs:
|
||||
alwaysApply: false
|
||||
---
|
||||
# Text-to-Speech
|
||||
|
||||
| **Service** | **Free Tier** | **Pricing Model** | **Docs** |
|
||||
|----------------------|-----------------------|--------------------------------------------------------------|---------------------------------------------------------------------|
|
||||
| **Amazon Polly** | 5M std + 1M neural | ~$4 /M (std), ~$16 /M (neural) after free tier | [Polly Docs](https://aws.amazon.com/polly/) |
|
||||
| **Google Cloud TTS** | 4M std + 1M WaveNet | ~$4 /M (std), ~$16 /M (WaveNet) pay-as-you-go | [Cloud TTS Docs](https://cloud.google.com/text-to-speech) |
|
||||
| **Azure TTS** | 500K neural ongoing | ~$15 /M (neural), discount at higher volumes | [Azure TTS Docs](https://azure.microsoft.com/products/cognitive-services/text-to-speech/) |
|
||||
| **IBM Watson TTS** | 10K chars Lite plan | ~$0.02 /1K (i.e. ~$20 /M). Enterprise options available | [IBM Watson Docs](https://www.ibm.com/cloud/watson-text-to-speech) |
|
||||
| **ElevenLabs** | 10K chars monthly | From ~$5/mo (30K chars) up to $330/mo (2M chars). Enterprise | [ElevenLabs Docs](https://elevenlabs.io) |
|
||||
|
||||
## Example Python Code
|
||||
|
||||
### Amazon Polly
|
||||
```python
|
||||
import boto3
|
||||
|
||||
polly = boto3.client("polly", region_name="us-east-1",
|
||||
aws_access_key_id="YOUR_AWS_ACCESS_KEY_ID",
|
||||
aws_secret_access_key="YOUR_AWS_SECRET_ACCESS_KEY")
|
||||
|
||||
resp = polly.synthesize_speech(
|
||||
Text="Hello from Polly!",
|
||||
OutputFormat="mp3",
|
||||
VoiceId="Joanna"
|
||||
)
|
||||
|
||||
with open("polly.mp3", "wb") as f:
|
||||
f.write(resp["AudioStream"].read())
|
||||
```
|
||||
|
||||
### Google Cloud TTS
|
||||
```python
|
||||
from google.cloud import texttospeech
|
||||
|
||||
client = texttospeech.TextToSpeechClient()
|
||||
input_text = texttospeech.SynthesisInput(text="Hello from Google Cloud TTS!")
|
||||
voice = texttospeech.VoiceSelectionParams(language_code="en-US")
|
||||
audio_cfg = texttospeech.AudioConfig(audio_encoding=texttospeech.AudioEncoding.MP3)
|
||||
|
||||
resp = client.synthesize_speech(input=input_text, voice=voice, audio_config=audio_cfg)
|
||||
|
||||
with open("gcloud_tts.mp3", "wb") as f:
|
||||
f.write(resp.audio_content)
|
||||
```
|
||||
|
||||
### Azure TTS
|
||||
```python
|
||||
import azure.cognitiveservices.speech as speechsdk
|
||||
|
||||
speech_config = speechsdk.SpeechConfig(
|
||||
subscription="AZURE_KEY", region="AZURE_REGION")
|
||||
audio_cfg = speechsdk.audio.AudioConfig(filename="azure_tts.wav")
|
||||
|
||||
synthesizer = speechsdk.SpeechSynthesizer(
|
||||
speech_config=speech_config,
|
||||
audio_config=audio_cfg
|
||||
)
|
||||
|
||||
synthesizer.speak_text_async("Hello from Azure TTS!").get()
|
||||
```
|
||||
|
||||
### IBM Watson TTS
|
||||
```python
|
||||
from ibm_watson import TextToSpeechV1
|
||||
from ibm_cloud_sdk_core.authenticators import IAMAuthenticator
|
||||
|
||||
auth = IAMAuthenticator("IBM_API_KEY")
|
||||
service = TextToSpeechV1(authenticator=auth)
|
||||
service.set_service_url("IBM_SERVICE_URL")
|
||||
|
||||
resp = service.synthesize(
|
||||
"Hello from IBM Watson!",
|
||||
voice="en-US_AllisonV3Voice",
|
||||
accept="audio/mp3"
|
||||
).get_result()
|
||||
|
||||
with open("ibm_tts.mp3", "wb") as f:
|
||||
f.write(resp.content)
|
||||
```
|
||||
|
||||
### ElevenLabs
|
||||
```python
|
||||
import requests
|
||||
|
||||
api_key = "ELEVENLABS_KEY"
|
||||
voice_id = "ELEVENLABS_VOICE"
|
||||
url = f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}"
|
||||
headers = {"xi-api-key": api_key, "Content-Type": "application/json"}
|
||||
|
||||
json_data = {
|
||||
"text": "Hello from ElevenLabs!",
|
||||
"voice_settings": {"stability": 0.75, "similarity_boost": 0.75}
|
||||
}
|
||||
|
||||
resp = requests.post(url, headers=headers, json=json_data)
|
||||
|
||||
with open("elevenlabs.mp3", "wb") as f:
|
||||
f.write(resp.content)
|
||||
```
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
---
|
||||
description: Guidelines for using PocketFlow, Utility Function, Vector Databases
|
||||
globs:
|
||||
alwaysApply: false
|
||||
---
|
||||
# Vector Databases
|
||||
|
||||
|
||||
Below is a table of the popular vector search solutions:
|
||||
|
||||
| **Tool** | **Free Tier** | **Pricing Model** | **Docs** |
|
||||
| --- | --- | --- | --- |
|
||||
| **FAISS** | N/A, self-host | Open-source | [Faiss.ai](https://faiss.ai) |
|
||||
| **Pinecone** | 2GB free | From $25/mo | [pinecone.io](https://pinecone.io) |
|
||||
| **Qdrant** | 1GB free cloud | Pay-as-you-go | [qdrant.tech](https://qdrant.tech) |
|
||||
| **Weaviate** | 14-day sandbox | From $25/mo | [weaviate.io](https://weaviate.io) |
|
||||
| **Milvus** | 5GB free cloud | PAYG or $99/mo dedicated | [milvus.io](https://milvus.io) |
|
||||
| **Chroma** | N/A, self-host | Free (Apache 2.0) | [trychroma.com](https://trychroma.com) |
|
||||
| **Redis** | 30MB free | From $5/mo | [redis.io](https://redis.io) |
|
||||
|
||||
---
|
||||
## Example Python Code
|
||||
|
||||
Below are basic usage snippets for each tool.
|
||||
|
||||
### FAISS
|
||||
```python
|
||||
import faiss
|
||||
import numpy as np
|
||||
|
||||
# Dimensionality of embeddings
|
||||
d = 128
|
||||
|
||||
# Create a flat L2 index
|
||||
index = faiss.IndexFlatL2(d)
|
||||
|
||||
# Random vectors
|
||||
data = np.random.random((1000, d)).astype('float32')
|
||||
index.add(data)
|
||||
|
||||
# Query
|
||||
query = np.random.random((1, d)).astype('float32')
|
||||
D, I = index.search(query, k=5)
|
||||
|
||||
print("Distances:", D)
|
||||
print("Neighbors:", I)
|
||||
```
|
||||
|
||||
### Pinecone
|
||||
```python
|
||||
import pinecone
|
||||
|
||||
pinecone.init(api_key="YOUR_API_KEY", environment="YOUR_ENV")
|
||||
|
||||
index_name = "my-index"
|
||||
|
||||
# Create the index if it doesn't exist
|
||||
if index_name not in pinecone.list_indexes():
|
||||
pinecone.create_index(name=index_name, dimension=128)
|
||||
|
||||
# Connect
|
||||
index = pinecone.Index(index_name)
|
||||
|
||||
# Upsert
|
||||
vectors = [
|
||||
("id1", [0.1]*128),
|
||||
("id2", [0.2]*128)
|
||||
]
|
||||
index.upsert(vectors)
|
||||
|
||||
# Query
|
||||
response = index.query([[0.15]*128], top_k=3)
|
||||
print(response)
|
||||
```
|
||||
|
||||
### Qdrant
|
||||
```python
|
||||
import qdrant_client
|
||||
from qdrant_client.models import Distance, VectorParams, PointStruct
|
||||
|
||||
client = qdrant_client.QdrantClient(
|
||||
url="https://YOUR-QDRANT-CLOUD-ENDPOINT",
|
||||
api_key="YOUR_API_KEY"
|
||||
)
|
||||
|
||||
collection = "my_collection"
|
||||
client.recreate_collection(
|
||||
collection_name=collection,
|
||||
vectors_config=VectorParams(size=128, distance=Distance.COSINE)
|
||||
)
|
||||
|
||||
points = [
|
||||
PointStruct(id=1, vector=[0.1]*128, payload={"type": "doc1"}),
|
||||
PointStruct(id=2, vector=[0.2]*128, payload={"type": "doc2"}),
|
||||
]
|
||||
|
||||
client.upsert(collection_name=collection, points=points)
|
||||
|
||||
results = client.search(
|
||||
collection_name=collection,
|
||||
query_vector=[0.15]*128,
|
||||
limit=2
|
||||
)
|
||||
print(results)
|
||||
```
|
||||
|
||||
### Weaviate
|
||||
```python
|
||||
import weaviate
|
||||
|
||||
client = weaviate.Client("https://YOUR-WEAVIATE-CLOUD-ENDPOINT")
|
||||
|
||||
schema = {
|
||||
"classes": [
|
||||
{
|
||||
"class": "Article",
|
||||
"vectorizer": "none"
|
||||
}
|
||||
]
|
||||
}
|
||||
client.schema.create(schema)
|
||||
|
||||
obj = {
|
||||
"title": "Hello World",
|
||||
"content": "Weaviate vector search"
|
||||
}
|
||||
client.data_object.create(obj, "Article", vector=[0.1]*128)
|
||||
|
||||
resp = (
|
||||
client.query
|
||||
.get("Article", ["title", "content"])
|
||||
.with_near_vector({"vector": [0.15]*128})
|
||||
.with_limit(3)
|
||||
.do()
|
||||
)
|
||||
print(resp)
|
||||
```
|
||||
|
||||
### Milvus
|
||||
```python
|
||||
from pymilvus import connections, FieldSchema, CollectionSchema, DataType, Collection
|
||||
import numpy as np
|
||||
|
||||
connections.connect(alias="default", host="localhost", port="19530")
|
||||
|
||||
fields = [
|
||||
FieldSchema(name="id", dtype=DataType.INT64, is_primary=True),
|
||||
FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=128)
|
||||
]
|
||||
schema = CollectionSchema(fields)
|
||||
collection = Collection("MyCollection", schema)
|
||||
|
||||
emb = np.random.rand(10, 128).astype('float32')
|
||||
ids = list(range(10))
|
||||
collection.insert([ids, emb])
|
||||
|
||||
index_params = {
|
||||
"index_type": "IVF_FLAT",
|
||||
"params": {"nlist": 128},
|
||||
"metric_type": "L2"
|
||||
}
|
||||
collection.create_index("embedding", index_params)
|
||||
collection.load()
|
||||
|
||||
query_emb = np.random.rand(1, 128).astype('float32')
|
||||
results = collection.search(query_emb, "embedding", param={"nprobe": 10}, limit=3)
|
||||
print(results)
|
||||
```
|
||||
|
||||
### Chroma
|
||||
```python
|
||||
import chromadb
|
||||
from chromadb.config import Settings
|
||||
|
||||
client = chromadb.Client(Settings(
|
||||
chroma_db_impl="duckdb+parquet",
|
||||
persist_directory="./chroma_data"
|
||||
))
|
||||
|
||||
coll = client.create_collection("my_collection")
|
||||
|
||||
vectors = [[0.1, 0.2, 0.3], [0.2, 0.2, 0.2]]
|
||||
metas = [{"doc": "text1"}, {"doc": "text2"}]
|
||||
ids = ["id1", "id2"]
|
||||
coll.add(embeddings=vectors, metadatas=metas, ids=ids)
|
||||
|
||||
res = coll.query(query_embeddings=[[0.15, 0.25, 0.3]], n_results=2)
|
||||
print(res)
|
||||
```
|
||||
|
||||
### Redis
|
||||
```python
|
||||
import redis
|
||||
import struct
|
||||
|
||||
r = redis.Redis(host="localhost", port=6379)
|
||||
|
||||
# Create index
|
||||
r.execute_command(
|
||||
"FT.CREATE", "my_idx", "ON", "HASH",
|
||||
"SCHEMA", "embedding", "VECTOR", "FLAT", "6",
|
||||
"TYPE", "FLOAT32", "DIM", "128",
|
||||
"DISTANCE_METRIC", "L2"
|
||||
)
|
||||
|
||||
# Insert
|
||||
vec = struct.pack('128f', *[0.1]*128)
|
||||
r.hset("doc1", mapping={"embedding": vec})
|
||||
|
||||
# Search
|
||||
qvec = struct.pack('128f', *[0.15]*128)
|
||||
q = "*=>[KNN 3 @embedding $BLOB AS dist]"
|
||||
res = r.ft("my_idx").search(q, query_params={"BLOB": qvec})
|
||||
print(res.docs)
|
||||
```
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
---
|
||||
description: Guidelines for using PocketFlow, Utility Function, Viz and Debug
|
||||
globs:
|
||||
alwaysApply: false
|
||||
---
|
||||
# Visualization and Debugging
|
||||
|
||||
Similar to LLM wrappers, we **don't** provide built-in visualization and debugging. Here, we recommend some *minimal* (and incomplete) implementations These examples can serve as a starting point for your own tooling.
|
||||
|
||||
## 1. Visualization with Mermaid
|
||||
|
||||
This code recursively traverses the nested graph, assigns unique IDs to each node, and treats Flow nodes as subgraphs to generate Mermaid syntax for a hierarchical visualization.
|
||||
|
||||
{% raw %}
|
||||
```python
|
||||
def build_mermaid(start):
|
||||
ids, visited, lines = {}, set(), ["graph LR"]
|
||||
ctr = 1
|
||||
def get_id(n):
|
||||
nonlocal ctr
|
||||
return ids[n] if n in ids else (ids.setdefault(n, f"N{ctr}"), (ctr := ctr + 1))[0]
|
||||
def link(a, b):
|
||||
lines.append(f" {a} --> {b}")
|
||||
def walk(node, parent=None):
|
||||
if node in visited:
|
||||
return parent and link(parent, get_id(node))
|
||||
visited.add(node)
|
||||
if isinstance(node, Flow):
|
||||
node.start and parent and link(parent, get_id(node.start))
|
||||
lines.append(f"\n subgraph sub_flow_{get_id(node)}[{type(node).__name__}]")
|
||||
node.start and walk(node.start)
|
||||
for nxt in node.successors.values():
|
||||
node.start and walk(nxt, get_id(node.start)) or (parent and link(parent, get_id(nxt))) or walk(nxt)
|
||||
lines.append(" end\n")
|
||||
else:
|
||||
lines.append(f" {(nid := get_id(node))}['{type(node).__name__}']")
|
||||
parent and link(parent, nid)
|
||||
[walk(nxt, nid) for nxt in node.successors.values()]
|
||||
walk(start)
|
||||
return "\n".join(lines)
|
||||
```
|
||||
{% endraw %}
|
||||
|
||||
|
||||
For example, suppose we have a complex Flow for data science:
|
||||
|
||||
```python
|
||||
class DataPrepBatchNode(BatchNode):
|
||||
def prep(self,shared): return []
|
||||
class ValidateDataNode(Node): pass
|
||||
class FeatureExtractionNode(Node): pass
|
||||
class TrainModelNode(Node): pass
|
||||
class EvaluateModelNode(Node): pass
|
||||
class ModelFlow(Flow): pass
|
||||
class DataScienceFlow(Flow):pass
|
||||
|
||||
feature_node = FeatureExtractionNode()
|
||||
train_node = TrainModelNode()
|
||||
evaluate_node = EvaluateModelNode()
|
||||
feature_node >> train_node >> evaluate_node
|
||||
model_flow = ModelFlow(start=feature_node)
|
||||
data_prep_node = DataPrepBatchNode()
|
||||
validate_node = ValidateDataNode()
|
||||
data_prep_node >> validate_node >> model_flow
|
||||
data_science_flow = DataScienceFlow(start=data_prep_node)
|
||||
result = build_mermaid(start=data_science_flow)
|
||||
```
|
||||
|
||||
The code generates a Mermaid diagram:
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph sub_flow_N1[DataScienceFlow]
|
||||
N2['DataPrepBatchNode']
|
||||
N3['ValidateDataNode']
|
||||
N2 --> N3
|
||||
N3 --> N4
|
||||
|
||||
subgraph sub_flow_N5[ModelFlow]
|
||||
N4['FeatureExtractionNode']
|
||||
N6['TrainModelNode']
|
||||
N4 --> N6
|
||||
N7['EvaluateModelNode']
|
||||
N6 --> N7
|
||||
end
|
||||
|
||||
end
|
||||
```
|
||||
|
||||
## 2. Call Stack Debugging
|
||||
|
||||
It would be useful to print the Node call stacks for debugging. This can be achieved by inspecting the runtime call stack:
|
||||
|
||||
```python
|
||||
import inspect
|
||||
|
||||
def get_node_call_stack():
|
||||
stack = inspect.stack()
|
||||
node_names = []
|
||||
seen_ids = set()
|
||||
for frame_info in stack[1:]:
|
||||
local_vars = frame_info.frame.f_locals
|
||||
if 'self' in local_vars:
|
||||
caller_self = local_vars['self']
|
||||
if isinstance(caller_self, BaseNode) and id(caller_self) not in seen_ids:
|
||||
seen_ids.add(id(caller_self))
|
||||
node_names.append(type(caller_self).__name__)
|
||||
return node_names
|
||||
```
|
||||
|
||||
For example, suppose we have a complex Flow for data science:
|
||||
|
||||
```python
|
||||
class DataPrepBatchNode(BatchNode):
|
||||
def prep(self, shared): return []
|
||||
class ValidateDataNode(Node): pass
|
||||
class FeatureExtractionNode(Node): pass
|
||||
class TrainModelNode(Node): pass
|
||||
class EvaluateModelNode(Node):
|
||||
def prep(self, shared):
|
||||
stack = get_node_call_stack()
|
||||
print("Call stack:", stack)
|
||||
class ModelFlow(Flow): pass
|
||||
class DataScienceFlow(Flow):pass
|
||||
|
||||
feature_node = FeatureExtractionNode()
|
||||
train_node = TrainModelNode()
|
||||
evaluate_node = EvaluateModelNode()
|
||||
feature_node >> train_node >> evaluate_node
|
||||
model_flow = ModelFlow(start=feature_node)
|
||||
data_prep_node = DataPrepBatchNode()
|
||||
validate_node = ValidateDataNode()
|
||||
data_prep_node >> validate_node >> model_flow
|
||||
data_science_flow = DataScienceFlow(start=data_prep_node)
|
||||
data_science_flow.run({})
|
||||
```
|
||||
|
||||
The output would be: `Call stack: ['EvaluateModelNode', 'ModelFlow', 'DataScienceFlow']`
|
||||
@@ -0,0 +1,112 @@
|
||||
---
|
||||
description: Guidelines for using PocketFlow, Utility Function, Web Search
|
||||
globs:
|
||||
alwaysApply: false
|
||||
---
|
||||
# Web Search
|
||||
|
||||
We recommend some implementations of commonly used web search tools.
|
||||
|
||||
| **API** | **Free Tier** | **Pricing Model** | **Docs** |
|
||||
|---------------------------------|-----------------------------------------------|-----------------------------------------------------------------|------------------------------------------------------------------------|
|
||||
| **Google Custom Search JSON API** | 100 queries/day free | $5 per 1000 queries. | [Link](https://developers.google.com/custom-search/v1/overview) |
|
||||
| **Bing Web Search API** | 1,000 queries/month | $15–$25 per 1,000 queries. | [Link](https://azure.microsoft.com/en-us/services/cognitive-services/bing-web-search-api/) |
|
||||
| **DuckDuckGo Instant Answer** | Completely free (Instant Answers only, **no URLs**) | No paid plans; usage unlimited, but data is limited | [Link](https://duckduckgo.com/api) |
|
||||
| **Brave Search API** | 2,000 queries/month free | $3 per 1k queries for Base, $5 per 1k for Pro | [Link](https://brave.com/search/api/) |
|
||||
| **SerpApi** | 100 searches/month free | Start at $75/month for 5,000 searches| [Link](https://serpapi.com/) |
|
||||
| **RapidAPI** | Many options | Many options | [Link](https://rapidapi.com/search?term=search&sortBy=ByRelevance) |
|
||||
|
||||
## Example Python Code
|
||||
|
||||
### 1. Google Custom Search JSON API
|
||||
```python
|
||||
import requests
|
||||
|
||||
API_KEY = "YOUR_API_KEY"
|
||||
CX_ID = "YOUR_CX_ID"
|
||||
query = "example"
|
||||
|
||||
url = "https://www.googleapis.com/customsearch/v1"
|
||||
params = {
|
||||
"key": API_KEY,
|
||||
"cx": CX_ID,
|
||||
"q": query
|
||||
}
|
||||
|
||||
response = requests.get(url, params=params)
|
||||
results = response.json()
|
||||
print(results)
|
||||
```
|
||||
|
||||
### 2. Bing Web Search API
|
||||
```python
|
||||
import requests
|
||||
|
||||
SUBSCRIPTION_KEY = "YOUR_BING_API_KEY"
|
||||
query = "example"
|
||||
|
||||
url = "https://api.bing.microsoft.com/v7.0/search"
|
||||
headers = {"Ocp-Apim-Subscription-Key": SUBSCRIPTION_KEY}
|
||||
params = {"q": query}
|
||||
|
||||
response = requests.get(url, headers=headers, params=params)
|
||||
results = response.json()
|
||||
print(results)
|
||||
```
|
||||
|
||||
### 3. DuckDuckGo Instant Answer
|
||||
```python
|
||||
import requests
|
||||
|
||||
query = "example"
|
||||
url = "https://api.duckduckgo.com/"
|
||||
params = {
|
||||
"q": query,
|
||||
"format": "json"
|
||||
}
|
||||
|
||||
response = requests.get(url, params=params)
|
||||
results = response.json()
|
||||
print(results)
|
||||
```
|
||||
|
||||
### 4. Brave Search API
|
||||
```python
|
||||
import requests
|
||||
|
||||
SUBSCRIPTION_TOKEN = "YOUR_BRAVE_API_TOKEN"
|
||||
query = "example"
|
||||
|
||||
url = "https://api.search.brave.com/res/v1/web/search"
|
||||
headers = {
|
||||
"X-Subscription-Token": SUBSCRIPTION_TOKEN
|
||||
}
|
||||
params = {
|
||||
"q": query
|
||||
}
|
||||
|
||||
response = requests.get(url, headers=headers, params=params)
|
||||
results = response.json()
|
||||
print(results)
|
||||
```
|
||||
|
||||
### 5. SerpApi
|
||||
```python
|
||||
import requests
|
||||
|
||||
API_KEY = "YOUR_SERPAPI_KEY"
|
||||
query = "example"
|
||||
|
||||
url = "https://serpapi.com/search"
|
||||
params = {
|
||||
"engine": "google",
|
||||
"q": query,
|
||||
"api_key": API_KEY
|
||||
}
|
||||
|
||||
response = requests.get(url, params=params)
|
||||
results = response.json()
|
||||
print(results)
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user