init add to majority vote

This commit is contained in:
zachary62 2025-03-31 19:43:32 -04:00
parent 130ada3c23
commit e303c3ad9b
4 changed files with 166 additions and 0 deletions

View File

@ -0,0 +1,77 @@
# Extended Thinking
This project demonstrates an extended thinking mode implementation that enables LLMs to solve complex reasoning problems by thinking step-by-step. It's designed to improve problem-solving accuracy through deliberate reasoning.
## Features
- Improves model reasoning on complex problems
- Works with models like Claude 3.7 Sonnet that support extended thinking
- Solves problems that direct prompting often fails on
- Provides detailed reasoning traces for verification
## Getting Started
1. Install the required packages:
```bash
pip install -r requirements.txt
```
2. Set up your API key:
```bash
export ANTHROPIC_API_KEY="your-api-key-here"
```
3. Run a test problem to see thinking mode in action:
```bash
python main.py
```
4. Try your own reasoning problem:
```bash
python main.py --"Your complex reasoning problem here"
```
## How It Works
The implementation uses a self-looping Chain of Thought node that allows an LLM to think through complex problems step by step:
```mermaid
flowchart LR
cot[ChainOfThoughtNode] -->|"continue"| cot
```
Each time the node loops, it:
1. Reads the problem and previous thoughts
2. Generates the next thought or final solution
3. Decides whether more thinking is needed
This approach helps LLMs solve problems that would be difficult with a single-pass approach.
## Example Thinking Process
Example Problem from [Quant Interview](https://www.youtube.com/watch?v=SCP7JptxPU0):
```
You work at a shoe factory. In front of you, there are three pairs of shoes (six individual shoes) with the following sizes: two size 4s, two size 5s, and two size 6s. The factory defines an "acceptable pair" as two shoes that differ in size by a maximum of one size (e.g., a size 5 and a size 6 would be an acceptable pair). If you close your eyes and randomly pick three pairs of shoes without replacement, what is the probability that you end up drawing three acceptable pairs?
```
Below is an example of how Claude 3.7 Sonnet to solve this complex problem, and get the correct result:
```
========================
All structured answers: ['0.333', '0.333', '0.333', '0.6', '0.333']
Majority vote => 0.333
Frequency => 4
========================
=== Final Answer ===
0.333
====================
```
## Files
- [`main.py`](./main.py): Implementation of the ...
- [`utils.py`](./utils.py): Simple wrapper for calling the Anthropic model

View File

@ -0,0 +1,67 @@
# main.py
from pocketflow import BatchNode, Flow
import collections
from utils import call_llm
import yaml
class MajorityVoteNode(BatchNode):
def prep(self, shared):
question = shared.get("question", "(No question provided)")
attempts_count = shared.get("num_tries", 3)
return [question for _ in range(attempts_count)]
def exec(self, single_question: str):
prompt = f"""
You are a helpful assistant. Please answer the user's question below.
Question: {single_question}
Return strictly using the following YAML structure:
```yaml
thinking: |
(Your thinking process here)
answer: 0.123 # Final answer as a decimal with 3 decimal places
```"""
raw_response = call_llm(prompt)
yaml_part = raw_response.split("```yaml")[1].split("```")[0].strip()
parsed = yaml.safe_load(yaml_part)
# Validate we have at least 'answer' field
if not isinstance(parsed, dict) or 'answer' not in parsed:
raise RuntimeError(f"Missing 'answer' in YAML: {parsed}")
# Return only the 'answer' field for the majority vote.
return str(parsed['answer'])
def exec_fallback(self, prep_res, exc):
return None
def post(self, shared, prep_res, exec_res_list):
# Count frequency for non-None answers
exec_res_list = [res for res in exec_res_list if res is not None]
counter = collections.Counter(exec_res_list)
best_answer, freq = counter.most_common(1)[0]
# Store final
shared["majority_answer"] = best_answer
print("========================")
print("All structured answers:", exec_res_list)
print("Majority vote =>", best_answer)
print("Frequency =>", freq)
print("========================")
# End the flow
return "end"
if __name__ == "__main__":
shared = {
"question": """You work at a shoe factory. In front of you, there are three pairs of shoes (six individual shoes) with the following sizes: two size 4s, two size 5s, and two size 6s. The factory defines an "acceptable pair" as two shoes that differ in size by a maximum of one size (e.g., a size 5 and a size 6 would be an acceptable pair). If you close your eyes and randomly pick three pairs of shoes without replacement, what is the probability that you end up drawing three acceptable pairs?""",
"num_tries": 5
}
majority_node = MajorityVoteNode()
flow = Flow(start=majority_node)
flow.run(shared)
print("\n=== Final Answer ===")
print(shared["majority_answer"])
print("====================")

View File

@ -0,0 +1,2 @@
pocketflow>=0.0.1
anthropic>=0.15.0 # For Claude API access

View File

@ -0,0 +1,20 @@
from anthropic import Anthropic
import os
def call_llm(prompt):
client = Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY", "your-api-key"))
response = client.messages.create(
model="claude-3-7-sonnet-20250219",
max_tokens=10000,
messages=[
{"role": "user", "content": prompt}
]
)
return response.content[0].text
if __name__ == "__main__":
print("## Testing call_llm")
prompt = "In a few words, what is the meaning of life?"
print(f"## Prompt: {prompt}")
response = call_llm(prompt)
print(f"## Response: {response}")