add majority vote tutorial
This commit is contained in:
zachary62
parent
e303c3ad9b
commit
6cefbfea3b
|
|
@ -78,6 +78,7 @@ From there, it's easy to implement popular design patterns like ([Multi-](https:
|
|||
| [Supervisor](https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/pocketflow-supervisor) | ★☆☆ <br> *Beginner* | Research agent is getting unreliable... Let's build a supervision process|
|
||||
| [Parallel](https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/pocketflow-parallel-batch) | ★☆☆ <br> *Beginner* | A parallel execution demo that shows 3x speedup |
|
||||
| [Parallel Flow](https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/pocketflow-parallel-batch-flow) | ★☆☆ <br> *Beginner* | A parallel image processing demo showing 8x speedup with multiple filters |
|
||||
| [Majority Vote](https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/pocketflow-majority-vote) | ★☆☆ <br> *Beginner* | Improve reasoning accuracy by aggregating multiple solution attempts |
|
||||
| [Thinking](https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/pocketflow-thinking) | ★☆☆ <br> *Beginner* | Solve complex reasoning problems through Chain-of-Thought |
|
||||
| [Memory](https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/pocketflow-chat-memory) | ★☆☆ <br> *Beginner* | A chat bot with short-term and long-term memory |
|
||||
| [MCP](https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/pocketflow-mcp) | ★☆☆ <br> *Beginner* | Agent using Model Context Protocol for numerical operations |
|
||||
|
|
|
|||
|
|
@ -1,13 +1,14 @@
|
|||
# Extended Thinking
|
||||
# Majority Vote Reasoning
|
||||
|
||||
This project demonstrates an extended thinking mode implementation that enables LLMs to solve complex reasoning problems by thinking step-by-step. It's designed to improve problem-solving accuracy through deliberate reasoning.
|
||||
This project demonstrates a majority vote implementation that enables LLMs to solve complex reasoning problems by aggregating multiple independent attempts. It's designed to improve problem-solving accuracy through consensus-based reasoning.
|
||||
|
||||
## Features
|
||||
|
||||
- Improves model reasoning on complex problems
|
||||
- Works with models like Claude 3.7 Sonnet that support extended thinking
|
||||
- Solves problems that direct prompting often fails on
|
||||
- Improves model reliability on complex problems through multiple attempts
|
||||
- Works with models like Claude 3.7 Sonnet
|
||||
- Solves problems that single attempts often fail on
|
||||
- Provides detailed reasoning traces for verification
|
||||
- Uses a consensus approach to reduce the impact of occasional reasoning errors
|
||||
|
||||
## Getting Started
|
||||
|
||||
|
|
@ -21,34 +22,34 @@ pip install -r requirements.txt
|
|||
export ANTHROPIC_API_KEY="your-api-key-here"
|
||||
```
|
||||
|
||||
3. Run a test problem to see thinking mode in action:
|
||||
3. Run a test problem to see majority voting in action:
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
4. Try your own reasoning problem:
|
||||
```bash
|
||||
python main.py --"Your complex reasoning problem here"
|
||||
python main.py --problem "Your complex reasoning problem here" --tries 5
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
The implementation uses a self-looping Chain of Thought node that allows an LLM to think through complex problems step by step:
|
||||
The implementation uses a MajorityVoteNode that processes multiple attempts and finds consensus:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
cot[ChainOfThoughtNode] -->|"continue"| cot
|
||||
mv[MajorityVoteNode]
|
||||
```
|
||||
|
||||
Each time the node loops, it:
|
||||
1. Reads the problem and previous thoughts
|
||||
2. Generates the next thought or final solution
|
||||
3. Decides whether more thinking is needed
|
||||
The MajorityVoteNode:
|
||||
1. Makes multiple independent attempts to solve the same problem
|
||||
2. Collects structured answers from each attempt
|
||||
3. Determines the most frequent answer as the final solution
|
||||
4. Returns the consensus answer
|
||||
|
||||
This approach helps LLMs solve problems that would be difficult with a single-pass approach.
|
||||
This approach helps overcome occasional reasoning errors that might occur in individual attempts.
|
||||
|
||||
|
||||
## Example Thinking Process
|
||||
## Example Problem
|
||||
|
||||
Example Problem from [Quant Interview](https://www.youtube.com/watch?v=SCP7JptxPU0):
|
||||
|
||||
|
|
@ -56,7 +57,7 @@ Example Problem from [Quant Interview](https://www.youtube.com/watch?v=SCP7JptxP
|
|||
You work at a shoe factory. In front of you, there are three pairs of shoes (six individual shoes) with the following sizes: two size 4s, two size 5s, and two size 6s. The factory defines an "acceptable pair" as two shoes that differ in size by a maximum of one size (e.g., a size 5 and a size 6 would be an acceptable pair). If you close your eyes and randomly pick three pairs of shoes without replacement, what is the probability that you end up drawing three acceptable pairs?
|
||||
```
|
||||
|
||||
Below is an example of how Claude 3.7 Sonnet to solve this complex problem, and get the correct result:
|
||||
Below is an example of how the majority vote approach uses Claude 3.7 Sonnet to solve this complex problem:
|
||||
|
||||
```
|
||||
========================
|
||||
|
|
@ -70,8 +71,9 @@ Frequency => 4
|
|||
====================
|
||||
```
|
||||
|
||||
This shows that 4 out of 5 attempts yielded the same answer (0.333), which is chosen as the final solution.
|
||||
|
||||
## Files
|
||||
|
||||
- [`main.py`](./main.py): Implementation of the ...
|
||||
- [`main.py`](./main.py): Implementation of the majority vote node and flow
|
||||
- [`utils.py`](./utils.py): Simple wrapper for calling the Anthropic model
|
||||
|
||||
|
|
@ -1,8 +1,9 @@
|
|||
# main.py
|
||||
import argparse
|
||||
from pocketflow import BatchNode, Flow
|
||||
import collections
|
||||
from utils import call_llm
|
||||
import yaml
|
||||
|
||||
class MajorityVoteNode(BatchNode):
|
||||
def prep(self, shared):
|
||||
question = shared.get("question", "(No question provided)")
|
||||
|
|
@ -53,9 +54,18 @@ answer: 0.123 # Final answer as a decimal with 3 decimal places
|
|||
return "end"
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Set up argument parser
|
||||
parser = argparse.ArgumentParser(description="Run majority vote reasoning on a problem")
|
||||
parser.add_argument("--problem", type=str, help="Your reasoning problem to solve")
|
||||
parser.add_argument("--tries", type=int, default=5, help="Number of attempts to make (default: 5)")
|
||||
args = parser.parse_args()
|
||||
|
||||
# Default problem if none provided
|
||||
default_problem = """You work at a shoe factory. In front of you, there are three pairs of shoes (six individual shoes) with the following sizes: two size 4s, two size 5s, and two size 6s. The factory defines an "acceptable pair" as two shoes that differ in size by a maximum of one size (e.g., a size 5 and a size 6 would be an acceptable pair). If you close your eyes and randomly pick three pairs of shoes without replacement, what is the probability that you end up drawing three acceptable pairs?"""
|
||||
|
||||
shared = {
|
||||
"question": """You work at a shoe factory. In front of you, there are three pairs of shoes (six individual shoes) with the following sizes: two size 4s, two size 5s, and two size 6s. The factory defines an "acceptable pair" as two shoes that differ in size by a maximum of one size (e.g., a size 5 and a size 6 would be an acceptable pair). If you close your eyes and randomly pick three pairs of shoes without replacement, what is the probability that you end up drawing three acceptable pairs?""",
|
||||
"num_tries": 5
|
||||
"question": args.problem if args.problem else default_problem,
|
||||
"num_tries": args.tries
|
||||
}
|
||||
|
||||
majority_node = MajorityVoteNode()
|
||||
|
|
|
|||
Loading…
Reference in New Issue