update para

This commit is contained in:
zachary62
2025-03-20 17:33:10 -04:00
parent 24782d1af1
commit 3544dd5440
9 changed files with 305 additions and 4 deletions
+12 -1
View File
@@ -38,4 +38,15 @@ Parallel took: 1.00 seconds
- Good for: Rate-limited APIs, maintaining order
- **Parallel**: Total time ≈ longest single item time
- Good for: I/O-bound tasks, independent operations
- Good for: I/O-bound tasks, independent operations
## Tech Dive Deep
- **Python's GIL** prevents true CPU-bound parallelism, but LLM calls are I/O-bound
- **Async/await** overlaps waiting time between requests
- Example: `await client.chat.completions.create(...)`
- See: [OpenAI's async usage](https://github.com/openai/openai-python?tab=readme-ov-file#async-usage)
For maximum performance and cost efficiency, consider using batch APIs:
- [OpenAI's Batch API](https://platform.openai.com/docs/guides/batch) lets you process multiple prompts in a single request
- Reduces overhead and can be more cost-effective for large workloads