Pocket Flow - LLM Framework in 100 Lines

A 100-line minimalist LLM framework for (Multi-)Agents, task decomposition, RAG, etc.

• Install via pip install pocketflow, or just copy the source codes (only 100 lines)

• 💡 Pro tip!! Build LLM apps with LLMs assistants (ChatGPT, Claude, Cursor.ai, etc.)

  • Claude Project (Sonnet 3.5 strongly recommended!):

    • Create a project and upload the docs to project knowledge

    • Set project custom instructions. For example:

      1. check "tool.md" and "llm.md" for the required functions.
      2. design the high-level (batch) flow and nodes.
      3. design the shared memory structure: define its fields, data structures, and how they will be updated.
      Think out aloud for above first and ask users if your design makes sense.
      4. Finally, implement. Start with simple, minimalistic codes without, for example, typing.
      

    • Ask it to build LLM application!

  • ChatGPT: Check out GPT assistant

    • GPT store seems to use older models. It's good at explaining but not good at coding.

    • Alternatively, send the docs to newer models like O1 for coding.

Documentation: https://minillmflow.github.io/PocketFlow/

Why Pocket Flow?

Pocket Flow is designed to be the framework used by LLMs. In the future, LLM projects will be self-programmed by LLMs themselves: Users specify requirements, and LLMs will design, build, and maintain. Current LLMs are:

1. 👍 Good at Low-level Details: LLMs can handle details like wrappers, tools, and prompts, which don't belong in a framework. Current frameworks are over-engineered, making them hard for humans (and LLMs) to maintain.

2. 👎 Bad at High-level Paradigms: While paradigms like MapReduce, Task Decomposition, and Agents are powerful, LLMs still struggle to design them elegantly. These high-level concepts should be emphasized in frameworks.

The ideal framework for LLMs should (1) strip away low-level implementation details, and (2) keep high-level programming paradigms. Hence, we provide this minimal (100-line) framework that allows LLMs to focus on what matters.

Pocket Flow is also a learning resource, as current frameworks abstract too much away.

How Does it Work?

The 100 lines capture what we see as the core abstraction of most LLM frameworks: a nested directed graph that breaks down tasks into multiple (LLM) steps, with branching and recursion for agent-like decision-making. From there, it’s easy to layer on more complex features.

• To learn more details, please check out documentation: https://minillmflow.github.io/PocketFlow/

• Beginner Tutorial: Text summarization for Paul Graham Essay + QA agent

  • Have questions for this tutorial? Ask LLM assistants through this prompt

• More coming soon ... Let us know you’d love to see!