ifoxhz/pigame

This is a Pi extension that uses an LLM to drive the game, starting with a simple 2D game. - ifoxhz/pigame

0 stars · 0 forks

README

pigame English | 中文 General game agent on Pi + LLM (e.g. DeepSeek). Design codename: GameMind. First demo game: Neon Snake. Remote: git@github.com:ifoxhz/pigame.git

Welcome. Contributions of any kind and any ideas are welcome — let’s make the Pi agent more fun.

Principle Pi + LLM = brain (strategy / when to stop) pigame = scaffolding only tools = eyes / hands / feet

Layer Role

Pi + LLM Decide what to do; judge when the goal is done

game_observe Eyes — see the screen → AgentState

game_move / game_wait Hands & feet — act / wait

npm run smoke:* / /game eat Test scaffolding only — hardcoded loops for CI; not the agent path

There is no game_eat_food / game_survive tool. The model loops observe → move itself.

Requirements

Tool Notes

Node.js ≥ 20 recommended (--experimental-strip-types)

npm Comes with Node

Pi CLI Required for the agent path (pi on PATH)

LLM in Pi e.g. DeepSeek configured in your Pi settings

WSLg / display Only for visible Chromium (HEADED=1)

Setup 1. Clone pigame git clone git@github.com:ifoxhz/pigame.git cd pigame # or your local folder name 2. Install pigame deps + Chromium npm install --prefix pigame npx --prefix pigame playwright install chromium Default game (no extra clone): pigame/games/snake.html — a vendored Neon Snake with agent mode (see below). Anyone who clones this repo gets the patched game automatically; Adapter loads it on /game connect. 3. Optional — upstream Mini Game Studio Only if you want the original continuous-loop game for comparison: git clone git@github.com:Digiman/mini-game-studio-ai-gen.git

HTTPS: https://github.com/Digiman/mini-game-studio-ai-gen.git

Quick test (full run once) A — One-time setup git clone git@github.com:ifoxhz/pigame.git cd pigame # or your local folder name

npm install --prefix pigame npx --prefix pigame playwright install chromium B — Pi + LLM (primary path) cd /path/to/pigame HEADED=1 pi -e ./.pi/extensions/pigame.ts In the Pi TUI (in order): /game connect /skill:eat-food

Or paste this prompt instead of the skill: Use only game_observe, game_move, and optionally game_wait. Prefer groundTruth grid coords. Eat one food: observe → move → repeat. When score increases, stop and report before/after score. Do not reverse 180°.

Watch the Playwright Chromium window (snake freezes between moves). When done: /game disconnect

After code changes in a live session: /reload, then /game disconnect → /game connect again. C — Optional: non-LLM smoke (scaffolding)

Adapter + perception + I/O only (no Pi / no LLM)

dist/pi-pigame/ — installable directory dist/pigame-0.1.0.tgz — npm tarball

Others install into Pi with: pi install /path/to/pigame/dist/pi-pigame

or

after you publish to npm:

npm publish ./dist/pigame-0.1.0.tgz

pi install npm:pigame

Agent-mode Snake (for Pi + LLM latency) Problem: Upstream Snake ticks on its own. While the LLM is in a long observe → think → move chain, the snake has already moved, so decisions are stale and play feels clumsy. What we ship: pigame/games/snake.html (see pigame/games/README.md)

Behavior Meaning

Auto gameLoop Paused after connect (setAgentMode(true))

game_observe World stays frozen (no time pass)

game_move setDir + exactly 1 grid step (MOVE_STEPS to change)

No special startup flag is required for agent mode — connect uses the vendored file by default.

Variable Default Meaning

SNAKE_HTML pigame/games/snake.html Override HTML path

MOVE_STEPS 1 Grid steps per game_move

HEADED unset 1 → show Chromium

Pi-driven play (primary path) This is the intended product flow: you open Pi; the LLM uses tools. Step A — Start Pi with the extension cd /path/to/pigame HEADED=1 pi -e ./.pi/extensions/pigame.ts

or: pi (auto-loads .pi/extensions/pigame.ts if the project is trusted)

Playwright opens Chromium and loads pigame/games/snake.html (agent-mode), then Start. World is frozen until each game_move. Do not open snake.html in your own browser — only the Playwright window is controlled. Optional: /game observe to sanity-check AgentState. Step C — Let the LLM drive (prompt) Example (DeepSeek / any model in Pi):

Use only game_observe, game_move, and optionally game_wait. Eat one food: observe → choose a direction → move → repeat. When score increases, stop and report before/after score. Do not reverse 180° from the previous move.

Or load the skill: /skill:eat-food. Step D — What the model should do game_observe → read player / food / score game_move → one step toward food game_observe → … … until the model decides the goal is done …

Slash commands (session scaffolding — not LLM tools)

Command Action

/game Status + help

/game connect Start Playwright Neon Snake

/game observe Compact AgentState (manual check)

/game state Full AgentState JSON

/game reset Restart round

/game disconnect Close browser

/game mock Switch to mock adapter

/game eat Test scaffolding — scripted auto-eat

/game survive Test scaffolding — scripted play until death

LLM tools

Tool Role

game_observe Eyes

game_move Hands / feet (up / down / left / right)

game_wait Wait

Test scaffolding (npm run smoke:*) These commands do not use Pi or an LLM. They run fixed policies (eatFoodLoop / surviveUntilEnd) to verify Adapter + Perception + I/O. Use them for CI and debugging — not as the agent design.

Eat one food (score 0 → 10), headless

Same with visible Chromium (WSLg)

Long run until death (or stop at a score)

Command What it checks (scaffolding)

npm run smoke Mock adapter / AgentState pipeline

npm run smoke:snake Playwright open + Start + key + screenshot

npm run smoke:perception CV fixture + live perception

npm run smoke:eat Scripted steer until score↑

npm run smoke:survive Scripted loop until game over (or MAX_SCORE)

npm run smoke:*:headed Same with HEADED=1

Environment variables (smoke / headed)

Variable Default Meaning

HEADED unset 1 / true → show Chromium

SNAKE_HTML vendored agent snake Path override

MOVE_STEPS 1 Steps per move (agent mode)

MAX_SCORE none Survive stops when score ≥ value

MAX_STEPS 50000 Survive step cap

STEP_MS 200 Visual delay between smoke moves

HOLD_MS 5000–8000 Keep window open after headed runs

Neon Snake has no clear — only death; MAX_SCORE is an artificial stop for tests.

Layout . ├── README.md / README.zh-CN.md ├── package.json # script proxies ├── docs/ # design & schema ├── pigame/ # main package │ ├── games/snake.html # DEFAULT agent-mode Neon Snake (vendored) │ ├── games/README.md │ ├── extensions/pigame.ts │ ├── skills/ │ ├── scripts/ # smoke:* (test scaffolding) │ └── src/ ├── .pi/extensions/pigame.ts # Pi entry └── mini-game-studio-ai-gen/ # optional upstream clone (SNAKE_HTML=…)

Design docs: docs/README.md.

Troubleshooting

Issue Fix

Missing pigame/games/snake.html Pull latest pigame — file is in-repo

Want upstream continuous loop SNAKE_HTML=./mini-game-studio-ai-gen/games/snake.html (clone Digiman first)

Snake runs by itself while LLM thinks You are not on agent-mode HTML; check default path / SNAKE_HTML

ENOENT package.json at root Use root scripts or cd pigame

No Chromium window HEADED=1 or *:headed; check WSLg / echo $DISPLAY

Opened Snake yourself, nothing happens Only Playwright’s window is controlled

Pi extension not loaded pi -e ./.pi/extensions/pigame.ts then /reload

Acknowledgments Thanks to Mini Game Studio (Digiman/mini-game-studio-ai-gen) for the original Neon Snake and HTML game collection. We vendor a patched copy under pigame/games/ for Pi + LLM agent-mode testing (paused auto loop + single-step moves).

License / status Work in progress (MVP: Pi + LLM drives Snake via observe / move). Phase notes: docs/implementation.md.

source