Show HN: LLM plays Pokémon (open sourced)
4 comments
·February 26, 2025minimaxir
See also: the AI Plays Pokemon project that went megaviral a year or so ago, using CNNs and RL instead of LLMs: https://github.com/PWhiddy/PokemonRedExperiments
> I believe that Claude Plays Pokemon isn't doing any of the memory parsing I spent a ton of time, they are just streaming the memory directly to Claude 3.7 and it is figuring it out
It is implied they are using structured Pokemon data from the LLM and saving it as a knowledge base. That is the only way they can get live Pokemon party data to display in the UI: https://www.twitch.tv/claudeplayspokemon
The AI Plays Pokemon project above does note some of the memory addresses where that data is contained, since it used that data to calculate the reward for the PPO.
adenta
On this page (https://excalidraw.com/#json=WrM9ViixPu2je5cVJZGCe,no_UoONhF...) linked from their twitch, it says: “ This info is all parsed directly from the RAM of the game, Claude Code is very good at this task”. I’m reading that as “we are pumping the RAM directly into the LLM”, but I could be mistaken.
minimaxir
I agree that's ambigiously worded. For example, I'm not sure if Claude could identify "MT MOON B1F" from the RAM data alone since internally world map areas are only known by IDs, while AI Plays Pokemon did annotate the corresponding area with a human-readable name. https://github.com/PWhiddy/PokemonRedExperiments/blob/master...
Though this RAM data could be in Claude's training data.
None4U
I suspect that means they wrote the memory parser using Claude (the Twitch description also mentions the LLM getting specific info)
I built a bot that plays Pokémon FireRed. It can explore, battle, and respond to game events. Farthest I made it was Viridian Forest.
I paused development a couple months ago, but given the launch of ClaudePlaysPokemon, decided to open source!