git404hub

what is sky-with-you fr?

akinia0315/sky-with-you — explained in plain English

Analysis updated 2026-05-18

18PythonAudience · developerComplexity · 4/5LicenseSetup · hard

tl;dr

An automation tool that gives an AI companion a physical presence in the PC game Sky: Children of the Light, letting it chat, react, and hold hands with you in real time.

vibe map

mindmap
  root((sky-with-you))
    What it does
      AI companion body
      Reads chat panel
      Reacts to hand holds
    Tech stack
      Python
      OCR
      Arduino HID
    Use cases
      Automate companion actions
      Study perception plus LLM setup
    Audience
      Sky game players
      Hobby automation builders
    Setup
      Windows PC
      Arduino board
      OpenRouter key

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Give an AI controlled companion a body that reacts to your Sky: Children of the Light gameplay in real time.

VIBE 2

Automate routine companion actions like lighting candles, bowing, and accepting teleport invites while you play.

VIBE 3

Study a working example of combining screen perception, an LLM, and hardware keyboard injection.

what's the stack?

PythonOpenCVOCRArduino HIDMCPOpenRouter

how it stacks up fr

akinia0315/sky-with-you1038lab/comfyui-agnes-aiadityaarsharma/wordpress-malware-removal
Stars181818
LanguagePythonPythonPython
Setup difficultyhardeasymoderate
Complexity4/52/53/5
Audiencedeveloperdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · hard time til it works · 1h+

Needs a Windows PC running the game plus an Arduino compatible board flashed with custom firmware and an OpenRouter API key.

MIT license: free to use, modify, and distribute for any purpose, including commercial use, as long as you keep the copyright notice.

in plain english

This project lets an AI companion play as a physical presence inside Sky: Children of the Light, a PC game. It watches the game's chat panel, replies to messages, lights and receives candles, bows, accepts teleport invitations, holds your hand when you reach out, and can say a line like come on, let's go home before guiding you back to your home area. The system runs entirely on the same Windows machine as the game and is built in three layers. A perception layer takes screenshots and combines visual matching with template recognition and text reading to build a shared picture of what is happening on screen, including which panels are open, whether a popup is showing, and what scene the game is in. A scheduling layer reads messages, asks a language model to write a reply based on a persona file the user sets up, and decides what action to take next, always working from that shared picture rather than the live screen. An execution layer sends the actual keyboard presses using an external Arduino board that plugs in over USB, so the game reads them as real hardware key presses instead of software simulated ones, since the game blocks software key simulation. This layer also handles Chinese text input through the clipboard, takes screenshots, and checks that the game window is in focus before pressing anything. Setting it up requires a Windows PC running the game, a small Arduino compatible board flashed with the included firmware, and an OpenRouter API key for the language model. You edit a persona file to describe who your AI companion is, then run two Python scripts in separate terminals. The project also documents several safety details worked out through trial and error, such as refusing to send key presses when the game window is not focused, and ignoring the automation's own messages when they get read back by the text recognition step so it does not talk to itself. The README notes this only reads the screen and sends keyboard input. It does not modify the game or read its memory, and automating gameplay this way may violate the game's terms of service, so use is at your own risk. The project builds on an earlier keyboard injection tool credited in the README. It is released under the MIT license.

prompts (copy fr)

prompt 1
Help me write a persona.txt file for my Sky: Children of the Light AI companion using this project's template.
prompt 2
Walk me through flashing the Arduino Leonardo with the sky_keyboard_v2.ino firmware from this repo.
prompt 3
Explain how the panel_detector.py perception layer combines OCR and template matching to build WorldState.
prompt 4
Show me how to set the OPENROUTER_API_KEY environment variable so sky-loop-v7.py can call the language model.

Frequently asked questions

what is sky-with-you fr?

An automation tool that gives an AI companion a physical presence in the PC game Sky: Children of the Light, letting it chat, react, and hold hands with you in real time.

What language is sky-with-you written in?

Mainly Python. The stack also includes Python, OpenCV, OCR.

What license does sky-with-you use?

MIT license: free to use, modify, and distribute for any purpose, including commercial use, as long as you keep the copyright notice.

How hard is sky-with-you to set up?

Setup difficulty is rated hard, with roughly 1h+ to a first successful run.

Who is sky-with-you for?

Mainly developer.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.