Skip to content

Commit 620722f

Browse files
adityachauhan0adityachauhan0
committed
merge: resolve conflict and consolidate branches into main
2 parents 46c5090 + 48e0473 commit 620722f

4 files changed

Lines changed: 138 additions & 19 deletions

File tree

‎GEMINI.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# GEMINI.md - Project Save State: Gamer-Cat 😼
2+
3+
## Project Context
4+
- **Goal:** Create a local, offline AI gaming buddy (VLM + LLM + STT + TTS).
5+
- **Constraint:** Must run under **16GB total system RAM**.
6+
- **Tech Stack:**
7+
- **Vision:** Moondream2 (via Ollama/llama.cpp)
8+
- **Brain:** Llama-3-8B (4-bit quantization)
9+
- **STT:** Faster-Whisper
10+
- **TTS:** Piper
11+
- **Status:** Planning complete. Repository initialized at `adityachauhan0/gamer-cat`.
12+
13+
## Current Progress
14+
- [x] `idea.md`: Initial vision.
15+
- [x] `technical_plan.md`: Resource budgeting (<12GB AI footprint).
16+
- [x] `implementation_plan.md`: 6-Phase development roadmap.
17+
- [x] GitHub Repository: `adityachauhan0/gamer-cat` (main branch).
18+
- [x] GitHub Pages: Landing page live.
19+
20+
## Next Directive
21+
- **Phase 1 Implementation:** Setup the Python environment on Windows and verify Moondream2/Llama-3 memory usage.
22+
23+
## Context for Gemini CLI (Windows)
24+
When restarting, prioritize local-first tools for Windows (e.g., `Ollama` for Windows, `PowerShell` scripts, and `venv` for Python).

‎README.md‎

Lines changed: 49 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,66 @@
1-
# GamerCat 🐱🎮
1+
# 😼 GAMER-CAT 🎮
22

3-
A real-time AI companion that sees what you're doing and talks to you.
3+
> **Your ultimate AI gaming buddy. No lag. No cloud. All vibes.**
44
5-
## Features
6-
- **Local Screen Perception:** Captures your screen every 10 seconds and describes it in 30 words.
7-
- **Voice-to-Voice:** Speak to GamerCat, and it responds with context-aware buddy talk.
8-
- **MCP Server:** Exposes screen context as a standard Model Context Protocol tool.
9-
- **Fully Offline:** Uses Ollama for vision/chat and Whisper for STT.
5+
Gamer-Cat is a local-first, offline AI companion that actually **sees** what you're doing. Whether you're clutching a 1v5 in Valorant or peacefully farming in Stardew Valley, Gamer-Cat is there to hype you up, give (sometimes questionable) advice, and keep you company.
106

11-
## Prerequisites
12-
1. **Ollama:** [Install Ollama](https://ollama.com/) and download the models:
7+
---
8+
9+
## 🚀 LEVEL UP YOUR SETUP
10+
11+
* **👁️ Visual Context:** Snaps your screen every 10s. It knows when you're in the lobby or in the heat of battle.
12+
* **🗣️ Real-time Voice:** Natural, low-latency voice interaction. Talk to it like a real homie.
13+
* **🔒 100% Offline:** Runs entirely on your rig. No telemetry, no data harvesting, no lag spikes from cloud APIs.
14+
* **MCP Server:** Exposes screen context as a standard Model Context Protocol tool.
15+
16+
---
17+
18+
## 🛠️ TECH STACK (The "Specs")
19+
20+
| Component | Tech |
21+
| :--- | :--- |
22+
| **The Brain** | Llama-3 (Ollama) |
23+
| **The Eyes** | Moondream (Ollama) |
24+
| **The Ears** | Faster-Whisper |
25+
| **The Voice** | pyttsx3 (SAPI5) |
26+
27+
---
28+
29+
## ⚙️ PREREQUISITES
30+
31+
1. **Ollama:** [Install Ollama](https://ollama.com/) and pull the models:
1332
```bash
1433
ollama pull moondream
1534
ollama pull llama3
1635
```
17-
2. **Python 3.10+**: Recommended.
18-
3. **Dependencies:**
36+
2. **Python 3.10+** (3.13 tested).
37+
3. **FFmpeg:** Required for audio processing.
38+
4. **Dependencies:**
1939
```bash
2040
pip install -r requirements.txt
2141
```
22-
*Note: On Windows, you might need to install `PyAudio` via a wheel if `pip install pyaudio` fails.*
42+
*Note: On Windows, ensure you have C++ Build Tools installed for PyAudio.*
2343

24-
## How to Run
25-
1. Start Ollama: `ollama serve` (if not already running).
26-
2. Run GamerCat:
27-
```bash
28-
python src/gamer_cat.py
29-
```
44+
---
45+
46+
## 👾 HOW TO RUN
47+
48+
1. **Start Ollama:** `ollama serve` (if not already running).
49+
2. **Launch Gamer-Cat:**
50+
```bash
51+
python src/gamer_cat.py
52+
```
53+
54+
---
55+
56+
## 📂 FILE STRUCTURE
3057

31-
## Files
3258
- `src/screen_capture.py`: Screen capture logic (Pillow).
3359
- `src/vision_engine.py`: Image-to-text via Ollama.
3460
- `src/voice_engine.py`: Whisper STT & pyttsx3 TTS.
3561
- `src/mcp_server.py`: MCP Server implementation.
3662
- `src/gamer_cat.py`: Main orchestration loop.
63+
64+
---
65+
66+
**Built with ❤️ for gamers who play too much.**

‎index.html‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8">
5+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
6+
<title>Gamer-Cat | Your AI Buddy</title>
7+
<style>
8+
body { background: #0b0e14; color: #fff; font-family: 'Inter', system-ui, -apple-system, sans-serif; display: flex; align-items: center; justify-content: center; height: 100vh; margin: 0; text-align: center; overflow: hidden; }
9+
.hero { max-width: 800px; padding: 3rem; border: 1px solid #30363d; border-radius: 16px; background: #161b22; box-shadow: 0 20px 50px rgba(0,0,0,0.8); position: relative; }
10+
.hero::before { content: ""; position: absolute; top: -2px; left: -2px; right: -2px; bottom: -2px; background: linear-gradient(45deg, #58a6ff, #bc8cff, #58a6ff); border-radius: 18px; z-index: -1; opacity: 0.3; }
11+
h1 { color: #58a6ff; font-size: 3.5rem; margin: 0; letter-spacing: -2px; }
12+
p { color: #8b949e; font-size: 1.4rem; line-height: 1.6; margin: 1.5rem 0; }
13+
.badge { background: #238636; padding: 6px 16px; border-radius: 20px; font-size: 0.9rem; font-weight: bold; text-transform: uppercase; letter-spacing: 1px; }
14+
.cta { margin-top: 2.5rem; display: inline-block; padding: 14px 32px; background: #238636; border: none; border-radius: 8px; color: #fff; text-decoration: none; font-weight: bold; font-size: 1.1rem; transition: transform 0.2s, background 0.2s; }
15+
.cta:hover { transform: translateY(-2px); background: #2ea043; }
16+
.specs { margin-top: 2rem; color: #484f58; font-family: monospace; font-size: 0.9rem; }
17+
</style>
18+
</head>
19+
<body>
20+
<div class="hero">
21+
<span class="badge">OFFLINE COMPANION</span>
22+
<h1>😼 GAMER-CAT</h1>
23+
<p>The first local-first AI buddy that <strong>sees</strong> your screen and talks back in real-time. Built to run alongside your favorite games.</p>
24+
<a href="https://github.com/adityachauhan0/gamer-cat" class="cta">VIEW SOURCE CODE</a>
25+
<div class="specs">SYSTEM REQS: 16GB RAM | LLAMA-3 | MOONDREAM-2</div>
26+
</div>
27+
</body>
28+
</html>

‎technical_plan.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Technical Plan: Gamer-Cat (Local Offline Companion)
2+
3+
## Objective
4+
Create a low-latency, offline AI companion that perceives the user's screen and interacts via voice, staying within 16GB RAM.
5+
6+
## 1. Resource Budget (Target: <16GB RAM)
7+
- **OS/System:** 2.0 GB
8+
- **Vision (Moondream2):** 2.0 GB (Smallest capable VLM for screen descriptions)
9+
- **Brain (Llama-3-8B-Q4_K_M):** 5.5 GB (The core personality)
10+
- **STT (Faster-Whisper-Small):** 1.0 GB (Transcribes user speech)
11+
- **TTS (Piper):** 0.5 GB (High-speed, low-resource voice synthesis)
12+
- **Buffer/Other:** 1.0 GB
13+
- **TOTAL:** ~12.0 GB (Safe margin of 4GB for the game/activity)
14+
15+
## 2. Component Breakdown
16+
17+
### A. Vision (The Eyes)
18+
- **Frequency:** Every 10 seconds.
19+
- **Tool:** `pyautogui` for screenshots + Moondream2.
20+
- **Output:** "The player is currently in a dark forest, fighting a giant spider with a sword."
21+
22+
### B. Brain (The Personality)
23+
- **System Prompt:** "You are a chill gaming buddy. You get text updates about what's on the screen. Don't be a robot; be a friend."
24+
- **Context Management:** We will keep a sliding window of the last 5-10 screen descriptions to maintain continuity.
25+
26+
### C. Voice (The Ears & Mouth)
27+
- **STT:** Faster-Whisper (runs on CPU/GPU) to detect when the user speaks.
28+
- **TTS:** Piper. It uses raw PCM audio and is significantly faster than Coqui or others for real-time feel.
29+
30+
## 3. Potential Bottlenecks
31+
- **GPU VRAM vs System RAM:** If running on a laptop with integrated graphics, the 16GB is shared. We must use 4-bit quantization (GGUF) strictly.
32+
- **CPU Spikes:** Running VLM inference every 10s might cause frame drops in heavy games. We may need to "nice" the process priority.
33+
34+
## 4. Next Steps
35+
1. Initialize a Python environment.
36+
2. Prototype the "Screen -> Moondream -> Text" pipeline.
37+
3. Test latency of STT + LLM + TTS chain.

0 commit comments

Comments
 (0)