Skip to content
BranchesTags

Repository files navigation

MyCelium

MyCelium Banner

Python React LangGraph ChromaDB

A knowledge graph-powered AI chatbot with persistent long-term memory.

MyCelium combines the power of LLMs with a knowledge graph to create an AI assistant that truly remembers. It extracts entities and relationships from conversations, stores them in a graph database, and uses semantic search to retrieve relevant context for every interaction.

Features

  • Knowledge Graph Memory - Automatically extracts entities and relationships from conversations using NetworkX and ChromaDB
  • Library (RAG Document Store) - Fast two-level knowledge base: chunk and embed documents for instant semantic search, then optionally promote results into the knowledge graph
  • Article Generation - Generate long-form articles from your knowledge graph with automatic outlining, section-by-section writing, and evolutionary optimization (genetic algorithm)
  • Multi-Source Ingestion - Import knowledge from Wikipedia, Project Gutenberg, bioRxiv, arXiv, web pages, PDFs, and more — with optional auto-translation
  • Interactive Graph Visualization - Explore and chat with your knowledge graph directly with a force-directed 2D visualization
  • Multiple Workspaces - Create isolated memory contexts for different topics or projects, and expose workspaces as callable tools for cross-workspace queries
  • Notes System - Create, search, and manage notes with semantic search (RAG)
  • Concept Clustering - Automatically identify themes and topics in your knowledge graph
  • Hot Topics & Connectors - Discover the most important and bridging nodes in your graph
  • Voice Calls - Live voice conversations with the AI via WebSocket (ASR + LLM + TTS streaming pipeline)
  • Text-to-Speech - Built-in streaming TTS powered by VibeVoice with 19 voices (optional, GPU-accelerated)
  • TTS Reader - Paste text or load a URL and have it read aloud paragraph by paragraph
  • Terminal - Workspace-aware tmux terminal in the browser with LLM-assisted command execution
  • Reddit Integration - Search and browse Reddit discussions
  • Emotional State System - The bot has emotions that evolve based on interactions
  • Growth Engine - Let the bot explore and expand its knowledge autonomously
  • theWay (Skills) - Teach the AI reusable skills with automatic discovery — skills are surfaced or injected based on message relevance
  • Chrome Extension - Browser sidebar for quick chat and one-click page ingestion

Usage Guide

Creating a Workspace

  1. Open the application in your browser
  2. Click the + button in the sidebar
  3. Enter a name for your workspace
  4. Click Create

Ingesting Content

You can add knowledge to your graph in several ways:

  • Chat naturally - The bot automatically extracts entities and relationships from conversations
  • Wikipedia - Ask the bot to ingest a Wikipedia article: "Ingest the Wikipedia article about quantum computing"
  • Web pages - Ask the bot to read a URL: "Summarize https://example.com/article" (if the page is too long, it will automatically ingest it into the graph first, then summarize)
  • Project Gutenberg - Ingest free ebooks: "Find and ingest 'Pride and Prejudice' from Gutenberg"
  • File upload - Upload PDFs, TXT, or Markdown files directly from the sidebar
  • arXiv - Search and ingest research papers: "Search arXiv for transformer architectures"

Ingestion targets: Use the sidebar toggle to choose where content goes:

  • Graph - Full entity extraction with LLM (default)
  • Library - Fast chunk-and-embed only (no LLM, instant)
  • Both - Ingest to graph and library simultaneously

Auto-translation: Set a target language in the sidebar to automatically translate all extracted entities and descriptions during ingestion.

Ingesting Content

Exploring the Graph

  • Click the Graph tab to see your knowledge graph visualization
  • Click on nodes to see their details and connections
  • Ask the bot to explore connections: "Traverse the graph from 'Machine Learning' and show me related concepts"

Graph Visualization

Using Notes

  • Click the Notes tab to manage your notes
  • The bot can create notes for you: "Create a note about our discussion on machine learning"
  • Search notes semantically: "Search my notes for neural networks"
  • Add and edit notes directly in the Notes tab, or ask the bot to help

Creating Notes Viewing Notes

Concepts & Hot Topics

  • Concepts - Automatically clusters your graph into themes
  • Hot Topics - Shows the most connected nodes in your graph
  • Connectors - Identifies bridge nodes that connect different topics

Concepts

Connectors

Growth Engine

  • Automated Research - The bot proactively searches for information to fill gaps in its knowledge
  • Recursive Expansion - You set the depth and breadth, and the bot explores related concepts

Growth Engine

Personality Sliders

  • Dynamic Emotions - Adjust the bot's emotional baseline (Happiness, Trust, Anger)
  • Motive System - The bot's responses change based on its current emotional state

Personality Sliders

Persona Configuration

  • Magic Persona Generator - Describe a character, and the AI will hallucinate a backstory, system prompt, and initial memories for it.
  • Roleplay Mode - The bot adopts the generated persona, including specific knowledge and emotional traits.

Skills (theWay)

Teach the AI reusable skills that it can look up and follow when prompted:

  1. Create a Skill - Go to the theWay tab (compass icon), click +, and define:

    • Title: e.g., "Email Writing"
    • Summary: Brief description for search (e.g., "Professional email format")
    • Instructions: Detailed step-by-step instructions the AI will follow

  2. Use a Skill - In chat, ask the AI to apply your skill:

    • "Use your email writing skill to write an email about the project update"
    • "Apply your code review skill to analyze this function"
    • "Follow your data analysis skill to interpret these numbers"

The AI will search for matching skills and follow the instructions you provided.

Automatic skill discovery: Skills can also be surfaced automatically based on message relevance. Configure thresholds in workspace settings:

  • Surface threshold (default 0.50) — shows skill title + summary as a hint
  • Auto-inject threshold (default 0.85) — injects the full skill instructions into context

Library (RAG Document Store)

The Library is a fast, two-level knowledge base for storing and searching documents without the overhead of full graph extraction:

  1. Add documents — Go to the Library tab, then upload files (PDF, TXT, MD) or ingest URLs. Documents are chunked and embedded instantly.
  2. Search — Semantic search returns the most relevant chunks across all your library sources.
  3. Promote to Graph — Select search results and extract entities/relationships into the knowledge graph with one click.
  4. Browse sources — View all ingested sources, read their chunks, or delete them.

The bot can also interact with the library via chat:

  • "Search the library for machine learning basics"
  • "Add this Wikipedia article to the library"
  • "Promote library results about neural networks to the graph"

Article Generation

Generate long-form articles from your knowledge graph:

  1. Go to the Notes tab and click the article generation button
  2. Enter a topic and choose a mode:
    • Existing — uses only your current knowledge graph
    • Research — searches external sources (Wikipedia, arXiv, web) to supplement
  3. The system clusters related graph nodes, creates a hierarchical outline, and writes each section using relevant graph context
  4. The finished article is saved as a note

Evolutionary optimization: After generating an article, use the Evolve feature to iteratively improve it using a genetic algorithm. The system scores articles on grounding, consistency, coherence, and completeness, then generates variants until the quality converges.

Voice Calls

Have a live voice conversation with the AI:

  1. Go to the Call tab (phone icon)
  2. Click to start a call — your microphone streams audio via WebSocket
  3. Speech is transcribed (ASR via Deepgram), sent to the LLM, and the response is streamed back as audio
  4. The full transcript is saved to your thread history

TTS Reader

Read text or web content aloud paragraph by paragraph:

  1. Go to the Reader tab (audio icon)
  2. Paste text or load a URL
  3. Playback proceeds paragraph by paragraph with controls to skip, pause, or adjust voice

Terminal

Access a workspace-aware terminal directly in the browser:

  1. Go to the Terminal tab
  2. An interactive tmux session opens, persistent across page refreshes
  3. Use the terminal chat to describe tasks in natural language — the LLM translates them to shell commands and executes them

Chrome Extension

Access Mycelium directly from your browser with the Chrome extension:

  • Sidebar Chat - Click the extension icon to open a chat panel in your browser sidebar
  • One-Click Ingestion - Ingest the current page into your knowledge graph with a single click
  • Page Context - Toggle "Include page" to send page content with your messages (auto-ingests if content exceeds limit)
  • Workspace & Thread Switching - Quickly switch between workspaces and threads without leaving your browser

Installation

1. Clone the Repository

git clone https://github.com/yourusername/graph-llm.git
cd graph-llm

2. Choose your Setup Method

Option 1: Running with Docker (Recommended)

  1. Prerequisites: Ensure you have Docker Desktop installed.

  2. Configure paths (Optional): Create a .env file to customize where your data is stored:

    cp .env.example .env
# Edit .env to set custom paths for memory data, logs, and config

    Example .env configuration:

    # Store memory data in a custom location
MEMORY_DATA_PATH=/path/to/your/memory-data

# Store logs in a custom location
LOGS_PATH=/path/to/your/logs

# Store config in a custom location (directory containing llm_config.json)
CONFIG_PATH=/path/to/your/config

    If you don't create a .env file, the default paths (./backend/memory_data, ./backend/logs, ./backend) will be used.

  3. Ensure config file exists: Before first run, make sure llm_config.json exists:

    cd backend
cp llm_config.example.json llm_config.json
# Edit llm_config.json with your settings
cd ..

  4. Run the application:

    docker-compose up --build

  5. Access the App:

  6. (Optional) Enable Text-to-Speech: See the Text-to-Speech section below.

Note: Config changes made through the UI or by editing llm_config.json will persist across container restarts.

Option 2: Manual Installation

Prerequisites:

  • Python 3.10+
  • Node.js 18+

Backend Setup (Manual)

cd backend

# Create virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Frontend Setup (Manual)

Ensure you have Node.js and npm installed:

  • Mac (Homebrew): brew install node
  • Windows: Download from nodejs.org
  • Linux (Ubuntu/Debian): sudo apt install nodejs npm

Then run the frontend:

cd frontend

# Install dependencies
npm install

Configure LLM Provider (Manual)

Copy the example config and customize it:

cd backend
cp llm_config.example.json llm_config.json
# Edit llm_config.json with your settings

See the Configuration section for provider-specific examples.

Chrome Extension Setup

  1. Open Chrome and navigate to chrome://extensions/
  2. Enable Developer mode (toggle in top right)
  3. Click Load unpacked
  4. Select the extension/ directory from this repository
  5. The Mycelium icon will appear in your browser toolbar
  6. Click the icon to open the sidebar chat panel

Note: The extension connects to http://localhost:8000 by default. Make sure the backend is running.

Configuration

Tip: You can configure these settings through the Global Settings menu in the application UI instead of manually editing the JSON file.

LLM Provider Configuration

Edit backend/llm_config.json with your preferred provider:

Data Storage Configuration

You can customize where MyCelium stores its data by creating a .env file in the project root:

cp .env.example .env

Then edit the .env file to set custom paths:

  • MEMORY_DATA_PATH: Where knowledge graphs, embeddings, and notes are stored (default: ./backend/memory_data)
  • LOGS_PATH: Where application logs are stored (default: ./backend/logs)
  • CONFIG_PATH: Directory containing llm_config.json (default: ./backend)

This is particularly useful when running with Docker, as it allows you to persist data outside the project directory or on different drives.

Ollama

{
  "provider": "ollama",
  "embedding_provider": "ollama",
  "ollama_base_url": "http://localhost:11434",
  "ollama_chat_model": "llama3.2",
  "ollama_embedding_model": "nomic-embed-text",
  "temperature": 0.7,
  "tts_enabled": false
}

Setup Ollama:

# Install Ollama from https://ollama.ai/
# Pull required models
ollama pull llama3.2
ollama pull nomic-embed-text

⚠️ Running with Docker? When running MyCelium in Docker while Ollama runs on your host machine, use host.docker.internal instead of localhost:

"ollama_base_url": "http://host.docker.internal:11434"

This is because localhost inside Docker refers to the container itself, not your host machine.

LM Studio

{
  "provider": "lmstudio",
  "embedding_provider": "lmstudio",
  "chat_base_url": "http://localhost:1234/v1",
  "chat_api_key": "lm-studio",
  "chat_model": "your-model-name",
  "embedding_base_url": "http://localhost:1234/v1",
  "embedding_api_key": "lm-studio",
  "embedding_model": "text-embedding-nomic-embed-text-v1.5",
  "temperature": 0.7,
  "tts_enabled": false
}

⚠️ Running with Docker? Use host.docker.internal instead of localhost:

"chat_base_url": "http://host.docker.internal:1234/v1",
"embedding_base_url": "http://host.docker.internal:1234/v1"

OpenAI

{
  "provider": "openai",
  "embedding_provider": "openai",
  "chat_base_url": "https://api.openai.com/v1",
  "chat_api_key": "sk-your-api-key",
  "chat_model": "gpt-4o",
  "embedding_base_url": "https://api.openai.com/v1",
  "embedding_api_key": "sk-your-api-key",
  "embedding_model": "text-embedding-3-small",
  "temperature": 0.7,
  "tts_enabled": false
}

Mixed Providers

You can use different providers for chat (LLM) and embeddings. For example, use OpenAI for chat and Ollama for embeddings:

{
  "provider": "openai",
  "embedding_provider": "ollama",
  "chat_base_url": "https://api.openai.com/v1",
  "chat_api_key": "sk-your-api-key",
  "chat_model": "gpt-4o",
  "ollama_base_url": "http://host.docker.internal:11434",
  "ollama_embedding_model": "nomic-embed-text",
  "temperature": 0.7,
  "tts_enabled": false
}

Text-to-Speech (Optional)

MyCelium includes a self-hosted streaming TTS service powered by VibeVoice-Realtime-0.5B from Microsoft. It runs as a separate Docker container with GPU acceleration and streams audio in real time.

Requirements

  • NVIDIA GPU with at least ~2GB VRAM
  • NVIDIA Container Toolkit (installation guide)
  • Docker with GPU support

Starting with TTS

The TTS service uses a Docker Compose profile so it won't start by default:

# Start everything including TTS
docker-compose --profile tts up --build

# Or start just the TTS service alongside an already-running stack
docker-compose --profile tts up tts

On the first run, the container will:

  1. Download the VibeVoice-Realtime-0.5B model weights (~1GB) from HuggingFace
  2. Download 19 voice preset files from GitHub

Both are cached in Docker volumes (tts_cache and tts_voices) so subsequent starts are fast.

Configuration

Once the TTS service is running, enable it in the UI:

  1. Open Settings (gear icon)
  2. Go to the Audio tab
  3. Toggle Enable TTS on
  4. The defaults should work out of the box:
    • Base URL: http://tts:8100/v1
    • Model: VibeVoice-Realtime-0.5B
    • Voice: en-Emma_woman
  5. Click Test Connection to verify — on success, the voice field becomes a dropdown with all available voices

Or set it directly in llm_config.json:

{
  "tts_base_url": "http://tts:8100/v1",
  "tts_model": "VibeVoice-Realtime-0.5B",
  "tts_voice": "en-Emma_woman",
  "tts_enabled": true
}

Custom Volume Paths

You can customize where TTS data is stored via .env:

TTS_CACHE_PATH=/path/to/hf-model-cache
TTS_VOICES_PATH=/path/to/voice-presets

Available Voices

19 voice presets are included, covering English, Chinese, French, German, Hindi, and Portuguese:

Voice Language Gender
en-Emma_woman English Female
en-Jess_woman English Female
en-River_woman English Female
en-Alice_woman English Female
en-Leo_man English Male
en-Dan_man English Male
zh-Xiaoxiao_woman Chinese Female
zh-Yunjian_man Chinese Male
fr-Denise_woman French Female
fr-Henri_man French Male
de-Amala_woman German Female
de-Conrad_man German Male
hi-Swara_woman Hindi Female
hi-Madhur_man Hindi Male
pt-Francisca_woman Portuguese Female
pt-Antonio_man Portuguese Male

Note: Emojis and special symbols are automatically stripped from text before sending to TTS to prevent garbled output.

Running Locally

If you prefer to run without Docker:

Start the Backend

cd backend
source venv/bin/activate  # If using virtual environment
uvicorn app.main:app --reload --port 8000

The API will be available at http://localhost:8000

Start the Frontend

cd frontend
npm run dev

The UI will be available at http://localhost:5173

API Endpoints

Core

Endpoint Method Description
/chat POST Send a message to the chatbot
/graph/{workspace_id} GET Get graph data for visualization
/debug/graph_check/{workspace_id} GET Debug graph data

Workspaces

Endpoint Method Description
/workspaces GET List all workspaces
/workspaces POST Create a new workspace
/workspaces/{id} DELETE Delete a workspace
/workspaces/{id}/rename POST Rename a workspace
/workspaces/{id}/stats GET Get workspace statistics
/workspaces/{id}/settings GET/POST Get or update workspace settings
/workspaces/{id}/emotions GET/POST Get or update emotional state
/workspaces/{id}/generate_persona POST Generate a persona for the workspace
/workspaces/available_tools GET List all available tools
/workspaces/exposed_tools GET List workspace tools exposed to other workspaces
/workspaces/{id}/generate_tool_description POST Generate a tool description for a workspace

Ingestion

Endpoint Method Description
/workspaces/{id}/upload POST Upload a file for ingestion
/workspaces/{id}/ingest-url POST Ingest content from a URL
/workspaces/{id}/ingest_status GET Get current ingestion status
/workspaces/{id}/ingest/stop POST Stop current ingestion

Threads

Endpoint Method Description
/threads/{workspace_id} GET List threads in a workspace
/threads POST Create a new thread
/threads/with_messages POST Create a thread with initial messages
/threads/{workspace_id}/{thread_id} DELETE Delete a thread
/threads/{workspace_id}/{thread_id}/history GET Get thread message history
/threads/{workspace_id}/{thread_id}/chat POST Send a message in a thread

Notes

Endpoint Method Description
/workspaces/{id}/notes GET List notes in a workspace
/workspaces/{id}/notes POST Create a new note
/workspaces/{id}/notes/{note_id} GET Get a specific note
/workspaces/{id}/notes/{note_id} PUT Update a note
/workspaces/{id}/notes/{note_id} DELETE Delete a note

Skills (theWay)

Endpoint Method Description
/workspaces/{id}/skills GET List skills in a workspace
/workspaces/{id}/skills POST Create a new skill
/workspaces/{id}/skills/{skill_id} GET Get a specific skill
/workspaces/{id}/skills/{skill_id} PUT Update a skill
/workspaces/{id}/skills/{skill_id} DELETE Delete a skill

Graph & Concepts

Endpoint Method Description
/concepts/{workspace_id} GET Get concept clusters
/concepts/generate POST Generate new concepts
/hot_topics/{workspace_id} GET Get highly connected nodes
/connectors/{workspace_id} GET Get connector/bridge nodes
/graph/{workspace_id}/chat POST Chat with graph context (streaming, returns retrieved nodes/edges)
/graph/{workspace_id}/node/{node_id} GET Get node details with neighbors
/workspaces/{id}/graph/export GET Export graph data
/workspaces/{id}/graph/import POST Import graph data

Library

Endpoint Method Description
/library/{workspace_id}/search POST Semantic search over library chunks
/library/{workspace_id}/sources GET List all library sources with chunk counts
/library/{workspace_id}/source/{source_id}/chunks GET Get all chunks for a source
/library/{workspace_id}/source/{source_id} DELETE Delete a source and its chunks
/library/{workspace_id}/stats GET Get library statistics
/library/{workspace_id}/upload POST Upload a file to the library
/library/{workspace_id}/ingest-url POST Ingest a URL into the library
/library/{workspace_id}/promote POST Search library and extract entities into the graph

Articles

Endpoint Method Description
/articles/generate POST Generate a long-form article (streaming NDJSON)
/articles/evolve POST Evolve an article note with genetic optimization (streaming NDJSON)

Growth Engine

Endpoint Method Description
/workspaces/{id}/contemplate POST Start contemplation/growth
/workspaces/{id}/contemplate/stop POST Stop contemplation
/workspaces/{id}/knowledge_gaps GET Get identified knowledge gaps

Scripts

Endpoint Method Description
/workspaces/{id}/scripts/generate POST Generate a script
/workspaces/{id}/scripts GET List scripts
/workspaces/{id}/scripts/{script_id} DELETE Delete a script

Audio (TTS)

Endpoint Method Description
/audio/speech POST Generate speech from text (streaming WAV)
/audio/stream GET Stream audio via query parameter
/audio/test GET Test TTS connectivity and list voices

Voice Call

Endpoint Method Description
/call/ws WebSocket Live voice call (mic audio in, TTS audio out)

Terminal

Endpoint Method Description
/terminal/{workspace_id}/ws WebSocket Interactive terminal session via tmux
/terminal/{workspace_id}/chat POST Natural language to shell commands (streaming)

System

Endpoint Method Description
/system/config GET/POST Get or update LLM configuration
/system/models GET List available models
/system/mcp/test POST Test MCP connection
/system/mcp/tools GET List MCP tools
/system/mcp/refresh POST Refresh MCP tools
/system/mcp/status GET Get MCP status

Tools Available to the Agent

The LangGraph agent has access to various tools:

  • Notes: create_note, read_note, update_note, delete_note, search_notes, list_notes
  • Graph: add_graph_node, update_graph_node, delete_graph_node, add_graph_edge, update_graph_edge, delete_graph_edge, search_graph_nodes, traverse_graph_node
  • Library: search_library, promote_library_search, add_gutenberg_book_to_library, add_wiki_article_to_library
  • Search: duckduckgo_search, search_images, search_concepts
  • Web: visit_page, ingest_web_page
  • Wikipedia: search_wikipedia, ingest_wikipedia_page
  • Books: search_books, search_gutenberg_books, ingest_gutenberg_book
  • Reddit: search_reddit, browse_subreddit, read_reddit_thread, get_reddit_user
  • Research: search_biorxiv, read_biorxiv_abstract, search_arxiv, read_arxiv_abstract, ingest_arxiv_paper
  • Learning: generate_lesson
  • Skills (theWay): lookup_skill, create_skill
  • Terminal: execute_terminal_command
  • Cross-workspace: consult_workspace (query other workspaces exposed as tools)

About

The living knowledge network

Resources

Readme
MIT license
Activity
Custom properties

Stars

12 stars

Watchers

0 watching

Forks

2 forks
Report repository

Releases

Packages

Contributors

Languages