Skip to content

Commit 52cee29

Browse files
authored
feat: MiniMax image OCR + registry-driven multimodal provider support (#423)
Adds MiniMax as an AI provider (text enhancement + vision OCR, global_en/cn_zh regions, openai/anthropic wire protocols) and generalizes AgentClient multimodal support while landing it: - API_PROVIDERS registry now declares each provider's wire protocol, supports_images capability, and optional regional endpoint map; _call_api branches on protocol, not provider names. - AgentClient.call_with_image() serves all image-capable providers (Anthropic, OpenAI, Gemini, MiniMax); video_visual frame OCR routes through it via a table-driven provider dispatch with registry-based auto-detection (SKILL_SEEKER_VISION_PROVIDER). - MiniMax adaptor resolves MINIMAX_API_REGION for enhancement requests (China-issued keys no longer 401 against the global endpoint). - Closes the UNIFICATION_PLAN "AgentClient multimodal support" deferred item; CLAUDE.md updated accordingly.
1 parent 09410e2 commit 52cee29

14 files changed

Lines changed: 766 additions & 120 deletions

.env.example

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,14 @@ GOOGLE_API_KEY=
1616
# Get your key from: https://platform.openai.com/api-keys
1717
OPENAI_API_KEY=
1818

19+
# MiniMax API (Optional)
20+
# Required for MiniMax enhancement and vision OCR
21+
MINIMAX_API_KEY=
22+
MINIMAX_API_REGION=global_en
23+
MINIMAX_API_PROTOCOL=openai
24+
MINIMAX_VISION_MODEL=MiniMax-M3
25+
# SKILL_SEEKER_VISION_PROVIDER=minimax
26+
1927
# GitHub Token (Optional, but recommended)
2028
# Increases rate limits from 60/hour to 5000/hour
2129
# Create token at: https://github.com/settings/tokens

CLAUDE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -184,9 +184,9 @@ Local codebase analysis features, all opt-out (`--skip-*` flags):
184184

185185
### Enhancement (AgentClient is the single AI transport)
186186

187-
Every text-based AI call goes through `AgentClient` (`src/skill_seekers/cli/agent_client.py`): central truncation gate, timeout policy, error classification. `API_PROVIDERS` (provider registry) and `AGENT_PRESETS` (local-agent command templates) live ONLY there. Adaptors declare provider/endpoint/model/prompt and route through `SkillAdaptor._enhance_skill_md_via_client` (atomic save with backup). `video_visual` frame classification is the documented multimodal exception (AgentClient is text-only).
187+
Every AI call goes through `AgentClient` (`src/skill_seekers/cli/agent_client.py`): central truncation gate, timeout policy, error classification. `API_PROVIDERS` (provider registry) and `AGENT_PRESETS` (local-agent command templates) live ONLY there. Each `API_PROVIDERS` entry declares its wire `protocol` (`anthropic`/`openai`/`google`) and `supports_images` capability — `_call_api` branches on the resolved protocol, NOT the provider name, so an OpenAI/Anthropic-compatible provider needs no new branch. Adaptors declare provider/endpoint/model/prompt and route through `SkillAdaptor._enhance_skill_md_via_client` (atomic save with backup). Multimodal image input goes through `AgentClient.call_with_image()` (used by `video_visual` frame OCR across all image-capable providers); it no longer bypasses AgentClient with a direct SDK call.
188188

189-
- **API mode** (if API key set): Anthropic, Google Gemini, OpenAI, Moonshot/Kimi — detected in registry order; `SKILL_SEEKER_PROVIDER` forces one. Models: `SKILL_SEEKER_MODEL` (global) or `ANTHROPIC_MODEL`/`GOOGLE_MODEL`/`OPENAI_MODEL`/`MOONSHOT_MODEL`; `ANTHROPIC_BASE_URL` for compatible endpoints.
189+
- **API mode** (if API key set): Anthropic, Google Gemini, OpenAI, Moonshot/Kimi, MiniMax — detected in registry order; `SKILL_SEEKER_PROVIDER` forces one. Models: `SKILL_SEEKER_MODEL` (global) or `ANTHROPIC_MODEL`/`GOOGLE_MODEL`/`OPENAI_MODEL`/`MOONSHOT_MODEL`/`MINIMAX_MODEL`; `ANTHROPIC_BASE_URL` for compatible endpoints. MiniMax adds `MINIMAX_API_REGION` (`global_en`/`cn_zh`) and `MINIMAX_API_PROTOCOL` (`openai`/`anthropic`). Vision OCR provider: `SKILL_SEEKER_VISION_PROVIDER` (`auto` picks the first image-capable provider with a key).
190190
- **LOCAL mode** (fallback): Claude Code, Kimi Code, Codex, Copilot, OpenCode, custom agents — command built by `build_local_agent_command()`.
191191
- Control: `--enhance-level 0` (off) / `1` (SKILL.md only) / `2` (default, balanced) / `3` (full)
192192
- Agent selection: `--agent claude|codex|copilot|opencode|kimi|custom`

docs/UNIFICATION_PLAN.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,9 @@ AgentClient is now the single AI transport for every text-based call:
106106
sys.exit removed); unified_scraper Phase-6 API save is atomic;
107107
doc_scraper's path already routed via adaptors.
108108
- video_scraper's reference cleaner goes through AgentClient.
109-
video_visual's frame classification is the DOCUMENTED EXCEPTION
110-
(multimodal; AgentClient is text-only today).
109+
video_visual's frame OCR goes through AgentClient.call_with_image()
110+
(multimodal support landed with the MiniMax image work; the former
111+
text-only exception is gone).
111112
- `build_local_agent_command()` in agent_client is the single
112113
preset-template/permissions-flag handler; LocalSkillEnhancer delegates.
113114

@@ -119,7 +120,9 @@ contracts — blind merges would silently change AI quality):
119120
- LocalSkillEnhancer terminal/background/daemon orchestration is kept; the
120121
remaining overlap with AgentClient._call_local is the prompt-file +
121122
subprocess loop (small).
122-
- AgentClient multimodal support (would absorb video_visual).
123+
- ~~AgentClient multimodal support (would absorb video_visual).~~ DONE —
124+
`AgentClient.call_with_image()` serves all image-capable providers;
125+
`_call_api` branches on a registry `protocol`/`supports_images`.
123126

124127
## Phase 3 — Enhancement consolidation (medium-large)
125128

@@ -277,7 +280,7 @@ quality.
277280
- Per-request ExecutionContext initialization at the MCP boundary —
278281
lower urgency since the converters' raw-config fallback (fixed earlier)
279282
plus the contextvars override (5b) cover the known divergences.
280-
- AgentClient multimodal support (absorbs video_visual).
283+
- ~~AgentClient multimodal support (absorbs video_visual).~~ DONE (see Phase 3).
281284

282285
## Phase 5 — Platform unification: config, dispatch, MCP (medium)
283286

docs/integrations/MINIMAX_INTEGRATION.md

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,23 @@ pip install skill-seekers[all-llms]
4040
### 3. Configure Environment
4141

4242
```bash
43-
export MINIMAX_API_KEY=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...
43+
export MINIMAX_API_KEY=your-api-key
44+
export MINIMAX_API_REGION=global_en
45+
export MINIMAX_API_PROTOCOL=openai
4446
```
4547

4648
Add to your `~/.bashrc`, `~/.zshrc`, or `.env` file for persistence.
4749

50+
Choose the region and protocol that match your account:
51+
52+
| Region | OpenAI-compatible base URL | Anthropic-compatible base URL |
53+
|--------|----------------------------|-------------------------------|
54+
| `global_en` | `https://api.minimax.io/v1` | `https://api.minimax.io/anthropic` |
55+
| `cn_zh` | `https://api.minimaxi.com/v1` | `https://api.minimaxi.com/anthropic` |
56+
57+
Set `MINIMAX_API_PROTOCOL` to `openai` or `anthropic`. Anthropic-compatible base
58+
URLs already end in `/anthropic`; do not append `/v1` to the configured base URL.
59+
4860
---
4961

5062
## Complete Workflow
@@ -182,6 +194,24 @@ response = client.chat.completions.create(
182194
)
183195
```
184196

197+
### Vision OCR for Video Frames
198+
199+
MiniMax-M3 accepts image input and can power the existing low-confidence OCR
200+
fallback used during visual video extraction. Select MiniMax as the vision
201+
provider, then run the normal video workflow:
202+
203+
```bash
204+
export SKILL_SEEKER_VISION_PROVIDER=minimax
205+
export MINIMAX_API_REGION=global_en
206+
export MINIMAX_API_PROTOCOL=openai
207+
208+
skill-seekers create --video-file demo.mp4 --visual --vision-ocr
209+
```
210+
211+
Use `MINIMAX_API_REGION=cn_zh` for the China endpoint. Both `openai` and
212+
`anthropic` protocols support the same tool path. `MINIMAX_VISION_MODEL`
213+
defaults to `MiniMax-M3`; MiniMax-M2.7 remains available for text workflows.
214+
185215
---
186216

187217
## API Reference
@@ -213,6 +243,11 @@ success = adaptor.enhance(skill_dir, api_key)
213243
| Variable | Description | Required |
214244
|----------|-------------|----------|
215245
| `MINIMAX_API_KEY` | Your MiniMax API key (JWT format) | Yes |
246+
| `MINIMAX_API_REGION` | API region: `global_en` or `cn_zh` | No; defaults to `global_en` |
247+
| `MINIMAX_API_PROTOCOL` | API protocol: `openai` or `anthropic` | No; defaults to `openai` |
248+
| `MINIMAX_MODEL` | Model override for text enhancement | No |
249+
| `MINIMAX_VISION_MODEL` | Model override for image OCR | No; defaults to `MiniMax-M3` |
250+
| `SKILL_SEEKER_VISION_PROVIDER` | Set to `minimax` to use MiniMax for vision OCR | No |
216251

217252
---
218253

src/skill_seekers/cli/adaptors/minimax.py

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,26 @@
88
"""
99

1010
from .openai_compatible import OpenAICompatibleAdaptor
11+
from skill_seekers.cli.minimax_config import (
12+
MINIMAX_DEFAULT_MODEL,
13+
MINIMAX_ENDPOINTS,
14+
resolve_minimax_endpoint,
15+
)
1116

1217

1318
class MiniMaxAdaptor(OpenAICompatibleAdaptor):
1419
"""MiniMax AI platform adaptor."""
1520

1621
PLATFORM = "minimax"
1722
PLATFORM_NAME = "MiniMax AI"
18-
DEFAULT_API_ENDPOINT = "https://api.minimax.io/v1"
19-
DEFAULT_MODEL = "MiniMax-M3"
23+
# Static fallback (global). Actual requests resolve region from
24+
# MINIMAX_API_REGION via _api_base_url() so China-issued keys reach
25+
# api.minimaxi.com instead of 401-ing against the global endpoint.
26+
DEFAULT_API_ENDPOINT = MINIMAX_ENDPOINTS["global_en"]["openai"]
27+
DEFAULT_MODEL = MINIMAX_DEFAULT_MODEL
2028
ENV_VAR_NAME = "MINIMAX_API_KEY"
2129
PLATFORM_URL = "https://platform.minimaxi.com/"
30+
31+
def _api_base_url(self) -> str:
32+
"""Resolve the OpenAI-compatible endpoint for the configured region."""
33+
return resolve_minimax_endpoint(protocol="openai")

src/skill_seekers/cli/adaptors/openai_compatible.py

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,14 @@ def _resolve_model(self) -> str:
4040
"""Model to use: ``config['custom_model']`` if set, else DEFAULT_MODEL."""
4141
return self.config.get("custom_model") or self.DEFAULT_MODEL
4242

43+
def _api_base_url(self) -> str:
44+
"""API base URL for requests and packaged metadata.
45+
46+
Defaults to the static ``DEFAULT_API_ENDPOINT``; subclasses override for
47+
region-aware routing (e.g. MiniMax global vs. China endpoints).
48+
"""
49+
return self.DEFAULT_API_ENDPOINT
50+
4351
def format_skill_md(self, skill_dir: Path, metadata: SkillMetadata) -> str:
4452
"""
4553
Format SKILL.md as system instructions (no YAML frontmatter).
@@ -153,7 +161,7 @@ def package(
153161
"version": "1.0.0",
154162
"created_with": "skill-seekers",
155163
"model": self._resolve_model(),
156-
"api_base": self.DEFAULT_API_ENDPOINT,
164+
"api_base": self._api_base_url(),
157165
}
158166

159167
zf.writestr(f"{self.PLATFORM}_metadata.json", json.dumps(metadata, indent=2))
@@ -226,7 +234,7 @@ def upload(self, package_path: Path, api_key: str, **kwargs) -> dict[str, Any]:
226234

227235
client = OpenAI(
228236
api_key=api_key,
229-
base_url=self.DEFAULT_API_ENDPOINT,
237+
base_url=self._api_base_url(),
230238
)
231239

232240
client.chat.completions.create(
@@ -294,7 +302,7 @@ def enhance(self, skill_dir: Path, api_key: str) -> bool:
294302
skill_dir,
295303
api_key,
296304
provider="openai",
297-
base_url=self.DEFAULT_API_ENDPOINT,
305+
base_url=self._api_base_url(),
298306
model=self._resolve_model(),
299307
system=(
300308
"You are an expert technical writer creating system "

0 commit comments

Comments
 (0)