Skip to content

Commit 5abba20

Browse files
committed
docs(readme): normalize structure to AIWG gold-standard template
Restructure README to match AIWG's section ordering: centered header with badges and nav, What Is, Why This Matters (Developers/Agents/ Operators), Core Capabilities, Quick Start, Session Persistence, Daemon Mode, Screen Inspection, Bot Detection Flags, Binary Search Order, Error Handling (table format), Documentation, Contributing, Community, License, Sponsors (3-column table), Acknowledgments, Back to Top. Content unchanged in substance — same API examples, same bot-detection rationale. Rearranged and expanded for consistency with the carbonyl and carbonyl-fleet readmes.
1 parent 2482297 commit 5abba20

1 file changed

Lines changed: 196 additions & 39 deletions

File tree

README.md

Lines changed: 196 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,72 @@
1-
<table align="center">
2-
<tbody>
3-
<tr>
4-
<td>
5-
<p></p>
6-
<pre>
1+
<div align="center">
2+
3+
<pre>
74
O O
85
\ /
96
O —— Cr —— O
107
/ \
11-
O O</pre>
12-
</td>
13-
<td><h1>carbonyl-agent</h1></td>
14-
</tr>
15-
</tbody>
16-
</table>
8+
O O
9+
</pre>
10+
11+
# carbonyl-agent
12+
13+
**Python automation SDK for the Carbonyl headless browser**
14+
15+
```bash
16+
pip install carbonyl-agent
17+
carbonyl-agent install
18+
```
19+
20+
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)
21+
[![Python](https://img.shields.io/badge/python-3.11%2B-blue?style=flat-square&logo=python&logoColor=white)](pyproject.toml)
22+
[![Carbonyl M147](https://img.shields.io/badge/carbonyl-M147-green?style=flat-square)](https://git.integrolabs.net/roctinam/carbonyl)
23+
24+
[**Get Started**](#-quick-start) · [**Session API**](#-session-persistence) · [**Daemon Mode**](#-daemon-mode) · [**Bot Detection**](#-bot-detection-flags) · [**Examples**](examples/)
25+
26+
</div>
27+
28+
---
29+
30+
## What carbonyl-agent Is
31+
32+
`carbonyl-agent` is the Python automation SDK for [Carbonyl](https://git.integrolabs.net/roctinam/carbonyl) — a Chromium-based headless browser that renders into terminal text. The SDK spawns Carbonyl via PTY, parses the screen via `pyte`, and exposes a high-level API for navigation, clicking, text extraction, and session persistence. It is designed for agent-driven web interaction: scripted scraping, automated form submission, and LLM-driven browsing loops that need a real browser but not a real display.
33+
34+
Unlike Playwright or Selenium, carbonyl-agent returns **terminal text**, not a DOM. This makes it fast (no screenshot decode), cheap (no GPU, no window server), and well-suited for the context windows of LLM-driven agents.
35+
36+
---
37+
38+
## Why This Matters
39+
40+
### For Developers
41+
42+
**A real browser, cheap and scriptable.** Most automation stacks require either a full display server (Selenium + Xvfb) or a heavyweight DevTools protocol (Playwright CDP). carbonyl-agent gives you Chromium rendering through a PTY — `pip install`, call `open()`, read `page_text()`. Named sessions persist cookies across runs; daemon mode keeps a browser warm across short-lived scripts.
43+
44+
### For Agents
45+
46+
**Rendered text is the native LLM format.** An LLM consuming `page_text()` gets the page as a human would read it in a terminal — headings, lists, table rows — without DOM noise or screenshot OCR. Built-in bot-detection evasion (Firefox UA, `AutomationControlled` suppression, HTTP/2 off) means agents aren't blocked by default on Akamai/Cloudflare-protected sites.
1747

18-
Python automation SDK for the [Carbonyl](https://git.integrolabs.net/roctinam/carbonyl) headless browser.
48+
### For Operators
1949

20-
## Install
50+
**Low footprint, no window server.** Runs in a safe-mode console, over SSH, or inside a container without X11/Wayland. Binary discovery is prioritized: env var → local install → PATH → Docker opt-in. Sessions and daemon sockets live under `~/.local/share/carbonyl/` with 0600/0700 permissions.
51+
52+
---
53+
54+
## Core Capabilities
55+
56+
1. **CarbonylBrowser** — spawn Carbonyl via PTY, `open()`, `drain()`, `page_text()`, `click()`, `send_key()`, `find_text()`, `click_text()`, `mouse_path()`
57+
2. **SessionManager** — named persistent profiles, `create` / `fork` / `snapshot` / `restore`, live-session detection
58+
3. **Daemon mode** — long-running Carbonyl exposed over a Unix socket; clients reconnect without losing state
59+
4. **ScreenInspector** — coordinate-grid rendering, region annotation, crosshairs for debugging click targets
60+
5. **Bot-detection evasion** — curated `_HEADLESS_FLAGS` set at spawn (UA spoof, webdriver suppression, HTTP/1.1 fallback)
61+
6. **Verified install**`carbonyl-agent install` downloads the runtime, verifies SHA256, optional `--checksum` pinning
62+
63+
---
64+
65+
## Quick Start
66+
67+
> **Prerequisites:** Python 3.11+. Linux (x86_64, aarch64) or macOS.
68+
69+
### Install
2170

2271
```bash
2372
pip install carbonyl-agent
@@ -29,7 +78,7 @@ carbonyl-agent install
2978
carbonyl-agent install --checksum <sha256-hex>
3079
```
3180

32-
## Quick Start
81+
### Your first script
3382

3483
```python
3584
from carbonyl_agent import CarbonylBrowser
@@ -41,21 +90,25 @@ print(b.page_text())
4190
b.close()
4291
```
4392

44-
All public API is importable directly from the package root:
93+
### Public API
94+
95+
All primary names importable directly from the package root:
4596

4697
```python
4798
from carbonyl_agent import (
4899
CarbonylBrowser, SessionManager, ScreenInspector,
49-
DaemonClient, start_daemon, stop_daemon,
100+
DaemonClient, start_daemon, stop_daemon, daemon_status,
50101
)
51102
```
52103

104+
---
105+
53106
## Session Persistence
54107

55108
Named sessions persist cookies, localStorage, and IndexedDB across browser restarts:
56109

57110
```python
58-
from carbonyl_agent import CarbonylBrowser, SessionManager
111+
from carbonyl_agent import CarbonylBrowser
59112

60113
b = CarbonylBrowser(session="myapp")
61114
b.open("https://example.com")
@@ -64,16 +117,18 @@ b.close()
64117
# Session data in ~/.local/share/carbonyl/sessions/myapp/
65118
```
66119

67-
### Session fork and snapshot
120+
### Fork and snapshot
68121

69122
Fork a logged-in session for parallel scraping, or snapshot to pin a known-good state:
70123

71124
```python
125+
from carbonyl_agent import SessionManager
126+
72127
sm = SessionManager()
73128
sm.create("base")
74129
# ... log in, accept cookies, etc. ...
75130

76-
# Fork: two independent profiles
131+
# Fork: two independent profiles that both start logged in
77132
sm.fork("base", "worker-1")
78133
sm.fork("base", "worker-2")
79134

@@ -83,11 +138,13 @@ sm.snapshot("base", "post-login")
83138
sm.restore("base", "post-login") # replaces profile with snapshot
84139
```
85140

86-
See `SessionManager` for the full API (list, destroy, exists, is_live, clean_stale_lock).
141+
See `SessionManager` for the full API: `list`, `destroy`, `exists`, `is_live`, `clean_stale_lock`.
142+
143+
---
87144

88145
## Daemon Mode
89146

90-
A long-running Carbonyl process exposed over a Unix socket. Clients reconnect without losing in-memory state:
147+
A long-running Carbonyl process exposed over a Unix socket. Clients reconnect without losing in-memory state — ideal for agent loops that want to amortize browser startup cost across many short scripts.
91148

92149
```python
93150
from carbonyl_agent import DaemonClient, start_daemon, stop_daemon
@@ -121,6 +178,10 @@ carbonyl-agent daemon attach myapp # interactive REPL
121178
carbonyl-agent daemon stop myapp
122179
```
123180

181+
Socket: `~/.local/share/carbonyl/daemons/<name>.sock` (mode 0600, parent dir 0700).
182+
183+
---
184+
124185
## Screen Inspection
125186

126187
Find text, debug click targets, and visualize coordinates:
@@ -142,18 +203,29 @@ matches = b.find_text("Continue") # [{col, row, end_col}, ...]
142203
print(si.annotate(marks=[(m["col"], m["row"]) for m in matches]))
143204
```
144205

206+
---
207+
145208
## Bot Detection Flags
146209

147210
`CarbonylBrowser` applies a curated `_HEADLESS_FLAGS` set at spawn time to minimize detection by commercial bot-detection engines (Akamai, Cloudflare, PerimeterX):
148211

149-
- Spoofed Firefox User-Agent (removes the "(Carbonyl)" marker and Chrome identifier)
212+
- Spoofed Firefox User-Agent (removes the `(Carbonyl)` marker and Chrome identifier)
150213
- `--disable-blink-features=AutomationControlled` (suppresses `navigator.webdriver=true`)
151-
- `--disable-http2` (HTTP/2 SETTINGS frame is a fingerprint used server-side)
152-
- Standard no-first-run, disable-sync, mock-keychain flags
214+
- `--disable-http2` (HTTP/2 SETTINGS frame is a server-side fingerprint)
215+
- Standard `--no-first-run`, `--disable-sync`, `--use-mock-keychain` flags
216+
217+
**If you hit bot-detection walls, do not remove these flags — they are the baseline.** For additional entropy, call `CarbonylBrowser.mouse_path([...])` to simulate organic mouse movement before interaction.
153218

154-
If you're hitting bot-detection walls, **do not remove these flags**. They are the baseline. For additional entropy, use `CarbonylBrowser.mouse_path([...])` to simulate organic mouse movement before interaction.
219+
---
220+
221+
## Binary Search Order
222+
223+
1. `CARBONYL_BIN` env var (explicit path)
224+
2. `~/.local/share/carbonyl/bin/<triple>/carbonyl` (installed by `carbonyl-agent install`)
225+
3. `carbonyl` on `$PATH`
226+
4. Docker fallback (requires `CARBONYL_ALLOW_DOCKER=1`)
155227

156-
## Docker Fallback (opt-in)
228+
### Docker fallback (opt-in)
157229

158230
When no local binary is installed, the SDK can fall back to `docker run fathyb/carbonyl` — but this is opt-in for supply-chain safety:
159231

@@ -164,19 +236,27 @@ python -c "from carbonyl_agent import CarbonylBrowser; CarbonylBrowser().open('h
164236

165237
Without `CARBONYL_ALLOW_DOCKER=1`, attempts to use Docker fallback raise `RuntimeError` with a clear message. The fallback pulls by pinned SHA256 digest, not a mutable `:latest` tag.
166238

239+
---
240+
167241
## Error Handling
168242

169243
Common exceptions:
170244

171-
- `ValueError` — invalid session name (path traversal, too long, empty)
172-
- `FileExistsError` — session already exists on `create()`
173-
- `KeyError` — session not found on `get()` / `destroy()` / `restore()`
174-
- `RuntimeError` — session is live when a destructive op is requested (destroy/fork/restore); also when Docker fallback is blocked
175-
- `pexpect.EOF` / `pexpect.TIMEOUT` — browser subprocess died or read timed out (handled internally by `drain()`; propagates on `send()`)
245+
| Exception | Raised when |
246+
|-----------|-------------|
247+
| `ValueError` | invalid session name (path traversal, too long, empty) |
248+
| `FileExistsError` | session already exists on `create()` |
249+
| `KeyError` | session not found on `get()` / `destroy()` / `restore()` |
250+
| `RuntimeError` | destructive op on a live session; Docker fallback blocked |
251+
| `pexpect.EOF` / `pexpect.TIMEOUT` | browser subprocess died or read timed out |
176252

177253
Retry pattern for flaky network:
178254

179255
```python
256+
import pexpect
257+
from carbonyl_agent import CarbonylBrowser
258+
259+
b = CarbonylBrowser()
180260
for attempt in range(3):
181261
try:
182262
b.open(url)
@@ -187,13 +267,90 @@ for attempt in range(3):
187267
b = CarbonylBrowser()
188268
```
189269

190-
## Binary Search Order
270+
---
191271

192-
1. `CARBONYL_BIN` env var (explicit path)
193-
2. `~/.local/share/carbonyl/bin/<triple>/carbonyl` (installed by `carbonyl-agent install`)
194-
3. `carbonyl` on `$PATH`
195-
4. Docker fallback (requires `CARBONYL_ALLOW_DOCKER=1`)
272+
## Documentation
273+
274+
- [CHANGELOG](CHANGELOG.md) — release history
275+
- [CONTRIBUTING](CONTRIBUTING.md) — dev setup, test suite, PR guidelines
276+
- [pyproject.toml](pyproject.toml) — dependencies, CLI entry points
277+
278+
### Related projects
279+
280+
- **[carbonyl](https://git.integrolabs.net/roctinam/carbonyl)** — the Chromium fork that produces the runtime binary
281+
- **[carbonyl-fleet](https://git.integrolabs.net/roctinam/carbonyl-fleet)** — server for managing N concurrent Carbonyl instances over PTY + Unix socket
282+
283+
---
284+
285+
## Contributing
286+
287+
PRs and issues welcome at [git.integrolabs.net/roctinam/carbonyl-agent](https://git.integrolabs.net/roctinam/carbonyl-agent) or [github.com/jmagly/carbonyl-agent](https://github.com/jmagly/carbonyl-agent).
288+
289+
- Run the test suite: `pytest`
290+
- Type-check: `mypy --strict src/`
291+
- Lint: `ruff check .`
292+
293+
---
294+
295+
## Community & Support
296+
297+
- **Issues**: [git.integrolabs.net/roctinam/carbonyl-agent/issues](https://git.integrolabs.net/roctinam/carbonyl-agent/issues)
298+
- **Discussions**: [github.com/jmagly/carbonyl-agent/discussions](https://github.com/jmagly/carbonyl-agent/discussions)
299+
300+
---
301+
302+
## License
303+
304+
**MIT License** — see [LICENSE](LICENSE).
305+
306+
---
307+
308+
## Sponsors
309+
310+
<table>
311+
<tr>
312+
<td width="33%" align="center">
313+
314+
### [Roko Network](https://roko.network)
315+
316+
**The Temporal Layer for Web3**
317+
318+
Enterprise-grade timing infrastructure for blockchain applications.
319+
320+
</td>
321+
<td width="33%" align="center">
322+
323+
### [Selfient](https://selfient.xyz)
324+
325+
**No-Code Smart Contracts for Everyone**
326+
327+
Making blockchain-based agreements accessible to all.
328+
329+
</td>
330+
<td width="33%" align="center">
331+
332+
### [Integro Labs](https://integrolabs.io)
333+
334+
**AI-Powered Automation Solutions**
335+
336+
Custom AI and blockchain solutions for the digital age.
337+
338+
</td>
339+
</tr>
340+
</table>
341+
342+
**Interested in sponsoring?** Open a discussion on [GitHub](https://github.com/jmagly/carbonyl-agent/discussions).
343+
344+
---
345+
346+
## Acknowledgments
347+
348+
Built on top of [Carbonyl](https://github.com/fathyb/carbonyl) by Fathy Boundjadj. The `roctinam/carbonyl` fork is actively maintained through the M147 Chromium line. PTY handling via [pexpect](https://github.com/pexpect/pexpect); terminal parsing via [pyte](https://github.com/selectel/pyte).
349+
350+
---
351+
352+
<div align="center">
196353

197-
## Changelog
354+
**[⬆ Back to Top](#carbonyl-agent)**
198355

199-
See [CHANGELOG.md](CHANGELOG.md) for release history.
356+
</div>

0 commit comments

Comments
 (0)