Non-intrusive AI-powered clients

[donmor]

I mean, yeah, there's many attempts to make plugins that integrate LLMs into TiddlyWiki, but there could be a way to enhance most TiddlyWiki with AIs without installing any plugin.

• A $:/LLM path for all AI-related data (these tiddlers can be deleted to reset AI-related stuff)
• $:/LLM/config.json contains OpenAI-compatible API URL, key (optional if is self-hosted), models (as an array), custom prompt (optional, default prompt is set by client)
• $:/LLM/logs/******.json are historical conversation data where a conversation can restore from
• Client provide its LLM interface (always available, using client's own config; or if $:/LLM/config.json is detected, using included configurations)
• Client implements 'New conversation', 'Continue conversation' (lists items under $:/LLM/logs/, and loads the chosen one), 'Configure' (configure current/new conversation), input box, 'Attach' (lists and selects tiddler to attach to the conversation, images are supported as well, selected tiddlers are shown above the input box), 'Send'
• Once 'Send' is pressed, client sends query to LLM, and provides all tiddlers as knoledge base. It then get response and shows up.
• Once a query or response is done, client append it to a log tiddler (or create one)
• Client tells LLM about tiddler data structure in prompt so that LLM can input/output tiddlers better.
• Long-click a generated paragraph to copy it / import it back to TiddlyWiki. (file attachments have 'Save as' instead of 'Copy')

We might even have something similiar for TWC :3c

[pmario]

IMO plugins are non-intrusive. The user has a choice to install it or not.

It seems what is proposed in the OP is intrusive, if it is hardcoded into the core. It would remove the "opt in" possibility.

The problem is there are a lot of different providers, which may have different preconditions, how to configure them. A session log can be huge. Especially if you need to copy images into the conversation, or if you use external tools like playwright (headless browser for debugging).

The logs go to several megabytes easy. For one session.

Native clients keep track of the session store and also "context tokes" which are active.

While the information and knowledge, that is created during a conversation session is valuable. IMO about 60% to 80% is noise. So keeping the noise in the wiki, IMO is waste of wiki storage.

Keeping it on disk is OK. May be it can be used later for "digging", but I doubt it. So being able to deleting it, without fear of loosing "value" is important.

I personally use a skill that is named: claude-handoff, It creates a session summary, that depends on the current session context. This context depends on the LLM model that is used by the client.

If it is eg: 1mio tokens the handoff file created is up to 500-800 lines, with a hard minimum of 250. It contains condensed (valuable) knowledge created during a session. This info can be used to start a new session, so the llm does not need to start from zero.

These handoff files are definitely worth keeping. They contain the history of a conversation with all user decisions recorded. So they will tell you later, what you "thought" about something. ...

Just some thoughts.
-mario

[linonetwo]

It's not just the era of LLM anymore; now it's the era of LLM agents. Agents will produce tons of text, measured in millions of words, and storing them in a knowledge base would cost more than it benefits.

Therefore, I suggest that conversations should still be primarily stored in external storage (such as SQLite / IndexedDB databases), with only the truly valuable parts being excerpted and added to the user's knowledge base.

And you are talking about tags for LLM, I think it is already handled in Jeremy's PR. And what needs more discussion is tags for LLM Agent: #9378