-
Notifications
You must be signed in to change notification settings - Fork 2.5k
(feat) Auto get free models from openrouter #2630
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
7807c28
68b8b64
8a34e68
9c671af
681697d
0e96807
2a72864
3770a06
0c042f5
2f7c056
0559a99
45857ec
b031815
c20d231
a211fa9
5c775d1
7d4e8b0
30aa0c0
9b9ca35
f4d8dff
ea4b9e4
0de746e
383f7e0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| import { expect } from "@playwright/test"; | ||
| import { testWithConfig } from "./helpers/test_helper"; | ||
|
|
||
| const testOpenRouter = testWithConfig({ | ||
| preLaunchHook: async () => { | ||
| process.env.OPENROUTER_API_KEY = "or-test-key"; | ||
| }, | ||
| }); | ||
|
|
||
| testOpenRouter( | ||
| "openrouter free models are listed in the model picker", | ||
| async ({ po }) => { | ||
| await po.setUp(); | ||
|
|
||
| await po.page.getByTestId("model-picker").click(); | ||
| await po.page.getByText("Other AI providers", { exact: true }).click(); | ||
| await po.page.getByText("OpenRouter", { exact: true }).click(); | ||
|
|
||
| await expect( | ||
| po.page.getByText("Free models", { exact: true }), | ||
| ).toBeVisible(); | ||
| await po.page.getByText("Free models", { exact: true }).click(); | ||
|
|
||
| await expect( | ||
| po.page.getByText("OpenRouter Free Models", { exact: true }), | ||
| ).toBeVisible(); | ||
|
|
||
| await expect( | ||
| po.page.getByRole("menuitem").filter({ hasText: "Free (OpenRouter)" }), | ||
| ).toBeVisible(); | ||
| }, | ||
| ); | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| import { describe, expect, it } from "vitest"; | ||
| import { buildOpenRouterFreeModels } from "@/ipc/shared/openrouter_free_models"; | ||
|
|
||
| describe("buildOpenRouterFreeModels", () => { | ||
| it("filters to free models and decorates display names", () => { | ||
| const models = [ | ||
| { | ||
| id: "paid/model", | ||
| name: "Paid Model", | ||
| pricing: { prompt: "0.01", completion: "0.02" }, | ||
| }, | ||
| { | ||
| id: "free/model", | ||
| name: "Awesome Model", | ||
| pricing: { prompt: "0", completion: "0" }, | ||
| context_length: 128_000, | ||
| top_provider: { max_completion_tokens: 16_384 }, | ||
| }, | ||
| { | ||
| id: "free/with-label", | ||
| name: "Free Model", | ||
| pricing: { prompt: 0, completion: 0, image: 0 }, | ||
| }, | ||
| ]; | ||
|
|
||
| const result = buildOpenRouterFreeModels(models); | ||
|
|
||
| expect(result).toHaveLength(2); | ||
| const awesome = result.find((model) => model.apiName === "free/model"); | ||
| expect(awesome?.displayName).toBe("Awesome Model"); | ||
| expect(awesome?.contextWindow).toBe(128_000); | ||
| expect(awesome?.maxOutputTokens).toBe(16_384); | ||
| expect(awesome?.tag).toBe("Free"); | ||
|
|
||
| const labeled = result.find((model) => model.apiName === "free/with-label"); | ||
| expect(labeled?.displayName).toBe("Free Model"); | ||
| expect(labeled?.dollarSigns).toBe(0); | ||
| }); | ||
|
|
||
| it("sanitizes external model name and description fields", () => { | ||
| const models = [ | ||
| { | ||
| id: "free/sanitized", | ||
| name: " <img src=x onerror=alert(1)>Safe Name (free) ", | ||
| description: "<script>alert(1)</script> Useful model ", | ||
| pricing: { prompt: 0, completion: 0 }, | ||
| }, | ||
| ]; | ||
|
|
||
| const result = buildOpenRouterFreeModels(models); | ||
|
|
||
| expect(result).toHaveLength(1); | ||
| expect(result[0]?.displayName).toBe("Safe Name"); | ||
| expect(result[0]?.description).toBe("alert(1) Useful model"); | ||
| }); | ||
| }); |
|
coderdylanbates marked this conversation as resolved.
|
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -271,6 +271,17 @@ export const MODEL_OPTIONS: Record<string, ModelOption[]> = { | |||||||||||||||||||||||||||
| temperature: 0, | ||||||||||||||||||||||||||||
| dollarSigns: 1, | ||||||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||||||
| // https://openrouter.ai/qwen/qwen3-coder | ||||||||||||||||||||||||||||
| { | ||||||||||||||||||||||||||||
| name: "qwen/qwen3-coder:free", | ||||||||||||||||||||||||||||
| displayName: "Qwen3 Coder", | ||||||||||||||||||||||||||||
| description: "Use for free (data may be used for training)", | ||||||||||||||||||||||||||||
| maxOutputTokens: 32_000, | ||||||||||||||||||||||||||||
| contextWindow: 196_608, | ||||||||||||||||||||||||||||
| temperature: 0, | ||||||||||||||||||||||||||||
| dollarSigns: 1, | ||||||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Free model The newly added
Suggested change
Was this helpful? React with 👍 or 👎 to provide feedback. |
||||||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||||||
| // https://openrouter.ai/z-ai/glm-5 | ||||||||||||||||||||||||||||
| { | ||||||||||||||||||||||||||||
| name: "z-ai/glm-5", | ||||||||||||||||||||||||||||
| displayName: "GLM 5", | ||||||||||||||||||||||||||||
|
|
@@ -280,6 +291,16 @@ export const MODEL_OPTIONS: Record<string, ModelOption[]> = { | |||||||||||||||||||||||||||
| temperature: 0.7, | ||||||||||||||||||||||||||||
| dollarSigns: 2, | ||||||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||||||
| // https://openrouter.ai/mistralai/devstral-2512 | ||||||||||||||||||||||||||||
| { | ||||||||||||||||||||||||||||
| name: "mistralai/devstral-2512:free", | ||||||||||||||||||||||||||||
| displayName: "Devstral 2", | ||||||||||||||||||||||||||||
| description: "Use for free (data may be used for training)", | ||||||||||||||||||||||||||||
| maxOutputTokens: 32_000, | ||||||||||||||||||||||||||||
| contextWindow: 200_000, | ||||||||||||||||||||||||||||
| temperature: 0.7, | ||||||||||||||||||||||||||||
| dollarSigns: 2, | ||||||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Free model The newly added
Suggested change
Was this helpful? React with 👍 or 👎 to provide feedback. |
||||||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||||||
|
Comment on lines
276
to
+303
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Incorrect Both new free-tier models are given non-zero Both entries should use
Suggested change
And for
Suggested change
Prompt To Fix With AIThis is a comment left during a code review.
Path: src/ipc/shared/language_model_constants.ts
Line: 276-303
Comment:
**Incorrect `dollarSigns` on new `:free` hardcoded models**
Both new free-tier models are given non-zero `dollarSigns` values (`1` and `2`), but because the deduplication in `language_model_helpers.ts` uses `[...freeModels, ...hardcodedModels]`, hardcoded entries always **overwrite** the API-fetched entries with `dollarSigns: 0`. The result is that these two models end up with `dollarSigns: 1` / `dollarSigns: 2` in the merged list, so `PriceBadge` will display `$` / `$$` for them inside the "Free models" submenu — contradicting the "free" label.
Both entries should use `dollarSigns: 0`:
```suggestion
// https://openrouter.ai/qwen/qwen3-coder
{
name: "qwen/qwen3-coder:free",
displayName: "Qwen3 Coder",
description: "Use for free (data may be used for training)",
maxOutputTokens: 32_000,
contextWindow: 196_608,
temperature: 0,
dollarSigns: 0,
},
```
And for `mistralai/devstral-2512:free`:
```suggestion
dollarSigns: 0,
```
How can I resolve this? If you propose a fix, please make it concise. |
||||||||||||||||||||||||||||
| { | ||||||||||||||||||||||||||||
| name: "z-ai/glm-4.7", | ||||||||||||||||||||||||||||
| displayName: "GLM 4.7", | ||||||||||||||||||||||||||||
|
|
@@ -321,7 +342,7 @@ export const MODEL_OPTIONS: Record<string, ModelOption[]> = { | |||||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||||||
| { | ||||||||||||||||||||||||||||
| name: "free", | ||||||||||||||||||||||||||||
| displayName: "Free (OpenRouter)", | ||||||||||||||||||||||||||||
| displayName: "Free", | ||||||||||||||||||||||||||||
| description: "Selects from one of the free OpenRouter models", | ||||||||||||||||||||||||||||
| tag: "Free", | ||||||||||||||||||||||||||||
| // These are below Gemini 2.5 Pro & Flash limits | ||||||||||||||||||||||||||||
|
|
@@ -505,7 +526,6 @@ export const FREE_OPENROUTER_MODEL_NAMES = MODEL_OPTIONS.openrouter | |||||||||||||||||||||||||||
| (model) => model.name.endsWith(":free") || model.name.endsWith("/free"), | ||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||
| .map((model) => model.name); | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| export const PROVIDER_TO_ENV_VAR: Record<string, string> = { | ||||||||||||||||||||||||||||
| openai: "OPENAI_API_KEY", | ||||||||||||||||||||||||||||
| anthropic: "ANTHROPIC_API_KEY", | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -11,6 +11,7 @@ import { | |||||||||||||||||||||||||||||||||
| MODEL_OPTIONS, | ||||||||||||||||||||||||||||||||||
| PROVIDER_TO_ENV_VAR, | ||||||||||||||||||||||||||||||||||
| } from "./language_model_constants"; | ||||||||||||||||||||||||||||||||||
| import { getOpenRouterFreeModels } from "./openrouter_free_models"; | ||||||||||||||||||||||||||||||||||
| /** | ||||||||||||||||||||||||||||||||||
| * Fetches language model providers from both the database (custom) and hardcoded constants (cloud), | ||||||||||||||||||||||||||||||||||
| * merging them with custom providers taking precedence. | ||||||||||||||||||||||||||||||||||
|
|
@@ -148,6 +149,15 @@ export async function getLanguageModels({ | |||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| if (providerId === "openrouter") { | ||||||||||||||||||||||||||||||||||
| const freeModels = getOpenRouterFreeModels(); | ||||||||||||||||||||||||||||||||||
| const dedupe = new Map<string, LanguageModel>(); | ||||||||||||||||||||||||||||||||||
| [...freeModels, ...hardcodedModels].forEach((model) => { | ||||||||||||||||||||||||||||||||||
| dedupe.set(model.apiName, model); | ||||||||||||||||||||||||||||||||||
| }); | ||||||||||||||||||||||||||||||||||
| hardcodedModels = Array.from(dedupe.values()); | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
devin-ai-integration[bot] marked this conversation as resolved.
Comment on lines
+152
to
+159
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Deduplication direction discards API-fetched data The array spread This defeats the PR's goal of pulling current metadata from OpenRouter at runtime. The spread order should be reversed so that API data overrides the hardcoded fallback:
Suggested change
Prompt To Fix With AIThis is a comment left during a code review.
Path: src/ipc/shared/language_model_helpers.ts
Line: 152-159
Comment:
**Deduplication direction discards API-fetched data**
The array spread `[...freeModels, ...hardcodedModels]` means that when the same `apiName` exists in both lists, `Map.set` is called last with the **hardcoded** entry, so hardcoded values always win. The API-fetched `contextWindow`, `maxOutputTokens`, `dollarSigns: 0`, and `tag: "Free"` metadata are silently discarded in favour of the static constants file.
This defeats the PR's goal of pulling current metadata from OpenRouter at runtime. The spread order should be reversed so that API data overrides the hardcoded fallback:
```suggestion
if (providerId === "openrouter") {
const freeModels = getOpenRouterFreeModels();
const dedupe = new Map<string, LanguageModel>();
[...hardcodedModels, ...freeModels].forEach((model) => {
dedupe.set(model.apiName, model);
});
hardcodedModels = Array.from(dedupe.values());
}
```
How can I resolve this? If you propose a fix, please make it concise. |
||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| return [...hardcodedModels, ...customModels]; | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test expects "Free (OpenRouter)" but
language_model_constants.ts:234now showsdisplayName: "Free"- the test will fail.Prompt To Fix With AI