Skip to content

Add telemetry sampling for non-Pro users #2642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wwwillchen merged 1 commit into from
Feb 12, 2026
Merged

Add telemetry sampling for non-Pro users #2642

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions src/hooks/useSettings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { queryKeys } from "@/lib/queryKeys";

const TELEMETRY_CONSENT_KEY = "dyadTelemetryConsent";
const TELEMETRY_USER_ID_KEY = "dyadTelemetryUserId";
const DYAD_PRO_STATUS_KEY = "dyadProStatus";

export function isTelemetryOptedIn() {
return window.localStorage.getItem(TELEMETRY_CONSENT_KEY) === "opted_in";
Expand All @@ -19,6 +20,10 @@ export function getTelemetryUserId(): string | null {
return window.localStorage.getItem(TELEMETRY_USER_ID_KEY);
}

export function isDyadProUser(): boolean {
return window.localStorage.getItem(DYAD_PRO_STATUS_KEY) === "true";
}

let isInitialLoad = false;

export function useSettings() {
Expand Down Expand Up @@ -121,4 +126,9 @@ function processSettingsForTelemetry(settings: UserSettings) {
} else {
window.localStorage.removeItem(TELEMETRY_USER_ID_KEY);
}
// Store Pro status for telemetry sampling
window.localStorage.setItem(
DYAD_PRO_STATUS_KEY,
hasDyadProKey(settings) ? "true" : "false",
);
}
20 changes: 19 additions & 1 deletion src/renderer.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ import { router } from "./router";
import { RouterProvider } from "@tanstack/react-router";
import { PostHogProvider } from "posthog-js/react";
import posthog from "posthog-js";
import { getTelemetryUserId, isTelemetryOptedIn } from "./hooks/useSettings";
import {
getTelemetryUserId,
isTelemetryOptedIn,
isDyadProUser,
} from "./hooks/useSettings";

// Initialize i18next before any rendering
import "./i18n";
Expand Down Expand Up @@ -87,6 +91,20 @@ const posthogClient = posthog.init(
event.properties["$ip"] = null;
}

// For non-Pro users, only send 10% of events (but always send errors)
if (!isDyadProUser()) {
const isErrorEvent =
event?.event === "$exception" ||
event?.event?.toLowerCase().includes("error") ||

@cubic-dev-ai cubic-dev-ai Bot Feb 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The optional chaining stops before .includes, so event?.event?.toLowerCase() can return undefined and then .includes() throws. Guard the string check (or optional-chain the result) before calling .includes() to avoid runtime errors in telemetry sampling.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At src/renderer.tsx, line 98:

<comment>The optional chaining stops before `.includes`, so `event?.event?.toLowerCase()` can return `undefined` and then `.includes()` throws. Guard the string check (or optional-chain the result) before calling `.includes()` to avoid runtime errors in telemetry sampling.</comment>

<file context>
@@ -87,6 +91,20 @@ const posthogClient = posthog.init(
+      if (!isDyadProUser()) {
+        const isErrorEvent =
+          event?.event === "$exception" ||
+          event?.event?.toLowerCase().includes("error") ||
+          event?.properties?.$exception_type ||
+          event?.properties?.error;
</file context>
Suggested change
event?.event?.toLowerCase().includes("error") ||
(typeof event?.event === "string" &&
event.event.toLowerCase().includes("error")) ||
Fix with Cubic

event?.properties?.$exception_type ||
event?.properties?.error;

@github-actions github-actions Bot Feb 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM | complexity | Found by: Code Health, Endorsed by: Correctness, UX

Magic number for sampling rate

The sampling rate 0.1 (10%) is a magic number embedded in the callback. If this rate needs tuning, someone has to hunt through PostHog init config to find it.

💡 Suggestion: Extract to a named constant, e.g. const NON_PRO_TELEMETRY_SAMPLE_RATE = 0.1; defined near the top of the file.

if (!isErrorEvent && Math.random() > 0.1) {
console.debug("Non-Pro user: sampling out event", event?.event);
return null;

@github-actions github-actions Bot Feb 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM | race-condition | Found by: Correctness, Endorsed by: Code Health, UX

Pro status in localStorage may be stale at PostHog init time

isDyadProUser() reads from localStorage, but processSettingsForTelemetry() (which writes the Pro status) is called from a React hook. PostHog is initialized at module level before React renders. On first app launch (or after clearing storage), the Pro status won't be in localStorage yet, so all users will be treated as non-Pro until settings are fetched and processed. This means some early startup events could be incorrectly sampled for Pro users.

💡 Suggestion: Add a comment documenting this known timing gap, e.g. // Note: on first launch, Pro status may not be in localStorage yet. This is acceptable since it only affects telemetry sampling during the brief startup window.

}
Comment on lines +96 to +105

@gemini-code-assist gemini-code-assist Bot Feb 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current sampling logic for non-Pro users will also sample crucial PostHog events like $identify and $pageview, which can lead to data integrity issues.

  • $identify events: Sampling these will cause user identification to fail for 90% of non-Pro user sessions. This makes it impossible to reliably associate other telemetry events with a specific user, skewing all user-centric analytics.
  • $pageview events: Sampling these will make user flow analysis (like funnels and paths) unreliable for this user segment, as most navigation actions won't be recorded.

These special events should always be sent, just like error events, to ensure the collected data is accurate and useful. The suggested change updates the logic to prevent these events from being sampled.

        const shouldAlwaysSend =
          event?.event === "$exception" ||
          event?.event === "$identify" ||
          event?.event === "$pageview" ||
          event?.event?.toLowerCase().includes("error") ||
          event?.properties?.$exception_type ||
          event?.properties?.error;

        if (!shouldAlwaysSend && Math.random() > 0.1) {
          console.debug("Non-Pro user: sampling out event", event?.event);
          return null;
        }

}

console.debug(
"Telemetry opted in - UUID:",
telemetryUserId,
Expand Down
Loading