fix(Segment Membership): Stagger the recurring count refresh fan-out by khvn26 · Pull Request #7943 · Flagsmith/flagsmith

khvn26 · 2026-07-02T18:27:04Z

Thanks for submitting a PR! Please check the boxes below:

I have read the Contributing Guide.
I have added information to docs/ if required so people know about the feature.
I have filled in the "Changes" section below.
I have filled in the "How did you test this code" section below.

Changes

Contributes to #7942

In this PR, we distribute per-organisation segment count refreshes to avoid a thundering herd problem when ClickHouse has to compute segment counts for all projects at roughly the same time.

We introduce a new setting:

SEGMENT_MEMBERSHIP_REFRESH_PROJECT_STAGGER_WINDOW_HOURS (default 1) — the window organisations are spread across. Validated at startup to not exceed SEGMENT_MEMBERSHIP_REFRESH_INTERVAL_HOURS so consecutive refresh cycles won't overlap.

How did you test this code?

Added unit tests to assert the fan-out is staggered.

refresh_all_segment_counts enqueued every project's refresh at once, so the task processor ran a burst of concurrent `FROM IDENTITIES FINAL` count queries that could exhaust ClickHouse's memory and get an unrelated query killed by the overcommit tracker. Spread organisations across a configurable stagger window and cluster each org's projects a few seconds apart, streaming in organisation order so worker memory stays flat. Same-org counts stay consistently fresh; cross-org load is bounded. beep boop Claude-Session: https://claude.ai/code/session_01EgZ5iHpDASZzCapiHRxHLB

vercel · 2026-07-02T18:27:06Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jul 3, 2026 12:06pm

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
flagsmith-frontend-preview	Ignored	Preview	Jul 3, 2026 12:06pm
flagsmith-frontend-staging	Ignored	Preview	Jul 3, 2026 12:06pm

coderabbitai · 2026-07-02T18:27:14Z

📝 Walkthrough

Walkthrough

This PR adds a setting for staggering segment membership refresh scheduling and updates refresh_all_segment_counts to enqueue live-segment projects with ordered, delayed dispatches. The settings module now validates the stagger window against the refresh interval. Unit tests cover the no-live-segments path and the staggered delay behaviour. Documentation references for refresh events are updated to match the new task line numbers.

Cohort / File(s)	Change Summary
Settings — `api/app/settings/common.py`	Adds `SEGMENT_MEMBERSHIP_REFRESH_PROJECT_STAGGER_WINDOW_HOURS` and validates it against the refresh interval.
Task logic — `api/segment_membership/tasks.py`	Selects ordered live-segment projects and enqueues each with a calculated `delay_until`.
Tests — `api/tests/unit/segment_membership/test_unit_segment_membership_tasks.py`	Adds coverage for the empty case and for ordered, evenly spaced delays.
Docs — `docs/docs/deployment-self-hosting/observability/_events-catalogue.md`	Updates "Logged at" line references for completed, failed, and skipped events.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant refresh_all_segment_counts
  participant enqueue_membership_refresh

  refresh_all_segment_counts->>refresh_all_segment_counts: build ordered live-segment project queryset
  refresh_all_segment_counts->>refresh_all_segment_counts: calculate spacing from stagger window
  loop for each project
    refresh_all_segment_counts->>enqueue_membership_refresh: enqueue with delay_until
  end

Related issues: None specified in the provided context.

Related PRs: None specified in the provided context.

Suggested labels: backend, segment-membership, scheduling

Suggested reviewers: None specified in the provided context.

Poem

A staggered queue begins to hum,
With delays set from window sum,
Projects ordered, one by one,
Until the refresh work is done.

_{Comment @coderabbitai help to get the list of available commands.}

beep boop Claude-Session: https://claude.ai/code/session_01EgZ5iHpDASZzCapiHRxHLB

codecov · 2026-07-02T18:39:26Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.63%. Comparing base (cbc295a) to head (d3d2db9).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7943   +/-   ##
=======================================
  Coverage   98.63%   98.63%           
=======================================
  Files        1497     1497           
  Lines       59151    59189   +38     
=======================================
+ Hits        58345    58383   +38     
  Misses        806      806

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Exercises the early return in refresh_all_segment_counts when no organisation has a live segment, restoring 100% patch coverage. beep boop Claude-Session: https://claude.ai/code/session_01EgZ5iHpDASZzCapiHRxHLB

github-actions · 2026-07-02T18:49:35Z

Docker builds report

Image	Build Status	Security report
`ghcr.io/flagsmith/flagsmith-e2e:pr-7943`	Finished ✅	Skipped
`ghcr.io/flagsmith/flagsmith-api-test:pr-7943`	Finished ✅	Skipped
`ghcr.io/flagsmith/flagsmith-frontend:pr-7943`	Finished ✅	Results ✅
`ghcr.io/flagsmith/flagsmith-api:pr-7943`	Finished ✅	Results ✅
`ghcr.io/flagsmith/flagsmith:pr-7943`	Finished ✅	Results ✅
`ghcr.io/flagsmith/flagsmith-private-cloud:pr-7943`	Finished ✅	Results ✅

coderabbitai

Actionable comments posted: 1

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e610eecc-eebc-4870-bc06-54099d2a22aa

📥 Commits

Reviewing files that changed from the base of the PR and between 31c9dc9 and 452b576.

📒 Files selected for processing (4)

api/app/settings/common.py
api/segment_membership/tasks.py
api/tests/unit/segment_membership/test_unit_segment_membership_tasks.py
docs/docs/deployment-self-hosting/observability/_events-catalogue.md

github-actions · 2026-07-02T18:56:32Z

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

4 passed

Details

4 tests across 3 suites
19 seconds
452b576
🔄 Run: #18043 (attempt 1)

5 tests across 4 suites
39.9 seconds
d3d2db9
🔄 Run: #18048 (attempt 1)

github-actions · 2026-07-02T19:00:20Z

Visual Regression

19 screenshots compared. See report for details.
View full report

…s slot With a fixed project stagger and an org_spacing that shrinks as organisations grow, an organisation with enough projects pushed its later refreshes past the next organisation's slot, colliding at scale. Accumulate a monotonic delay so each organisation starts at its slot or after the previous org's last project, whichever is later. beep boop Claude-Session: https://claude.ai/code/session_01EgZ5iHpDASZzCapiHRxHLB

emyller

The idea is sound. I'd like to discuss the approach a bit, and consider distributing projects evenly rather than grouping by organisations first — my visual imagination of this patch currently leads to a weird fragmenting of turns.

emyller

The two code suggestions remain, but neither is a blocker now.
The design LGTM now but we may need to revisit it down the road.

…s the window Per review, drop the per-organisation grouping (and its cumulative-delay overflow handling and STAGGER_SECONDS setting) in favour of spreading all live-segment projects evenly across the stagger window, ordered by organisation so an org's projects still refresh contiguously. Simpler, and no org-slot overflow to guard. beep boop Claude-Session: https://claude.ai/code/session_01EgZ5iHpDASZzCapiHRxHLB

coderabbitai

Actionable comments posted: 1

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 9b705641-f921-406f-8e91-d75dfbe030b0

📥 Commits

Reviewing files that changed from the base of the PR and between 8df9042 and 4c06af4.

📒 Files selected for processing (4)

api/app/settings/common.py
api/segment_membership/tasks.py
api/tests/unit/segment_membership/test_unit_segment_membership_tasks.py
docs/docs/deployment-self-hosting/observability/_events-catalogue.md

💤 Files with no reviewable changes (1)

api/app/settings/common.py

coderabbitai

Actionable comments posted: 2

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 6d8973b4-0122-4be2-b5ab-02e584f5a285

📥 Commits

Reviewing files that changed from the base of the PR and between 4c06af4 and d3d2db9.

📒 Files selected for processing (4)

api/app/settings/common.py
api/segment_membership/tasks.py
api/tests/unit/segment_membership/test_unit_segment_membership_tasks.py
docs/docs/deployment-self-hosting/observability/_events-catalogue.md

💤 Files with no reviewable changes (1)

api/app/settings/common.py

emyller

Stunning stagger

vercel Bot deployed to Preview – docs July 2, 2026 18:27 View deployment

github-actions Bot added api Issue related to the REST API docs Documentation updates fix and removed docs Documentation updates labels Jul 2, 2026

chore(Segment Membership): Drop redundant stagger comments

d5a21c7

beep boop Claude-Session: https://claude.ai/code/session_01EgZ5iHpDASZzCapiHRxHLB

github-actions Bot added docs Documentation updates fix and removed fix docs Documentation updates labels Jul 2, 2026

test(Segment Membership): Cover the no-live-segments refresh path

452b576

Exercises the early return in refresh_all_segment_counts when no organisation has a live segment, restoring 100% patch coverage. beep boop Claude-Session: https://claude.ai/code/session_01EgZ5iHpDASZzCapiHRxHLB

khvn26 marked this pull request as ready for review July 2, 2026 18:47

khvn26 requested review from a team as code owners July 2, 2026 18:47

khvn26 requested review from emyller and removed request for a team July 2, 2026 18:47

flagsmith-engineering Bot assigned emyller Jul 2, 2026

github-actions Bot added docs Documentation updates fix and removed fix docs Documentation updates labels Jul 2, 2026

github-actions Bot added fix and removed fix labels Jul 2, 2026

coderabbitai Bot reviewed Jul 2, 2026

View reviewed changes

Comment thread api/segment_membership/tasks.py Outdated

vercel Bot deployed to Preview – docs July 2, 2026 19:05 View deployment

github-actions Bot added docs Documentation updates fix and removed fix docs Documentation updates labels Jul 2, 2026

emyller requested changes Jul 2, 2026

View reviewed changes

Comment thread api/segment_membership/tasks.py

Comment thread api/segment_membership/tasks.py Outdated

emyller previously approved these changes Jul 3, 2026

View reviewed changes

khvn26 dismissed emyller’s stale review via 4c06af4 July 3, 2026 12:02

github-actions Bot added the docs Documentation updates label Jul 3, 2026

vercel Bot deployed to Preview – docs July 3, 2026 12:02 View deployment

github-actions Bot added fix and removed fix docs Documentation updates labels Jul 3, 2026

khvn26 force-pushed the fix/stagger-segment-count-refresh branch from 4c06af4 to d3d2db9 Compare July 3, 2026 12:06

github-actions Bot added the docs Documentation updates label Jul 3, 2026

vercel Bot deployed to Preview – docs July 3, 2026 12:06 View deployment

github-actions Bot added fix and removed fix docs Documentation updates labels Jul 3, 2026

coderabbitai Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread api/segment_membership/tasks.py

khvn26 requested a review from emyller July 3, 2026 12:11

coderabbitai Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread api/segment_membership/tasks.py

Comment thread api/tests/unit/segment_membership/test_unit_segment_membership_tasks.py

khvn26 changed the title ~~fix(Segment Membership): Stagger the recurring count-refresh fan-out~~ fix(Segment Membership): Stagger the recurring count refresh fan-out Jul 3, 2026

emyller approved these changes Jul 3, 2026

View reviewed changes

khvn26 merged commit 9105ae0 into main Jul 3, 2026
35 checks passed

khvn26 deleted the fix/stagger-segment-count-refresh branch July 3, 2026 12:58

flagsmith-engineering Bot mentioned this pull request Jul 2, 2026

chore(main): release 2.252.0 #7926

Merged

Uh oh!

Conversation

khvn26 commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

How did you test this code?

Uh oh!

vercel Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Sequence Diagram(s)

Uh oh!

codecov Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Docker builds report

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!