Skip to content

Asynchronous materialization of samples view#7734

Open
XingY wants to merge 31 commits into
developfrom
fb_asyncSamplesTable
Open

Asynchronous materialization of samples view#7734
XingY wants to merge 31 commits into
developfrom
fb_asyncSamplesTable

Conversation

@XingY

@XingY XingY commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Rationale

On server start or after sample designer update, the first request to load the sample grid for a large sample type triggers a synchronous full table materialization, which can take a long time if the number of sample rows is large. If the user navigates away before it completes, the HTTP request is cancelled mid-flight, aborting the SELECT INTO. The materialized view is never populated — subsequent visits repeat the same blocking behavior indefinitely.

This change decouples view population from the request. When the materialized view is not ready (never built, or stale due to pending incremental updates), the request falls back immediately to direct JOINs and submits a background rebuild. The background task runs to completion regardless of user navigation, so the next visit uses the fast materialized path.

Related Pull Requests

Changes

  • ExpMaterialTableImpl: replaced the synchronous getMaterializedSQL() call with the async-aware path.

Comment thread experiment/src/org/labkey/experiment/api/ExpMaterialTableImpl.java Outdated
Comment thread experiment/src/org/labkey/experiment/api/ExpMaterialTableImpl.java
Comment thread core/src/org/labkey/core/admin/AdminController.java Outdated
* Returns true if the global (non-transactional) materialized view is LOADED and has no pending synchronous work.
* Use this to decide whether to use the fast materialized path or fall back to direct JOINs.
*/
public boolean isReadyToUse()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems redundant to have isReadyToUse() and tryGetFromSqlIfLoaded()? Is there a caller of isReadyToUse() that doesn't immediately want to proceed to use the table?

sql = tryGetFromSqlIfLoaded();
if (null == sql)
     start asyncLoading
     sql = getJoinSQL(selectedColumns)
...

@XingY XingY Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a case when there is a lookup defined on samples table, getJoinCondition is called again even though it was previously already determined:

if (assertEnabled || !map.containsKey(colTableAlias))

Could we actually remove the "assert" check here? Absent of it, the lookup join sql won't be attempted again and we can simplify the materialization logic for samples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants