Skip to content

[rust] Add RPC message wrappers for extended operations#630

Open
gnuhpc wants to merge 3 commits into
apache:mainfrom
gnuhpc:pr/3-rpc-messages-extended
Open

[rust] Add RPC message wrappers for extended operations#630
gnuhpc wants to merge 3 commits into
apache:mainfrom
gnuhpc:pr/3-rpc-messages-extended

Conversation

@gnuhpc

@gnuhpc gnuhpc commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

What

Adds message wrappers for the remaining 1.x RPC APIs:

  • KV snapshot lifecycle: acquire/release/drop lease, list, metadata, latest snapshots, lake snapshot
  • Server management: add/remove server tag, rebalance + progress + cancel, get cluster health, list remote log manifests
  • Producer offsets: register/get/delete
  • ScanKv (API 1061): full KV-table bucket scan request/response

Wired into rpc/message/mod.rs. scan_kv.rs is consumed by the KvBatchScanner in the final PR of the series.

Stack

Part 3/6, stacked on #629 (pr/2-rpc-messages-core) → #628. All target main; each branch compiles on its predecessor, so diffs shrink to this branch's unique files as the base PRs merge.

🤖 Generated with Claude Code

@warmbupt @claude
[rust] Sync Fluss 1.x proto definitions and ApiKey registry
7c0f61f 
- Update fluss_api.proto with all 1.x message types (ACLs, KV snapshots,
  producer offsets, cluster config, rebalance, server tags, etc.)
- Add optional fields: rack, remote_data_dir, leader_epoch, agg_mode, etc.
- Register 24 new ApiKey variants (1023-1064) in api_key.rs
- Update build.rs prost bytes config for new proto fields
- Add None defaults in convert.rs and partition.rs
- Update pre-existing message wrappers that reference renamed proto fields /
  ApiKey variants so the crate still builds:
    * create_partition.rs: ignore_if_exists -> ignore_if_not_exists
    * get_latest_lake_snapshot.rs: ApiKey::GetLatestLakeSnapshot -> GetLakeSnapshot
    * list_databases.rs: populate new include_summary field
    * lookup.rs: PbLookupReqForBucket.key -> keys; new LookupRequest fields

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jun 19, 2026
Open
Open
fresh-borzoni
fresh-borzoni reviewed Jun 19, 2026
View reviewed changes

@fresh-borzoni fresh-borzoni left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnuhpc Thank you for the PR, left comment.
The same problems as in #629, follow the comments there, please.

@gnuhpc gnuhpc mentioned this pull request Jun 19, 2026
Merged
warmbupt and others added 2 commits June 20, 2026 02:45
@warmbupt @claude
[rust] Add RPC message wrappers for core DB/table operations
a6bb958 
Add 9 RPC message wrapper types:
- alter_database, alter_table (DDL operations)
- get_table_stats (table statistics)
- list_database_summaries (database listing with summaries)
- create_acls, list_acls, drop_acls (ACL management)
- describe_cluster_configs, alter_cluster_configs (cluster configuration)

Each wrapper follows the standard pattern: a request struct wrapping the
proto-generated type, implementing RequestBody (tying to ApiKey and ResponseBody),
WriteType, and ReadType.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@warmbupt
[rust] Add RPC message wrappers for extended operations
9d2fbff 
Add message wrappers for the remaining 1.x RPC APIs:
- KV snapshot lifecycle: acquire/release/drop lease, list, metadata,
  latest snapshots, lake snapshot
- Server management: add/remove server tag, rebalance + progress +
  cancel, get cluster health, list remote log manifests
- Producer offsets: register/get/delete
- ScanKv (API 1061): full KV-table bucket scan request/response
@gnuhpc gnuhpc force-pushed the pr/3-rpc-messages-extended branch from ea7c1da to 9d2fbff Compare June 19, 2026 18:54
@gnuhpc

gnuhpc commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto updated #629 and proactively applied the same proto::*/Default cleanup that the maintainer requested on #629 (#629 (review)):

Domain types added (crates/fluss/src/metadata/):

  • kv_snapshot_lease.rs — KvSnapshotLeaseForTable + KvSnapshotLeaseForBucket
  • producer_offsets.rs — ProducerTableOffsets + BucketOffset
  • Extended table.rs — added to_pb/from_pb to existing TableBucket

Wrapper signatures: 3 wrappers that took raw proto::Pb* in new() now take domain types (acquire_kv_snapshot_lease, register_producer_offsets, release_kv_snapshot_lease).

#[derive(Default)] dropped on the 17 arg-taking wrappers in this PR (kept on get_cluster_health which is param-less, matching the pre-existing list_databases.rs precedent).

Not changed (yet): scan_kv.rs retains Vec<proto::PbScanReqForBucket> in its new() for now. The wrapper has no consumer on this branch (its caller KvBatchScanner lands in #633), and making it pub(crate) here triggers cargo clippy --all-targets -- -D warnings failures from dead-code lints. I'll address it on #633 when the consumer arrives — either by adding a ScanReqForBucket domain type or by making the wrapper crate-private once it actually has a use site.

cargo build + clippy (-D warnings) + 548 unit tests pass locally. Pushed 9d2fbff.

@fresh-borzoni

fresh-borzoni commented Jun 19, 2026

Copy link
Copy Markdown
Member

@gnuhpc Can you pls, rebase/cherry-pick on top of current main and fix proto visibility in a similar fashion as I did in previous PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fresh-borzoni fresh-borzoni fresh-borzoni left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@gnuhpc @fresh-borzoni @warmbupt