Skip to content

[rust] Add RPC message wrappers for core DB/table operations#629

Merged
fresh-borzoni merged 2 commits into
apache:mainfrom
gnuhpc:pr/2-rpc-messages-core
Jun 19, 2026
Merged

[rust] Add RPC message wrappers for core DB/table operations#629
fresh-borzoni merged 2 commits into
apache:mainfrom
gnuhpc:pr/2-rpc-messages-core

Conversation

@gnuhpc

@gnuhpc gnuhpc commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

What

Adds request/response message wrappers for the core 1.x DB/table admin RPCs: AlterClusterConfigs, AlterDatabase, AlterTable, CreateAcls, DescribeClusterConfigs, DropAcls, GetTableStats, ListAcls, ListDatabaseSummaries.

Each is a thin wrapper over the generated proto:: types (from #628) wired into rpc/message/mod.rs. The admin client methods that consume them land in a later PR.

Stack

Part 2/6 of the Fluss 1.x Rust support series, stacked on #628 (pr/1-proto-sync). Each branch in the stack compiles on top of its predecessor, so all PRs here target main; once #628 merges, this PR's diff automatically reduces to just the 9 message files unique to this branch.

🤖 Generated with Claude Code

@fresh-borzoni fresh-borzoni left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnuhpc Thank you for the PR, we need to address a couple of issues here.

We shouldn't use proto::* in any public FlussAdmin signature.
Please, define domain types mirroring Java's with conversion in the wrapper layer

inner_request: proto::GetTableStatsRequest {
table_id,
buckets_req,
target_columns: vec![],

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we hardcode target_columns: vec![]?
There's no way to ask for specific columns with this, is it intentional?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not intentional — fixed in 720d069. GetTableStatsRequest::new now takes target_columns: Vec<i32> and threads it through. Single existing call site (the integration test on #632) will pass an explicit vec![] to preserve current behavior. Same drop-silently-features pattern also fixed on #631 (alter_table was hardcoding vec![] for drop/rename/modify columns); will be in the next force-push on that PR.

use bytes::{Buf, BufMut};
use prost::Message;

#[derive(Debug, Default)]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we drop #[derive(Default)] on the arg-taking requests?

default() just builds empty(invalid requests), and the rest of the wrappers are #[derive(Debug)] only, so it's inconsistent

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 720d069. Dropped #[derive(Default)] on all 7 arg-taking wrappers added by this PR (alter_cluster_configs, alter_database, alter_table, create_acls, drop_acls, get_table_stats, list_acls).

Kept Default on the param-less ones (describe_cluster_configs, list_database_summaries) since they match the pre-existing list_databases.rs precedent and default() produces a valid request there — happy to drop those too if you'd prefer strict uniformity.

Same change applied to the 18 arg-taking wrappers on #630.

@gnuhpc gnuhpc force-pushed the pr/2-rpc-messages-core branch from 784f39d to 720d069 Compare June 19, 2026 17:43
@gnuhpc

gnuhpc commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the proto::*-in-public-API feedback in 720d069. Summary of what changed:

Domain types introduced under crates/fluss/src/metadata/ (mirroring Java with to_pb / from_pb conversion in the wrapper layer):

  • config.rsAlterConfig + AlterConfigOpType (Set/Delete/Append/Subtract)
  • acl.rsAclInfo / AclFilter + ResourceType (Any/Cluster/Database/Table), OperationType (Any/All/Read/Write/Create/Drop/Alter/Describe), PermissionType (Any/Allow)
  • table_change.rsAddColumn / DropColumn / RenameColumn / ModifyColumn + ColumnPositionType (currently Last only, matching Java's ColumnPositionType.java — proto reserves First=1, After=3 for future use)
  • table_stats.rsBucketStatsRequest

Enum int codes are taken verbatim from the Java source (AlterConfigOpType.java, ResourceType.java, OperationType.java, PermissionType.java, ColumnPositionType.java).

Wrapper changes: all 7 arg-taking wrappers' new() now accept domain types and call .to_pb() to populate inner_request. Wrappers still hold proto::* internally (matches the existing PartitionSpec/DatabaseDescriptor pattern).

Stack-wide sweep: since the same proto::*-leak class of issue exists on #630 and #631 (the FlussAdmin admin client surface in #631 was where most leaks lived), I'm restacking the rest of the chain with the same cleanup now — will post here once each PR is force-pushed.

Tests: added *_pb_roundtrip unit tests for each new domain type (matching the PartitionSpec/DatabaseDescriptor test style). cargo test -p fluss-rs --lib passes 544 (15 new).

Thanks for the review!

@gnuhpc gnuhpc force-pushed the pr/2-rpc-messages-core branch from 720d069 to a6bb958 Compare June 19, 2026 18:45
Add 9 RPC message wrapper types:
- alter_database, alter_table (DDL operations)
- get_table_stats (table statistics)
- list_database_summaries (database listing with summaries)
- create_acls, list_acls, drop_acls (ACL management)
- describe_cluster_configs, alter_cluster_configs (cluster configuration)

Each wrapper follows the standard pattern: a request struct wrapping the
proto-generated type, implementing RequestBody (tying to ApiKey and ResponseBody),
WriteType, and ReadType.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@fresh-borzoni fresh-borzoni force-pushed the pr/2-rpc-messages-core branch from a6bb958 to 3e2c989 Compare June 19, 2026 22:37

@fresh-borzoni fresh-borzoni left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnuhpc Thank you for the changes, LGTM
I pushed commit to tighten visibility for protos in your new code and a couple of pre-existing issues. Hope it's okay with you

@fresh-borzoni fresh-borzoni force-pushed the pr/2-rpc-messages-core branch from 3e2c989 to f0b606e Compare June 19, 2026 22:41
@fresh-borzoni fresh-borzoni force-pushed the pr/2-rpc-messages-core branch from f0b606e to 775ee07 Compare June 19, 2026 22:46
@fresh-borzoni fresh-borzoni merged commit 9d682e3 into apache:main Jun 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants