HDDS-15616. ListBuckets API ignores MaxBuckets parameter and returns all buckets

[Gargi-jais11]

What changes were proposed in this pull request?

S3 test test_list_buckets_paginated is failing today as ListBuckets pagination using MaxBuckets and ContinuationToken on GET / (ListAllMyBuckets) is not supported.

creates 2 buckets and calls list_buckets(MaxBuckets=1).

Expected: 1 bucket per page + ContinuationToken when more exist.
Ozone: Ignores MaxBuckets and returns both buckets → assert len == 1 fails with 2.

Simple example:

Buckets: [bucket-A, bucket-B]
Client: list_buckets(MaxBuckets=1) 

AWS: { Buckets: [bucket-A], ContinuationToken: "..." }
Ozone: { Buckets: [bucket-A, bucket-B] } ← 2 items, no token 

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15616

How was this patch tested?

Added UT and IT.
Also tested manually.
Before Fix:

// no continuation token is given and send all the buckets stored.
bash-5.1$ aws s3api list-buckets --max-buckets 1 --endpoint-url http://s3g:9878 --output json
{
    "Buckets": [
        {
            "Name": "buck1",
            "CreationDate": "2026-06-30T07:04:27.544Z"
        },
        {
            "Name": "buck2",              -----------------------------> wrong behaviour
            "CreationDate": "2026-06-30T07:04:30.460Z"
        },
        {
            "Name": "buck3",       -----------------------------> wrong behaviour
            "CreationDate": "2026-06-30T07:04:35.556Z"
        },
        {
            "Name": "buck4",           -----------------------------> wrong behaviour
            "CreationDate": "2026-06-30T07:04:40.205Z"
        },
        {
            "Name": "buck5",        -----------------------------> wrong behaviour
            "CreationDate": "2026-06-30T07:04:43.890Z"
        }
    ],
    "Owner": {
        "DisplayName": "om",
        "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
    }
}

After Fix:

bash-5.1$ aws s3api list-buckets --max-buckets 1 --endpoint-url http://s3g:9878/ --output json
{
    "Buckets": [
        {
            "Name": "buck1",
            "CreationDate": "2026-06-23T08:51:13.848Z"
        }
    ],
    "Owner": {
        "DisplayName": "om",
        "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
    },
    "ContinuationToken": "000000056275636b31-f239795a4c8be1e364face563bdc554b5559cbc1978becf3086b5079c2c44850"
}

bash-5.1$ aws s3api list-buckets --max-buckets 2 --endpoint-url http://s3g:9878/ --output json
{
    "Buckets": [
        {
            "Name": "buck1",
            "CreationDate": "2026-06-23T08:51:13.848Z"
        },
        {
            "Name": "buck2",
            "CreationDate": "2026-06-23T08:51:16.495Z"
        }
    ],
    "Owner": {
        "DisplayName": "om",
        "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
    }
}

bash-5.1$ ozone sh bucket create /s3v/buck3
bash-5.1$ aws s3api list-buckets --max-buckets 2 --endpoint-url http://s3g:9878/ --output json
{
    "Buckets": [
        {
            "Name": "buck1",
            "CreationDate": "2026-06-23T08:51:13.848Z"
        },
        {
            "Name": "buck2",
            "CreationDate": "2026-06-23T08:51:16.495Z"
        }
    ],
    "Owner": {
        "DisplayName": "om",
        "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
    },
    "ContinuationToken": "000000056275636b32-0880ee15253ee89c74cbc13b73c026a3ecd9529d0682e4d61e85dde392eb3ed3"
}

bash-5.1$ aws s3api list-buckets --max-buckets 1 --continuation-token 000000056275636b32-0880ee15253ee89c74cbc13b73c026a3ecd9529d0682e4d61e85dde392eb3ed3 \
    --endpoint-url http://s3g:9878/ --output json
{
    "Buckets": [
        {
            "Name": "buck3",
            "CreationDate": "2026-06-23T08:52:25.003Z"
        }
    ],
    "Owner": {
        "DisplayName": "om",
        "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
    }
}
bash-5.1$ aws s3api list-buckets --max-buckets 2 --endpoint-url http://s3g:9878/
{
    "Buckets": [
        {
            "Name": "buck1",
            "CreationDate": "2026-06-23T08:51:13.848Z"
        },
        {
            "Name": "buck2",
            "CreationDate": "2026-06-23T08:51:16.495Z"
        }
    ],
    "Owner": {
        "DisplayName": "om",
        "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
    },
    "ContinuationToken": "000000056275636b32-0880ee15253ee89c74cbc13b73c026a3ecd9529d0682e4d61e85dde392eb3ed3"
}