Skip to content

Support MPI distributed arrays where some processes have no data#26

Merged
tskisner merged 3 commits into
mainfrom
proc_nodata
Jun 30, 2026
Merged

Support MPI distributed arrays where some processes have no data#26
tskisner merged 3 commits into
mainfrom
proc_nodata

Conversation

@tskisner

Copy link
Copy Markdown
Member

When working with large arrays distributed across a fixed-size MPI communicator, it is sometimes necessary to handle the case where a sliced / reduced array results in no local data on some processes. These changes accomodate that situation by allowing construction of FlacArray instances where some processes have zero streams. Routines for I/O to HDF5 and Zarr are also updated for the case where some processes have no local data.

When working with large arrays distributed across a fixed-size
MPI communicator, it is sometimes necessary to handle the case
where a sliced / reduced array results in no local data on some
processes.  These changes accomodate that situation by allowing
construction of FlacArray instances where some processes have
zero streams.  Routines for I/O to HDF5 and Zarr are also updated
for the case where some processes have no local data.
@tskisner tskisner requested a review from keskitalo June 29, 2026 18:12

@keskitalo keskitalo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems uncontroversial. Thanks for adding support for empty local dataset.

Comment thread src/flacarray/array.py Outdated
Comment thread src/flacarray/decompress.py
Comment thread src/flacarray/hdf5.py
@tskisner tskisner merged commit e114176 into main Jun 30, 2026
13 of 14 checks passed
@tskisner tskisner deleted the proc_nodata branch June 30, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants