-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rechunk group to group #79
Comments
Hi @valpesendorfer! 👋 Thanks for this interesting issue. You're correct: right now, we only support Zarr arrays or flat groups (no groups within group). The challenge here is that Zarr support infinitely deep nesting of groups. It's hard for me to think of the right behavior. So I have a question for you. What API syntax (call to |
👋 and may I add, thanks for all your work. First, good to know I'm not missing something essential, I've started with zarrs only recently. To answer your Q, I don't know exactly ... But ideally, the manual work I need to do in iterating over groups would be taken care of by In this specific case, I don't want to do anything fancy, meaning I want to keep the same structure, just with the So to specify target_chunks = {
'band': (256, 256, 5),
} This could be extended to each group that has an array Or, more verbosely, using a nested dictionary with a group : array : chunk syntax, something like target_chunks = {
"/full/path/to/group1": {
"band": (256, 256, 5),
},
"/full/path/to/group2": {
"band": (256, 256, 5),
}
} Which could be generated dynamically: zarr_raw = zarr.open("raw.zarr", mode="r")
target_chunks = { group: {"band": (256, 256, 5)} for group in zarr_raw.group_keys()} EditOk, I see now the code above doesn't work for nested groups ... that would require something smarter. I've only worked with groups as exemplified above, and I don't think there's any reason for me to go deeper. |
Hello,
I'm experimenting with multi-group zarrs, where each group represents a separate tile. The tile themselves are structured the same way (same dimensions, variables, etc) and are created with
xarray
.Here's a small example of how it can look like:
I was wondering if there's a way to either
band
array of each group into a new zarr that has the same structure, orThe only way I was successful in rechunking the groups was in iterating over each one, and running
rechunk
individually with the group path (i.e.target.zarr/group
) set astarget_store
.Thanks
The text was updated successfully, but these errors were encountered: