Unified storage

[cgwalters]

Status quo

See the filesystem docs.

Goals

• Direct support for zstd:chunked
• Be able to trivially and efficiently podman run <booted image>
• Also be able to use that image for booting local builds (but note this one relates to reflink by default (if possible) when copying between containers-storage: instances containers/container-libs#144 )

Proposal 1 (Phase 0): optionally pull first into c/storage

• Pull the image using podstorage which is podman/skopeo aka container-libs (note this REALLY wants reflink by default (if possible) when copying between containers-storage: instances containers/container-libs#144)
• Then if using ostree, pull from that storage into ostree via containers-storage: transport
• Then if using composefs-rs, pull from that storage

User experience

• For the upgrade/switch path, we will check if the image is already present in the bootc-owned container storage. If so, we will use the unified path. To onboard to this feature, use the command: bootc image set-unified - this just re-pulls (from the original source, usually registry) the currently booted container image but this time into the bootc owned storage.

• For the install path, a new flag such as --experimental-unified-storage will enable the feature during installation.

This will start out as an opt-in experimental mode. We initially considered adding a configuration file or option to enable the mode (possibly within the origin file), but decided against it for now. Instead, we are opting for automatic detection during switch/upgrade and a flag for installation. This approach will be revisited once the feature becomes more robust and stable.

Basically our initial fetch we pull the base OS image via just directly doing a podman pull or skopeo copy the same way we pull LBIs.

From there, we do a reflink/hardlink copy into the ostree store (i.e. This would duplicate metadata, but not data on disk. Reflinks should always work. If we have to do hardlinks it could potentially confuse the st_nlink count if any LBIs happen to share layers. But that seems unlikely.

We'll also need to teach the garbage collection code about this.

Incremental transition

Probably the best way to do this is to change things so on upgrade, we only pull layers into the LBI storage if they didn't already exist in the ostree store or so.

More info

Today this project uses ostree-containers for the base image.

Much more recently, we landed logically bound app images which ended up requiring a bootc-owned containers/storage instance.

This left us in a confusing situation where we have two image storage backends used by bootc - one for the booted host, and one for LBIs.

The code they don't share is mostly containers/storage.

This issue tracks unification: Having bootc leverage more of containers/storage (which is going to likely require containers/storage improvements.

This issue also intersects #1016 in that we would need to get progress out of podman pulls

Note: I did a draft PR for this a while ago #215

Update 20250632: See also #1190 for transitioning to a "native composefs" storage.

Random notes

• https://issues.redhat.com/browse/RHEL-75827 is why we don't have the AIS enabled by default

[jlebon]

Would containers-storage be another potential backend? Some considerations there of course (mixing images of different trust level and duplication) but it seems natural to have the OS container go there just like any other image so then you'd manage it with podman as usual.

[cgwalters]

Potentially. That said, it was an intentional decision so far to separate application container storage from the OS. This ties into larger architectural decisions taken. Specifically today, we tend to view containers as optional. For example, we'd need to ensure things like podman image prune -a don't just blow away your OS too. I could imagine the OS being represented as an always-running container or something internally? But it needs careful design and thought.

[cheese]

How about a transactional-update/BTRFS backend? It is straightforward and naturally supports apply-live.

[vrothberg]

@cgwalters, do you want to track this separately from the podman backend?

[cgwalters]

@cgwalters, do you want to track this separately from the podman backend?

I've reworked this issue to now be about unified storage.

How about a transactional-update/BTRFS backend? It is straightforward and naturally supports apply-live.

Long story short, the goal here is to make composefs the technological heart of things. Depending on BTRFS specifically doesn't give us a lot of value versus what we already have, augmented with composefs. "Live apply" type things are related but distinct.

[cgwalters]

In the short term one thing we could do is basically add an opt-in flag (part of the origin? bootc.conf? something) that changes the flow for fetching the base image to:

• pull into bootc-owned c/storage (where LBIs go today)
• pull from that via ostree-container path into the ostree repo (this will suck, it will be a full physical copy for now) via the containers-storage: transport
• Then we optimize that

[cgwalters]

pull from that via ostree-container path into the ostree repo (this will suck, it will be a full physical copy for now) via the containers-storage: transport

Was thinking about this more and honestly we can probably rely on bypassing all the containers/storage/overlay abstractions as is done in https://github.com/containers/bootc/pull/215/files#diff-d61a00deb019ae32a7f695e1323119424ce1a268fb34d0639f3e74fb2ed78fedR244 - especially for a storage version we "own".

[cgwalters]

Just to cross-reference a few things, I also have been working in the background on https://github.com/cgwalters/composefs-oci which is basically a "clean slate experimental playground" looking at "what might it look like if we made composefs the heart of things" - which it basically isn't today for ostree or c/storage (it's kind of awkwardly glued on the side to both).

In reality there's a lot to do to productize that, and the most practical short term will likely continue to be improving c/storage and strategically "hollowing out" ostree where we can; xref ostreedev/ostree#3291