Cache l2 tables #31
Merged
Cache l2 tables #31
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AkihiroSuda
reviewed
Oct 15, 2024
AkihiroSuda
reviewed
Oct 15, 2024
AkihiroSuda
reviewed
Oct 15, 2024
AkihiroSuda
reviewed
Oct 15, 2024
6 tasks
AkihiroSuda
reviewed
Oct 17, 2024
AkihiroSuda
reviewed
Oct 17, 2024
Starting with Read benchmark, since testing lima show that all time is
spent on reading the image. The benchmark measures the time to open a
compressed and uncompressed qcow2 images without a backing file and read
the entire image. This matches the common use of this library in lima.
Example run:
% go test -bench Read
BenchmarkRead/qcow2-12 1 1957581083 ns/op 137.13 MB/s 4297616648 B/op 98474 allocs/op
BenchmarkRead/qcow2_zlib-12 1 5352411666 ns/op 50.15 MB/s 5034875288 B/op 278804 allocs/op
Signed-off-by: Nir Soffer <[email protected]>
This is a simple generic thread safe LRU cache with the minimum functionality for caching limited number of qcow2 L2 tables. Signed-off-by: Nir Soffer <[email protected]>
We read the L2 table from storage for reading every cluster. This
duplicates the number of I/Os. Now we read every L2 table exactly once
and cache it. With the default cluster size (64 KiB), images up to 8 GiB
will use 1 MiB for the L2 tables cache.
With this change reading entire image is 35.9 times faster for qcow2
image and 1.6 times faster for compressed qcow2 image. Testing
converting image to raw format as part of limactl create shows similar
improvement.
Before:
% go test -bench Read
BenchmarkRead/qcow2-12 1 1957581083 ns/op 137.13 MB/s 4297616648 B/op 98474 allocs/op
BenchmarkRead/qcow2_zlib-12 1 5352411666 ns/op 50.15 MB/s 5034875288 B/op 278804 allocs/op
% limactl create --tty=false qcow2.yaml
3.50 GiB / 3.50 GiB [-------------------------------------] 100.00% 407.48 MiB/s
% limactl create --tty=false qcow2.zlib.yaml
3.50 GiB / 3.50 GiB [-------------------------------------] 100.00% 218.10 MiB/s
After:
% go test -bench Read
BenchmarkRead/qcow2-12 21 54562403 ns/op 4919.79 MB/s 1183486 B/op 46 allocs/op
BenchmarkRead/qcow2_zlib-12 1 3452944500 ns/op 77.74 MB/s 736156816 B/op 178774 allocs/op
% limactl create --tty=false qcow2.yaml
3.50 GiB / 3.50 GiB [-------------------------------------] 100.00% 4.49 GiB/s
% limactl create --tty=false qcow2-zlib.yaml
3.50 GiB / 3.50 GiB [-------------------------------------] 100.00% 330.29 MiB/s
Signed-off-by: Nir Soffer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We read the L2 table from storage for reading every cluster. This
duplicates the number of I/Os. Now we read every L2 table exactly once
and cache it. With the default cluster size (64 KiB), images up to 8 GiB
will use 1 MiB for the L2 tables cache.
With this change reading entire image is 35.9 times faster for qcow2
image and 1.6 times faster for compressed qcow2 image. Testing
converting image to raw format as part of limactl create shows similar
improvement.
Before:
After:
Part-of #32