Skip to content

Trace2:Gvfs:Experiment: read-cache: trace threading and cache-tree extension #158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 10, 2019
Merged

Trace2:Gvfs:Experiment: read-cache: trace threading and cache-tree extension #158

merged 2 commits into from
Jul 10, 2019

Conversation

@jeffhostetler
Copy link

This series adds per-thread tracing to the do_read_index() and adds regions around the cache-tree extension. The intent is to show / confirm that index read time is now bounded by the cache-tree
extension (now that we have Ben's multi-threading changes (both the N-1 cache-entry and
the single extension worker threads)).

@jeffhostetler
trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache
99d2703 
Add trace2_thread_start() and trace2_thread_exit() events to the worker
threads used to read the index.  This gives per-thread perf data.

These workers were introduced in:
abb4bb8 read-cache: load cache extensions on a worker thread
77ff112 read-cache: load cache entries on worker threads

Signed-off-by: Jeff Hostetler <[email protected]>
@jeffhostetler
trace2:gvfs:experiment: read-cache: time read/write of cache-tree ext…
4e28740 
…ension

Add regions around code to read and write the cache-tree extension
when the index is read or written.

This is an experiment and may be dropped in future releases if
we don't need it anymore.

This experiment demonstrates that it takes more time to parse and
deserialize the cache-tree extension than it does to read the
cache-entries.

Commits [1] and [2] spreads cache-entry reading across N-1 cores
and dedicates a single core to simultaneously read the index extensions.

Local testing (on my machine) shows that reading the cache-tree extension
takes ~0.28 seconds.  The 11 cache-entry threads take ~0.08 seconds.
The main thread is blocked for 0.15 to 0.20 seconds waiting for the
extension thread to finish.

Let's use this commit to gather some telemetry and confirm this.

My point is that improvements, such as index V5 which makes the
cache entries smaller, may improve performance, but the gains may
be limited because of this extension.  And that we may need to
look inside the cache-tree extension to truly improve do_read_index()
performance.

[1] abb4bb8 read-cache: load cache extensions on a worker thread
[2] 77ff112 read-cache: load cache entries on worker threads

Signed-off-by: Jeff Hostetler <[email protected]>
kewillford
kewillford approved these changes Jul 9, 2019
View reviewed changes
Copy link
Member

@kewillford kewillford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@jeffhostetler jeffhostetler merged commit 87a148b into microsoft:vfs-2.22.0 Jul 10, 2019
@derrickstolee derrickstolee mentioned this pull request Jul 22, 2019
Merged
derrickstolee added a commit to derrickstolee/VFSForGit that referenced this pull request Jul 24, 2019
@derrickstolee
Update Git to include tracing updates
bd08590 
Includes changes from:

* microsoft/git#158
* microsoft/git#159
* microsoft/git#160
* microsoft/git#162

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee added a commit to microsoft/VFSForGit that referenced this pull request Jul 24, 2019
@derrickstolee
Merge pull request #1342: Update Git to include tracing updates
43b98d1 
Includes changes from:

* microsoft/git#158 (experimental tracing: trace threading and cache-tree extension)
* microsoft/git#159 (experimental tracing: prime_cache_tree())
* microsoft/git#160 (experimental tracing: checkout and reset perf)
* microsoft/git#162 (support building with GCC v9.x)
derrickstolee pushed a commit that referenced this pull request Jun 1, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
10ece56 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Jul 20, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
d65d8e8 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Jul 27, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
716ccc7 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Jul 28, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
833d2df 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Oct 6, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
630bf9f 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Oct 6, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
5c6a780 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Oct 6, 2020
@jeffhostetler @dscho
Merge trace2 experimental regions
1e748e8 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Oct 9, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
8a4ae8f 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Oct 9, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
e901c94 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Oct 16, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
a0b8f63 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
derrickstolee pushed a commit that referenced this pull request Oct 19, 2020
@jeffhostetler @derrickstolee
Merge trace2 experimental regions
3f8df51 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
mjcheetham pushed a commit that referenced this pull request Dec 15, 2020
@jeffhostetler @mjcheetham
Merge trace2 experimental regions
40a602e 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
mjcheetham pushed a commit that referenced this pull request Dec 15, 2020
@jeffhostetler @mjcheetham
Merge trace2 experimental regions
28f9345 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Dec 18, 2020
@jeffhostetler @dscho
Merge trace2 experimental regions
2df6b9f 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Dec 24, 2020
@jeffhostetler @dscho
Merge trace2 experimental regions
2eadc98 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Dec 28, 2020
@jeffhostetler @dscho
Merge trace2 experimental regions
b2fc1c3 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Mar 4, 2021
@jeffhostetler @dscho
Merge trace2 experimental regions
6fd8f4b 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Sep 18, 2024
@jeffhostetler @dscho
Merge trace2 experimental regions
29ec6f4 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Sep 24, 2024
@jeffhostetler @dscho
Merge trace2 experimental regions
372d8e1 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Oct 8, 2024
@jeffhostetler @dscho
Merge trace2 experimental regions
95af07b 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
mjcheetham pushed a commit that referenced this pull request Dec 3, 2024
@jeffhostetler @mjcheetham
Merge trace2 experimental regions
abd49af 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Dec 17, 2024
@jeffhostetler @dscho
Merge trace2 experimental regions
df778d9 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Dec 18, 2024
@jeffhostetler @dscho
Merge trace2 experimental regions
658e328 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jan 1, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
81e582e 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jan 1, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
fcc85f8 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jan 1, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
98bfe82 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Feb 10, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
a25cba0 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Feb 27, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
2cade4b 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Mar 5, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
50f90d8 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
mjcheetham pushed a commit that referenced this pull request Mar 12, 2025
@jeffhostetler @mjcheetham
Merge trace2 experimental regions
04bfc3c 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
mjcheetham pushed a commit that referenced this pull request Mar 17, 2025
@jeffhostetler @mjcheetham
Merge trace2 experimental regions
c27058b 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jun 6, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
e0a2911 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jun 11, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
a08b098 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jun 13, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
cae4936 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jun 16, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
161573a 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jun 16, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
9bc5517 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jun 16, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
307f037 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Jul 8, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
f717255 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Aug 5, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
9eae5ca 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Aug 5, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
7e4abde 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Aug 8, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
a171e92 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Aug 8, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
89f644a 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Aug 13, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
e6216b9 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Aug 19, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
ca20ad2 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Oct 17, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
343d7c7 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Oct 17, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
d2a99aa 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
dscho pushed a commit that referenced this pull request Oct 28, 2025
@jeffhostetler @dscho
Merge trace2 experimental regions
bb327a9 
Includes gvfs-specific commits from these pull requests:

	#158
	#159
	#160
	#164

Signed-off-by: Derrick Stolee <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dscho dscho dscho approved these changes

@derrickstolee derrickstolee Awaiting requested review from derrickstolee

+1 more reviewer

@kewillford kewillford kewillford approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jeffhostetler @dscho @kewillford