Git Fetch fails for repos with too many branches with v2.51.0

[tarunramsinghani]

Existing issues matching what you're seeing

• I was not able to find an open or closed issue matching what I'm seeing

Git for Windows version

git version 2.51.0.windows.1
cpu: x86_64
built from commit: 4d21a77b98af5cf479d8b6f863c2aa94257cd4e1
sizeof-long: 4
sizeof-size_t: 8
shell-path: D:/git-sdk-64-build-installers/usr/bin/sh
feature: fsmonitor--daemon
libcurl: 8.15.0
OpenSSL: OpenSSL 3.2.4 11 Feb 2025
zlib: 1.3.1
SHA-1: SHA1_DC
SHA-256: SHA256_BLK
default-ref-format: files
default-hash: sha1

Windows version

Windows 11

Windows CPU architecture

x86_64 (64-bit)

Additional Windows version information

Microsoft Windows [Version 10.0.26100.4946]

Options set during installation

using the portable version of git but it can be reproed in any manner

Other interesting things

None

Terminal/shell

git-cmd.exe

Commands that trigger the issue

git init
git remote add origin https://github.com/microsoft/azure-pipelines-tasks.git
git fetch

// Does not repros with packaed-refs in the repo hence initialize a new repo.

Expected behaviour

Git fetch should succeed without any errors. It succeeded with v2.50.1

Actual behaviour

Get error as below

 * [new branch]            users/zqian/update-npmversion -> origin/users/zqian/update-npmversion
 * [new branch]            users/zqian/upgrade-helm3.7 -> origin/users/zqian/upgrade-helm3.7
error: cannot lock ref 'refs/remotes/origin/users/ajya/sqlservermoduleimportM146': Unable to create 'C:/Repos/taskstest1/.git/refs/remotes/origin/users/ajya/sqlservermoduleimportM146.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

C:\Repos\taskstest1>git --version
git version 2.51.0.windows.1

Repository

Multiple repoes, Example provided above
https://github.com/microsoft/azure-pipelines-tasks.git

[rimrul]

The root cause seems to not be "many branches", but case collisions on case-insensitive filesystems, when using the "files" ref backend.

This is probably the same issue as https://lore.kernel.org/git/829c6e09-c834-461f-820d-981512a02a76@app.fastmail.com/T/#mcb563796beabd96fdcdc00e2f94636ca633b0557 and https://lore.kernel.org/git/aLbNIRRn_Mm1IKp9@pks.im/T/#t

You can probably work around this by using git init --ref-format=reftable
or git clone --ref-format=reftable.
A patch series that should fix this in future versions has been submitted to the mailing list.

[dscho]

git init
git remote add origin https://github.com/microsoft/azure-pipelines-tasks.git
git fetch

I can reproduce!

However, this is far from a minimal example. It downloaded almost 700 megabyte, which took so long that I cannot justify working on this more.

The next steps would be to reduce this to a script that first creates a bare repository with many refs (but only few, if any, blobs) and then fetches that into another bare repository. The easiest to accomplish that would probably to generate input for fast-import that generates a configurable number of refs, each of which has a distinct, single commit that has the empty tree as its root tree (i.e. no files). To see how that input looks like, you could create a new worktree with a branch that has no commits yet git worktree add --orphan, and then commit a change with git commit --allow-empty, and then look at the output of git fast-export of that branch.

Again, I cannot justify spending too much time on this, so if the task of making this easily reproducible is left to me, I cannot work on this at all.

[tarunramsinghani]

@dscho the download happens the first time, but the git fetch will keep failing on the same repo over and over again and init and download is not required each time. I will try to see if I can have repro outside of this repo tomorrow.

[mofreh345]

اطفاء شبكة مربوطه في الراس في الثلاثاء، ٢ سبتمبر ٢٠٢٥، ٧:٠٩ م Tarun Ramsinghani < ***@***.***> كتب:

…

*tarunramsinghani* left a comment (git-for-windows/git#5804) <#5804 (comment)> @dscho <https://github.com/dscho> the download happens the first time, but the git fetch will keep failing on the same repo over and over again and init and download is not required each time. I will try to see if I can have repro outside of this repo tomorrow. — Reply to this email directly, view it on GitHub <#5804 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BQAQWQU4F5INHISUJFAMGND3QW6MBAVCNFSM6AAAAACFNSGNWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENBVHE3DEOBSHA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[rimrul]

This is probably the same issue as https://lore.kernel.org/git/829c6e09-c834-461f-820d-981512a02a76@app.fastmail.com/T/#mcb563796beabd96fdcdc00e2f94636ca633b0557

I now realise, that that email is you reporting the same issue on macOS.

[dscho]

@dscho the download happens the first time, but the git fetch will keep failing on the same repo over and over again and init and download is not required each time. I will try to see if I can have repro outside of this repo tomorrow.

@tarunramsinghani have you tried @rimrul 's suggestion yet?

[tarunramsinghani]

@dscho Yes tried the git init --ref-format=reftable and it seems to resolve the problem, now I just need to see if this is suported for older versions as well before adopting it as our prodcut supports customer using older versions of git as well.

[rimrul]

I just need to see if this is suported for older versions as well

It's available since Version 2.45.0.

[dscho]

I just need to see if this is suported for older versions as well

It's available since Version 2.45.0.

Well, to be fair, the manual page says:

--ref-format=<format>:
Specify the given ref storage <format> for the repository. The valid values are:

• files for loose files with packed-refs. This is the default.
• reftable for the reftable format. This format is experimental and its internals are subject to change.

That "experimental" label is probably not really confidence-inducing.

At the same time, that format is based on JGit's reftable support, therefore the format itself is highly unlikely to change at this point. There have been some recent performance fixes, though, and therefore the "experimental" label might still be merited (for the implementation, though, not for the format itself).

In any case, it looks as if there are movements on the Git mailing list towards a fix.

[hlms]

The root cause seems to not be "many branches", but case collisions on case-insensitive filesystems, when using the "files" ref backend.

This is probably the same issue as https://lore.kernel.org/git/829c6e09-c834-461f-820d-981512a02a76@app.fastmail.com/T/#mcb563796beabd96fdcdc00e2f94636ca633b0557 and https://lore.kernel.org/git/aLbNIRRn_Mm1IKp9@pks.im/T/#t

You can probably work around this by using git init --ref-format=reftable or git clone --ref-format=reftable. A patch series that should fix this in future versions has been submitted to the mailing list.

Yes for the collision on case-insensitive filesytem.

To reproduce the issue, need to create two branches on remote having same name and different case: E.g. ABC-1 and abc-1

And then git pull will fail in Windows machine:

error: cannot lock ref 'refs/remotes/origin/ABC-1': Unable to create 'C:/Repo/.git/refs/remotes/origin/ABC-1.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

[dscho]

@hlms thank you for chiming in. At this point in time, I think the problem is well understood, and it is also well established that this is not a Windows-specific problem, but a general problem Git has with case-insensitive filesystems (probably because the initial as well as the current maintainer dismiss those filesystems, having a choice range of swearwords to characterize the design).

The best way forward would probably be to try to help out over in https://lore.kernel.org/git/20250902-587-git-fetch-1-fails-fetches-on-case-insensitive-repositories-v1-0-35e69bbb507d@gmail.com/#r (reply as instructed at the bottom of the page, in particular paying attention to force the email to be plaintext only, to avoid having the email outright rejected). I am sure that reviews, suggestions for improvements, especially for the test case added in https://lore.kernel.org/git/20250902-587-git-fetch-1-fails-fetches-on-case-insensitive-repositories-v1-1-35e69bbb507d@gmail.com/, would be welcome.

I am not sure how well it would be received to mention that this is kind of a pressing matter, seeing as the usual tangential side conversations are on display in that thread, the core Git contributors seem to kinda like their ways.