GitError Class:Zlib on cloning General

[c42f]

I've been fairly frequently getting a mysterious GitError(Code:ERROR, Class:Zlib, error inflating zlib stream) on julia-nightly while using julia-buildpkg.

I think this might possibly only occur on Juila-nightly. Example:

https://github.com/JuliaArrays/StaticArrays.jl/pull/840/checks?check_run_id=1272952752

Run julia-actions/julia-buildpkg@latest
     Cloning registry from "https://github.com/JuliaRegistries/General.git"
25l    Fetching: [>                                        ]  0.0 %
    Fetching: [==========>                              ]  22.5 %
    Fetching: [================>                        ]  38.2 %
    Fetching: [=========================>               ]  60.5 %
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Zlib, error inflating zlib stream)
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:52
  [2] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.GitTools /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/GitTools.jl:153
  [3] (::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:998
  [4] mktempdir(fn::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:711
  [5] mktempdir
    @ ./file.jl:709 [inlined]
  [6] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:978
  [7] clone_or_cp_registries
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:971 [inlined]
  [8] #add#6
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:32 [inlined]
  [9] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:28 [inlined]
 [10] #add#5
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [11] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [12] add(regs::Vector{String}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25
 [13] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25 [inlined]
 [14] #add#1
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24 [inlined]
 [15] add(reg::String)
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24
 [16] top-level scope
    @ none:1

caused by: GitError(Code:ERROR, Class:Zlib, error inflating zlib stream)
Stacktrace:
  [1] macro expansion
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/LibGit2/src/error.jl:110 [inlined]
  [2] clone(repo_url::SubString{String}, repo_path::String, clone_opts::LibGit2.CloneOptions)
    @ LibGit2 /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/LibGit2/src/repository.jl:459
  [3] clone(repo_url::SubString{String}, repo_path::String; branch::String, isbare::Bool, remote_cb::Ptr{Nothing}, credentials::LibGit2.CachedCredentials, callbacks::Dict{Symbol, Tuple{Ptr{Nothing}, Any}})
    @ LibGit2 /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/LibGit2/src/LibGit2.jl:580
  [4] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.GitTools /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/GitTools.jl:143
  [5] (::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:998
  [6] mktempdir(fn::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:711
  [7] mktempdir
    @ ./file.jl:709 [inlined]
  [8] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:978
  [9] clone_or_cp_registries
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:971 [inlined]
 [10] #add#6
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:32 [inlined]
 [11] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:28 [inlined]
 [12] #add#5
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [13] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [14] add(regs::Vector{String}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25
 [15] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25 [inlined]
 [16] #add#1
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24 [inlined]
 [17] add(reg::String)
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24
 [18] top-level scope
    @ none:1
25h
Error: Process completed with exit code 1.

[c42f]

May possibly be related to the fact that we're using libgit2 for this

#6

And julia nightly was upgraded to use libgit2-1.0.0

JuliaLang/julia#35233

which has a bug which also bit the rust people and seems like it could be similar

rust-lang/cargo#8258

I tried to reproduce outside of julia-buildpkg, but haven't had any luck so far. Here's the script I'm using:

#!/bin/bash

export JULIA_PKG_SERVER=""
export JULIA_DEPOT_PATH=$(mktemp -d)
trap "rm -Rf $JULIA_DEPOT_PATH" EXIT

julia --color=yes -e 'using Pkg; Pkg.Registry.add("General")'

[c42f]

By the way, I've seen this bug in nightly CI for more than one package.

[SaschaMann]

Could you rerun the build and see if it still happens?

I asked on Slack and Elliot said it may have been fixed in libgit2 1.0.1, which Julia nightly should be using now.

[c42f]

Rerunning it appears this may have been fixed. I'll reopen if I see it again.

[c42f]

I've just seen this error again:

https://github.com/JuliaArrays/StaticArrays.jl/pull/848/checks?check_run_id=1351636719

[SaschaMann]

I have no idea what causes this, sorry. Perhaps open an issue on Pkg and see if they have an idea?

[DilumAluthge]

It seems that when this happens, it is intermittent. So I think something like #13 should fix this.

[DilumAluthge]

Also, even if cloning General fails, we should still continue the build. This is what #13 does.