Skip to content

hash/crc32: commit 18dbe5b broke kubernetes unit tests #74767

@dims

Description

@dims

Go version

go version go1.26-devel_6fbad4be75

Output of go env in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/dsrinivas/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/dsrinivas/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3560074278=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/home/dsrinivas/junk/go.mod'
GOMODCACHE='/home/dsrinivas/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/dsrinivas/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/dsrinivas/golang-go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/dsrinivas/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/dsrinivas/golang-go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.26-devel_6fbad4be75 Fri Jul 25 17:43:10 2025 -0700'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

longer context - kubernetes/kubernetes#133224

Kubernetes runs a set of CI jobs, one of which uses the golang tip with unit tests. Since 18dbe5b landed, the CI job is broken.

This commit is titled add AVX512 IEEE CRC32 calculation and we can recreate the problem outside the CI environment in a GCP vm of type c4d-highmem-8-lssd

What did you see happen?

Here's the smallest repro i have so far

the test code is here:

package main

import (
        "bytes"
        "compress/gzip"
        "io"
        "testing"
)

const (
        defaultGzipThresholdBytes       = 128 * 1024
        defaultGzipContentEncodingLevel = gzip.DefaultCompression
)

func gzipContent(data []byte, level int) []byte {
        buf := &bytes.Buffer{}
        gw, err := gzip.NewWriterLevel(buf, level)
        if err != nil {
                panic(err)
        }
        if _, err := gw.Write(data); err != nil {
                panic(err)
        }
        if err := gw.Close(); err != nil {
                panic(err)
        }
        return buf.Bytes()
}

func gunzipContent(data []byte) ([]byte, error) {
        gr, err := gzip.NewReader(bytes.NewReader(data))
        if err != nil {
                return nil, err
        }
        defer gr.Close()
        return io.ReadAll(gr)
}

func TestGzipLargePayload(t *testing.T) {
        // Create a large payload that's slightly bigger than the threshold
        largePayload := bytes.Repeat([]byte("0123456789abcdef"), defaultGzipThresholdBytes/16+1)

        // Original content with a prefix
        originalContent := []byte(": " + string(largePayload))

        // Compress the content
        compressed := gzipContent(originalContent, defaultGzipContentEncodingLevel)

        // Decompress the content
        decompressed, err := gunzipContent(compressed)
        if err != nil {
                t.Fatalf("Failed to decompress: %v", err)
        }

        // Compare the decompressed content with the original
        if !bytes.Equal(decompressed, originalContent) {
                t.Error("Decompressed content does not match original")
        }

        // Optional: Log some stats
        t.Logf("Original size: %d bytes", len(originalContent))
        t.Logf("Compressed size: %d bytes", len(compressed))
        t.Logf("Compression ratio: %.2f%%", float64(len(compressed))/float64(len(originalContent))*100)
}

Running go test -v against the above test works fine. BUT we add -race as well in the CI jobs, so go test -v -race will fail.

dsrinivas@instance-20250726-165248:~/junk$ ../golang-go/bin/go version
go version go1.26-devel_6fbad4be75 Fri Jul 25 17:43:10 2025 -0700 linux/amd64
dsrinivas@instance-20250726-165248:~/junk$ ../golang-go/bin/go test -v
=== RUN   TestGzipLargePayload
    gzipbr0ke_test.go:61: Original size: 131090 bytes
    gzipbr0ke_test.go:62: Compressed size: 312 bytes
    gzipbr0ke_test.go:63: Compression ratio: 0.24%
--- PASS: TestGzipLargePayload (0.00s)
PASS
ok      example.com/m   0.002s
dsrinivas@instance-20250726-165248:~/junk$ ../golang-go/bin/go test -v -race
=== RUN   TestGzipLargePayload
    gzipbr0ke_test.go:52: Failed to decompress: gzip: invalid checksum
--- FAIL: TestGzipLargePayload (0.01s)
FAIL
exit status 1
FAIL    example.com/m   0.015s

What did you expect to see?

expect TestGzipLargePayload to pass. Reverting that one single commit works fine for me.

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugReportIssues describing a possible bug in the Go implementation.CriticalA critical problem that affects the availability or correctness of production systems built using Go

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions