Illegal instruction using openblas 0.3.13+4 on 1.7-dev

[sethaxen]

After JuliaLang/julia#40343 I am unable to call certain LinearAlgebra functions anymore. An example:

julia> using LinearAlgebra

julia> A = randn(4, 4)
4×4 Matrix{Float64}:
  1.19616   -0.832824  -0.779283   2.62967
 -0.520257  -1.05399    0.282232  -0.0441448
 -1.38003   -0.580062  -2.14204    0.224513
 -0.837389   0.238154  -0.107394  -2.57343

julia> eigen(A)

signal (4): Illegal instruction: 4
in expression starting at REPL[3]:1
dlapy2_64_ at /Users/saxen/projects/julia/usr/lib/libopenblas64_.0.3.13.dylib (unknown line)
Allocations: 2231132 (Pool: 2230671; Big: 461); GC: 3
Illegal instruction: 4

If I check out the previous commit on master and build Julia, everything works fine.

I tried to generate an RR bug report but got the error:

ERROR: RR not available on this platform

My version info:

julia> versioninfo()
Julia Version 1.7.0-DEV.845
Commit 58fba2b356 (2021-04-04 23:29 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, ivybridge)
Environment:
  JULIA_EDITOR = code

[pablosanjose]

Uhm, looks bad. Unfortunately I cannot confirm on my machine (also a mac). I get no error with latest nightly.

julia> versioninfo()
Julia Version 1.7.0-DEV.847
Commit fedefe913a (2021-04-06 03:03 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = code

I also tried with the vs/40279 branch without error.

[KristofferC]

If I check out the previous commit on master and build Julia, everything works fine.

Can you try do make clean; rm -rf usr between the builds. Can you also try downloading the nightly build and see if it repros there.

[sethaxen]

If I check out the previous commit on master and build Julia, everything works fine.

Can you try do make clean; rm -rf usr between the builds. Can you also try downloading the nightly build and see if it repros there.

I cleaned several times and still saw the error, but just to be safe, I made a fresh clone of the repo and am makeing it now. EDIT: Yes, same error on a fresh build.

I see the same error on the nightly build:

julia> using LinearAlgebra

julia> eigen(randn(10, 10))

signal (4): Illegal instruction: 4
in expression starting at REPL[2]:1
dlapy2_64_ at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libopenblas64_.0.3.13.dylib (unknown line)
Allocations: 2245082 (Pool: 2244614; Big: 468); GC: 3
Illegal instruction: 4

julia> versioninfo()
Julia Version 1.7.0-DEV.847
Commit fedefe913a (2021-04-06 03:03 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, ivybridge)
Environment:
  JULIA_EDITOR = code

[jaakkor2]

I can't even start Julia-1.7 nightly on my Mac

$ exec '/Applications/Julia-1.7.app/Contents/Resources/julia/bin/julia'
ERROR: Unable to load dependent library /Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/julia/libopenblas64_.dylib
Message: dlopen(/Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/julia/libopenblas64_.dylib, 6): Symbol not found: _clock_gettime
  Referenced from: /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia//libgfortran.5.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia//libgfortran.5.dylib
Unable to load "/Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/julia/libopenblas64_.dylib"
┌ Error: No loaded BLAS libraries were built with ILP64 support
└ @ LinearAlgebra.BLAS /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.7/LinearAlgebra/src/blas.jl:154
Quitting.

[Process completed]

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM)2 Duo CPU     P7350  @ 2.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, penryn)

Julia-1.6.0 is fine.

[giordano]

@jaakkor2 can you also please try on the commit before JuliaLang/julia#40343?

[jaakkor2]

@giordano , only if I can get a download link to a nightly before JuliaLang/julia#40343, as my Mac is from 2009 and I cannot likely compile anything in a reasonable time in it.

[giordano]

@giordano , only if I can get a download link to a nightly before JuliaLang/julia#40343

There you go: https://julialangnightlies-s3.julialang.org/bin/mac/x64/1.7/julia-89fff18d59-mac64.dmg

[jaakkor2]

@giordano , only if I can get a download link to a nightly before JuliaLang/julia#40343

There you go: https://julialangnightlies-s3.julialang.org/bin/mac/x64/1.7/julia-89fff18d59-mac64.dmg

Same error with julia-89fff18d59-mac64.dmg as with latest nightly (creation time difference between the two is only 6 hours). More nightlies to test? From mid-February before libblastrampoline went in?

[giordano]

Find a commit and replace the first 10 characters of the hash in the URL. I think these tarballs are available for 60 or 90 days.

[ViralBShah]

For the folks who are seeing this issue, can you confirm that 1.6 works fine (I think it should be but just checking).

[giordano]

Determing which is the illegal instruction would give us some more clues of what's going on. Look for example in dmesg log if there is a line referencing it

[sethaxen]

For the folks who are seeing this issue, can you confirm that 1.6 works fine (I think it should be but just checking).

Yes, Julia 1.6 works fine for me. As does the commit immediately before JuliaLang/julia#40343.

Determing which is the illegal instruction would give us some more clues of what's going on. Look for example in dmesg log if there is a line referencing it

Here's the output of sudo dmesg that looks relevant:

Waking up reference: 20605Thread waiting on reference 20605 woke upevaluation result: 20605, allowed, cache,  1617663216Waking up reference: 20606Thread waiting on reference 20606 woke upevaluation result: 20606, allowed, cache,  1617663217Waking up reference: 20607Thread waiting on reference 20607 woke upevaluation result: 20607, allowed, cache,  1617663217AMFI: '/Users/saxen/projects/julia/usr/lib/libopenlibm.3.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libopenlibm.3.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20608Thread waiting on reference 20608 woke upevaluation result: 20608, allowed, cache,  1617663217Waking up reference: 20609Thread waiting on reference 20609 woke upevaluation result: 20609, allowed, cache,  1617663217AMFI: '/Users/saxen/projects/julia/usr/lib/libunwind.1.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libunwind.1.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20610Thread waiting on reference 20610 woke upevaluation result: 20610, allowed, cache,  1617663217AMFI: '/Users/saxen/projects/julia/usr/lib/libLLVM.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libLLVM.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20611Thread waiting on reference 20611 woke upevaluation result: 20611, allowed, cache,  1617662974Waking up reference: 20612Thread waiting on reference 20612 woke upevaluation result: 20612, allowed, cache,  1617663808Waking up reference: 20613Thread waiting on reference 20613 woke upevaluation result: 20613, denied, skip-cache,  0Library load (/private/var/folders/5v/8c378lxx4ddg21zlmzy88dgm0000gp/T/tmp.KMeniZ) rejected: library load disallowed by system policy
purgatory_cleaner_cb:19279: remove_purgatory_entry failed on 0x254c940-dead for file-id 39110976 err 2
Waking up reference: 20614Thread waiting on reference 20614 woke upevaluation result: 20614, allowed, cache,  1617730927AMFI: '/Users/saxen/projects/julia/usr/lib/libpcre2-8.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libpcre2-8.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20615Thread waiting on reference 20615 woke upevaluation result: 20615, allowed, cache,  1617663352AMFI: '/Users/saxen/projects/julia/usr/lib/libgmp.10.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libgmp.10.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20616Thread waiting on reference 20616 woke upevaluation result: 20616, allowed, cache,  1617663368AMFI: '/Users/saxen/projects/julia/usr/lib/libmpfr.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmpfr.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20617Thread waiting on reference 20617 woke upevaluation result: 20617, allowed, cache,  1617663372AMFI: '/Users/saxen/projects/julia/usr/lib/libblastrampoline.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libblastrampoline.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20618Thread waiting on reference 20618 woke upevaluation result: 20618, allowed, cache,  1617663427AMFI: '/Users/saxen/projects/julia/usr/lib/libopenblas64_.0.3.13.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libopenblas64_.0.3.13.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20619Thread waiting on reference 20619 woke upevaluation result: 20619, allowed, cache,  1617663485AMFI: '/Users/saxen/projects/julia/usr/lib/libgfortran.5.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libgfortran.5.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20620Thread waiting on reference 20620 woke upevaluation result: 20620, allowed, cache,  1617663485AMFI: '/Users/saxen/projects/julia/usr/lib/libquadmath.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libquadmath.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20621Thread waiting on reference 20621 woke upevaluation result: 20621, allowed, cache,  1617663485AMFI: '/Users/saxen/projects/julia/usr/lib/libcholmod.3.0.14.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcholmod.3.0.14.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20622Thread waiting on reference 20622 woke upevaluation result: 20622, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libamd.2.4.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libamd.2.4.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20623Thread waiting on reference 20623 woke upevaluation result: 20623, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libcolamd.2.9.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcolamd.2.9.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20624Thread waiting on reference 20624 woke upevaluation result: 20624, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparseconfig.5.8.1.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparseconfig.5.8.1.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20625Thread waiting on reference 20625 woke upevaluation result: 20625, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libccolamd.2.9.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libccolamd.2.9.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20626Thread waiting on reference 20626 woke upevaluation result: 20626, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libcamd.2.4.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcamd.2.4.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20627Thread waiting on reference 20627 woke upevaluation result: 20627, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparse_wrapper.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparse_wrapper.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20628Thread waiting on reference 20628 woke upevaluation result: 20628, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libnghttp2.14.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libnghttp2.14.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20629Thread waiting on reference 20629 woke upevaluation result: 20629, allowed, cache,  1617663429AMFI: '/Users/saxen/projects/julia/usr/lib/libcurl.4.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcurl.4.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20630Thread waiting on reference 20630 woke upevaluation result: 20630, allowed, cache,  1617663429AMFI: '/Users/saxen/projects/julia/usr/lib/libssh2.1.0.1.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libssh2.1.0.1.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20631Thread waiting on reference 20631 woke upevaluation result: 20631, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libz.1.2.11.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libz.1.2.11.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20632Thread waiting on reference 20632 woke upevaluation result: 20632, allowed, cache,  1617663429AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedtls.2.24.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedtls.2.24.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20633Thread waiting on reference 20633 woke upevaluation result: 20633, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedx509.2.24.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedx509.2.24.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20634Thread waiting on reference 20634 woke upevaluation result: 20634, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedcrypto.2.24.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedcrypto.2.24.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20635Thread waiting on reference 20635 woke upevaluation result: 20635, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libdSFMT.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libdSFMT.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20636Thread waiting on reference 20636 woke upevaluation result: 20636, allowed, cache,  1617663407AMFI: Process does not have a code signature, allowing core dump for pid 73884