yarn install sporadically fails with integrity check failure

[kayneb]

Do you want to request a feature or report a bug?
bug

What is the current behavior?
When running yarn install (on our CI) we are finding that random dependencies fail to download due to integrity check failures:

e.g.

yarn install --pure-lockfile error https://registry.yarnpkg.com/react-style-proptype/-/react-style-proptype-3.2.1.tgz: Integrity check failed for "react-style-proptype" (computed integrity doesn't match our records, got "sha512-znIMso2zjWI0V3Ne7hQcOB5HOXdPpJ0HsGw4GBs/uEmVLaN46acxHi5B0PXwOnEzSQ46hp0arB+khsFh/sVIxg==")

If the current behavior is a bug, please provide the steps to reproduce.
Have a somewhat yarn project setup with a somewhat large number of dependencies - keep yarn installing until integrity check fails.

What is the expected behavior?
No integrity checks occur

Please mention your node.js, yarn and operating system version.
node: v10.9.0
yarn: v1.10.0
OS: NixOS (Linux)

This could possibly be occurring due to network failures on our end, but I don't consider this likely considering our build machines are EC2 instances with solid internet connections.

As a workaround for now, we are rolling back to v1.9.4

[arcanis]

As a workaround for now, we are rolling back to v1.9.4

Are you sure this doesn't happen with the 1.9.4? Afaik this is caused by the npm registry sometimes returning 500 at random (the npm client retries until it works, but we don't do this yet), so this should affect all Yarn releases.

[kayneb]

I think integrity checking was added with 1.10.0. See https://github.com/yarnpkg/yarn/pull/5042/files?utf8=%E2%9C%93&diff=unified and https://twitter.com/yarnpkg/status/1037631294170189824

[arcanis]

I know :) But the integrity errors are caused by the registry returning 500 errors that are not properly reported as such.

[Dahaden]

@arcanis We have been testing for flakiness recently, just reverting yarn back to 1.9.4 has taken our builds from a 36% success rate to 88%

[arcanis]

@Dahaden Can you share details regarding the errors? In particular:

• do you use --frozen-lockfile?
• did your lockfile contain the new integrity field, or was it the same one than for the 1.9.4?

The 1.10 ships with the new integrity feature, which requires us to migrate your lockfiles by querying the npm registry for each dependency in order to add the field. If done during CI, it could increase the opportunities for the registry to return an 500 somewhere. If the integrity fields were added before being checked-in, that would alleviate the issue (since the CI wouldn't have to query the network for this anymore) cc @imsnif

If you have the opportunity to retry the 1.10, can you add the following line into a .yarnrc file at the root of your repository? This should prevent Yarn from adding the integrity field if it is missing, which will help us determine if this is the cause of the issue or not.

unsafe-disable-integrity-migration true

[imsnif]

Yeah, as @arcanis said, the migration to 1.10 is more network intensive. If on a flaky network (or if the npm registry is having networking issues at the time) it might explain it. Following.

[arcanis]

This should hopefully be mitigated with #6413

[kayneb]

We did populate the integrity field in the lockfile prior to pushing, so it should be all good.
Awesome thanks for that improvement @arcanis! Will definitely look to bump to 1.11.0, including this fix.

[fqborges]

If you have the opportunity to retry the 1.10, can you add the following line into a .yarnrc file at the root of your repository? This should prevent Yarn from adding the integrity field if it is missing, which will help us determine if this is the cause of the issue or not.

unsafe-disable-integrity-migration true

In my case, this .yarnrc workaround did the job.

[szrenwei]

@Dahaden Can you share details regarding the errors? In particular:

• do you use --frozen-lockfile?
• did your lockfile contain the new integrity field, or was it the same one than for the 1.9.4?

The 1.10 ships with the new integrity feature, which requires us to migrate your lockfiles by querying the npm registry for each dependency in order to add the field. If done during CI, it could increase the opportunities for the registry to return an 500 somewhere. If the integrity fields were added before being checked-in, that would alleviate the issue (since the CI wouldn't have to query the network for this anymore) cc @imsnif

If you have the opportunity to retry the 1.10, can you add the following line into a .yarnrc file at the root of your repository? This should prevent Yarn from adding the integrity field if it is missing, which will help us determine if this is the cause of the issue or not.

unsafe-disable-integrity-migration true

add a .yarnrc file also solved my problem, thanks.

[shunia]

unsafe-disable-integrity-migration true

is working

[viviansteller]

yarn config set unsafe-disable-integrity-migration true -g

[sibelius]

One package that keeps failing the integrity check is this one https://github.com/nodkz/mongodb-memory-server

I hope this helps