Yarn install never ends with (complex ?) workspaces

[no-more]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
When launching yarn install in workspace directory it never ends(waited all night). But if I rollback to v1.3.2 it completes in 42s.

If the current behavior is a bug, please provide the steps to reproduce.
The project is on a private repo so I can't provide a link.

What is the expected behavior?
Should end within minutes.

Please mention your node.js, yarn and operating system version.
node: v9.11.1
Yarn: 1.6.0, 1.5.1, 1.3.2
Os: Ubuntu v9.11.1 or windows 10

I'm not sure if it could be related but the packages are referenced as "package-a": "git+ssh://git@bitbucket.org/companyname/package-a.git#0.1.1-beta.81",

[no-more]

I created a similar public repo on github:

nomore-test-workspace.zip

I have the same issue, with version 1.3.2 installation completes in less than 30s (cache empty).
But with version 1.6.0 it does not seems to end.

[rally25rs]

Trying to dig into this a bit but debugging all the recursion is tricky. The change that causes this to slow is somewhere between v1.3.2 and v1.4.0.

The code seems to just be cycling between package-request and pacakge-resolver recursively building up the entire tree, which gets very deep by the time you build test-m in the attached repro example. If you remove test-g through test-m then the install will finish, but you can also see it print out that it is resolving zone.js 3 or 4 times. It seems to be exponentially worse as you add back in the packages (test-f depends on test-e so all the resolves that happened for test-e happen again).

Just thinking through it, I suspect this might have something to do with resolutions because yarn always has to build the entire tree, just incase you had a resolution that said test-m/**/test-g/test-f/**/test-b/test-a: "v1.2.3" so the entire resolution tree building recursive thing can't be exited just because a lib has been resolved before. Just a thought anyway... no concrete evidence yet...

[rally25rs]

I bisected the performance change back to [this commit] (6adbd47) which strangely is mine, yet I don't remember making these specific code changes when fixing peerDep warnings 😕

I'll... uh... guess I'l try to figure out what I did here...

[rally25rs]

Oh, yeah, based on my comments in the original PR #5088

The code used to return when it had already resolved a package, and only build a partial dependency tree, so finding peerDeps wouldnt work if that part of the dep tree wasn't filled out.

If we put that logic back, then it's going to break peerDep resolution and I'm pretty sure will break resolutions in the way I mentioned above.

We might need to find some more efficient way to build the entire dep tree...

[no-more]

Hi, thanks for your investigation.

This example is only representing a portion of the project I'm working with which contains much more dependencies between packages (but no cycling). I submitted this one because it's the smallest case which reproduce the issue.

Do you think that removing resolution section would change something ? I'm not sure but I think I've tried this case with no more success. But maybe I'm wrong. Could you please confirm if you think that's related I could create another test case without resolution sections.

Thanks.

[rally25rs]

Do you think that removing resolution section would change something?

I don't think the resolutions would matter.

[rally25rs]

oops, clicked wrong button, accidental close :embarrassed: