node_modules and symlinking dependencies #6
Comments
Some node packages like babel traverse the complete node_modules tree to find dependencies i.e. for plugins. However the absolute and resolved path to the module is used thus no node_modules folder can be found in the parent folder and the dependencies can't be found. Due to this, lots of dependencies of downstream dependencies are simply wrong. In our case, some babel packages won't find their plugins but without using symlinks they to. This hack ensures symlinks are only used for dependencies to keep the overall space consumption low. In the package we're interested in building, all symlinks will be dereferenced and the contents copied to the main node_modules directory. TODO: generation of template package nix code (top)
|
Unfortunately I don't know of any good solutions for this. Yarn itself ran into the same problem, and that's why yarn copies all the files (it used to use symlinks, but then it ran into these problems). There is an open issue about changing yarn to use hard links instead (but symlinks still won't work). There were some attempts at changing npm/yarn/node so that this sort of symlink behavior would be possible: nodejs/node#9719 However, they didn't seem to really go anywhere, which I think is a huge shame. |
|
Is it possible to find out how many packages do this? If it’s just a few, we might be able to patch them into coercion. |
|
@Profpatsch I guess you could try asking the Yarn team, since they ran into this problem already. Other than that, the only way I know of to figure out the impact is to just wait and see for ourselves how many packages break. As far as patching goes, my understanding is that big and popular packages like Babel cause symlinks to break, and I'm not sure if it's even possible to patch them to make it work. Though I guess most packages that have this kind of behavior are used as |
|
I think this is also relevant: https://github.com/pnpm/pnpm They use a combination of hardlinks and symlinks to ensure that only a single copy of an npm library exists on disk. |
|
working fix in https://github.com/milahu/nodejs-hide-symlinks |
mayflower@16212ba
In yet another horrible aberration in the node ecosystem, some packages manually traverse out of their own scope to find plugins of some sort?
When will this madness end and what should we do about it.
The text was updated successfully, but these errors were encountered: