How does npm handle conflicting package versions?

Photo by JOSHUA COLEMAN on Unsplash

If you find this post useful, don’t forget to give me your 👏 (s) and follow me on twitter https://twitter.com/imDongCHEN

Problem

If the same module is listed both as a dependency of an application and one of its dependencies’ dependency, but their listed version is different, how would npm handle the case? That is, we have a dependency tree like:

Background

The problem comes up as we upgraded to React 16. While we are happy that we finally embrace all the new features of React, we found an issue in our app. In Chrome dev tool, if we press command and p and search react.js, we find multiple react packages in our bundle! Although our app has upgraded to React 16, some other libraries we use probably still stick with React 15. This is really bad as it definitely increases bundle size. Worse, Dan Abrmov has warned that including multiple versions of react could cause problems.

Our app suffers from multiple React bundled in the output

Ask yourself the following questions. If you have any confusion with answer to these questions, this article is for you.

1. When you are developing a reusable react component, should you include react as a dependency?
2. What would happen if your react app includes a react component that also has a dependency on react? Will react be bundled twice?
3. What would happen if you upgrade your app from using react@15 to react@16, but some of your third party react components still rely on react@15? In other words, what would happen if you have different versions of dependencies?

Let’s do some experiment to answer these questions.

Experiment

The experiment code is available in GitHub

Treatment #1: app and module has no shared dependency

We start with a basic setting. We have an app that has dependency of moduleA and moduleB. Neither module themselves has any dependency. The two modules are hosted on GitHub. Npm can install packages directly from GitHub so I don’t need to publish these two packages to npm registry.

Trick: you can do npm install cdbean/moudleA to install a GitHub hosted package

package.json looks like this

The result should be obvious. Npm installs moduleA and moduleB as root dependency (under node_modules). In package-lock.json we’ll

Treatment #2: app and module has the same dependency

Now our app upgrades moduleB to v2 and moduleB@v2 adds dependency on moduleA. I put moduleB@v2 in a separate git branch v2 to simulate that it is a different package version. package.json in app looks like:

Now run npm install. The result is the same as basic setting! Although moduleB has a dependency on moduleA, npm is smart enough to avoid installing another moduleA under moduleB/node_modules. Therefore, moduleA will only be bundled once in the output file. Nice! Below is the package-lock.json

What is going on here?

When installing dependencies npm checks if that package of the same version has been installed in the root. If it is, npm will skip that; otherwise, it will install the package in the root, even though this dependency is only required by a dependent module but not required by the parent app. The process of moving dependency to the root is known as “hoist”.

Treatment #3: app and module has conflicting dependency

Now moduleA has a new version v2, and moduleB upgrades to v3, which depends on moduleA@v2. Yet app still has dependency on moduleA@v1. This is what looks like in package.json:

The result of npm install is that moduleA@v1 is installed in node_modules/ and moduleA@v2 is installed in node_modules/moduleB/node_modules/. The output bundle includes both versions of moduleA. The package-lock.json looks like:

What is going on here?

Npm installs dependencies in the order they are listed in package.json. It first encounters moduleA and goes ahead installing it because it’s installed yet. When it encounters moduleB, it tries to install moduleA@v2 but notes moduleA@v1 has been installed. So npm installs moduleA@v2 as a dependency of moduleB. In the output bundle, both versions of moduleA will exist.

What if we manually install moduleB first and then moduleA? As described above, npm will first install moduleA@v2 as the top dependency because it’s not installed yet. When it encounters moduleA, it will try to install moduleA but find another version is already installed in the root node_module. What will npm do? It turns out npm removes moduleB and its dependencies and installs moduleA@v1 instead. Interesting behavior.

When installing packages using npm install, even though moduleB is listed above moduleA, npm is smart enough to install things correctly. I am not sure what exactly happens. But npm will run npm dedupe after installation. Maybe some magic happens there.

Treatment #4: multiple modules has conflicting dependencies

What if the app adds another dependency, moduleC, that depends on moduleA@v2. It should be obvious that moduleA@v1 will still be installed as the app dependency, and moduleA@v2 will be installed as both moduleB’s dependency and moduleC’s dependency. Question is will moduleA@v2 be bundled twice in the output?

The answer is yes!

This is what happens in our React app described in the background section. While our app uses React 16, we have several third party react components that still use React 15. That causes our output bundle to include both React 16, and several copies of React 15!!

Solution: PeerDependencies

peerDependencies is the current solution in community to solve such problem. Peer dependencies assume parent app will install this dependency and the module will not install its own. If the parent app does not have this dependency installed, npm will throw a warning. React is a good use case of peerDependencies as we can safely assume any app that imports a react component will install react itself. In order to let the module to be most compatible, we should loosen the version restriction of the peer dependency as much as possible. For example, in moduleB, we can declare peer dependencies as such in its package.json:

Conclusion

To answer the questions in the background section:

1. Always declare react as a peer dependency if your module is designed to be reused.
2. If your dependency has the same dependency as your app, things would be fine. But if they declare different versions, multiple versions will be bundled in the output. In case of React, not only the react library will be duplicately bundled, but your app and some components may use different instances of React, which is likely to cause issues. To avoid that, use peer dependency.

If you find this post useful, don’t forget to give me your 👏 (s) and follow me on twitter https://twitter.com/imDongCHEN