Switching Between Branches with Different node_modules Folders

Every once in a while, I find myself in the situation of needing to change quite a few JavaScript dependencies. And oftentimes, this means that I would need to switch between two branches with rather different node_modules folders—which is a hassle.

You too? Well, in this post, I will share how to do just that. 🙂

Context

Imagine you are working on a project team for quite a while. This would mean that your project codebase already exists, and also that you would have most of your tooling set up already. Now, say that, for some reason, you want to swap out some JavaScript dependency, which comes with several peer dependencies. In my case, one example has been moving from a custom webpack setup to @wordpress/scripts, which means changes in the tooling itself, but also related things such as which webpack or Babel dependencies I (still) need, versus which ones now live abstracted behind @wordpress/scripts. And then there’s things like ESLint plugins and presets, and potential related dependencies on that front.

While I was working on this new branch, I still had to fix bugs or tweak some things here and there in the production codebase, meaning the master branch. Switching between my new feature branch and master, however, comes with a headache—or at least some time wasted waiting for a lot of node_modules to be generated according to the changes in the package.json file.

Now, how can one do this? At all, but also in an efficient way? I see two methods for that, which I am going to illustrate in this post:

• building dependencies from scratch;
• switching between two node_modules folders.

Each method can be approached manually or (semi-)automated.

The (Slow) Manual Way

I guess, we all know how to do this the straightforward and manual, but at the same time error-prone and slow way.

Every time you check out one or the other branch, you manually install/update all JavaScript dependencies as per your package.json file. If you want to do that from scratch, then this would mean to delete the node_modules folder first. This takes time. A lot of time, actually. And depending on your package manager—npm vs. Yarn—you might even end up with different package versions than the ones you had before.

Ensuring Package Versions

Using Yarn, this is all good, but npm will install whatever most recent package releases match your version constraints. Yes, even with a lock file! However, you can (and, in this case, should) use npm ci. Not many people seem to know this. In contrast to the npm install command—or short: npm i— this will install whatever you have in your lock file, exactly as you have it in the lock file! The “ci” stands for continuous integration, and is intended to be used with, right, CI services/tools such as Travis CI. The idea is that you use the exact same package versions on your CI stage than what you had locally when you created and committed the lock file. At the same time, this allows for quicker CI runs or builds, because you can make perfect use of caching (if your CI service/tools supports that and if you have that set up).

Updating Dependencies

If you don’t want to delete all you got, you can also just (try to) install/update your JavaScript packages with an existing node_modules folder. This should work most of the time, but not always. It also needs a bit of time, although not as much as killing it all and installing it all fom scratch, of course. Again, depending on what tools you use (and how), you might end up with different versions, and thus an updated lock file. Usually, you don’t want this.

Switching Between Different node_modules Folders

The second method that I see is to have two versions of node_modules. After checking out one or the other branch, you would ensure that the actual node_modules folder in your project is the one that matches the requirements of the branch you just checked out. You could do that in several ways again. For example, you have two folders, one of which is the actual node_modules folder, and the other has a different name. You would then rename both folders accordingly, so that node_modules contains what the current branch or rather the package.json (and lock file) expects. You could maybe also have two folders with custom names, and then symlink the one you want to be the node_modules folder. I don’t know if this works fine in all cases, and I would not recommend doing that.

Using a Script

Now, pretty much all of what we did before by hand, we can also do in an automated way. No matter if you want to delete and re-install, or just update on-the-fly, or if you want to switch between two versions of node_modules, you can put this in some script—bash, JavaScript, PHP, whatever—and run it on demand. You could even go that far and make it run automatically, for example, using the post-checkout githook.

User interaction can be reduced so much, but you still have to wait for the process to complete. And you still might have issues here and there, according to some leftover packages or version incompatibilities, or issues with the filesystem because files/folders are in use.

Yarn to the Rescue

But … in case you are using Yarn, there is a much more straightforward and efficient way to do exactly what you want: make individual branches use individual node_modules folders.

How? Well, Yarn comes with a --modules-folder CLI option, which you can also put in a .yarnrc config file. ?

The process is super easy:

1. Check out your feature branch.
2. Create a branch-specific folder to contain your JavaScript dependencies, for example, node_modules-123, with “123” being the issue reference, or node_modules-scripts or whatever.
3. Make sure that you have a .yarnrc file, and that it specifies to use that new folder: --modules-folder ./node_modules-123
4. Adapt your package.json file to your needs.
5. Maybe delete your lock file…?
6. Install all dependencies.
7. Commit both the changed package.json and yarn.lock file, as well as the new .yarnrc config file to your feature branch.
8. That’s it! You can now safely switch between branches, and all your dependencies are what you need them to be.

This is just one more reason why I like Yarn so much better than npm. 😉

Bonus: What about Composer?

That’s all cool, right? So, wouldn’t it be nice if we could do the same for Composer?

Well, actually, we can! Composer provides a config.vendor-dir setting, so you could do the same as above. Just specify your custom name for the vendor folder in your composer.json file, like so:

{
    "config": {
        "vendor-dir": "vendor-scripts"
    }
}