How to fix npm install failed to remove some directories

Introduction

One of the pain points of working with Node and NPM projects is that the NPM CLI does not delete the packages you have removed and it still lingers.

This can cause blow outs in your node_modules folder!

I recently had this issue when I was trying to uninstall gulp from my project.

$ npm ls
├─┬ pkg-that-just-requires-gulp@1.0.0
│ └── UNMET DEPENDENCY gulp@^3.9.0
└──  extraneous error: ENOENT, open '/Users/<username>/my-project/node_modules/vinyl-fs/package.json

npm ERR! extraneous: vinyl-fs /Users/<username>/my-project/node_modules/vinyl-fs
npm ERR! error in /Users/<username>/my-project//node_modules/vinyl-fs: ENOENT, open '/Users/<username>/my-project/node_modules/vinyl-fs/package.json'
npm ERR! missing: gulp@^3.9.0, required by pkg-that-just-requires-gulp@1.0.0

When you’re seeing a message like “npm install failed to remove some directories”, it’s likely an issue with file permissions, file paths, or issues with npm caches.

Here are a few ways to troubleshoot and resolve the problem:

Run in admin mode

Run npm with sudo - If you are on a Unix-based system, such as Linux or macOS, you might need to run npm with administrator privileges.

You can do this by prefixing your npm commands with sudo. For example: sudo npm install.

• Fix the file permissions: Sometimes npm can’t access certain directories because it doesn’t have the right permissions. You can fix the file permissions by running

sudo chown -R $USER /path/to/directory 

(replace /path/to/directory with the correct path).

On windows

Follow these steps to do this:

• Click the Start button (or press the Windows key), search for “Command Prompt” or “cmd”.
• Right-click on the “Command Prompt” result, and then select “Run as administrator”.
• You’ll likely get a User Account Control (UAC) prompt asking if you want to allow the Command Prompt to make changes to your system. Click “Yes” to proceed.

Now you can run your npm command (npm install, for example). Running the command prompt as an administrator should give npm the permissions it needs to access the necessary directories.

Clear NPM cache

Alternatively, avoid this issue by not running npm as root, which can cause permissions issues.

• Clear the npm cache: It’s possible the npm cache might be causing the issue. You can clear it by running npm cache clean –force.

Delete node_modules and package-lock.json

• Delete node_modules and package-lock.json: Try deleting the node_modules folder and package-lock.json file in your project directory, and then run npm install again. This allows npm to recreate these files without any potential issues from the old files.

Check path is valid

• Check the path: Make sure the path to your project does not have any spaces, as this can sometimes cause problems with npm.

Upgrade NPM and Node

Update npm and Node.js: If none of the above solutions work, you might want to consider updating npm and Node.js to the latest versions, as the issue could be a bug that’s been fixed in a more recent version.

Using Yarn

Another option going forward is to move over to use Yarn instead of NPM.

It has similar functionalities to npm, but I find it can be more reliable in certain situations.

You can install Yarn globally using npm with this command:

npm install -g yarn

Once Yarn is installed, you can use it to install your project dependencies. Instead of using npm install, you would use yarn install. Yarn reads from the same package.json file as npm, so it will install the same dependencies.

Before switching to Yarn, remember to delete the node_modules folder and package-lock.json file (if they exist) from your project directory, as these are specific to npm and could cause conflicts. Yarn will generate its own versions of these when you run yarn install.

Also, remember that switching package managers can have implications for your project and your team if you are working with others. Make sure everyone is aware of the change and is comfortable with using Yarn.

In case you face issues with Yarn or decide to switch back to npm later, you can uninstall Yarn using npm:

npm uninstall -g yarn

Manual delete

In my case, I had to do a manual delete. This seems to be caused by .bin directories in a nested node_modules (node_modules/*/node_modules/.bin).

Now I may have two packages with the same .bin declaration, or due to npm being unable to flatten the module hierarchy due to incompatible dependencies.

├─┬ gulp@3.9.0
…
│ ├─┬ gulp-util@3.0.7
…
│ │ ├─┬ dateformat@1.0.12
…
│ │ │   ├─┬ read-pkg-up@1.0.1
…
│ │ │   │   ├─┬ load-json-file@1.1.0
…
│ │ │   │   │ └── strip-bom@2.0.0
…
│ └─┬ vinyl-fs@0.3.14
…
│   ├─┬ strip-bom@1.0.0
…

We can see that strip-bom is included twice:

strip-bom@1.0.0 declares a bin field of “strip-bom”: “cli.js”, so that creates a nested .bin in vinyl-fs (node_modules/vinyl-fs/node_modules/.bin).

Here’s my step-by-step guide to solve this issue:

• Manually delete problematic directories: As a first step, you can try manually deleting the directories that npm is having trouble with. In the example provided, this would be the nested node_modules/.bin directories that contain strip-bom.

• Update packages - If manually deleting problematic directories doesn’t help, you might try updating the packages involved in the conflict. In my example, updating gulp and vinyl-fs packages could potentially resolve the conflict.

You can update individual packages using npm by running

npm update <package-name>.

Replacing package-name with the name of the package you want to update.

• Install conflicting packages at the root level: As a last resort, you can try installing the conflicting packages at the root level of your project.

For example, you could run

npm install strip-bom@2.0.0

(replace with the latest version if necessary). This will force npm to install strip-bom in the top-level node_modules directory and hopefully resolve the conflict.

• Reinstall your node_modules - If the above methods did not work, I would just go for the nuke approach - delete node_modules directory and package-lock.json file, then run npm install again.

Summary

Working with NPM can be a pain on Node and JavaScript projects. A common issue that I can see is that NPM can sometimes fail to remove directories causing blowouts in the size of node_modules folder.

Some ways we can fix this is to make sure that we are running under permissions that can delete files/ folders. Alternatively, we can clear NPM cache, make sure we are running the latest version of NPM and Node.

If all of that does not work, then I would suggest to manually go in and delete the node_modules folder and package-lock.json and start fresh!