[Solved] Git ignore node_modules not working (.gitignore)

Introduction

If you ever worked with Node apps and Git you might come across the .gitignore file. This file just lets you tell git to ignore certain files or folders when you are checking in code.

A common folder that we do not want to check-in to source control (git) is the node_modules folder.

This folder just contains all the packages and dependencies thats used with the application. Since are not touching this folder it makes sense to not check it in - since I have seen these folders go up into the Gigabyte sizes on some apps.

Ok great! The project now is that sometimes using the .gitignore file is not ignoring our node_modules folder!

This post will go over reasons to add .gitignore to your project to ignore node_modules folder!

What is the node_modules folder?

The node_modules folder is a directory in a Node.js project that contains all the dependencies (packages) required for the project. When a package is installed using npm (Node Package Manager), its files are saved in the node_modules folder and can be imported into the project to be used

How do I add .gitignore?

Typically, to get started with .gitignore (on Mac OSX or Linux distros):

1. Open the Terminal
2. Go to the root of your project (the location of your Git repository)
3. Create a .gitignore file for your repository.

$ touch .gitignore

We then update the .gitignore file to ignore node_modules/

An example of my own gitignore file - consider the node_modules/ (we need to keep the slash to ignore child folders aswell)

# My .gitignore file

### Node ###

# Ignore node_modules and its child directories
node_modules/

# Ignore Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*
.pnpm-debug.log*

# Bower dependency directory (https://bower.io/)
bower_components

# Optional npm cache directory
.npm

# Optional eslint cache
.eslintcache

# Optional stylelint cache
.stylelintcache

# dotenv environment variable files
.env
.env.development.local
.env.test.local
.env.production.local
.env.local

# ignore the dist folder
dist

Then we add .gitignore file to git:

$ git add .gitignore

Finally add a commit message and commit:

$ git commit −m "modified .gitignore to add bin/"

Now if the above is not working, here are some common reasons why .gitignore might not be working, and how to fix them:

1. Make sure the .gitignore file is in the root folder. The .gitignore file must be located in the root directory of the repository you want to ignore files in.
2. Check the syntax is correct - Ensure that each ignore rule is on a new line and does not include any extra spaces.
3. Verify that the node_modules folder is not already tracked
4. Make sure that the .gitignore file are not to specific and covers the search correctly
5. Check the the .gitignore file encoding - make sure it is UNICODE instead of ASCII!
6. Check if there is a global .gitignore file overriding your project file

1. Make sure the .gitignore file is in the root folder.

The .gitignore file must be located in the root directory of the repository you want to ignore files in. As an example, consider the following folder structure:

my-app/
├─ node_modules/
├─ public/
│  ├─ favicon.ico
│  ├─ index.html
│  ├─ robots.txt
├─ src/
│  ├─ index.css
│  ├─ index.js
├─ .gitignore
├─ package.json
├─ README.md

We can see that our .gitignore file is on the root of our project (“my-app”).

2. Check the syntax is correct

One thing to check is that your .gitignore file follows correct syntax.

Some rules to consider:

• Each line in a gitignore file specifies a pattern
• Make sure that each ignore rule is on a new line and does not include any extra spaces. For example, node_modules/ (extra space at the end would not work)
• Consider the forward slash (/) for node_modules - the forward slash means to include child folders aswell

3. Verify that the node_modules folder is not already tracked

If the files you want to ignore were already added and committed to your repository, they will not be ignored by Git.

You will first have to remove them from source control and check in the removal - then add your .gitignore file.

As an example to do this, run this command in the terminal:

git rm -r --cached node_modules

The above command will will remove the node_modules folder from your Git repository. Now we then can commit and pus the changes.

git commit -m "Remove node_modules"

Then push your changes to the remote repository.

git push

So after the above steps, your node_modules folder will be ignored - given that you have a .gitignore file with node_modules/ specified to be ignored.

Alternative nuke approach

Now if the above did not work, we can go the full nuke method and clear everything out:

1. Delete node_modules (or move it somewhere outside from the project directory)
2. Commit the changes (there will be a tons of deletion from node_modules) This step will remove the files from source control.
3. Add node_modules to .gitignore again
4. Commit gitignore
5. Re-run npm install or restore the node_modules directory.

4. Make sure that the .gitignore file are not to specific and covers the search correctly

For example, node_modules/package-name will only ignore that specific package, while node_modules/ will ignore all contents within the node_modules folder.

Tip: .gitignore will not consider symlinks

One thing of note when writing out the ignore patterns in your gitignore file is that it does not consider symlinks.

Symlinks are just redirect files (sort of like shortcuts). For example, ~/.myfile.txt -> ~/.myfile2.txt.

When Git reads your .gitignore file - it will not follow the symlink!

5. Check the the .gitignore file encoding - make sure it is UNICODE instead of ASCII

This is usually a issue with Windows systems, we just need to check that the file is of correct encoding (prefer ASCII):

1. Open up Powershell and check on the git status:

PS> git status

2. Create a function to get the file encoding: Get-FileEncoding

function Get-FileEncoding {
    param ( [string] $FilePath )

    [byte[]] $byte = get-content -Encoding byte -ReadCount 4 -TotalCount 4 -Path $FilePath

    if ( $byte[0] -eq 0xef -and $byte[1] -eq 0xbb -and $byte[2] -eq 0xbf )
        { $encoding = 'UTF8' }  
    elseif ($byte[0] -eq 0xfe -and $byte[1] -eq 0xff)
        { $encoding = 'BigEndianUnicode' }
    elseif ($byte[0] -eq 0xff -and $byte[1] -eq 0xfe)
         { $encoding = 'Unicode' }
    elseif ($byte[0] -eq 0 -and $byte[1] -eq 0 -and $byte[2] -eq 0xfe -and $byte[3] -eq 0xff)
        { $encoding = 'UTF32' }
    elseif ($byte[0] -eq 0x2b -and $byte[1] -eq 0x2f -and $byte[2] -eq 0x76)
        { $encoding = 'UTF7'}
    else
        { $encoding = 'ASCII' }
    return $encoding
}

3. Test .gitignore’s encoding:

PS> Get-FileEncoding .gitignore

4. Change the encoding to ASCII:

PS> Set-Content .gitignore -Encoding Ascii -Value (Get-Content .gitignore)

5. Confirm:

PS> git status

6. Check the order of precedence- eg if there is a global .gitignore file overriding your project file

There is a order of precedence for .gitignore - Git looks for ignore patterns from multiple places and knowing them would benefit us.

Generally from highest order to lowest is as follows:

1. Patterns read from the command line for those commands that support them.
2. Patterns read from a .gitignore file in the same directory as the path, or in any parent directory (up to the top-level of the working tree), with patterns in the higher level files being overridden by those in lower level files down to the directory containing the file. These patterns match relative to the location of the .gitignore file.
3. Patterns read from your git directory ($GIT_DIR/info/exclude).
4. Patterns read from the global configuration variable core.excludesFile.

To find the global .gitignore file we can run the following command in the terminal:

git config --global core.excludesFile

Now if that returns empty, then its probably using the default locations:

For Linux or Mac OSX:

~/.config/git/ignore

For Windows:

%USERPROFILE%\.config\git\ignore

We can update the global configurations as follows:

For linux or Mac OSX:

git config --global core.excludesFile '~/.gitignore'

For Windows cmd:

git config --global core.excludesFile "%USERPROFILE%\.gitignore"

Summary

To not include files in Git/ source control we can use the .gitignore file. Commonly, we want to ignore the node_modules folder since it is just contains packages and dependencies we are going to use in the project and will not contain code changes.

In this post, I went through step by step on how to fix gitignore not ignoring the node_modules folder. Firstly we need to make sure that the .gitignore file is in the root of the project directory, check the syntax, make sure the file is of correct encoding (ASCII) and check the order of precedence - eg not getting overrides.

References

https://git-scm.com/docs/gitignore