Slow npm install on self-hosted GitHub runners in AWS Cloud (Windows)

We are experiencing slow execution times for the npm install command on our self-hosted GitHub runners in AWS Cloud. Here is our setup:

Infrastructure: We use here module and Packer images.

Environment:

• Windows Core 2022 image as base for our AMIs
• PowerShell 7.1
• Latest version of GitHub runner (v2.317.0)
• Node.js version v20.14.0 and npm v10.7.0

Issue:

The npm install command runs extremely slow on our self-hosted GitHub runners. For instance, it takes around 5 minutes compared to 1 minute on GitHub-hosted runners. The process spends most of its time extracting packages from tgz files in npm cache.

Troubleshooting Steps Taken:

• ReFS Partition: Attempted to run everything from a ReFS partition.
• Different AWS volumes: Tried various types of volumes mounted from AWS to the runner.
• Manual Execution: Cloning the repository and running npm install manually on the EC2 instance finishes in under a minute—showing it's not a hardware or software issue but something related to the GitHub runner integration.

Any advice?

Answer

After much debugging we established the cause of the problem. In the packer script, github runner is configured to run as scheduled task. See script here. If we run it as windows service under NETWORK_SERVICE credentials - it installs npm modules much faster. I will post fixed code as soon as we find suitable solution for this code.