Docker Installation with N8N & NPM

From Sea of Fate
Jump to navigationJump to search

Introduction

After the convoluted Walnut install of N8N was reversed the decision was made to install N8N on Lychee, as it is already a Linux host and it was likely to have other AI, automation scripts and applications added over time. N8N is installed as a Docker image and that was a part of the problem for walnut, a Windows 11 host, Docker on Linux should be a lot easier.

Installation of N8N and NPM

To get the N8N running on Lychee the will be a few separate steps.

  • The first goal is to install Docker Engine and Docker Compose plugin to run containers.
  • Second step is to install and configure N8N
  • The next step is to install NPM
  • Once NPM is installed we need to get a SSL cert downloaded raisin and get it configured to update on a 90 day cycle
  • We need to copy the script from Raisin to Lychee as it is renewed
  • NPM must be made to terminate the SSL connection and forward to N8N

Docker Installation

Docker has it's own repository so it's certificates needs to be allowed. First we need to update System and Install Prerequisites 

sudo apt update && sudo apt upgrade -y
sudo apt install -y ca-certificates curl gnupg

Add Docker's Official GPG Key: 

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

Add Docker's Official APT Repository: 

echo \ "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ "$(lsb_release -cs)" stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/nul

Install Docker Engine and Docker Compose Plugin: 

sudo apt update 
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Add your user (nigel) to the docker group: (Allows running Docker commands without sudo) 

sudo usermod -aG docker nigel

Important: Log out and log back in (or reboot Lychee) for group changes to take effect. After re-login/reboot to verify Docker Installation we run a hello world app 

docker run hello-world

Assuming we get a suitable "Hello world" response from Docker is now installed and we can move on to installing the applications.

Install N8N

Docker application should normally have two files. The first is an .env file that sets up the environment that the application will run in. examples of the sort of environment variables that will be defined are

  • DB_PASSWORD=your_secure_password,
  • N8N_HOST=my.n8n.com
  • APP_VERSION=1.0

The second file is a docker-compose.yml file that defines what the application will be and how its different parts interact. It will specify

  • Services: Each container that makes up your application (e.g., a web server, a database, a Redis cache, n8n itself).
  • Images: Which Docker image each service uses.
  • Build Contexts: If you're building custom images from a Dockerfile.
  • Ports: How container ports are mapped to host ports.
  • Volumes: How data is persisted (mapped to host directories or named volumes).
  • Networks: How services communicate with each other.
  • Dependencies: The order in which services should start.
  • Environment variables: These can also be defined directly in the YAML, but if they are sensitive or change frequently, it's better to get them from an .env file.

We will need To create and encription key (just a long string of text that cant easily be guessed). The easiest method to get this generated is to use the random hex generator supplied with openssl. There are a few variants that could be used. 

openssl rand -base64 32 # generates 32 random bytes and encodes them in to a base 64 number with an output string of 43 characters
openssl rand -base64 64 # generates 64 random bytes and encodes them in to a base 64 number with an output string of 86 characters 
openssl rand -hex 32 # generates 32 random bytes and encodes them in to a base 16 (hex) number with an output string of 64 characters
openssl rand -hex 64 # generates 64 random bytes and encodes them in to a base 16 (hex) number with an output string of 128 characters

This random number generator is built in to openssl so it will be available on Lychee. we will also need a username and password for the N8N user. The first job is to create a Project Directory for n8n: 

mkdir ~/n8n && cd ~/n8n

When we have those prerequisites we create the .env file no need for sudo as this is within the homedir of the logged on user 

nano .env

copy in the following into the file 

 N8N_HOST=lychee.seaoffate.net # Use Lychee's internal hostname for now
 N8N_PORT=5678
 N8N_PROTOCOL=http             # Initial setup on HTTP
 N8N_ENCRYPTION_KEY=your_strong_encryption_key # CRITICAL!
 DB_TYPE=sqlite

 N8N_BASIC_AUTH_ACTIVE=true
 N8N_BASIC_AUTH_USER=your_secure_user
 N8N_BASIC_AUTH_PASSWORD=your_secure_password

 N8N_DEFAULT_BINARY_DATA_MODE=filesystem
 N8N_RUNNERS_ENABLED=true
 N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true

Save and exit. we then create a yaml file with 

nano docker-compose.yml

and it will contain the following 

 version: '3.8'

 services:
   n8n:
     image: n8nio/n8n:latest
     restart: always
     ports:
       - "5678:5678" # Map host port 5678 to container port 5678
     environment:
       - N8N_HOST=${N8N_HOST}
       - N8N_PORT=${N8N_PORT}
       - N8N_PROTOCOL=${N8N_PROTOCOL}
       - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
       - DB_TYPE=${DB_TYPE}
       - N8N_BASIC_AUTH_ACTIVE=${N8N_BASIC_AUTH_ACTIVE}
       - N8N_BASIC_AUTH_USER=${N8N_BASIC_AUTH_USER}
       - N8N_BASIC_AUTH_PASSWORD=${N8N_BASIC_AUTH_PASSWORD}
       - N8N_DEFAULT_BINARY_DATA_MODE=${N8N_DEFAULT_BINARY_DATA_MODE}
       - N8N_RUNNERS_ENABLED=${N8N_RUNNERS_ENABLED}
       - N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=${N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS}
     volumes:
       - n8n_data:/home/node/.n8n

 volumes:
   n8n_data:

Save and exit. Note these are YML files and consistent indentation matters. We can now start the application with 

docker compose up -d

And check the status with 

docker ps
docker logs n8n-n8n-1

N8N will now be running and will work from the localhost with http on port 5678. To test open a browser on the on the local desktop and browse to http://localhost:5678 there should be the owner information screen show where the new owner can add the credentials. Enter some new username (email address) and password, although this is localhost this is still http only.

Installing NPM

N8N works on the localhost with http and we want it to work as https and across the network. Cloudflare as a proxy for our webservers will only accept properly terminated SSL connections with a cert that it can verify if it is set to full strict, it will not need to verify the cert if it is set to full as all that is required for full is that the connection is encrypted end to end. We have Cloudflare set to full strict. We have not been able to get the SSL engine on N8N to terminate the connection to the satisfaction of Cloudflare so we will use another Reverse Proxy, the instructions that were being followed recommended NPM as a Nginx install with a web interface. It is supposed to be easier to configure than the regular Nginx in that it has a WebGUI instead of text files as used on Raisin, it also has it's config contained in a database. While I am not totally convinced it is any easier than the more regular Nginx I thought I would try it.

The application will need a database and user so to set them up ssh to mandarin and login to MySQL with 

 sudo mysql -u root -p

Then create a DB and user 

 CREATE DATABASE npm_db;
CREATE USER 'npm_user'@'192.168.100.27' IDENTIFIED BY 'your_strong_npm_db_password';
GRANT ALL PRIVILEGES ON npm_db.* TO 'npm_user'@'192.168.100.27';
FLUSH PRIVILEGES;
EXIT;

Store the username and password in Keepass. If Lychee was in a different network to mandarin we would need to set rule in Pfsense for the Production Interface

  • Action: Pass
  • Interface: Production
  • Address Family: IPv4
  • Protocol: TCP
  • Source: Single host or Alias: 192.168.100.27 (Lychee's IP)
  • Destination: Single host or Alias: 192.168.100.8 (mandarin's IP)
  • Destination Port Range: MySQL (3306)
  • Description: Allow NPM on Lychee to connect to MariaDB on Mandarin

The next prerequisite will be to create a shared network for the two applications to run. First stop and remove both current Docker Compose setups (to clean up old networks): 

cd ~/n8n/ && docker compose down -v # This will remove n8n's data too, be aware if you had workflows
cd ~/npm/ && docker compose down -v # This will remove NPM's data too

Create the shared network manually (once): 

docker network create my_shared_network --driver bridge

With the prerequires done we can proceed to the main installation. As before the first thing to do is create a dir to it to run in: 

mkdir ~/npm && cd ~/npm

Create a docker-compose.yml 

nano docker-compose.yml

copy the following in to the .yml 

 version: '3.8'

 services:
   app:
     image: 'jc21/nginx-proxy-manager:latest'
     restart: unless-stopped
     ports:
       - '80:80'    # Public HTTP port for NPM
       - '443:443'  # Public HTTPS port for NPM
       - '81:81'    # Admin UI port for NPM
     environment:
       DB_MYSQL_HOST: "192.168.100.8" # <--- MANDARIN'S IP
       DB_MYSQL_PORT: 3306
       DB_MYSQL_USER: "your_npm_db_user" # <--- Must match user created on Mandarin
       DB_MYSQL_PASSWORD: "your_strong_npm_db_password" # <--- Must match password created on Mandarin
       DB_MYSQL_NAME: "npm_db" # <--- Must match DB created on Mandarin
     volumes:
       - ./data:/data
       - ./letsencrypt:/etc/letsencrypt # NPM uses this for its certs
     networks: # <--- Connect to the manually created shared network
       - my_shared_network

 networks: # <--- Define the shared network as external
   my_shared_network:
     external: true

save and exit. Before we start the applications again we need to modify the N8N docker-compose.yml so that it will start with the new shared network 

cd ~/n8n
nano nano docker-compose.yml

add the new lines for the shared network so that it now has the following text 

 version: '3.8'

 services:
   n8n:
     image: n8nio/n8n:latest
     restart: always
     ports:
       - "5678:5678" # Map host port 5678 to container port 5678
     environment:
       - N8N_HOST=${N8N_HOST}
       - N8N_PORT=${N8N_PORT}
       - N8N_PROTOCOL=${N8N_PROTOCOL}
       - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
       - DB_TYPE=${DB_TYPE}
       - N8N_BASIC_AUTH_ACTIVE=${N8N_BASIC_AUTH_ACTIVE}
       - N8N_BASIC_AUTH_USER=${N8N_BASIC_AUTH_USER}
       - N8N_BASIC_AUTH_PASSWORD=${N8N_BASIC_AUTH_PASSWORD}
       - N8N_DEFAULT_BINARY_DATA_MODE=${N8N_DEFAULT_BINARY_DATA_MODE}
       - N8N_RUNNERS_ENABLED=${N8N_RUNNERS_ENABLED}
       - N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=${N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS}
     volumes:
       - n8n_data:/home/node/.n8n
     networks:
       - my_shared_network
 

 volumes:
   n8n_data:

 networks: # <--- CHANGE THIS BLOCK
   my_shared_network: # <--- Reference the external network
     external: true
Retrieved from "https://wiki.seaoffate.net/index.php?title=Docker_Installation_with_N8N_%26_NPM&oldid=527"