UpLoad Server

Introduction

We need a method of easily getting image files uploaded from both LAN and Internet ready to be edited and displayed on the Webservers. For most of the webservers there will not be too many images to be uploaded so direct access with a FTPS server on the webserver host will be good enough. However, for the Plum (Photo) there will be a significant amount of images to be uploaded, it is after all the Piwgo photo webserver and some pre processing and backups will be needed. We expect to do the backups from Backup Server. There will be some changes to the backup server/plum webserver to reverse the storage we will have to change the documentation for them as well.

Photo Management

The general idea is to have an upload server as a staging point to get rid of duplicates, organise metatags and maybe enhance some of the photos and eventually transfer to the Piwigo website on Plum (Photo), we will separate these duties on to this server. To make it all a self contained object we will make this a desktop Unbuntu install rather than the usual server installs. It is uncertain if that is going to be fast enough to actually edit photos but it should be good enough to edit any metatags.

Workflow Outline

The first thing to do is to dump all photos on the physical desktop PC on the 4tb drive it doesn't matter if there are duplicates as they will be addressed as part of this workflow. The important thing is to make sure there are none missing. Once there are photos to upload simply copy them to Satsuma via the SMB shares Import Export, they have been mapped to a pair of network drives (S: is Import and T: is Export). These network shares are on SMB so will only ever work on the LAN and they have been blocked from the internet. The photos are on Satsuma dir /mnt/images/Import and /mnt/images/Export. Once the files are on Satsuma they will be checked for duplicates and previously known, all new unique files will be moved to a separate input directory for processing by Digikam. When there are a few photos the user can login to the plum desktop and start Digikam (I was going to use Shotwell but it would appear that Digikam is a better fit). Digikam should pull the photos an copy them in to its own directory. The user edits the photos or adds tags to them and when finished they should be exported to another staging area to be sent to Plum (Photo). While this leaves us with several copies of the same photos we should be able to have a script periodically delete the various copies, as a long term feature the photos will be left on the Digikam working directory (it is likely that Digikam will show loads of errors if it's working directory is cleared). The running of the various scripts will be tracked with some XML log files to cope with errors. The main processes flow will be tracked by a full database on Mandarin the MySQL Server. Once Photos are added to the final staging area a script will copy them on to Plum (Photo). As they appear on Plum the photos can be displayed or not as defined by the Piwigo software. Also the photos will be copied by the Backup Server Strawberry from a NFS share from Plum to Offsite backup.

Satsuma Setup

Satsuma is a Ubuntu desktop install to allow for some graphical applications to be run, we will have to see how well that runs.

Samba Install & A ZFS Hard Drive

To allow photos to be uploaded in the simplest method is to use SMB protocol and then setup network drives on any windows PC. It should be noted that SMB is not very secure so it should only be allowed on the LAN and make appropriate rules in the firewall. To install Samba log in to the host Satsuma

sudo apt update
sudo apt install samba

Setup a drive for samba to use by adding a separate drive in Proxmox from one of the ZFS datasets. Then start and login to the VM Satsuma. once we are logged in we need to identify the disk so run

lsblk

Look through the listings for the block devices to identify the new disk. Look for the new disk (e.g., /dev/sdb). The output will show the UUID (e.g., UUID="a1b2c3d4-e5f6-7890-1234-567890abcdef" Use fdisk to create a partition on the disk.

sudo fdisk /dev/sdb 

1. Replace /dev/sdb with your disk. Inside fdisk:

• n (New partition)
• p (Primary partition)
• 1 (Partition number 1)
• Enter (Default first sector)
• Enter (Default last sector)
• w (Write changes and exit)

When that is done we need to format. To Format the partition with ext4 filesystem:

sudo mkfs.ext4 /dev/sdb1 # Replace /dev/sdb1 with your partition

We will now need to crate the mount point

sudo mkdir /mnt/images

To use the partition UUID to mount it:

sudo mount UUID="YOUR_PARTITION_UUID" /mnt/images 

1. Replace with your YOUR_PARTITION_UUID

The mount like this will disappear on reboot so to make the mount persistent we would need to edit the /etc/fstab

sudo nano /etc/fstab

WE need to add the line UUID="YOUR_PARTITION_UUID" /mnt/images ext4 defaults 0 2 # Replace with your UUID Save and close the file now the drive is set and formated we need to add the dirs for Samba to use

sudo mkdir /mnt/images/import
sudo mkdir /mnt/images/export

if the group photoshare has not been created yet

sudo groupadd photoshare 

If the users aren't yet created

sudo adduser photoup
sudo adduser nigel

add the users to photoshare

sudo usermod -aG photoshare nigel
sudo usermod -aG photoshare photoup

Then we can set the permissions of the drive

sudo chown nigel:photoshare /mnt/images/export
sudo chown nigel:photoshare /mnt/images/import

and allow full access

sudo chmod 765 /mnt/images/import
sudo chmod 765 /mnt/images/export

now that the users are ready and the had drive has been added and configured we can setup Samba

sudo nano /etc/samba/smb.conf

Scroll down towards the bottom and add [Import]

       comment = Share for importing files
       path = /mnt/images/import
       browseable = yes
       read only = no
       guest ok = no
       valid users = nigel, photoup # Users with import permissions
       create mask = 0664 # Permissions for new files
       directory mask = 0775 # Permissions for new directories

[Export]

       comment = Share for exporting files
       path = /mnt/images/export
       browseable = yes
       read only = yes # Export share is read-only
       guest ok = no
       valid users = nigel, photoup # Users with export permissions

Save and close the file then restart Samba

sudo systemctl restart smbd

Set the password within Samba

sudo smbpasswd -a photoup
sudo smbpasswd -a nigel

We will now need