How to Fix UISP “Application is loading. 1 min remaining…” Again… fluentd Error

There is an issue upgrading to UISP version 2.4.188 on Ubuntu 22.04. If you manually run an update from the command line, you’ll receive an error stating:

Error response from daemon: unknown log opt 'fluentd-async-connect' for fluentd log driver
ERROR: Failed to start Postgres DB.
UISP install script failed. Attempting rollback…
Restoring previous configuration

The Resolution

To fix the problem, we can do one of two things:

  1. Downgrade Docker.
  2. Change fluentd-async-connect to fluentd-async in docker-compose files.

The commands are taken from the comments in the following link.

https://community.ui.com/releases/UISP-Application-2-4-188/dee1603c-9f36-413c-aad6-cd6a9fc68258?page=2

Downgrade Docker

Use apt to downgrade Docker.

sudo apt install docker-ce=5:27.5.1-1~ubuntu.22.04~jammy docker-ce-cli=5:27.5.1-1~ubuntu.22.04~jammy docker-ce-rootless-extras=5:27.5.1-1~ubuntu.22.04~jammy

After Docker is downgraded, you can manually run the update again.

sudo ~unms/app/unms-cli update

Modify Docker Compose File

All we need to do is replace fluentd-async-connect with fluentd-async in the docker-compose file in the unms user directory. We can do this with sed.

sudo sed -i.orig 's/fluentd-async-connect/fluentd-async/g' ~unms/app/docker-compose.yml
sudo sed -i.orig 's/fluentd-async-connect/fluentd-async/g' ~unms/app/docker-compose.yml.template

After we run both sed commands, start or update UISP.

sudo ~unms/app/unms-cli update

How to Fix UISP “Application is loading. 1 min remaining…”

After a recent update, UISP may have stopped working.

If we run sudo ~unms/app/unms-cli status, we receive the following error:

WARN[0000] /home/unms/app/docker-compose.yml: the attribute version is obsolete, it will be ignored, please remove it to avoid potential confusion
Error response from daemon: Container 174b07e5e39d311a326c69497f1b2e1ae0eedcb067c9dada6e546ab556aad371 is restarting, wait until the container is running

It appears that there is a bug upgrading to 2.4.155.
https://community.ui.com/questions/UISP-v2-4-143-and-v2-4-155-UISP-Fails-to-start-Geomagnetism-date-comparison-BUG/0cb32012-9afb-41d4-9f37-6a9ba94c4c36

The Fix

To fix the issue, manually update from the command line again.

sudo ~unms/app/unms-cli update

Collecting Logs

There are a couple ways to check logs. We can view docker logs for specific containers using

docker logs unms

We can collect all of the logs with

sudo tar -cvjSf /tmp/uisp-logs.tar.bz2 /home/unms/data/logs

If you receive an error about bzip2 not being found, either install it sudo apt install bzip2 or change tar to tar -cvz

This will put all the logs in /tmp. You can download them with scp, sftp, winscp or something.

https://help.ui.com/hc/en-us/articles/115015690107-UISP-How-to-Find-Logs-Report-Bugs

Backup UISP Application Backup Files with Rsync

UISP runs inside of a docker container. To copy out the backup files we need to use the “docker cp” command.

sudo docker cp unms:/home/app/unms/data/unms-backups ./uisp-backups

This will copy the backups into ./uisp-backups directory.

On an Ubuntu system, docker needs sudo permissions. If you copy the backups with the above command, the backup files will be assigned to the root user and you will not be able to use your normal user to manipulate the files.

You can either add your current user to the Docker group, or change the files owner

sudo chown username:username -R ./uisp-backups/

We can now copy all the automatic backups with rsync

sudo rsync -a ./uisp-backups -e "ssh -p 22" backupuser@backuphost:/backups

You can also automate this with Cron by doing something like

1 1 * * 1 docker cp unms:/home/app/unms/data/unms-backups ~/uisp-backups && rsync -a ~/uisp-backups -e "ssh -p 22" backupuser@backuphost:/backups

Every Monday at 1:01AM, copy the current UISP automatic backups, then use rsync to copy them to a remote server.

This expects that the current user has permissions to call Docker without sudo.