UniFi-Video Service failing to Start

Do note that the UniFi-Video software is no longer supported should be migrated to the UniFi Protect systems.

Running “systemctl status unifi-video” gives the following error.

(unifi-video) checking for system.properties and truststore files…
…fail!

It appears that the error is due to an unsupported version of Java. Installing an older version of Java resolves the issue.

You can check which versions of Java are available with

apt-cache show openjdk-8-jre

The versions that showed up for me were

8u282-b08-0ubuntu1
8u252-b09-1ubuntu1 

Installing the last one fixed the issue for now.

sudo apt-get install openjdk-8-jre-headless=8u252-b09-1ubuntu1

https://old.reddit.com/r/Ubiquiti/comments/l30jm5/unifi_video_31013_not_compatible_with_openjdk_180/

Notes on Migrating UniFi-Video NVR Disk

Recently upgraded a UniFi-Video VM by creating a new VM and restoring the backup from the previous one. There was a separate virtual disk used for storing all the videos. Although once I reattached it, I had to change the owner to the unifi-video user. lt was reporting that the owner/group was 127 and 134, looks like that was the user ID from the previous VM.

Example output from ls -hla command

drwxr-xr-x 3 127 134 4.0K Jan 16 2020 176de761-8399-3bed-8ceb-917cbe4d25bd
drwxr-xr-x 2 127 134 4.0K Sep 10 12:28 21d506ef-dded-31f2-baf8-c109a7e653b9
drwxr-xr-x 4 127 134 4.0K Jan 1 2020 1a3afc4c-a030-39a1-b139-857444c8b71c

Ended up changing the owner to the unifi-video owner with the chown command. /path/to/videos is the mount point of the video drive.

chown -R unifi-video:unifi-video /path/to/videos

Configuring UFW Firewall on Ubuntu for UniFi and UniFi-Video

You should be able to copy and paste all the following commands in. You can check the status of the commands after with

sudo ufw status

Allowing access to UniFi ports

sudo ufw allow 3478/UDP
sudo ufw allow 5514/UDP
sudo ufw allow 8080/TCP
sudo ufw allow 8443/TCP
sudo ufw allow 8880/TCP
sudo ufw allow 8843/TCP
sudo ufw allow 6789/TCP
sudo ufw allow 27117/TCP
sudo ufw allow 10001/UDP
sudo ufw allow 1900/UDP

UniFI Video ports

sudo ufw allow 7080/TCP
sudo ufw allow 7443/TCP
sudo ufw allow 7445/TCP
sudo ufw allow 7446/TCP
sudo ufw allow 7447/TCP
sudo ufw allow 10001/UDP
sudo ufw allow 6666/TCP
sudo ufw allow 7004/UDP
sudo ufw allow 7442/TCP

Helpful links

https://help.ui.com/hc/en-us/articles/217875218-UniFi-Video-Ports-Used
https://help.ui.com/hc/en-us/articles/218506997-UniFi-Ports-Used

MongoDB – dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)

Ran into an issue when trying to figure out some problems with UniFi and UniFi-Video. Upgrading both to the latest version was causing problems, because they both needed different versions of MongoDB.

Was getting the following error when trying to install MongoDB. Think I was trying to install Mongo following the instructions on their site and then it caused issues with apt.

dpkg: error processing archive /var/cache/apt/archives/mongodb-org-server_4.2.5_amd64.deb (--unpack):
trying to overwrite '/usr/bin/mongod', which is also in package mongodb-server-core 1:3.6.3-0ubuntu1.1
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)

Removed the following file

rm /etc/apt/sources.list.d/mongodb-org-4.2.list

and was able to use apt again to install MongoDB. Didn’t end up solving my problem because I technically needed two versions, but at least apt was being nice again. Ended up installing the UniFi controller docker container and can run both on the same server that way.

Auto renew ssl cert for UniFI and UniFi-Video

The following script was taken from here

Added unifi-video support. Script uses letsencrypt to get the cert and automatically updates the UniFi and UniFi-Video Keystores.

Would be a good idea to check and make sure the the UniFi-Video cameras reconnect and still work after running script.

Installing Let’s Encrypt

Install Let’s Encrypt with the following

sudo apt install letsencrypt

And generate a cert for your domain with

sudo certbot certonly -d unifi.domain.com

Executing Script to Renew Certificate

Copy the script at the bottom of this post and put it in a file called gen-unifi-cert.sh
Run the script to insert the cert into the UniFi and UniFi-Video services.

sudo ./gen-unifi-cert.sh -e email@domain.com -d unifi.domain.com

You can run it with no or the -h argument to show the options and arguments to use.

./gen-unifi-cert.sh -h

Setup Cron Job

You should be able to add the following to a cronjob to auto renew the certificate. Replace path to script and domain name.

30 2 * * * /root/gen-unifi-cert.sh -r -d unifi.domain.com

UniFi SSL Cert Renew Script

#!/usr/bin/env bash
# Added support to do UniFi and UniFi controllers at the same time using the same cert.
# Original script from https://git.sosdg.org/brielle/lets-encrypt-scripts/raw/branch/master/gen-unifi-cert.sh
# More info here https://www.reddit.com/r/Ubiquiti/comments/43v23u/using_letsencrypt_with_the_unifi_controller/ 
# And here https://www.reddit.com/r/Ubiquiti/comments/43v23u/using_letsencrypt_with_the_unifi_controller/
# Modified script from here: https://github.com/FarsetLabs/letsencrypt-helper-scripts/blob/master/letsencrypt-unifi.sh
# Modified by: Brielle Bruns <bruns@2mbit.com>
# Download URL: https://source.sosdg.org/brielle/lets-encrypt-scripts
# Version: 1.7
# Last Changed: 09/26/2018
# 02/02/2016: Fixed some errors with key export/import, removed lame docker requirements
# 02/27/2016: More verbose progress report
# 03/08/2016: Add renew option, reformat code, command line options
# 03/24/2016: More sanity checking, embedding cert
# 10/23/2017: Apparently don't need the ace.jar parts, so disable them
# 02/04/2018: LE disabled tls-sni-01, so switch to just tls-sni, as certbot 0.22 and later automatically fall back to http/80 for auth
# 05/29/2018: Integrate patch from Donald Webster <fryfrog[at]gmail.com> to cleanup and improve tests
# 09/26/2018: Change from TLS to HTTP authenticator

# Location of LetsEncrypt binary we use.  Leave unset if you want to let it find automatically
#LEBINARY="/usr/src/letsencrypt/certbot-auto"

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

function usage() {
  echo "Usage: $0 -d <domain> [-e <email>] [-r] [-i]"
  echo "  -d <domain>: The domain name to use."
  echo "  -e <email>: Email address to use for certificate."
  echo "  -r: Renew domain."
  echo "  -i: Insert only, use to force insertion of certificate."
}

while getopts "hird:e:" opt; do
  case $opt in
    i) onlyinsert="yes";;
    r) renew="yes";;
    d) domains+=("$OPTARG");;
    e) email="$OPTARG";;
    h) usage
       exit;;
  esac
done

DEFAULTLEBINARY="/usr/bin/certbot /usr/bin/letsencrypt /usr/sbin/certbot
  /usr/sbin/letsencrypt /usr/local/bin/certbot /usr/local/sbin/certbot
  /usr/local/bin/letsencrypt /usr/local/sbin/letsencrypt
  /usr/src/letsencrypt/certbot-auto /usr/src/letsencrypt/letsencrypt-auto
  /usr/src/certbot/certbot-auto /usr/src/certbot/letsencrypt-auto
  /usr/src/certbot-master/certbot-auto /usr/src/certbot-master/letsencrypt-auto"

if [[ ! -v LEBINARY ]]; then
  for i in ${DEFAULTLEBINARY}; do
    if [[ -x ${i} ]]; then
      LEBINARY=${i}
      echo "Found LetsEncrypt/Certbot binary at ${LEBINARY}"
      break
    fi
  done
fi

# Command line options depending on New or Renew.
NEWCERT="--renew-by-default certonly"
RENEWCERT="-n renew"

# Check for required binaries
if [[ ! -x ${LEBINARY} ]]; then
  echo "Error: LetsEncrypt binary not found in ${LEBINARY} !"
  echo "You'll need to do one of the following:"
  echo "1) Change LEBINARY variable in this script"
  echo "2) Install LE manually or via your package manager and do #1"
  echo "3) Use the included get-letsencrypt.sh script to install it"
  exit 1
fi

if [[ ! -x $( which keytool ) ]]; then
  echo "Error: Java keytool binary not found."
  exit 1
fi

if [[ ! -x $( which openssl ) ]]; then
  echo "Error: OpenSSL binary not found."
  exit 1
fi

if [[ ! -z ${email} ]]; then
  email="--email ${email}"
else
  email=""
fi

shift $((OPTIND -1))
for val in "${domains[@]}"; do
        DOMAINS="${DOMAINS} -d ${val} "
done

MAINDOMAIN=${domains[0]}

if [[ -z ${MAINDOMAIN} ]]; then
  echo "Error: At least one -d argument is required"
  usage
  exit 1
fi

if [[ ${renew} == "yes" ]]; then
  LEOPTIONS="${RENEWCERT}"
else
  LEOPTIONS="${email} ${DOMAINS} ${NEWCERT}"
fi

#if [[ ${onlyinsert} != "yes" ]]; then
if [[ ${onlyinsert} == "yes" ]]; then
  echo "Firing up standalone authenticator on TCP port 80 and requesting cert..."
  ${LEBINARY} --server https://acme-v01.api.letsencrypt.org/directory \
              --agree-tos --standalone --preferred-challenges http ${LEOPTIONS}
fi

#if [[ ${onlyinsert} != "yes" ]] && md5sum -c "/etc/letsencrypt/live/${MAINDOMAIN}/cert.pem.md5" &>/dev/null; then
if [[ ${onlyinsert} == "yes" ]] && md5sum -c "/etc/letsencrypt/live/${MAINDOMAIN}/cert.pem.md5" &>/dev/null; then
  echo "Cert has not changed, not updating controller."
  exit 0
else
  echo "Cert has changed or -i option was used, updating controller..."
  TEMPFILE=$(mktemp)
  CATEMPFILE=$(mktemp)

  # Identrust cross-signed CA cert needed by the java keystore for import.
  # Can get original here: https://www.identrust.com/certificates/trustid/root-download-x3.html
  cat > "${CATEMPFILE}" <<'_EOF'
-----BEGIN CERTIFICATE-----
MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow
PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD
Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O
rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq
OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b
xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw
7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD
aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV
HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG
SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69
ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr
AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz
R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5
JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo
Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ
-----END CERTIFICATE-----
_EOF

  md5sum "/etc/letsencrypt/live/${MAINDOMAIN}/cert.pem" > "/etc/letsencrypt/live/${MAINDOMAIN}/cert.pem.md5"
  echo "Using openssl to prepare certificate..."
  cat "/etc/letsencrypt/live/${MAINDOMAIN}/chain.pem" >> "${CATEMPFILE}"
  openssl pkcs12 -export  -passout pass:aircontrolenterprise \
          -in "/etc/letsencrypt/live/${MAINDOMAIN}/cert.pem" \
          -inkey "/etc/letsencrypt/live/${MAINDOMAIN}/privkey.pem" \
          -out "${TEMPFILE}" -name unifi \
          -CAfile "${CATEMPFILE}" -caname root

  echo "Stopping Unifi and UniFi-Video controllers..."
  systemctl stop unifi unifi-video  

  echo "Removing existing certificate from Unifi protected keystore..."
  keytool -delete -alias unifi -keystore /usr/lib/unifi/data/keystore -deststorepass aircontrolenterprise
  echo "Removing existing certificate from Unifi-Video protected keystore..."
  keytool -delete -alias unifi -keystore /usr/lib/unifi-video/data/keystore -deststorepass ubiquiti
  # following lines are needed for unifi-video
 
  echo "Inserting certificate into Unifi keystore..."
  keytool -trustcacerts -importkeystore \
          -deststorepass aircontrolenterprise \
          -destkeypass aircontrolenterprise \
          -destkeystore /usr/lib/unifi/data/keystore \
          -srckeystore "${TEMPFILE}" -srcstoretype PKCS12 \
          -srcstorepass aircontrolenterprise \
          -alias unifi

  echo "Inserting certificate into Unifi-Video keystore..."
  keytool -trustcacerts -importkeystore \
          -deststorepass ubiquiti \
          -destkeypass ubiquiti \
          -destkeystore /usr/lib/unifi-video/data/keystore \
          -srckeystore "${TEMPFILE}" -srcstoretype PKCS12 \
          -srcstorepass aircontrolenterprise \

          rm -f "${TEMPFILE}" "${CATEMPFILE}"

  mv /usr/lib/unifi-video/data/ufv-truststore{,.old} # Delete old unifi-video keystore
  sleep 5
  echo "Starting Unifi and UniFi-Video controllers..."
  systemctl start unifi unifi-video 

  echo "Done!"
fi

Errors Renewing Lets Encrypt Certificate for UniFi-Video

Had an issue with the Lets Encrypt cert for a UniFi-Video server.  When renewing the cert and reimporting it into the UniFi-Video keystore, the certification was showing out of date.

Issue ended up being something with certbot.

When certbot runs it generates a new cert.pem, chain.pem, fullchain.pem and privkey.pem and puts them in the “/etc/letsencrypt/live/unifi.domain.com/” directory.

The privkey.pem and cert.pem are used to create the keys.p12 file which gets imported into the UniFi-Video keystore.

Apparently the .pem files in “/etc/letsencrypt/live/unifi.domain.com/” are symbolic links to files in “/etc/letsencrypt/archive/unifi.domain.com/”

Upon inspection of the archive directory, multiple cert.pem and privkey.pem files were found with the names cert1.pem, cert2.pem, cert3.pem etc.  Looking at the creation date of the file revealed the symbolic link was referring to an old “cert1.pem” file.

Work around was to stop the unifi-video service and reimport the cert using the latest .pem files in the archive directory.

echo ubiquiti | openssl pkcs12 -export -inkey /etc/letsencrypt/archive/unifi.yourdomain.com/privkey2.pem -in /etc/letsencrypt/archive/unifi.yourdomain.com/cert2.pem -name airvision -out /usr/lib/unifi-video/data/keys.p12 -password stdin
echo y | keytool -importkeystore -srckeystore /etc/letsencrypt/archive/unifi.yourdomain.com/keys.p12 -srcstoretype pkcs12 -destkeystore /usr/lib/unifi-video/data/keystore -storepass ubiquiti -srcstorepass ubiquiti

Remove the old ufv-truststore and start the service.

mv /usr/lib/unifi-video/data/ufv-truststore{,.old}
systemctl start unifi-video

Worked like a charm.

Add a SSL Certificate to Ubiquiti UniFi-Video server using Lets Encrypt

Install certbot

sudo apt-get install python-certbot

Generate certificate.  Change unifi.yourdomain.com to the domain name you have pointing to your UniFi-Video controller.

sudo certbot certonly -d unifi.yourdomain.com

Certbot will create the files in “/etc/letsencrypt/live/unifi.yourdomain.com/”

Now you should stop the unifi service.

systemctl stop unifi-video

The following two commands create and install the keystore for the UniFi-Video application.  These commands were copied from here.  Thanks scobber!

echo ubiquiti | openssl pkcs12 -export -inkey /etc/letsencrypt/live/unifi.yourdomain.com/privkey.pem -in /etc/letsencrypt/live/unifi.yourdomain.com/cert.pem -name airvision -out /usr/lib/unifi-video/data/keys.p12 -password stdin
echo y | keytool -importkeystore -srckeystore /etc/letsencrypt/live/unifi.yourdomain.com/keys.p12 -srcstoretype pkcs12 -destkeystore /usr/lib/unifi-video/data/keystore -storepass ubiquiti -srcstorepass ubiquiti

Remove or rename the Trusted Store.  If you don’t, the cameras will connect, but will not record.  The controller will rebuild the ufv-truststore when it starts up and the cameras will be able to record.

mv /usr/lib/unifi-video/data/ufv-truststore{,.old}

Start the UniFi-Video service

systemctl start unifi-video

Now you can check it by going to https://unifi.yourdomain.com:8443

Ubuntu UniFi server running out of space on /run

Had an issue that /run was randomly running out of space which in turn would interfere with the unifi-video service causing it to run, but not record.

/run looks like a tmpfs or ramdisk that Ubuntu sets up.  So you can do a “temporary” fix by remounting the tmpfs with a larger size.  Example below.  If /run is a 2GB directory, you can remount changing the size from 2GB to 2.5GB.

sudo mount -t tmpsfs tmpfs /run -o remount,size=2500M

Note that it is a temporary fix and goes away after a reboot.

The issue ended up being that the WiFi UniFi controller was setup to auto backup everything once a week.  So as it was backing stuff up, it would eat up the available space in the tmpfs, think there may be an issue with the size of the UniFi data and maybe not being able to fit it all in RAM?

Running the following command

df -h --max=1 /var | sort

shows the following

1.1M /run/udev
2.5G /run/
2.5G /run/unifi   <-- UniFi controller
4.0K /run/initramfs
8.0K /run/network
12K /run/user
288K /run/samba
404K /run/systemd

Looking inside the unifi directory shows the following folders.  Looks like the they are temp files.

200M /run/unifi/ExpTmp351719567129045774
696M /run/unifi/ExpTmp3406220793759111216
1.6G /run/unifi/ExpTmp3368400690321364109
0 /run/unifi/work
2.5G /run/unifi

Running an ls inside the folder shows

-rw-r----- 1 unifi unifi 13971807 Jul 2 02:30 db.gz
-rw-r----- 1 unifi unifi 1665223462 Jul 2 02:56 db_stat.gz

Looking inside the UniFi controller it is set to auto backup on Monday at 2:30AM

Looks like “/run/unifi” is used as a temporary folder to create the backups and when it is completed converts it to a .unf file and moves it to “/usr/lib/unifi/data/backup/autobackup/”  So the left over temp files were never completed or something caused them to stop working.  Maybe the backup was big enough to take up all the tmpfs space and cause the backup to fail.

Resolution.  To fix the problem I turned the data retention down to a week and deleted the temp files in /var/unifi.  Had to sudo su to root.  sudo wasn’t cutting it for some reason.

sudo su
rm -rf /var/unifi/Exp*
exit

Should be all good now.

Add extra drive to UniFi Video Controller

Find your disk drive using fdisk or some other utility.  More info here.

fdisk -l

Example:

steve@ubuntu:~$ sudo fdisk -l
Disk /dev/sda: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe96d30af

Device     Boot    Start      End  Sectors Size Id Type
/dev/sda1  *        2048 58720255 58718208  28G 83 Linux
/dev/sda2       58722302 67106815  8384514   4G  5 Extended
/dev/sda5       58722304 67106815  8384512   4G 82 Linux swap / Solaris


Disk /dev/sdb: 557 GiB, 598074195968 bytes, 1168113664 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
steve@ubuntu:~$

The drive is the 500GiB drive /dev/sdb

Create a partition on the drive.  Warning!  Be careful when formatting drives and make sure you don’t format the wrong one!

echo ';' | sfdisk /dev/sdb

You should now see the partition when you run fdisk -l again.  If you run into issues mounting the drive, try formating the drive with a different utility i.e.(fdisk, cfdisk, gparted etc.)

steve@ubuntu:~$ sudo fdisk -l
Disk /dev/sdb: 557 GiB, 598074195968 bytes, 1168113664 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x717f99ba

Device     Boot Start        End    Sectors  Size Id Type
/dev/sdb1        2048 1168113663 1168111616  557G 83 Linux   <--New Partition
steve@ubuntu:~$

Create mount point

mkdir -p /video/mount/point

Now mount the partition

mount /dev/sdb1 /video/mount/point

Allow unifi-video user to own the mount point

chown -R video/mount/point

Add the drive to /etc/fstab to auto mount on system bootup.  More info here.

Change /dev/sdXx to your drive and /MOUNTLOCATION to your mount location.

sudo echo "/dev/sdXx /MOUNTLOCATION ext4 rw,defaults 0 0 " >> /etc/fstab

Example:

sudo echo "/dev/sdb1 /video/mount/point ext4 rw,defaults 0 0 " >> /etc/fstab

If you have issues running the above command, you can manually add it to the file with

sudo vi /etc/fstab

add the following line to the bottom of the file.  Change sdXx to your drive and /MOUNTLOCATION to the mount location of the drive.

/dev/sdXx /MOUNTLOCATION ext4 rw,defaults 0 0

You can reboot the system to make sure that the drive mounts properly.

Now you can log into the UniFi Video controller and set the Recording Path by going to Settings>SYSTEM CONFIGURATION>CONFIGURE and set the Recording Path to /video/mount/point

Tip. In the SYSTEM CONFIGURATION menu you can hover over the Disk and it’ll show you your total amount of recording space.

Install UniFi Video 3.8.5 on Ubuntu or Debian

You can run all the commands from the terminal, or ssh into the server

See here if you need to setup SSH on the server.

Install prerequisites

sudo apt-get install mongodb mongodb-server openjdk-8-jre-headless jsvc

Download UniFi Video installer

Note the Debian package works on Ubuntu, and has been tested on the latest Ubuntu

wget https://dl.ubnt.com/firmwares/ufv/v3.8.5/unifi-video.Debian7_amd64.v3.8.5.deb

Install package

sudo dpkg -i unifi-video.Debian7_amd64.v3.8.5.deb

Login to the UniFi Video controller using your web browser and going to the following address to finish configuring the NVR.

https://your-server-address:7443