While the UniFi controller is nice and everything, it does make it hard to see if a device is already adopted. At least if you have a ton of sites. Fortunately, we can search the database directly to find out if a UniFi is already adopted and which site it is assigned to.
Connect to Mongo DB
First we need to connect to MongoDB. And then we need to use the ace database.
mongo -port 27117
use ace
List all the devices on the controller
This command will list all the devices on the controller. Regardless of which site they are assigned to.
db.device.find({}, { site_id:"", ip : "", name :"", mac:""})
It is sometimes helpful to pull an APK from a working device so you can install it on a different device. These commands should work on an emulator, phone, tablet, or other Android device. You just need to be able to connect with ABD.
Connect to device with ADB
View installed apps
Find path for APK
Pull/Download APK
View Installed Apps
This will display a list of all the installed packages.
adb shell pm list packages
Find path for specific App/APK
Replace com.android.apk with the app of interest.
adb shell pm path com.android.apk
Pull APK to local machine
Pull/Download the APK of interest to your local machine with the following command. Change the path “/data/app/…” to the path returned from the previous command.
adb shell pull /data/app/info/base.apk
You can view the following link for more information.
In this case our second user id is 11. To get a list of APKs installed for our second user we would specify the –user= option
adb shell pm list packages --user=11
To get the path for the app we would run it with
adb shell pm path --user=11 com.android.apk
Split APKs
Split APKs can be slightly more difficult to manage, mainly due to the fact that there are multiple APKs to keep track of.
When you run the “pm path” command, it should return multiple APKs. Use the pull command like normal, but download each APK.
You’ll need to use a split APK installer to install all the APKs.
PowerShell script for Pulling/Downloading all APKs on Device
The following PowerShell script will download all APKs for a specific user and put them in their own folders.
Copy the contents to a .ps1 file
Enable ps1 execution policy if not already enabled
Run PowerShell script.
This script will pull all the APKs off of a device and put them in the current folder. It will also download split APKs.
# adbapkbackup uses adb to get a list of all the APKs you have on a phone and then
# Creates folders for each app and downloads the APKs for those apps.
# Copy and save code as a ps1 file
# Enable ps1 scripts to run on your computer by launching an Admin promopt and running
# set-executionpolicy remotesigned
# If you are in a secondary profile, add and/or modify
# "--user 15"
# to your user id
# adb shell pm list users
# If in secondary profile, add "--user 15" after packages before > apklist.txt
adb shell pm list packages --user 15 > apklist.txt
$apks = ((Get-Content .\apklist.txt)) -replace 'package:',''
ForEach ($apk in $apks) {
echo "APK is $apk"
md $apk
# If in secondary profile, add "--user 15" after path, before $file
adb shell pm path $apk
$filepath = ((adb shell pm path --user 15 $apk | % {$_.replace("package:","")}))
ForEach ($lapk in $filepath | % {$_.replace("package:","")}) {
echo "pulling $lapk $apk"
adb pull $lapk $apk
}
}
Create docker compose file with the following options
vi docker-compose.yml
Change the TZ to your time zone. If you have issues with the graphs, most likely something is off with the time zone between this container and Grafana/LibreNMS server
A note on SSL/TLS certificates. If you have an SSL certificate for LibreNMS, you can use it for grafana. If you run into issues, try copying the cert (fullchain.pem, privkey.pem) to /etc/grafana/
In Grafana, go to Configuration -> Data Sources -> Add data source
Set Name for Data Source
URL should be https://your_librenms_url/api/v0
Add Custom HTTP Header
Header field should be “X-Auth-Token”
Value field should contain the API token we created in LibreNMS
Save and Test If you receive any errors, refer to the Troubleshooting part at the end.
Add RRDReST Data Source
In Grafana, go to Configuration -> Data Sources -> Add data source
Set Name for Data Source
URL needs to be your docker container IP address (Steps above)
Save and Test (Should return “Unprocessable Entity”)
Import Dashboard into Grafana
Now we need a dashboard to present our data.
Go to Create -> Import
Upload JSON file (Download from here or PasteBin )
Under RRDReST API , select our RRDReST Data Source
Under LibreNMS API , select our LibreNMS Data Source
Click Import
You should now be able to view your dashboard and use the drop down menus to select devices
Troubleshooting
There were a couple of issues I ran into while trying to get everything working together.
RRDReST shows 404 Not Found
Issue: When trying to run RRDReST with uvicorn, I was never able to access the rrd files, even the test rrd files that are included when installing RRDReST. I am guessing it is either a permisions issue, or something unable to access the files. Work around: Install RRDReST via Docker container.
Error Adding LibreNMS API
Issue: Get a “JSON API: Bad Request” when trying to set up the LibreNMS API Data Source in Grafana.
Work around: Install a valid SSL Certificate and set up a DNS record so you can access LibreNMS at librenms.yourdomain.com.
More info: I would assume that “Skip TLS Verify” would work with or without a valid certificate, but it would not work for me. There are potentially some other options with modifying how Nginx or Apache is set up that would get this working. If you setup Grafana to use a SSL certificate, you may need to copy the certificate files (fullchain.pem, privkey.pem) to /etc/grafana/ and run “chown root:grafana *.pem” to let grafana have access to the files.
Running the ./validate.php script returns the following error
[FAIL] Python3 module issue found: 'Required packages: ['PyMySQL!=1.0.0', 'python-dotenv', 'redis>=3.0', 'setuptools', 'psutil>=5.6.0', 'command_runner>=1.3.0']
Package not found: The 'command_runner>=1.3.0' distribution was not found and is required by the application
'
[FIX]:
pip3 install -r /opt/librenms/requirements.txt
Running the [FIX] throws an error saying gcc failed with exit status 1.
The Solution
Fortunately this issue is easy to resolve.
First we need to install python3-devel
sudo yum install python3-devel
Next, as the librenms user, run the pip command to install the requirements.
Some of the UniFi UNVR’s have system files on a USB drive. There seem to be a number of the drives failing recently, rendering the UNVR inoperable. Fortunately it is easy to replace. The following steps should preserve the video recordings.
Remove the USB drive (use a heat gun or screw driver to break the glue that is holding the USB drive)
Install new USB drive
Temporarily remove UNVR HDDs (this may not be necessary, but rather be safe then sorry.)
Boot UNVR with new USB drive. (Give it a little time to format and copy contents to the new USB drive. Should not take more then 30 minutes.)
Setup the UNVR like it was before
Power off the UNVR again
Reinstall the HDD’s
Power on the UNVR
Log in and reconfigure the users
Note on Recovery
You could potentially mount the failed or failing USB drive on a Linux machine copy off a UniFi backup. Unfortunately, the UniFi Protect backup does not preserve the users. Just the video groups. You will probably have to resend invites to users.
Note on the video storage drives
It sounds like the UniFi Protect system will try to read the drives and if it can preserve the data or read the raid information it will try to use that. That is what it sounds like at least from the forums. More info on drive management.
When using the above to run a matrix server, it can be confusing how to verify and check which version of Postgres you are running. Fortunately this is really easy to check.
Run
sudo matrix-postgres-cli --version
And you should get the version of Postgres that is being used.