How to Bypass NVIDIA NVENC Limits on RTX Cards on Linux

It appears that NVIDIA has limited the number of NVEncoding streams on consumer GPUs. Guess it is so people have to buy the more expensive professional cards.

Fortunately, the limit is only applied to the driver, and there is a patch available that let’s us bypass the limiter.

https://github.com/keylase/nvidia-patch

Install Patch

This assumes you already have the driver installed. If you do not, or run into issues with the commands below, refer to the above link.

Download the tool

https://github.com/keylase/nvidia-patch/archive/refs/heads/master.zip

wget https://github.com/keylase/nvidia-patch/archive/refs/heads/master.zip

Unzip the file

unzip nvidia-patch-master.zip

Run the patch script

cd nvidia-patch-master
sudo bash ./patch.sh

And we are finished!

Further reading

NVIDIA has a matrix of which cards support how many streams etc.

https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new

And while we are on the topic of artificial limits, check out the vGPU license bypass

https://github.com/KrutavShah/vGPU_LicenseBypass

Send an Email with Node.JS

In this post, we will be using Node.JS and the nodemailer library to send email. We need to have an email account with an email provider to send email. Gmail or some other email provider should work.

Prerequisites

First lets install some tools

sudo apt install nodejs npm

Now lets install nodemailer

npm install nodemailer

Writing the Code to Send Email

Now that we have nodemailer installed, we can write or copy our code. Create a file called maill.js and make it look similar to the following.

// We can pass in the email text as an argument
const emailText = process.argv.slice(2);
// Or we can just have it as a variable
// const emailText = "NodeJS test email message."
console.log("args " + args)

const nodemailer = require("nodemailer");

const transporter = nodemailer.createTransport({
  host: "mail.emailserver.com",
  port: 465,    //  If your email server does not support TLS, change to 587
  secure: true, // If you are using port 587, change to false.  Upgrade later with STARTTLS
  auth: {
    user: "smtpuser@emailserver.com",
    pass: "notpassword)",
  },
});

const mailOptions = {
  from: 'user@emailserver.com',
  to: "touser@email.com",
  subject: 'Test Email using NodeJS',
  text: `${emailText}`
};

transporter.sendMail(mailOptions, function(error, info){
  if (error) {
    console.log(error);
  } else {
    console.log('Email sent: ' + info.response);
  }
});

Update the following variables

  • host: to your host email server
  • user: to the email user that is sending email. It should have an account on the email server
  • pass: password for your email user that is sending the email
  • from: email address that is sending the email
  • to: email account(s) you are sending email to
  • subject: subject of your email

Now we can proceed to send email

Sending Email

We can now run the code by saving our file and running it directly with NodeJS

nodejs ./mail.js "This is the body text for the email"

Hit Return and look for the email. If something went wrong, it should throw an error.

You can change the emailText variable if you would rather have the message body inside the code.

Code Explanation and Notes

A little explanation on the code.

The second line “const emailText = process.argv.slice(2);” is used to pass in a command line argument to use as the text for the body of the email. You can delete the line and uncomment line 4 if you would rather use a variable inside the code.

Your email server should support using SSL/TLS on port 465. If it does not, you may need to use STARTTLS which uses port 587, and then set secure to false. STARTTLS should upgrade the connection to be encrypted. But it’s opportunistic. You can read more about STARTTLS, SSL/TLS here https://mailtrap.io/blog/starttls-ssl-tls/

You can change the “to: ” in the mailOptions object to an array of email addresses to send the email to multiple people at once.

to: ["email1@email.com", "email2@email.com", "etc"],

JavaScript check if a string matches any element in an Array

In the following code we will be checking a string and check if any of the words in the string match some or any elements in an array.

We can imagine that our “stringToCheck” variable is an email or message response. We want to know if it contains a mention to afternoon, tomorrow, or evening. Presumably so we can automate something. Note that the matches are case sensitive.

// Check if any text in a string matches an element in an array

const stringToCheck = "Let's grab lunch tomorrow";
const arrayToCompareTo =["afternoon", "tomorrow", "evening"];

// We are checking to see if our string "stringToCheck" 
// Has any of the words in "arrayToCompareTo".
// If it does, then the result is true.  Otherwise false.
const resultsOfCompare = arrayToCompareTo.some(checkVariable => stringToCheck.includes(checkVariable));

if (resultsOfCompare == true){
    console.log(stringToCheck + " Contains a value in our Array " + arrayToCompareTo);
} else {
    console.log(stringToCheck + " Does NOT contain a value in our Array " + arrayToCompareTo);
}

More examples and ways to do it are available at the following link.

https://stackoverflow.com/questions/37428338/check-if-a-string-contains-any-element-of-an-array-in-javascript

Setting up Databricks Dolly on Ubuntu with GPU

This is a quick guide for getting Dolly running on an Ubuntu machine with Nvidia GPUs.

You’ll need a good internet connection and around 35GB of hard drive space for the Nvidia driver, Dolly (12b model) and extras. You can use the smaller models to take up less space. The 8 billion parameter model uses about ~14GB of space while the 3 billion parameter one is around 6GB

Install Nvidia Drivers and CUDA

sudo apt install nvidia-driver nvidia-cuda-toolkit

Reboot to activate the Nvidia driver

reboot

Install Python

Python should already be installed, but we do need to install pip.

Once pip is installed, then we need to install numpy, accelerate, and transformers

sudo apt install python3-pip
pip install numpy
pip install accelerate>=0.12.0 transformers[torch]==4.25.1

Run Dolly

Run a python console. If you run it as administrator, it should be faster.

python3

Run the following commands to set up Dolly.

import torch
from transformers import pipeline

generate_text = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")

# Alternatively, If you want to use a smaller model run

generate_text = pipeline(model="databricks/dolly-v2-3b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")

Notes:

  1. If you have issues, you may want/need to specify an offload folder with offload_folder=”.\offloadfolder”. An SSD is preferable.
  2. If you have lots of RAM, you can take out the “torch_dtype=torch.bfloat16”
  3. If you do NOT have lots of ram (>32GB), then you may only be able to run the smallest model

Alternatively, if we don’t want to trust_remote_code, we can download this file, and run the following

from instruct_pipeline import InstructionTextGenerationPipeline
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-12b", padding_side="left")
model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-12b", device_map="auto")

generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)

Now we can ask Dolly a question.

generate_text("Your question?")

Example:

>>> generate_text("Tell me about Databricks dolly-v2-3b?")
'Dolly is the fully managed open-source engine that allows you to rapidly build, test, and deploy machine learning models, all on your own infrastructure.'

Further information is available at the following two links.

https://github.com/databrickslabs/dolly
https://huggingface.co/databricks/dolly-v2-3b
https://huggingface.co/databricks/dolly-v2-12b

Install and Use OpanAI Whisper

These commands work for Ubuntu. Should be simple to change for other Linux distros.

Install Nvidia and CUDA drivers

sudo apt install nvidia-driver-530 nvidia-cuda-toolkit

Reboot so the system uses the driver.

Install pip and ffmpeg

sudo apt install python3-pip
sudo apt install ffmpeg

Now we can install whisper with

pip install -U openai-whisper

Run Whisper

After it is installed, it should be able to run it like

whisper audio.mp3 --model medium

Change out medium to the model you would like to use. It will then download the model and then work get to work on transcribing it. The .en models i.e. medium.en, seem to perform better then the other ones. If you are using English that is.

If you receive a “Command ‘whisper’ not found” error, you may not have ~/.local/bin in your user PATH. Either add ~/.local/bin to your PATH, or run whisper with the full path

~/.local/bin/whisper audio.mp3 --model medium

OpenAI Whisper GitHub link.
https://github.com/openai/whisper

Simple JavaScript Object – Code Example

Below is a code example for creating a basic object and using a function to calculate the fuel economy.

// New object Car
const car = {
    make: 'Honda',
    model: 'Civic',
    topSpeed: 100,
    tankCapacity: 10,
    range: 300,
    MPG: function() {
        this.mpg = this.range / this.tankCapacity;
        return this.mpg
    }
}

car.MPG();  // We need to call this to calculate the MPG, otherwise we get undefined

console.log(`My car is a ${car.make + " " + car.model }, can go ${car.topSpeed}/MPH, and gets ${car.mpg}/MPG `)

// Alternatively we can call the function car.MPG() directly.  
// This keeps us from having to run the function before logging.
console.log(`My car is a ${car.make + " " + car.model }, can go ${car.topSpeed}/MPH, and gets ${car.MPG()}/MPG `)

Setup Remote Syslog on Cisco

Configure Logging

First we need to drop into configuration mode

conf t

Now we run the following command. Change ip-address to the address of you remote syslog server.

logging host ip-address

You will want to make sure that your time/timezone is correct.

https://community.cisco.com/t5/networking-knowledge-base/how-to-configure-logging-in-cisco-ios/ta-p/3132434

Set timezone

Change UTC and 0 to your your timezone and how many hours off UTC you are. For example for EST you would do EST -5

clock timezone UTC 0

Here are just the commands

terminal config
logging on
logging logserveraddress
clock timezone UTC 0
quit
wr

Setting up Databricks Dolly on Windows with GPU

The total process can take awhile to setup Dolly. You’ll need a good internet connection and around 50GB of hard drive space.

Install Nvidia CUDA Toolkit

You’ll need to install the CUDA Toolkit to take advantage of the GPU. The GPU is much faster than just using the CPU.

https://developer.nvidia.com/cuda-downloads

Install Git

Install git from the following site.

https://git-scm.com/downloads

Download Dolly

Download Dolly with git.

git lfs install 
git clone https://huggingface.co/databricks/dolly-v2-12b

Install Python

We’ll also need Python installed if it is not already.
https://www.python.org/downloads/release/

Next we’ll need the following installed

py.exe -m pip install numpy
py.exe -m pip install accelerate>=0.12.0 transformers[torch]==4.25.1
py.exe -m pip install numpy --pre torch --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu117 --user

The last one is needed to get Dolly to utilize a GPU.

Run Dolly

Run a python console. If you run it as administrator, it should be faster.

py.exe

Run the following commands to set up Dolly.

import torch
from transformers import pipeline

generate_text = pipeline(model="databricks/dolly-v2-3b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")

# Or to use the full model run

generate_text = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")

Note: if you have issues, you may want/need to specify an offload folder with offload_folder=”.\offloadfolder”. An SSD is preferable.
Also if you have lots of RAM, you can take out the “torch_dtype=torch.bfloat16”

Alternatively, if we don’t want to trust_remote_code, we can do run the following

from instruct_pipeline import InstructionTextGenerationPipeline
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-3b", padding_side="left")
model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-3b", device_map="auto")

generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)

Now can ask Dolly a question.

generate_text("Your question?")

Example:

>>> generate_text("Tell me about Databricks dolly-v2-3b?")
'Dolly is the fully managed open-source engine that allows you to rapidly build, test, and deploy machine learning models, all on your own infrastructure.'

Further information is available at the following two links.

https://github.com/databrickslabs/dolly
https://huggingface.co/databricks/dolly-v2-3b