Harden SSH for AlmaLinux 9 (RHEL, Fedora)

These steps are taken from the following link. They have other guides for hardening Ubuntu, Debian etc.

https://www.sshaudit.com/hardening_guides.html#rocky9

You will need to become the root user, use either su – or sudo -i

First we need to regenerate the RSA and ED25519 keys

rm /etc/ssh/ssh_host_*
ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_rsa_key -N ""

Next, remove the small Diffie-Hellman moduli. The moduli file contains prime numbers and generators. Removing the smaller numbers should help increase security as it makes attempting to factor the private keys harder.

awk '$5 >= 3071' /etc/ssh/moduli > /etc/ssh/moduli.safe
mv /etc/ssh/moduli.safe /etc/ssh/moduli

We can now specify which key exchange, ciphers, and algorithms to use.

Add the following to “/etc/crypto-policies/back-ends/opensshserver.config”

# Restrict key exchange, cipher, and MAC algorithms, as per sshaudit.com
# hardening guide.
KexAlgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,gss-curve25519-sha256-,diffie-hellman-group16-sha512,gss-group16-sha512-,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha256

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

MACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com

HostKeyAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256

RequiredRSASize 3072

CASignatureAlgorithms sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256

GSSAPIKexAlgorithms gss-curve25519-sha256-,gss-group16-sha512-

HostbasedAcceptedAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-256

PubkeyAcceptedAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256

Finally, restart the ssh server

systemctl restart sshd

Other helpful links

https://www.ssh.com/academy

https://www.redhat.com/en/blog/primes-parameters-and-moduli

https://security.stackexchange.com/questions/79043/is-it-considered-worth-it-to-replace-opensshs-moduli-file

Configuring Firewalld with Ansible

We’ll be using Ansible to change and maintain our firewall settings on a server.

The playbook will do the following.

  1. Set the default zone to drop (Drops all external traffic to server)
  2. Set a zone for internal access
  3. Allow access from RFC1918 addresses to internal zone (Any local IP address will be able to access the server)
  4. Enable the services and ports specified in the vars section
  5. Disable the services listed in firewall_disable_services variable

Modify the variables as needed for your server(s). You can also add or move the variables to the inventory or host_vars files.

If you need to create an inventory file, refer to the first part of this post

BE CAREFUL CHANGING FIREWALL SETTINGS!!! IMPROPER SETTINGS COULD RENDER THE SERVER INACCESSIBLE!!!

Playbook for firewalld

Change the variables under the vars section

---
- name: Configure firewalld
  hosts: rhel
  gather_facts: yes
  become: yes

  vars: 
    firewall_allowed_ips:
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 192.168.0.0/16
    firewall_allowed_services:
      - ssh
      - https
      - snmp
    firewall_allowed_ports:
      - "2222/tcp"
    firewall_disable_services:
      - cockpit
      - dhcpv6-client
      - mdns
      - samba-client

  tasks: 
  - name: Set default zone to drop
    ansible.builtin.command: firewall-cmd --set-default-zone=drop
    register: default_zone_set
    changed_when:
      - '"ZONE_ALREADY_SET" not in default_zone_set.stderr'

  - name: Enable and allow access to internal zone from RFC1918 addresses
    ansible.posix.firewalld:
      source: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: enabled
    with_items: "{{ firewall_allowed_ips }}"

  - name: Disable unused services for internal zone
    ansible.posix.firewalld:
      service: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: disabled
    with_items: "{{ firewall_disable_services }}"


  - name: Set services for internal zone
    ansible.posix.firewalld:
      service: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: enabled
    with_items: "{{ firewall_allowed_services }}"

  - name: Set custom ports for internal zone
    ansible.posix.firewalld:
      port: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: enabled
    with_items: "{{ firewall_allowed_ports }}"

Helpful links

https://docs.ansible.com/ansible/latest/collections/ansible/posix/firewalld_module.html#parameter-source

https://stackoverflow.com/questions/51563643/how-to-change-firewalld-zone-using-ansible

https://www.middlewareinventory.com/blog/ansible-firewalld/

Ansible Playbook for Updating Mikrotik RouterOS

This playbook is for updating Mikrotik routers. It will update both the RouterOS version and the firmware.

The playbook executes in the following order.

  1. Check for RouterOS Updates
  2. Update RouterOS (Router will reboot if there is an update)
  3. Sleep 120 seconds to allow the router(s) to boot up
  4. Check current firmware version, and if there is an available upgrade
  5. Update firmware
  6. Reboot router to apply firmware upgrade

This playbook attempts to be smart and will not reboot a router if there is not an update available. Routers that have updates available will reboot twice. Once to apply the RouterOS version, and the second time to apply the firmware.

Prerequisites

You should already have an inventory file and the Ansible RouterOS collection installed. If not, check out the following post.

Setup Ansible host file and RouterOS collection

Playbook

Here is the playbook.
A quick command syntax note, RouterOS 7 and newer typically use slashes / between commands. i.e. /system/package/update/install. Older versions of RouterOS have spaces in the command path i.e. /system package update install Since this still works on newer versions, we use it here.

---
- name: Mikrotik RouterOS and Firmware Upgrades
  hosts: routers
  gather_facts: false
  tasks:

# Update RouterOS version.  Mikrotik update/install command automatically reboots the router
  - name: Check for RouterOS updates
    community.routeros.command:
      commands:
        - /system package update check-for-updates
    register: system_update_print

  - name: Update RouterOS version
    community.routeros.command:
      commands:
        - /system package update install
    when: system_update_print is not search('System is already up to date')

# Check if firmware needs an upgrade, upgrade and reboot.
  - name: Sleeping for 120 seconds.  Giving time for routers to reboot.
    ansible.builtin.wait_for:
      timeout: 120
    delegate_to: localhost
      
  - name: Check Current firmware
    community.routeros.command:
      commands:
        - ':put [/system routerboard get current-firmware]'
    register: firmware_current

  - name: Check Upgrade firmware 
    community.routeros.command:
      commands:
        - ':put [/system routerboard get upgrade-firmware]'
    register: firmware_upgrade

  - name: Upgrade firmware
    community.routeros.command:
      commands:
        - ':execute script="/system routerboard upgrade"'
    when: firmware_current != firmware_upgrade

  - name: Wait for firmware upgrade and then reboot
    community.routeros.command:
      commands:
        - /system routerboard print
    register: Reboot_Status
    until: "Reboot_Status is search(\"please reboot\")"
    notify:
      - Reboot Mikrotik
    retries: 3
    delay: 15
    when: firmware_current != firmware_upgrade

  handlers:
    - name : Reboot Mikrotik
      community.routeros.command:
        commands:
          - ':execute script="/system reboot"'

Run the playbook with

ansible-playbook -i routers.ini mikrotik_update.yaml

Change routers.ini to your router inventory.
mikrotik_update.yaml to whatever you end up calling the playbook.

Setup Ansible and Mikrotik RouterOS

https://docs.ansible.com/ansible/devel/collections/community/routeros/

https://github.com/ansible-collections/community.routeros

Install the RouterOS collection.

ansible-galaxy collection install community.routeros

Create inventory

vi inventory.ini
[routers]
mikrotik ansible_host=192.168.88.1

[routers:vars]
ansible_connection=ansible.netcommon.network_cli
ansible_network_os=community.routeros.routeros
ansible_user=admin
ansible_ssh_pass=
ansible_ssh_port=22

If you are using a custom SSH port, be sure that ansible-pylibssh is installed.

pip install ansible-pylibssh

Simple Playbook

This simple playbook will print the system resources. Playbook is taken from here.

---
- name: RouterOS test with network_cli connection
  hosts: routers
  gather_facts: false
  tasks:

  - name: Gather system resources
    community.routeros.command:
      commands:
        - /system resource print
    register: system_resource_print

  - name: Show system resources
    debug:
      var: system_resource_print.stdout_lines

  - name: Gather facts
    community.routeros.facts:

  - name: Show a fact
    debug:
      msg: "First IP address: {{ ansible_net_all_ipv4_addresses[0] }}"

Migrate CentOS 8 Stream to AlmaLinux 8

https://wiki.almalinux.org/documentation/migration-guide

Update CentOS 8 Stream

sudo dnf update -y

Download and run the almalinux-deploy script

curl -O https://raw.githubusercontent.com/AlmaLinux/almalinux-deploy/master/almalinux-deploy.sh
sudo bash almalinux-deploy.sh -d

You’ll need to run with the -d “downgrade” option if you are migrating from CentOS 8 Stream. https://github.com/AlmaLinux/almalinux-deploy/tree/master?tab=readme-ov-file#roadmap

You may need to remove packages if there are conflicts. On one instance, there were issues and I needed to remove grafana and llvm-compat-libs.

sudo yum remove grafana llvm-compat-libs

After those errors are fixed, rerun.

sudo bash almalinux-deploy.sh -d

Once the script finishes

sudo reboot

Once it comes back up, check the Linux version

cat /etc/*release

Example output

AlmaLinux release 8.9 (Midnight Oncilla)
AlmaLinux release 8.9 (Midnight Oncilla)
NAME="AlmaLinux"
VERSION="8.9 (Midnight Oncilla)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.9"
PLATFORM_ID="platform:el8"
PRETTY_NAME="AlmaLinux 8.9 (Midnight Oncilla)"

Install Node.js 18 on AlmaLinux 8

List available Node.js versions available.

dnf module list nodejs
AlmaLinux 8 - AppStream
Name           Stream           Profiles                                     Summary
nodejs         10 [d][x]        common [d], development, minimal, s2i        Javascript runtime
nodejs         12 [x]           common [d], development, minimal, s2i        Javascript runtime
nodejs         14 [x]           common [d], development, minimal, s2i        Javascript runtime
nodejs         16 [x]           common [d], development, minimal, s2i        Javascript runtime
nodejs         18 [x]           common [d], development, minimal, s2i        Javascript runtime
nodejs         20 [x]           common [d], development, minimal, s2i        Javascript runtime

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

As we can see above, nodejs 18 is disabled. Enable it with

sudo dnf module enable nodejs:18

Now we can install with

sudo dnf install nodejs

You may need to uninstall older versions.

Ansible Playbook for Linux SNMP

This playbook is for installing and configuring SNMP on Ubuntu or RedHat machines

Change the snmp_location and snmp_contact etc. variables. Or define them in the inventory file, or pass them in as –extra-vars. –extra-vars=”snmp_location=’location address’ snmpv3_user=incredigeek …etc”

Couple of notes

  • We check to see if a read only SNMPv3 user has been created. If so, we don’t create a new one.
  • The snmpd service is stopped and started each time this is run
  • You will still need to allow SNMP through the firewall. Ubuntu or Fedora
---
- name: Linux SNMP Config
  hosts: all
  gather_facts: yes
  become: yes

# Install SNMPv3 on RHEL or Debian/Ubuntu
# Disable SNMP v1 and v2 on RHEL
# Configure SNMPv3 user

  vars:

    # Change these!
    snmp_location: My SNMP location
    snmp_contact: My SNMP contact info
    snmpv3_pass: mypassword
    snmpv3_user: incredigeek

    # These are used to disable the default public community.
    cmnt: '#'
    cmnt_lines:
      - com2sec notConfigUser
      - group   notConfigGroup
      - view    systemview
      - access  notConfigGroup 


  tasks: 

    - name: Check if SNMPv3 user exists
      ansible.builtin.lineinfile:
        path: /etc/snmp/snmpd.conf
        regexp: '^rouser'
        state: absent
      check_mode: yes
      changed_when: false
      register: snmpv3_user_exists

    - name: Stop SNMPD Service
      ansible.builtin.service:
        name: snmpd
        state: stopped

    - name: RHEL SNMP Config
      block:
        - name: Install SNMP RHEL
          ansible.builtin.dnf:
            name:
              - net-snmp
              - net-snmp-utils
            state: present

        - name: Disable public snmp community RHEL
          replace:
            path: /etc/snmp/snmpd.conf
            regexp: '^{{ item }}'
            replace: '{{ cmnt }} {{ item }}'
          loop: "{{ cmnt_lines }}"
        - name: Set SNMP Location
          ansible.builtin.lineinfile:
            path: /etc/snmp/snmpd.conf
            regexp: '^syslocation.*'
            line: "syslocation {{ snmp_location }}"

        - name: Set SNMP Contact
          ansible.builtin.lineinfile:
            path: /etc/snmp/snmpd.conf
            regexp: '^syscontact.*'
            line: "syscontact {{ snmp_contact }}"
        - name: Setup SNMPv3 user for RHEL
          shell: net-snmp-create-v3-user -ro -a SHA -A '{{ snmpv3_pass }}' -x '{{ snmpv3_pass }}' -X AES {{ snmpv3_user }}
          when: not snmpv3_user_exists.found

      when: ansible_os_family == "RedHat"

    - name: Debian SNMP Config
      block:
        - name: Install SNMP on Debian
          ansible.builtin.apt:
            pkg:
            - snmp
            - snmpd
            - libsnmp-dev

        - name: Modify available from address
          ansible.builtin.lineinfile:
            path: /etc/snmp/snmpd.conf
            regexp: '^agentAddress udp:127\.0\.0\.1:161'
            line: 'agentAddress udp:161,udp6:[::1]:161'

        - name: Set SNMP Location
          ansible.builtin.lineinfile:
            path: /etc/snmp/snmpd.conf
            regexp: '^sysLocation.*'
            line: "sysLocation {{ snmp_location }}"

        - name: Set SNMP Contact
          ansible.builtin.lineinfile:
            path: /etc/snmp/snmpd.conf
            regexp: '^sysContact.*'
            line: "sysContact {{ snmp_contact }}"
        - name: Setup SNMPv3 user for Debian
          shell: net-snmp-config --create-snmpv3-user -ro -a SHA -A '{{ snmpv3_pass }}' -x '{{ snmpv3_pass }}' -X AES {{ snmpv3_user }}
          when: not snmpv3_user_exists.found

      when: ansible_os_family == "Debian"

    - name: Enable SNMPD Service
      ansible.builtin.service:
        name: snmpd
        enabled: true

    - name: Start SNMPD Service
      ansible.builtin.service:
        name: snmpd
        state: started

Ansible Playbook to Detect OS version

This playbook can be used to report the Linux Distribution, OS Family, Distribution Version, and Distribution Major Version. This can be helpful for verifying all operating systems are up to date, or for working out what to use in other playbooks.

You will need to already have an inventory file.

Playbook yaml file

The playbook is very simple. Copy and paste the following contents into a file named “os_info.yaml”

---
- hosts: all
  gather_facts: yes
  become: false
  tasks:
  - name: Distribution
    debug: msg=" distribution {{ ansible_distribution }} - os_family {{ ansible_os_family}} - distribution_version {{ansible_distribution_version}} - distribution_major_version {{ ansible_distribution_major_version }}"

If we wanted to, we could break out each Ansible variable in its own debug line. I prefer having them all on a single line.

Running the Playbook

Run the playbook like any other playbook. Change inventory.ini to your inventory file. If your inventory file is encrypted, use the –ask-vault-pass option.

ansible-playbook -i inventory.ini os_info.yaml 

Results

Here are some example results.

 ---------------------
< TASK [Distribution] >
 ---------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||


ok: [almalinux_server01] => {
    "msg": " distribution AlmaLinux - os_family RedHat - distribution_version 9.3 - distribution_major_version 9"
}
ok: [fedora_server01] => {
    "msg": " distribution Fedora - os_family RedHat - distribution_version 39 - distribution_major_version 39"
}
ok: [centos_server] => {
    "msg": " distribution CentOS - os_family RedHat - distribution_version 7.9 - distribution_major_version 7"
}
ok: [ubuntu_serevr01] => {
    "msg": " distribution Ubuntu - os_family Debian - distribution_version 20.04 - distribution_major_version 20"
}