Author Archives: Jack

Configuring DKIM for Postfix

Fighting spam can be tricky. In addition to SPF records, DKIM is nearly mandatory to help prevent sent emails from being classified as spam. Beginning February of 2024, both Google and Yahoo will require DMARC, which require either SPF or DKIM; and in some cases for a high volume of emails (5,000+), both.

In this tutorial, we will look at signed outbound messages with DKIM via use of the open source project OpenDKIM. If you followed my previous tutorial on Postfix + Dovecot + Mysql/MaraiDB, you may have multiple domain names, so this guide will assume you will want to configure separate DKIM keys for each domain name you are hosting.

Step 1: Install OpenDKIM

First, update packages for your distribution.

sudo apt-get update && sudo apt-get upgrade

Install OpenDKIM and OpenDKIM tools. OpenDKIM-tools has a utility to generate the keys we will use.

sudo apt-get install opendkim opendkim-tools

Step 2: Created trusted hosts configuration file for OpenDKIM

First, create a file that OpenDKIM will use that defines the trusted hosts that can send messages.

sudo mkdir /etc/opendkim
sudo vi /etc/opendkim/TrustedHosts

Add the IP addresses and fqdn of the server sending messages by typing i to change into insert mode in vi.

127.0.0.1
localhost
192.168.1.2
mail.mydomain.com

Type :wq to commit the changes in vi.

Step 3: Modify OpenDKIM configuration file

Modify the opendkim.conf configuration file

sudo vi /etc/opendkim.conf

Search for #Canonicalization simple and uncomment the line by removing the # symbol.

Search for #Mode and remove the # symbol to uncomment the line. Ensure the line is configured with s for signing outbound emails or sv for verifying dkim keys on sent and received emails.

If you have subdomains, search for #SubDomains and remove the # and change to yes. For example:

SubDomains              yes

Search for Socket local:/var/run/opendkim/opendkim.sock and comment the line by adding a # in front of the line.

Search for #Socket inet:8891t@localhost and uncomment the line. If the line does not exist in your document, then add the following at the end of your document.

Socket inet:8891@localhost

Next, add the following lines to reference our DKIM configurations for each domain:

KeyTable /etc/opendkim/KeyTable
SigningTable /etc/opendkim/SigningTable
ExternalIgnoreList /etc/opendkim/TrustedHosts
InternalHosts /etc/opendkim/TrustedHosts

Type :wq to save the change and close the file

Step 4: Configure Postfix

sudo vi /etc/postfix/main.cf

Add the following lines to the end of the file:

milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

Type :wq to save the changes and close the file.

Step 5: Restart services to use the changes (optional)

Execute the following to apply the changes if you wish to add domain names at a later time. If not, you can skip this step.

sudo /etc/init.d/opendkim restart
sudo /etc/init.d/postfix reload
sudo /etc/init.d/postfix restart

Step 5/6: Create a DKIM key for a domain

Run the following command to create a new folder and change directory to it for where we will generate our key used to sign the outgoing emails.

sudo mkdir -p /etc/opendkim/keys/mydomain.com
cd /etc/opendkim/keys/mydomain.com

Execute the following command to generate the key:

sudo opendkim-genkey -r -d mydomain.com

Delegate access to the opendkim user and group to access the key (note, if you modified the user in your opendkim.conf file, you will want to use that instead)

sudo chown opendkim:opendkim default.private

Step 7: Reference the key via OpenDKIM KeyTable

Modify the Keytable with vi

sudo vi /etc/opendkim/KeyTable

Add the following line to the file to define your selector. In this example, we will call the selector default, but if your domain requires multiple DKIM keys, ensure you make this unique. You can modify the file by pressing i to enter insert mode in vi:

default._domainkey.mydomain.com mydomain.com:default:/etc/opendkim/keys/mydomain.com/default.private

Type :wq to write and quite vi

Step 8: Specify the domain in your OpenDKIM SigningTable

Open the SigningTable file via vi

sudo vi /etc/opendkim/SigningTable

Add the following line to the file by pressing i to enter insert mode (changing default if specified a different selector earlier on):

mydomain.com default._domainkey.mydomain.com

Type :wq to write and quite vi

Step 8: Update your services to apply the changes

Restart your services to begin signing your messages:

sudo /etc/init.d/opendkim restart
sudo /etc/init.d/postfix reload
sudo /etc/init.d/postfix restart

Step 9: Update DNS

Get the DNS record values we need to publish by executing the following command:

sudo cat /etc/opendkim/keys/mydomain.com/default.txt

Create a new TXT record within your nameservers and specify the value between the quotes (don't include the quotes). I.e.:

v=DKIM1; h=sha256; k=rsa; s=email; p=ABCDEFG.....

Note: I choose to update DNS last as once you update DNS, any servers that would receive mail before you apply the previous configuration may discard your emails. Then again, you didn't have DKIM before, so you were probably going to junk mail anyways ;^)

Credits

Shoutout to Diego on stackoverflow, edoceo, and suenotek for consolidating a lot of these steps:
postfix - Using DKIM in my server for multiple domains (websites) - Ask Ubuntu

How To: Installing and Configuring OpenDKIM for multiple domains with Postfix on Linux | Edoceo

Roundcube mail app and SPF, DKIM & DMARC on Ubuntu 20.04 (suenotek.com)

How to generate a root certificate and create a self-signed server certificate issued from the root

This is going to be a quick tutorial, but here's a quick way to generate a root certificate, server certificate, and bundle them together via pfx file. This can be useful to validate scenarios where a certificate chain is required. For this tutorial, we'll be using the openssl utility, which can be freely downloaded here: Win32/Win64 OpenSSL Installer for Windows - Shining Light Productions (slproweb.com)

Generate the Root Certificate

Execute the following command to generate a key for the root certificate:

openssl ecparam -out root.key -name prime256v1 -genkey

Execute the following command to generate a certificate signing request. Note: During this step, you will be prompted to specify several certificate attributes; for the common name, you can specify the name you'd like as the issuer (i.e. MyCorp)

openssl req -new -sha256 -key root.key -out root.csr

Execute the following to generate the public certificate. During this step, you'll specify the validity of the root certificate (you may want this to be longer than 365 days as the root).

openssl x509 -req -sha256 -days 3650 -in root.csr -signkey root.key -out root.crt

Generate the Server Certificate

Execute the following command to generate a private key for the server certificate:

openssl ecparam -out server-cert.key -name prime256v1 -genkey

Execute the following command to generate a certificate signing request. Note: During this step, you will be prompted to specify several certificate attributes; for the common name, specify the FQDN to your server. You do not need to start the value of the common name with CN=

openssl req -new -sha256 -key server-cert.key -out server-cert.csr

Execute the following command to generate the public certificate for the server certificate. During this step, you'll specify the validity of the server certificate. Generally speaking, the validity of this certificate would be much shorter than your root.

openssl x509 -req -in server-cert.csr -CA root.crt -CAkey root.key -CAcreateserial -out server-cert.crt -days 365 -sha256

Verify certificate chain

Optionally, you can verify the issuer or expiry dates of the server certificate is correct via the following command:

openssl x509 -in server-cert.crt -text -noout

Generate PFX from Root and Server certificate

Execute the following command to generate a PFX file containing the public and private keys of the server certificate as well as public key of the root certificate. Note, you will be prompted for a password for the PFX file, which can increase security when needing to move these sensitive files around.

openssl pkcs12 -export -out mycert.pfx -inkey server-cert.key -in server-cert.crt -certfile root.crt

Using SDRplay RSPduo with RTLSDR-Airplay and a RaspberryPi

One of the side projects I have is rebroadcasting local ATC (Air Traffic Control) audio from my local airport to LiveATC.net. I previously had an RTL-SDR dongle connected to a RaspberryPi 1 Model B, which then rebroadcasted to LiveATC via IceCast. While I've had success the past few years broadcasting, overhead plans were really the only thing that was clear; being distant from the airport, receiving broadcasts from the tower were a slim to none at best.

In doing a bit of research I settled on purchasing a SDRplay RSPduo and Raspberry Pi 4, which seems to help with noise. Pairing the SDRplay with the newest version of RTLSDR-Airplay, I was able to achieve much clearer audio/hear things I couldn't before. While I'm using the SDRplay RSPduo, this guide can be used for their other devices such as the Rsp1a and RSPdx as well (likely others as this guide ages). Here's a reflection on how I got things setup.

Update Raspbian packages

First, update your Linux packages to latest version. I'm running the latest version of Raspbian / Debian.

sudo apt-get update && sudo apt-get upgrade

Disable WiFi/Bluetooth

This is optional, but I figured I'd disable the radios on the RaspberryPi to further mitigate as much possible noise as possible. First, you can disable both radios by editing /boot/config.txt via the vi text editor (this can actually be configured by placing this file on your SD-Card you attach to your Raspberry Pi during first-time boot). Official details on the boot overlays can be found here.

sudo vi /boot/config.txt

Once in vi, press i to insert the following lines:

dtoverlay=disable-bt
dtoverlay=disable-wifi

Press the escape key and then type :wq to write the changes to the file and exit vi.

Lastly, execute the following command to disable the UART bluetooth service.

sudo systemctl disable hciuart

Download & Install RSP Control Library + Driver

First, you will want to grab the latest SDRplay Drivers and Libraries. You can do this by navigating to SDRplay's website and selecting RSPduo and ARM Raspberry Pi OS for the download. Then click the API button. Now this is kinda difficult if you are SSHed into the Pi, so I'd find the latest version from their website and then use the following commands below to remotely download the software (substituting in the version number to grab the latest download) and install it and reboot after install (rebooting after installation is strongly recommended).

Execute the following commands:

# Navigate to home directory
cd ~
# Download latest API Library + Driver
wget https://www.sdrplay.com/software/SDRplay_RSP_API-ARM32-3.07.2.run
# Provide execution rights to install the software
chmod 755 ./SDRplay_RSP_API-ARM32-3.07.2.run
# Run the installer
./SDRplay_RSP_API-ARM32-3.07.2.run
# Reboot the machine
sudo reboot now

Build and install SoapySDR from source

In this section, we need to install SoapySDR which is a vendor and platform neutral SDR support library. Essentially this means that instead of needing a bunch of developers to write integrations into all the different SDRs, other software can leverage these interfaces to skip worrying about device compatibility and focus on what the application needs to do. As we'll see later, RTLSDR-Airband does exactly this to provide support for tons of different SDRs. Kudos to the PothosWare team for enabling developers all over the world to build all sorts of SDR projects!

So, to get this installed, we need to clone the source code from their GitHub repo and compile the project. Official documentation on this process can be found on their wiki, but I'm going to try and simplify everything here.

Since Raspberry Pi doesn't come with Git, I am going to use wget and unzip to do this, but if you don't mind installing Git, that'd be the easier way to "clone" down the latest source code from GitHub (make sure you replace versions where appropriate, at time of writing this, 0.8.0 is the latest version).

# Install dependencies needed to build this project
sudo apt-get install cmake g++ libpython-dev python-numpy swig
# Make sure we are back in our home directory
cd ~
# Grab latest tarball from GitHub
wget -O soapy-sdr-0.8.0.tar.gz https://github.com/pothosware/SoapySDR/archive/soapy-sdr-0.8.0.tar.gz
# Extract the tarball (this is like unzipping a .zip on Windows)
tar xvfz soapy-sdr-0.8.0.tar.gz

# Change directories into the new SoapSDR folder
cd SoapySDR
# Make a new folder called build
mkdir build

# Change directories into the build folder
cd build

# Execute cmake build automation
cmake ..

# Make installer (-j4 parameter increases build threads to make compilation quicker)
make -j4
# Make the installer copy files to right locations
sudo make install
sudo ldconfig #needed on debian systems
# Navigate back to home directory
cd ~
# Delete the SoapySDR folder since we are done with it
rm -R SoapySDR

At this point, you should be able to execute the SoapySDRUtil command and see the version you installed.

SoapySDRUtil --info

You should get something like this:

Build and install SoapySDR Play module from source

Now that we have SoapySDR installed, we need to install the module to allow it to control the SDRplay device. Similiar to SoapySDR install, we'll pull down the latest files from the SoapySDR Play Module GitHub repo, build the installer, execute it, and verify that all went well. Official instructions can be found on their wiki as well.

# Make sure we are back in our home directory
cd ~
# Grab latest tarball from GitHub
wget -O SoapySDRPlay.zip https://github.com/pothosware/SoapySDRPlay3/archive/refs/heads/master.zip
# Unzip the archive
unzip SoapySDRPlay.zip
# Change directories into the new SoapSDR folder
cd SoapySDRPlay3-master
# Make a new folder called build
mkdir build
# Change directories into the build folder
cd build
# Execute cmake build automation
cmake ..
# Make installer
make
# Make the installer copy files to right locations
sudo make install
sudo ldconfig #needed on debian systems
# Navigate back to home directory
cd ~
# Delete the SoapySDR folder since we are done with it
rm -R SoapySDRPlay3-master

Plug in SDRplay RSPduo device and verify we see it

If you haven't already, go ahead and plug in your SDRplay RSPduo. Next, let's verify we see it using the SopaySDRUtil command.

SoapySDRUtil --probe="driver=sdrplay"

You should see something like this and you should see your device and hardware version (note the hardware hardware= value as you may need that later). In addition, one thing that is neat about the RSPduo is there are multiple tuners/antennas. You will be able to see these values in the probe output. Once you enable RTLSDR-Airplay, you'll notice active antennas are removed from the list of available devices.


pi@raspberrypi:~/RTLSDR-Airband-3.2.1 $ SoapySDRUtil --probe="driver=sdrplay"
######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

Probe device driver=sdrplay
[INFO] devIdx: 0
[INFO] hwVer: 3
[INFO] rspDuoMode: 1
[INFO] tuner: 1
[INFO] rspDuoSampleFreq: 0.000000

----------------------------------------------------
-- Device identification
----------------------------------------------------
  driver=SDRplay
  hardware=RSPduo
  sdrplay_api_api_version=3.070000
  sdrplay_api_hw_version=3

----------------------------------------------------
-- Peripheral summary
----------------------------------------------------
  Channels: 1 Rx, 0 Tx
  Timestamps: NO
  Other Settings:
     * RF Gain Select - RF Gain Select
       [key=rfgain_sel, default=4, type=string, options=(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)]
     * IQ Correction - IQ Correction Control
       [key=iqcorr_ctrl, default=true, type=bool]
     * AGC Setpoint - AGC Setpoint (dBfs)
       [key=agc_setpoint, default=-30, type=int, range=[-60, 0]]
     * ExtRef Enable - External Reference Control
       [key=extref_ctrl, default=true, type=bool]
     * BiasT Enable - BiasT Control
       [key=biasT_ctrl, default=true, type=bool]
     * RfNotch Enable - RF Notch Filter Control
       [key=rfnotch_ctrl, default=true, type=bool]
     * DabNotch Enable - DAB Notch Filter Control
       [key=dabnotch_ctrl, default=true, type=bool]

----------------------------------------------------
-- RX Channel 0
----------------------------------------------------
  Full-duplex: NO
  Supports AGC: YES
  Stream formats: CS16, CF32
  Native format: CS16 [full-scale=32767]
  Antennas: Tuner 1 50 ohm, Tuner 1 Hi-Z, Tuner 2 50 ohm
  Corrections: DC removal
  Full gain range: [0, 48] dB
    IFGR gain range: [20, 59] dB
    RFGR gain range: [0, 9] dB
  Full freq range: [0.001, 2000] MHz
    RF freq range: [0.001, 2000] MHz
    CORR freq range:  MHz
  Sample rates: 0.0625, 0.096, 0.125, 0.192, 0.25, ..., 6, 7, 8, 9, 10 MSps
  Filter bandwidths: 0.2, 0.3, 0.6, 1.536, 5, 6, 7, 8 MHz

Build and install RTLSDR-Airband from source

RTLSDR-Airband is an open source project that allows you to receive analog radio voice channels and produce audio streams which can be routed to various outputs, such as online streaming via Icecast server, PulseAudio server, Audio file, or Raw I/Q file. In our case, we are going to stream to an Icecast server in this example.

Similar to our previous section in SoapySDR, we need to download the latest source code, build and install RTLSDR, and then modify the configuration file. Official documentation can be found on the RTLSDR-Airplay GitHub Wiki.

# Install RTLSDR-Airplay dependencies
sudo apt-get install build-essential libmp3lame-dev libshout3-dev libconfig++-dev libfftw3-dev
# Navigate back to our home directory
cd ~
# Download the latest source from GitHub
wget -O RTLSDR-Airband-3.2.1.tar.gz https://github.com/szpajder/RTLSDR-Airband/archive/v3.2.1.tar.gz
# Extract the tarball
tar xvfz RTLSDR-Airband-3.2.1.tar.gz
# Change directory into the RTLSDR-Airband folder
cd RTLSDR-Airband-3.2.1
# Make the installer; this is specify to armv7 (32-bit Raspberry PI) with SoapySDR support.
# This removes RTLSDR support to avoid another dependency install (WITH_RTLSDR=0)
make PLATFORM=armv7-generic WITH_RTLSDR=0 WITH_SOAPYSDR=1
# Install the program
sudo make install

Configure RTLSDR-Airband

For my particular setup, I want to stream to an external icecast server. To do that, I recommend creating a backup of the default configuration file (as a backup).

# Rename the original config file as a backup
sudo mv /usr/local/etc/rtl_airband.conf /usr/local/etc/rtl_airband.conf.bak

Next, we can create a new configuration file with the proper configuration. Execute the following command to open vi.

sudo vi /usr/local/etc/rtl_airband.conf

Press i to go into insert mode and paste the following (replacing the values applicable to your environment; you may want to change the name of the stream, authentication parameters, and gain"). Also, note that we are using the first Antenna and specifying the hardware version of RSPduo from the previous step where we probed the SDRplay device (if you have a different SDRplay device, substitute that value accordingly).

# Configure IceCast Stream
devices:
({
  type = "soapysdr";
  index = 0;
  device_string = "driver=sdrplay,hardware=RSPduo";
  channel = 0;
  gain = 35;
  correction = 1;
  antenna = "Tuner 1 50 ohm";
  mode = "scan";
  channels: ( 
    {
      freqs = ( 133.8 );
      outputs: ( 
          {
             type = "icecast";
             server = "audio-in.myicecastserver.net";
             port = 8010;
             mountpoint = "station"
             username = "username"
             password = "mypassword";
             name = "Tower";
             description = "Tower - 133.8Mhz";
             genre = "ATC";
          }
       );
    }
 );
});

Type :wq to write and save the changes to the file.

Validate RTLSDR-Airband Configuration

Once you have your configuration, you can validate everything is ready to go by running RTLSDR-Airplay in foreground mode.

From their wiki: you will see simple text waterfalls, one per each configured channel. This is an example for three devices running in multichannel mode. The meaning of the fields is as follows:

User interface screenshot
  • The number at the top of each waterfall is the channel frequency. When running in scan mode, this will be the first one from the list of frequencies to scan.
  • The number before the forward slash is the current signal level
  • The number after the forward slash is the current noise level estimate.
  • If there is an asterisk * after the second number, it means the squelch is currently open.
  • If there is a > or < character after the second number, it means AFC has been configured and is currently correcting the frequency in the respective direction.

Execute the following command to start running in foreground mode:

# Test in foreground mode
/usr/local/bin/rtl_airband -f

Press Cntrl+C to break out of the stream once you are satisfied with your testing.

Enable RTLSDR-Airband to autostart

To enable RTLSDR-Airband to automatically start up each time your Raspberry Pi is rebooted, you can execute the following commands from within the RTLSDR-Airband directory.

sudo cp init.d/rtl_airband.service /etc/systemd/system
sudo chown root.root /etc/systemd/system/rtl_airband.service
sudo systemctl daemon-reload
sudo systemctl enable rtl_airband

Hurray! We are done!

If you made it this far you have completed all the steps! Enjoy your new streaming SDR solution!

Controlling a Haiku fan with a wall switch

TLDR: I wanted to control the light on Big Ass Fans' Haiku fan via physical wall switch, so this tutorial is going to go over how to pair a smart switch with Home Assistant software to provide a traditional light switch experience. Skip down to Setting up the wall switch to start if you want to skip my ramblings.

Here's a YouTube video if you don't like to read:

Longer story

In the background of many commercial buildings, silently lurking and judging us from above, lies what looks like possibly a recycled helicopter blade. Don't be fooled, these blades are no helicopter blade, they are years of engineered excellence in the makings. The company prides themselves on solid engineering and building a solid product for their customers. They are called Big Ass Fans.

For quite some time I've been eyeing their Haiku fan, which is their residential ceiling fan. Their fans look incredibly modern, operate almost completely silent, have a "SenseMe" feature that figures out when people are in the room to automatically do stuff, and they have an API that you can integrate into locally on the fan via WiFi (lose internet, no problem, you can still control your fan!). One of my biggest "beefs" with today's companies is they try to make things really proprietary and crappy, so seeing the company that takes pride in their product and allowing others to integrate into it remotely without internet is super "cool" 😉

You spin me right round

The fan itself is smart... too smart

One thing that's really interesting, is when you hook up the fan, to me, it's designed more like their commercial units where it needs to be constantly powered on; from there you remotely control the fan either by remote or their smart phone application. Both the remote and even the mobile app, work incredibly well and are extremely responsive, but the only tricky thing about the fan is in a residential setting, many folks have light/fan combos in their bedrooms, offices, and living rooms and if a guest walks into the room and flips the light switch, they are flipping power to the whole fan/light.

So....?

In many commercial settings, fans you typically set once and let em' rip, but with the residential play, you have grandparents, guests, friends, etc. that may come over. Since the remote is there, they go, how do I turn the lights on to this room? Unfortunately, there isn't a good answer here other than to put a plate on the wall and force your guests to check out the remote.

I personally find the remote a hassle since I have a small little corridor into one room, so when it's darker in the evenings, you grab the remote on the wall, walk through this dark area, and then aim the remote somewhere at the ceiling to turn it on (this is if you don't forget the remote in the room from before).

So...?

I am a "big fan" of having a smart home, but I want it to be super intuitive to the end user. I design everything to be used as if my grandparent is over and they have no idea what the heck is going on. In this case, I leveraged an open source project called Home Assistant and a Leviton Z-Wave switch to do the magic of controlling the fan like any other fan you'd buy at a big box store. More specifically, I really just needed to control the light on the fan, so this tutorial is going to go over how to control the light from the fan via the switch.

Setting up the wall switch

The first thing you'll need is a smart switch. It can be WiFi, Z-Wave, ZigBee, etc; it doesn't matter specifically what brand (odds are, it'll be compatible (here's the official list)), but you'll need a switch that allows you to control it via the computer or your phone. I used a dimmer switch specifically as the light on the Haiku allows several different levels of brightness.

Once you have the switch, what you'll want to do is wire up the fan so it constantly has power and also give power the switch. This does two things: 1) it allows the fan to be powered on regardless if your guest turns the light switch on/off 2) it allows the light switch to stay powered on so you use that to talk to your fan. Here's an example of how I wired my Leviton Z-Wave switch.

Here you can see I don't have anything connected to the red pin, or load. Typically, you'd have this connect back to the fans lights to turn them on/off, but the Haiku fan isn't wired like that.

On this side, you can see we only have the negative wire connected. It's hard to see, but in the box, I have all my neutral and negative wires capped together, which offers power to the fan 100% of the time, regardless of what this switch is doing.

Once you have the switch wired up and ready to go, it should literally do nothing when you turn it on/off, but your fan should stay on all the time.

Setting up Home Assistant Automation

This guide won't go into installing / setting up Home Assistant, rather more so around the automation scripts needed to get this all working. If you are interested in learning more about Home Assistant, you can check out their website here and I have a blog post on how to deploy Home Assistant on a Raspberry Pi here.

To get this working, you will need a couple of things:

  • Add your smart switch to Home Assistant
  • Install HACS
  • Install Haiku SenseMe Integration
  • Add two automation scripts
    • One to control light on/off events
    • One to control light brightness events
Add your smart switch

I won't go into details here too much since every switch will have a separate way to install (Z-Wave vs WiFi vs Zigbee for example), but here is a nice YouTube video on how to get things going (https://youtu.be/FtWFSuMdiSQ?t=353).

HACS

If you have used Home Assistant, it comes with many different native integrations out of the box. Unfortunately, many integrations are developed so quickly the HA (Home Assistant) team doesn't have time to vet them all, so they end up being maintained by the community. HACS helps install these integrations, so I'd recommend installing this.

Step-by-Step documentation on installation can be found here: Prerequisites | HACS

Haiku Integration

A few much smarter folks wrote up an integration for the Haiku fan called SenseME, which we need to install. Once HACS is installed, you can search for the integration via HACS and install the integration. Copied from their integration, here is how to install the integration:

  1. Go to Configuration -> Integrations.
  2. Click on the + ADD INTEGRATION button in the bottom right corner.
  3. Search for and select the SenseME integration.
  4. If any devices are discovered you will see the dialog below. Select a discovered device and click Submit and you are done. If you would prefer to add a device by IP address select that option, click Submit, and you will be presented with the dialog in step 5.
  5. If no devices were discovered or you selected the IP Address option the dialog below is presented. Here you can type in an IP address of undiscoverable devices.
  6. Repeat these steps for each device you wish to add.

Information on the SenseME integration can be found on their GitHub site here: mikelawrence/senseme-hacs: Haiku with SenseME fan integration for Home Assistant (github.com)

Once configuration is completed, you should see an entity for your fan listed that looks something like this.

On/Off Automation

This automation will first control On/Off behavior from your light switch.

  1. Go to Configuration -> Automations.
  2. Click on the + ADD Automation button in the bottom right corner.
  3. Click the START WITH AN EMPTY AUTOMATION button
  4. Click on the three dots in the top right corner and click Edit in YAML

5. Paste the following code; make sure you edit the names of each of your light switch entities (one for your fan light and one for the light switch on the wall):
light.your_light (the light for your wall) and light.fan_light (the light on the Haiku fan).

alias: Turn On/Off Haiku Fan/Wall Switch
description: ''
trigger:
  - platform: state
    entity_id: light.your_light, light.fan_light
    from: 'off'
    to: 'on'
  - platform: state
    from: 'on'
    to: 'off'
    entity_id: light.your_light, light.fan_light
condition: []
action:
  - service: light.turn_{{ trigger.to_state.state }}
    data:
      entity_id: |-
        {% if trigger.entity_id == 'light.your_light' %}
          light.fan_light
        {% elif trigger.entity_id == 'light.fan_light' %}
          light.your_light
        {% endif %}
mode: single

6. Click the SAVE button

Brightness Automation

This automation will first control On/Off behavior from your light switch.

  1. Go to Configuration -> Automations.
  2. Click on the + ADD Automation button in the bottom right corner.
  3. Click the START WITH AN EMPTY AUTOMATION button
  4. Click on the three dots in the top right corner and click Edit in YAML
  5. Paste the following code; make sure you edit the names of each of your light switch entities (one for your fan light and one for the light switch on the wall):
    light.your_light (the light for your wall) and light.fan_light (the light on the Haiku fan).
alias: Sync Haiku Fan/Wall Switch Brightness
description: ''
trigger:
  - platform: state
    entity_id: light.your_light, light.fan_light
    attribute: brightness
    for: '00:00:02'
condition:
  - condition: template
    value_template: '{{ trigger.to_state.attributes.brightness > 0}}'
action:
  - service: light.turn_on
    data:
      brightness: '{{ trigger.to_state.attributes.brightness }}'
      entity_id: |-
        {% if trigger.entity_id == 'light.your_light' %}
          light.fan_light
        {% elif trigger.entity_id == 'light.fan_light' %}
          light.your_light
        {% endif %}
mode: restart

6. Click the SAVE button

Testing!

At this point, whether you use your remote or the light switch, your lights should be in sync! Use the remote or the wall switch to the turn on/off the lights. Try using the switch to dim and it should adjust the brightness of the light (note: there may be a tiny delay after you make changes to the dimmer value as there's a 2second delay in the automation, which prevents the lights from going wonky).

Conclusion

Through the use of Home Assistant + any smart switch, we can easily control the Haiku fan with physical nobs and dials. While this tutorial only covers controlling the fan's light via a switch, the same principals can be used to add a second switch to control the fan speed.

For those that like physical knobs and dials to control your devices, hope this was helpful!

If you are thinking of buying a Big Ass Fan, please consider using my referral code so I can use them towards future reviews! https://bigassfans.referralrock.com/l/1JACKSTROMB57/

How to update Z-Wave JS Docker Container

This document is written to help those that are using Z-Wave JS and Home Assistant as Docker containers. This tutorial goes hand-in-hand with this: How to update Home Assistant Docker Container | Jack Stromberg

Validate your current version

First, validate what version of Z-Wave JS you are running. To do this, navigate to the Z-Wave JS webpage and hover over the i icon to validate what versions of the software you are running. The Z-Wave JS webpage can typically be accessed at http://yourip:8091.

Get the current name of your container and version

sudo docker ps

In running this command, note the NAME of your container as well as the IMAGE.

Stop and delete the container

Replace the name of the container in the command below with the value you had.

sudo docker stop zwave-js
sudo docker rm zwave-js

Update packages

Some versions of HA require newer versions of Python, Docker, etc. I may consider updating to latest package versions first.

sudo apt-get update
sudo apt-get upgrade

Pull the latest container from Docker Hub

Replace the value below with your IMAGE value you documented in the previous steps.

sudo docker pull zwavejs/zwavejs2mqtt:latest

Deploy the container

Make sure your replace the name and value of the image with the values in the previous step. In addition, ensure you specify the correct path to where you existing configuration files exist to have the container load your existing configurations.

sudo docker run -d --restart=always  -p 8091:8091 -p 3000:3000 --device=/dev/ttyACM0 --name="zwave-js" -e "TZ=America/Chicago" -v /home/docker/zwave-js:/usr/src/app/store zwavejs/zwavejs2mqtt:latest

Validate your version number

After a few minutes, navigate back to the Z-Wave JS page. Upon load, you should now be on the latest versions.

Notes:

You can find the latest, stable, and development builds out on docker hub here: https://hub.docker.com/r/zwavejs/zwavejs2mqtt

How to add buster-backports to a Raspberry Pi

What are backports?

Debian has a really good write up here on what backports are. Copying directly from their introduction paragraph:

You are running Debian stable, because you prefer the Debian stable tree. It runs great, there is just one problem: the software is a little bit outdated compared to other distributions. This is where backports come in.

Backports are packages taken from the next Debian release (called "testing"), adjusted and recompiled for usage on Debian stable. Because the package is also present in the next Debian release, you can easily upgrade your stable+backports system once the next Debian release comes out. (In a few cases, usually for security updates, backports are also created from the Debian unstable distribution.)

Backports cannot be tested as extensively as Debian stable, and backports are provided on an as-is basis, with risk of incompatibilities with other components in Debian stable. Use with care!

It is therefore recommended to only select single backported packages that fit your needs, and not use all available backports.

Once I enable backports will all packages use them?

No! Any new packages and updates to existing stable packages will prefer the stable releases. The only time you will leverage a new backport package is if you explicitly specify to pull from them.

How do I enable backports?

First you need to add the new backport source to your sources.list file. Edit the file in vi:

sudo vi /etc/apt/sources.list

Arrow down to the last row, press o to create a new line and then enter the following:

deb http://deb.debian.org/debian buster-backports main

Press escape and then type :wq to save the changes and exit via.

Next, we need to specify a keyserver to verify the authenticity of these packages. Note we use Ubuntu's key servers to validate the packages. Interestingly, Debian has a keyring to validate the packages, however the keyring doesn't contain the backports for buster on the raspberry pi at time of writing this. Ubuntu's servers will work fine to validate the authenticity of these packages and you will ultimately pull the packages from Debian rather than Ubuntu.

sudo bash
gpg --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC
gpg --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138

gpg --export 04EE7237B7D453EC | sudo apt-key add -
gpg --export 648ACFD622F3D138 | sudo apt-key add -
exit

How do I obtain a package from backport?

You can leverage one of the follow formats to specify the backport package:

apt install <package>/buster-backports
apt-get install <package>/buster-backports

or

apt install -t buster-backports <package>
apt-get install -t buster-backports <package>

or

aptitude install <package>/buster-backports

Establishing a GCP VPN Tunnel to Azure Virtual WAN; Active/Active BPG Configuration

This is a quick reflection of the steps I took to establish two IPSec tunnels between GCP’s VPC and Azure’s Virtual WAN VPN Gateway, propagating routes dynamically via BPG and ensuring High Availability. The design is fairly straightforward since both GCP and Azure offer the ability to established multiple connections to remote peers. When everything is said and done, you’ll end up with a diagram that conceptually looks something like this:

Note: It is recommended to complete the steps in this document in the outlined order to complete the least amount of steps. In this case, provide Azure Virtual WAN first, then configure GCP, then create the Azure Virtual WAN Site Links / Connections to GCP.

Note 2: If you have followed my previous guide on establishing an AWS VPN tunnel to Azure Virtual WAN , this guide will co-exist both connections and can skip the Create Azure Virtual WAN and Virtual WAN Hub sections.

Create Azure Virtual WAN and Virtual WAN Hub

On the Azure side, first we need to create a Virtual WAN resource and a Virtual WAN Hub, which will contain our VPN Gateway. If you have already created these, you can skip to the next session.

First, click the "Hamburger" icon and select Create a resource

Search for Virtual WAN and select it from the list in the marketplace.

Select Create

Specify the resource group and region you wish to deploy the Virtual WAN resource to. Specify a name for your Virtual WAN resource and click Review + Create

Click Create to start provisioning the Virtual WAN resource.

Once the resource is created, click Go to resource to navigate to your Virtual WAN resource.

On the Virtual WAN resource, select New Hub from the top menu.

Specify the name of the Hub and an address space that can be used for all the networking components Virtual WAN will deploy into the Virtual Hub. Click Next : Site to Site >

On the Site to Site tab, toggle Yes that you want to provision a VPN Gateway, and specify the scale units you need. Click the Review + create button when done.

Click the Create button to start provisioning the Hub and VPN Gateway. Please note this can take up to 30 minutes to complete.

Once the Virtual WAN Hub has been created, click the Menu icon and select All services (note: if you click the Go to resource button after the Virtual WAN Hub resource is created, it'll take you the properties of the Hub, which isn't where we want to be).

Search for Virtual WAN and select Virtual WANs.

Select your Virtual WAN resource.

Click on Hubs under Connectivity and select your Virtual WAN Hub.

Select VPN (Site to Site) under Connectivity and then click on the View/Configure link.

Set the Custom BGP IP addresses for each instance. Use the values below:

  • VPN Gateway Instance 0: 169.254.21.2
  • VPN Gateway Instance 1: 169.254.22.2

Click Edit once completed.

Configure GCP

Prerequisites

This guide assumes you have a VPC already (in my case, mine is called GCP-VPC with an address space of 10.60.0.0/16) and corresponding set of subnets for your servers.

Note: A GCP VPC is the equivalent of a VNet in Azure. One thing that is different between GCP and Azure is that in GCP you do not need to specify a subnet for your Gateways (i.e. “GatewaySubnet”).

Within the GCP Console, select Hybrid Connectivity -> VPN

Click Create VPN Connection

Select High-availability (HA) VPN and select Continue

Enter a name, select your VPC, and specify a region. Click Create & Continue.

Write down your Interface public IPs (we'll use these later) and check On-prem or Non Google Cloud for Peer VPN Gateway. Click Create & continue.

Select two interfaces and enter your Instance 0 and Instance 1 Public IP addresses from your Virtual WAN Hub's VPN Gateway. Click Create.

Click the dropdown for Cloud Router and select Create a new router

Enter a name and description for your router. For the ASN, enter a unique ASN to use (I used 64700 to differentiate from Azure as well as the ASN I used in the AWS example (which was 64512)). You can specify any supported ASN for this, however I would recommend against using 65515 specifically as this is reserved by Azure's VPN Gateways.

Note: Google ASN must be an integer between 64512 and 65534 or between 4200000000 and 4294967294 or 16550

Click the pencil icon to modify the first VPN tunnel.

Select the instance 0 VPN Gateway interface, enter a name, set the IKE version to IKEv2, enter a pre-shared key, and click Done.

Repeat the same steps for the second VPN tunnel, specifying instance 1 VPN Gateway interface, enter a name, set the IKE version to IKEv2, enter a pre-shared key, and click Done and then Create & continue.

Click the Configure button for the first BPG session.

Enter a name for the first BGP peer connecting to instance 0 gateway on Virtual WAN.

Specify Peer ASN of 65515 (this is Azure VWAN's BGP ASN), specify 169.254.21.1 for Cloud Router BGP IP and 169.254.21.2 as BGP peer IP (Azure VWAN's BGP Peer IP). Click the Save and continue button.

Click the Configure button and enter a name for the first BGP peer connecting to instance 1 gateway on Virtual WAN.

Specify Peer ASN of 65515 (this is Azure VWAN's BGP ASN), specify 169.254.22.1 for Cloud Router BGP IP and 169.254.22.2 as BGP peer IP (Azure VWAN's BGP Peer IP).

Click the Save BGP configuration button.

Configure Azure Virtual WAN VPN Site

On the Virtual WAN hub, select VPN (Site to site) and click + Create new VPN site

Specify a name for the VPN connection, enter GCP for vendor, and click Next : Links >

Specify the following values to define each VPN tunnel that should be created to connect to GCP's VPN interfaces.

Note: I entered 1000 for the link speed as a placeholder, but that doesn't mean the connection will be throttled down to 1Gbps.

  • First Link:
    • Link Name: gcp-east1-vpc-vpn-int0
    • Link Speed: 1000
    • Link provider name: GCP
    • Link IP address: <GCP VPN Interface 0 Public IP>
    • Link ASN: 64700
  • Second Link:
    • Link Name: gcp-east1-vpc-vpn-int1
    • Link Speed: 1000
    • Link provider name: GCP
    • Link IP address: <GCP VPN Interface 1 Public IP>
    • Link ASN: 64700

Click Create

Configure Virtual WAN VPN Connection

Once the Virtual WAN Hub has been created, click the Menu icon and select All services.

Search for Virtual WAN and select Virtual WANs.

Select your Virtual WAN resource.

Click on Hubs under Connectivity and select your Virtual WAN Hub.

Select VPN (Site to Site) under Connectivity and then click on the X to remove the Hub association filter.

Check the box for your VPN site and click Connect VPN sites

Specify the following information:

  • Pre-shared key (PSK): <use the same one you specified in GCP>
  • Protocol: IKEv2
  • IPsec: Custom
    • SA Lifetime in seconds: 36000
    • Phase 1 (IKE):
      • Encryption: GCMAES256
      • Integrity/PRF: SHA384
      • DH Group: ECP256
    • Phase 2 (IPSec):
      • Encryption: GCMAES256
      • IPSec Integrity: GCMAES256
      • PFS Gropu: ECP384
  • Propagate Default Route: Disable
  • Use policy based traffic selector: Disable

Click Connect.

Note: Here is the link of Supported IKE ciphers for GCP.

Verify Connectivity

From the Azure Side, we will review three different areas to validate connectivity and propagation of routes via BGP.

Note: I connected a virtual network to the Virtual WAN Hub to show further configuration. In this case, you'll see an additional IP address space of 10.51.0.0/16, which defines my connected VNet.

On the Azure Side, you should see the VPN Site’s Connectivity status change to Connected on the VPN (Site to site) blade of your Virtual WAN hub.

On the Routing blade, the Effective Routes will show you the learned VPC address space from GCP (10.60.0.0/16)

On a virtual machine in a connected VNet to the Virtual WAN Hub, you can pull the Effective Routes. Here I see the 10.60.0.0/16 route learned from both Instance 0 and Instance 1 gateways from the Virtual WAN Hub.

From the GCP Side, we can see the VPN tunnel status as well as Bgp session status now Established and Green on the Hybrid Connectivity -> VPN -> Cloud VPN Tunnels section.

If we switch over to Hybrid Connectivity -> Cloud Routers -> and select View on the logs column

Further, if creating a VM (instance) in GCP, you can view the Firewall and Route details to confirm you see the learned routes from the gateway (in our case, we see 10.51.0.0/16 and 10.50.0.0/24 learned from both BGP Peers):

Huzzah! Traffic! 🙂

Establishing an AWS VPN Tunnel to Azure Virtual WAN; Active/Active BPG Configuration

This is a quick reflection of the steps I took to establish two IPSec tunnels between AWS' VPG and Azure's Virtual WAN VPN Gateway, propagating routes dynamically via BPG and ensuring High Availability. The design itself is a bit interesting since AWS and Azure differ on how connections are established to remote peers. When everything is said and done, you'll end up with a diagram that conceptually looks something like this:

Note: It is recommended to start with the Virtual WAN side first since you cannot modify the IP address of a Customer Gateway in AWS

Create Azure Virtual WAN and Virtual WAN Hub

On the Azure side, first we need to create a Virtual WAN resource and a Virtual WAN Hub, which will contain our VPN Gateway. If you have already created these, you can skip to the next session.

First, click the "Hamburger" icon and select Create a resource

Search for Virtual WAN and select it from the list in the marketplace.

Select Create

Specify the resource group and region you wish to deploy the Virtual WAN resource to. Specify a name for your Virtual WAN resource and click Review + Create

Click Create to start provisioning the Virtual WAN resource.

Once the resource is created, click Go to resource to navigate to your Virtual WAN resource.

On the Virtual WAN resource, select New Hub from the top menu.

Specify the name of the Hub and an address space that can be used for all the networking components Virtual WAN will deploy into the Virtual Hub. Click Next : Site to Site >

On the Site to Site tab, toggle Yes that you want to provision a VPN Gateway, and specify the scale units you need. Click the Review + create button when done.

Click the Create button to start provisioning the Hub and VPN Gateway. Please note this can take up to 30 minutes to complete.

Configure customer BGP IP Address for Virtual WAN VPN Gateway Instances

Once provisioning is completed, navigate back to the Virtual WAN resource. You can do this by clicking the "Hamburger" icon and searching for Virtual WAN

Select your Virtual WAN resource.

You should now see your Virtual WAN Hub resource you provisioned. Select the Virtual WAN Hub.

On the Virtual WAN Hub, click on the View/Configure link.

On the View/Configure Gateway Configuration blade, specify 169.254.21.2 as the Custom BGP IP address for Instance 0 and 169.254.22.2 as the Custom BGP IP address for Instance 1. Notate the Public IP address uses for Instance 0 and 1 and then click Edit and Confirm to apply the changes.

Create Virtual WAN VPN Site

On the Virtual WAN Hub, click Create new VPN Site

Specify a name for your VPN Site to define the connection connecting to AWS. Click Next : Links >

On the Links tab, add two entries with the following values (to tell VWAN how to connect to each of the AWS Site-to-Site connections). Note: this is very similar to AWS' Customer Gateway section.

Link 1:

  • Link Name; AWS_Tunnel1
  • Link Speed: 1000
  • Link Provider Name: AWS
  • Link IP address: 1.1.1.1 (this is a placeholder value until we configure the AWS side)
  • Link BGP address: 169.254.21.1
  • Link ASN: 64512

Link 2:

  • Link Name; AWS_Tunnel2
  • Link Speed: 1000
  • Link Provider Name: AWS
  • Link IP address: 1.1.1.2 (this is placeholder value until we configure the AWS side)
  • Link BGP address: 169.254.22.1
  • Link ASN: 64512

Click Next: Review + Create >

Click Create

Click Go to resource once the links have finished being created.

Configure Phase 1/2 Proposals

Select your Virtual WAN hub on the Virtual WAN Overview blade.

Check the box for the new VPN Site Name and click the Connect VPN sites button

Specify the following configuration:

  • Pre-shared key (PSK): YourSecretKeyWithNumb3rs
    • Must be a 8-64 character string with alphanumeric, underscore(_), and dot(.). It cannot start with 0.
  • Protocol: IKEv2
  • IPSec: Custom
  • IKE Phase 1:
    • Encryption: GCMAES256
      • GCM algorithm is more efficient and can improve throughput on the Azure Gateways
    • Integrity/PRF: SHA256
    • DH Group: DHGroup14
  • IKE Phase 2 (ipsec):
    • IPSec Encryption: AES256
      • AWS does not support GCM algorithm for IPSec integrity at time of writing this, but if it is available, you may want to opt for that
    • IPSec Integrity: SHA256
    • PFS Group: PFS14

Click Connect

Configure AWS

Prerequisites

This guide assumes you have a VPC already (in my case, mine is called AWS-OHIO-VPC), a corresponding set of subnets for your servers, and a route table associated to your VPC.

Note: An AWS VPC is the equivalent of a VNet in Azure. One thing that is different between AWS and Azure is that in AWS you do not need to specify a subnet for your Gateways (i.e. "GatewaySubnet").

Create the Customer Gateways

Customer Gateways in AWS are the equivalent of a local network gateway that you'd associate to a connection for a traditional VPN Gateway in Azure. It is also the equivalent of a defined Site Link for Azure's Virtual WAN VPN configuration.

In this section, you will need to create two Customer Gateways. Specify the corresponding instance value obtained from the Configure Customer BPG IP address section. When creating the Customer Gateways ensure Dynamic routing is enabled and the BGP ASN is specified as 65515.

Configuration for the second Customer Gateway using the Instance 1 Gateway Public IP address.

Create a Virtual Private Gateway

Next we need to create an AWS Virtual Private Gateway. This is the equivalent of Azure's VPN Gateway.

Create VPN Connections

We need to create two VPN Connections, each VPN Connection linked to its corresponding Customer Gateway and VPC.

On the Inside IPv4 CIDR for Tunnel 1 on the first VPN Connection, ensure you use 169.254.21.0/30 as the BGP Peer addresses and 169.254.21.4/30 for the second tunnel. Due to the way that the VPN Connection works, we are using a placeholder value of 169.254.21.4/30 tunnel, which will never be used in practice since we cannot point it to leverage Azure's secondary VPN Gateway instance. This value must be specified as if we define the secondary BGP Peer address that will be created for the secondary instance in VWAN, you will receive an error that overlapping address space exists between this VPN Connection and the secondary VPN connection we create in AWS. Add the pre-shared key value you specified in Azure during this time as well.

When creating the second VPN connection, ensure 169.254.22.0/30 is specified for Inside IPv4 CIDR for Tunnel 1 and 169.254.22.4/30 is specified for Inside IPv4 CIDR for Tunnel 2 (which is again a placeholder value that won't be used).

Configure Route Table to Propagate Routes

To allow the learned routes from BGP propagate to the VPC, you need to enable route propagation on your Route Table.

Navigate to Route Tables and select your Route Table and click the Route Propagation tab and select Edit route propagation

Check the Propagate box and click Save

Update Azure

Update Azure Site Link IP addresses

As per the Configure Phase 1/2 Proposals section for Azure Virtual WAN, you specified 1.1.1.1 and 1.1.1.2 as a placeholder value for the Public IP addresses of the AWS VPN Gateway instances. We will need to update these addresses with the proper values.

Naviate to your Virtual WAN instance and select your Virtual WAN hub

Select VPN (Site to site) and choose click on the Site name you created

Click on the three dots (ellipsis) for AWS_Tunnel1 and click Edit Link.

Specify the proper IP address for Tunnel 1 on AWS Site-to-Site connection 1. Click Confirm.

Click on the three dots (ellipsis) for AWS_Tunnel2 and click Edit Link.

Specify the proper IP address for Tunnel 1 on AWS Site-to-Site connection 2. Click Confirm.

Verify connectivity

On the Azure Side, you should see the VPN Site's Connectivity status change to Connected

You can also select a Virtual Machine that may have it's virtual network attached to the VWAN Hub and validate you see learned routes from the VWAN Hub (and AWS) propagated into the VNet.

Tip: You can see the same route twice as we have both VPN Gateway instance BGP Peers actively connected to AWS. In the event you lose a peer, you would only see one route to one gateway listed.

On the AWS side, you can validate for each Site to Site VPN connection that you see Tunnel 1's status as UP and Tunnel 2's status as DOWN (remember, Tunnel 2 will always be listed as down because a fictitious BGP is specified).

Here you can see the secondary Site-to-Site connection with the same status: UP for Tunnel 1, DOWN for Tunnel 2

How to upgrade Home Assistant Z-Wave integration to Z-Wave JS for Docker

If you've been following my last two tutorials (Home Assistant + Docker + Z-Wave + Raspberry Pi | Jack Stromberg and How to update Home Assistant Docker Container | Jack Stromberg) on running Home Assistant via Docker and how to keep the container updated, you may have noticed that 2021 has been a big year for larger changes, with a surprising change coming to how Home Assistant handles Z-Wave Devices.

In Home Assistant v2021.2, Home Assistant announced the Z-Wave integration as deprecated in favor of a new integration called Z-Wave JS. In Home Assistant v2021.3, many fixes were implemented, with the notable limitation of Door Sensors being removed.

So why the change?

As per the Home Assistant v2021.2 announcement:

More and more people were concerned about the future of Z-Wave with Home Assistant; meanwhile the Z-Wave JS project was rapidly growing and gathering a large community around it. Long story short: Home Assistant and Z-Wave JS teamed up! And a lot of contributors jumped on the train!

This new integration is based on the same base principles as the OpenZWave integration: It is decoupled from Home Assistant. Instead of MQTT, the Z-Wave JS integration uses a WebSocket connection to a Z-Wave JS server.

This means, in order to use this new integration, you’ll need to run the Z-Wave JS server that sits in between your Z-Wave USB stick and Home Assistant. There are multiple options available for running the Z-Wave JS server, via Docker or manually, and there is also a Home Assistant add-on available.

So how do I upgrade?

This article reflects the steps I took to update my Z-Wave implementation.

Ensure you are running Home Assistant v2021.3.2 or greater

This will ensure you have support for most all sensors. You can find your Home Assistant version by selecting the Configuration gear on the left menu, and then selecting Info

Here you should see the version of Home Assistant (in my case 2021.3.2)

If you are not running the latest version, you can follow my upgrade steps here: How to update Home Assistant Docker Container | Jack Stromberg

Create a backup

It's rather critical to create a backup, especially in this case if you need to roll back to the older OpenZWave integration if you find many of your devices not being compatible. One downside in not using Home Assistant's OS is you don't have the "Supervisor" option to create a full backup.

To complete this step, I'd recommend checking out this blog post here which provides several options: Backing up Home Assistant | Tinkering with Home Automation (ceard.tech)

Alternatively, you can be extremely lazy and less cautious by simply copying the configuration folder containing your docker config:

sudo cp /home/docker/home-assistant/ /home/docker/home-assistant-backup/ -R

Update your Operating System

Execute the following commands against your machine:

sudo sh -c 'apt update && apt upgrade'

Make sure you restart your machine to ensure your kernel updates to the latest version:

sudo shutdown -r -t now

Document Z-Wave entity IDs

The easiest way to do this is to navigate to Developer Tools (hammer icon on the left menu) and then type node_id into the Attributes column's filter.

In this case, you'll want to write down the node_id and the name of the entity it maps to. If you want to do this quickly, you can single click on the table, press Control + A to select all contents, or cmd+a on a Mac, and copy the contents into Word or Excel (Excel works remarkably well).

Document & Comment Z-Wave Stick Hardware ID and Network Key

SSH to your server and find your configuration.yaml file (if using my tutorial it should be /home/docker/home-assistant/configuration.yaml). Open the file in vi

sudo vi configuration.yaml

Find the section of code labeled zwave: and copy the information (we'll need it later) as well as comment out the following lines like so:


#zwave:
#  usb_path: /dev/ttyACM0
#  network_key: "0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF, 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99"

Type :wq to write the changes to the file and quit vi

Uninstall Z-Wave integration

Navigate to Configuration -> Integrations

Click the three dots on the Z-Wave integration and select Delete

Click OK when prompted

Click OK on the prompt that you should restart home assistant (it won't restart home assistant at this point)

Restart Home Assistant

While we can restart Home Assistant from the web UI, we need to ensure that the Docker container running home assistant no longer needs access to your Z-Wave stick directly (Z-Wave JS Server will be what interfaces with the device directly). In this case, you will need to SSH into your Home Assistant server and stop / remove / start the container accordingly.

Stop the Docker container

sudo docker stop home-assistant

Remove the container

sudo docker rm home-assistant

Deploy the new container configuration, which removes any device mappings to your Z-Wave stick/device

sudo docker run --init -d --restart=always --name="home-assistant" -e "TZ=America/Chicago" -v /home/docker/home-assistant:/config --net=host homeassistant/raspberrypi4-homeassistant:stable

Install Z-Wave JS Server

Create a new directory for the zwave-js server configuration files

sudo mkdir /home/docker/zwave-js/

Run the docker container (the first port listed is for the Z-Wave JS Web Interface, the second port is the Z-Wave JS WebSocket listener)

sudo docker run -d --restart=always  -p 8091:8091 -p 3000:3000 --device=/dev/ttyACM0 --name="zwave-js" -e "TZ=America/Chicago" -v /home/docker/zwave-js:/usr/src/app/store zwavejs/zwavejs2mqtt:latest

Configure Z-Wave JS Server

Navigate to the JS Web Server

http://serverIP:8091/settings

On the settings page, enter the following configuration values (ensuring you substitute in the correct values obtained in the previous steps)

  • Serial Port: /dev/ttyACM0
  • Network Key: AABBCCDDEEFF00112233445566778899
    • Take your existing network key you obtained earlier and remove the 0x and "s to only leave one long hex string. For example:
      • Before
        • 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF, 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99
      • After
        • AABBCCDDEEFF00112233445566778899
  • Log Enabled: Disabled (toggled should be grayed)
  • Commands Timeout: 30 seconds
  • Disable MQTT Gateway: Can be enabled if you have no use for MQTT

Click the Home Assistant menu and set the following:

  • WS Server: Enabled
  • Server Port: 3000

Click SAVE

Verify you see devices

Click on the Control Panel icon on the top left of the Z-Wave JS Web UI. Verify that you see the amount of devices you previously had.

At this point, I would recommend waiting a few minutes / possibly hours to let the table populate with all the device information.

Install Z-Wave JS Integration

I would recommend a full refresh of the web page for Home Assistant and then navigate back to Configuration -> Integrations

Click the Add Integration button and search for Z-Wave JS

Click Submit to accept the URL as-is (assuming you are running the container on the same server running the Home Assistant container; if not, you can specify the IP address of the server hosting the Z-Wave JS Server container as well).

If all went well, you should see your Z-Wave devices and you can click Finish (Note: I wouldn't worry about specifying Areas since it's likely you have no idea what device is what at this point)

Update your Z-Wave Device Names in Home Assistant

The last step is to update your device names to match your existing device names. To do this, on the Configuration -> Integrations page, select the devices link on the Z-Wave JS integration tile

Next, select one of the items in your list. In my case, I'm going to select the first 1000W Dimmer I have.

On the device, you should see Node ID. This can be looked up on your list of devices you exported in the previous steps.

Click the Device Name (in my case 1000W Dimmer) and specify the correct information for the device. Once done, click Update

Rinse and repeat

Go through each of the devices you have and update their corresponding names. If you click Advanced settings, you can specify the area for the device as well.

Congrats!

If you've made it this far, you have successfully migrated to the latest Z-Wave integration for Home Assistant!

How to generate base64 encoded SSL certificates via PowerShell for Azure

Background

Many Azure services allow you to bring your own SSL Certificate to the cloud. While Azure provides an easy way to create and deploy resources through ARM templates, specification of what SSL certificate is a little less trivial since it's not as easy to specify an exported PEM or PFX file. In this case, Azure may look for the certificate in a base64 encoded format, so the certificate can be passed as a string (or list of characters) into the template.

Goal of this tutorial

This tutorial will walk through the commands needed to generate a self-signed certificate that is base64 encoded via PowerShell (Option 1) or base64 encode an existing PFX (Option 2), so that the certificate can be passed as a parameter into ARM templates in Azure.

Option 1: Generate and encode a self-signed certificate

Generate a self-signed certificate
$selfSignedCert = New-SelfSignedCertificate -DnsName *.azurewebsites.net -NotAfter (Get-Date).AddYears(2)
Export the self-signed certificate into PFX format from Certificate manager
$pwd = ConvertTo-SecureString -String "1234" -Force -AsPlainText
Export-PfxCertificate -cert $selfSignedCert.PSPath -FilePath "selfSignedCertificate.pfx" -Password $pwd
Convert the certificate to base64 encoding
$pfxBytes = Get-Content "selfSignedCertificate.pfx" -Encoding Byte
[System.Convert]::ToBase64String($pfxBytes) | Out-File "selfSignedCertificate.txt"

Option 2: Encode from a pre-existing pfx file

Convert the certificate to base64 encoding
$pfxBytes = Get-Content "selfSignedCertificate.pfx" -Encoding Byte
[System.Convert]::ToBase64String($pfxBytes) | Out-File "selfSignedCertificate.txt"

Result

At this point, if you open selfSignedCertificate.txt, you should see a long list of characters compromised of letters, numbers, and a few symbols, which is your base64 version of your certificate. See example below (...s denote I removed a large portion of the text, you won't see that in your file).

MIIKcQIBAzCCCi0GCSqGSIb3DQEHAaCCCh4EggoaMIIKFjCCBg8GCSqGSIb3DQEHAaCCBgAEggX8MIIF+DCCBfQGCyqGSIb3DQEMCgECoIIE/jCCBPowHAYKKoZIhvcNAQwBAzAOBAij81GovXchnAICB9AEggTYvVQbLThNVlLYiivGlD0uSASG3g6OaY9xF+c0BfZ1ZCHGKKQ3705CDkIy4.......jx9lSOAForjR+e1nNaBFfMGy+ONccoS0lnWvFIgggZG8RCZx2jQGMnPQdm4hPdmL3j2pUPMDswHzAHBgUrDgMCGgQUJpp3pnPr5/NXgyhYzi+rGzVkCJMEFBsqGkHSsFZaBXQ/bvR5DnhzgaekAgIH0A==

This text can be used-as within your templates now (although, in general, try to never code these values into your templates, these values should be passed as parameters into the template).