Category Archives: Raspberry Pi

How to upgrade Home Assistant Z-Wave integration to Z-Wave JS for Docker

If you’ve been following my last two tutorials (Home Assistant + Docker + Z-Wave + Raspberry Pi | Jack Stromberg and How to update Home Assistant Docker Container | Jack Stromberg) on running Home Assistant via Docker and how to keep the container updated, you may have noticed that 2021 has been a big year for larger changes, with a surprising change coming to how Home Assistant handles Z-Wave Devices.

In Home Assistant v2021.2, Home Assistant announced the Z-Wave integration as deprecated in favor of a new integration called Z-Wave JS. In Home Assistant v2021.3, many fixes were implemented, with the notable limitation of Door Sensors being removed.

So why the change?

As per the Home Assistant v2021.2 announcement:

More and more people were concerned about the future of Z-Wave with Home Assistant; meanwhile the Z-Wave JS project was rapidly growing and gathering a large community around it. Long story short: Home Assistant and Z-Wave JS teamed up! And a lot of contributors jumped on the train!

This new integration is based on the same base principles as the OpenZWave integration: It is decoupled from Home Assistant. Instead of MQTT, the Z-Wave JS integration uses a WebSocket connection to a Z-Wave JS server.

This means, in order to use this new integration, you’ll need to run the Z-Wave JS server that sits in between your Z-Wave USB stick and Home Assistant. There are multiple options available for running the Z-Wave JS server, via Docker or manually, and there is also a Home Assistant add-on available.

So how do I upgrade?

This article reflects the steps I took to update my Z-Wave implementation.

Ensure you are running Home Assistant v2021.3.2 or greater

This will ensure you have support for most all sensors. You can find your Home Assistant version by selecting the Configuration gear on the left menu, and then selecting Info

Here you should see the version of Home Assistant (in my case 2021.3.2)

If you are not running the latest version, you can follow my upgrade steps here: How to update Home Assistant Docker Container | Jack Stromberg

Create a backup

It’s rather critical to create a backup, especially in this case if you need to roll back to the older OpenZWave integration if you find many of your devices not being compatible. One downside in not using Home Assistant’s OS is you don’t have the “Supervisor” option to create a full backup.

To complete this step, I’d recommend checking out this blog post here which provides several options: Backing up Home Assistant | Tinkering with Home Automation (ceard.tech)

Alternatively, you can be extremely lazy and less cautious by simply copying the configuration folder containing your docker config:

sudo cp /home/docker/home-assistant/ /home/docker/home-assistant-backup/ -R

Update your Operating System

Execute the following commands against your machine:

sudo sh -c 'apt update && apt upgrade'

Make sure you restart your machine to ensure your kernel updates to the latest version:

sudo shutdown -r -t now

Document Z-Wave entity IDs

The easiest way to do this is to navigate to Developer Tools (hammer icon on the left menu) and then type node_id into the Attributes column’s filter.

In this case, you’ll want to write down the node_id and the name of the entity it maps to. If you want to do this quickly, you can single click on the table, press Control + A to select all contents, or cmd+a on a Mac, and copy the contents into Word or Excel (Excel works remarkably well).

Document & Comment Z-Wave Stick Hardware ID and Network Key

SSH to your server and find your configuration.yaml file (if using my tutorial it should be /home/docker/home-assistant/configuration.yaml). Open the file in vi

sudo vi configuration.yaml

Find the section of code labeled zwave: and copy the information (we’ll need it later) as well as comment out the following lines like so:


#zwave:
#  usb_path: /dev/ttyACM0
#  network_key: "0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF, 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99"

Type :wq to write the changes to the file and quit vi

Uninstall Z-Wave integration

Navigate to Configuration -> Integrations

Click the three dots on the Z-Wave integration and select Delete

Click OK when prompted

Click OK on the prompt that you should restart home assistant (it won’t restart home assistant at this point)

Restart Home Assistant

While we can restart Home Assistant from the web UI, we need to ensure that the Docker container running home assistant no longer needs access to your Z-Wave stick directly (Z-Wave JS Server will be what interfaces with the device directly). In this case, you will need to SSH into your Home Assistant server and stop / remove / start the container accordingly.

Stop the Docker container

sudo docker stop home-assistant

Remove the container

sudo docker rm home-assistant

Deploy the new container configuration, which removes any device mappings to your Z-Wave stick/device

sudo docker run --init -d --restart=always --name="home-assistant" -e "TZ=America/Chicago" -v /home/docker/home-assistant:/config --net=host homeassistant/raspberrypi4-homeassistant:stable

Install Z-Wave JS Server

Create a new directory for the zwave-js server configuration files

sudo mkdir /home/docker/zwave-js/

Run the docker container (the first port listed is for the Z-Wave JS Web Interface, the second port is the Z-Wave JS WebSocket listener)

sudo docker run -d --restart=always  -p 8091:8091 -p 3000:3000 --device=/dev/ttyACM0 --name="zwave-js" -e "TZ=America/Chicago" -v /home/docker/zwave-js:/usr/src/app/store zwavejs/zwavejs2mqtt:latest

Configure Z-Wave JS Server

Navigate to the JS Web Server

http://serverIP:8091/settings

On the settings page, enter the following configuration values (ensuring you substitute in the correct values obtained in the previous steps)

  • Serial Port: /dev/ttyACM0
  • Network Key: AABBCCDDEEFF00112233445566778899
    • Take your existing network key you obtained earlier and remove the 0x and “s to only leave one long hex string. For example:
      • Before
        • 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF, 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99
      • After
        • AABBCCDDEEFF00112233445566778899
  • Log Enabled: Disabled (toggled should be grayed)
  • Commands Timeout: 30 seconds
  • Disable MQTT Gateway: Can be enabled if you have no use for MQTT

Click the Home Assistant menu and set the following:

  • WS Server: Enabled
  • Server Port: 3000

Click SAVE

Verify you see devices

Click on the Control Panel icon on the top left of the Z-Wave JS Web UI. Verify that you see the amount of devices you previously had.

At this point, I would recommend waiting a few minutes / possibly hours to let the table populate with all the device information.

Install Z-Wave JS Integration

I would recommend a full refresh of the web page for Home Assistant and then navigate back to Configuration -> Integrations

Click the Add Integration button and search for Z-Wave JS

Click Submit to accept the URL as-is (assuming you are running the container on the same server running the Home Assistant container; if not, you can specify the IP address of the server hosting the Z-Wave JS Server container as well).

If all went well, you should see your Z-Wave devices and you can click Finish (Note: I wouldn’t worry about specifying Areas since it’s likely you have no idea what device is what at this point)

Update your Z-Wave Device Names in Home Assistant

The last step is to update your device names to match your existing device names. To do this, on the Configuration -> Integrations page, select the devices link on the Z-Wave JS integration tile

Next, select one of the items in your list. In my case, I’m going to select the first 1000W Dimmer I have.

On the device, you should see Node ID. This can be looked up on your list of devices you exported in the previous steps.

Click the Device Name (in my case 1000W Dimmer) and specify the correct information for the device. Once done, click Update

Rinse and repeat

Go through each of the devices you have and update their corresponding names. If you click Advanced settings, you can specify the area for the device as well.

Congrats!

If you’ve made it this far, you have successfully migrated to the latest Z-Wave integration for Home Assistant!

Home Assistant + Docker + Z-Wave + Raspberry Pi

Notice: Home Assistant has released a new integration called Z-Wave JS. You should be using that integration vs the older Z-Wave integration that this article covers. I will be updating this guide soon.

A few years back I had a SmartThings Hub and for the most part it worked great. It was simple to setup, can be accessed anywhere, and for the most part automatically updated itself. Unfortunately, with the acquisition of it by Samsung, it seems to have turned into bloatware with poor responsiveness, the mobile application’s UI is horrific, and they have a less than desirable security/privacy policy.

Luckily, the open source community has thrown together Home Assistant, an open source home automation project backed by hundreds/thousands of individuals. Over the years, they have now brought native support for mobile devices, at time of writing this there are 1500+ integrations for dang near any device, and the software puts you in control of who has access to and where your data is accessible.

The one trade-off though is while Home Assistant works well and is very extensible, the documentation and usability of the application can be overwhelming to understand for someone new to home automation, unfamiliar with Linux/Open Source technologies, or new to debugging/command line interfaces.

In this case, I’ve tried to document a crash course in getting Home Assistant up and running as quickly as possible for those that want to get started with Z-Wave devices and Home Assistant.

Hardware

You can leverage pretty much any hardware with Home Assistant, but here are the two items I used in my venture. Home Assistant has a full list of recommendations for what hardware to use for Home Assistant (https://www.home-assistant.io/getting-started/#suggested-hardware) as well as what Z-Wave controllers are supported (https://www.home-assistant.io/docs/z-wave/controllers/).

Update your Raspberry Pi

First things first, update your Raspberry Pi with the latest updates. Open up Terminal or SSH to your Raspberry Pi and execute the following command:

sudo apt-get update && sudo apt-get upgrade

Prepare your Z-Wave USB Stick

Plug in your Z-Wave USB stick. Once plugged in, we need to find the device path so that we can reference it for Home Assistant. Execute the lsusb command to find your device ID. In this case, you can see my device ID begins with 0658.

[email protected]:/dev# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Next, let’s find what the device path is for the USB stick. You can do this by executing the following command: dmesg | egrep ‘0658|acm’ Please note, if you purchased a difference device, 0658 may be a different number. In this case, you can see my device is presented on ttyACM0.

[email protected]:/dev# dmesg | egrep '0658|acm'
[    1.405327] usb 1-1.2: New USB device found, idVendor=0658, idProduct=0200, bcdDevice= 0.00
[    3.468875] cdc_acm 1-1.2:1.0: ttyACM0: USB ACM device
[    3.471348] usbcore: registered new interface driver cdc_acm
[    3.471359] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters

Install Docker

Home Assistant doesn’t require Docker, but by leveraging Docker you can easily copy/backup your configuration and simply redeploy the container if something goes wrong. As updates are made, you can simply remove your container and redeploy. To install Docker, execute the following command:

curl -sSL https://get.docker.com | sh

Deploy Home Assistant Docker Container

Once Docker is installed, you can deploy the container from Docker Hub. Docker Hub is a public repository that has tons of different prebuilt containers to deploy. Here you can find the official homeassistant containers: https://hub.docker.com/u/homeassistant

To deploy the container, execute the following line, replacing the following variables with your desired configuration:

  • –name=”the name of your container
  • -e “TM=YourTimezone
  • –device=/dev/ttyACM0
    • This allows the container to leverage the Z-Wave USB device. Make sure you specify the path to your device found in the previous step
  • -v /home/docker/home-assistant:/config
    • This is the path that the home assistant configuration files should be stored to. You can specify a fileshare or other path to place your configuration files.
  • –net=host homeassistant/raspberrypi4-homeassistant:stable
    • The first half of this is the container you wish to deploy and the second half is the version. You can find all of Home Assistant’s official containers here: https://hub.docker.com/u/homeassistant
sudo docker run --init -d --restart=always --name="home-assistant" -e "TZ=America/Chicago" --device=/dev/ttyACM0 -v /home/docker/home-assistant:/config --net=host homeassistant/raspberrypi4-homeassistant:stable

Setup Home Assistant

Give the container a few minutes to deploy and configure itself for the first time. After a few minutes, try opening your web browser and navigating to the IP address assigned to your machine, using port number 8123: http://192.168.1.2:8123/

When the page loads, it should first ask for your Name, Username, and Password. This is the username and password you will use to login to Home Assistant.

Next, specify the location of where your Home Assistant deployment is located. Oddly enough, you cannot type in a location, but you can place the pin near your location by dragging the map around and clicking once to set the pin.

Once you click Next, Home Assistant may have already found a few devices connected to your network. You can add them now or skip and add them later.

Tell Home Assistant to use your Z-Wave USB Stick

Although we granted access to the container to use the Z-Wave USB Stick, you need to tell Home Assistant how to leverage the device. To do so, you will need to open up Terminal or SSH to your machine and edit the configuration.yaml file to point to the device. Before we get into modifying the configuration.yaml file, first execute the following command to generate a Z-Wave Security Key. This key may be required by Z-Wave security devices (Door Locks, Keypads, etc), as an extra layer of security. More information on this can be found here: https://www.home-assistant.io/docs/z-wave/adding#network-key

Execute the following command via Terminal or SSH:

cat /dev/urandom | tr -dc '0-9A-F' | fold -w 32 | head -n 1 | sed -e 's/\(..\)/0x\1, /g' -e 's/, $//'

Once you execute the command, it should give you a string of characters that look something like:

“0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10”

Next, we need to edit the configuration.yaml file, which can be found in the path specified when the Docker container was deployed (using the -v parameter). For the purpose of this article, /home/docker/home-assistant/configuration.yaml is where the file is located. Using your favorite text editor, add the following lines of code:

zwave:
  usb_path: /dev/ttyACM0
  network_key: "0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10"
configuration.yaml file with Z-Wave configuration

Once saved, go back to Home Assistant and click the Gear icon and then select Server Controls

Select the Restart button to restart Home Assistant. Any time you make a change to the configuration.yaml file, you will need to restart Home Assistant to pickup the configuration changes.

Click OK to Restart

Upon restart, navigate back to the Gear icon and you should see a new entry in the Config portal for Z-Wave. If you do not see the “Z-Wave” section, scroll down to the troubleshooting step at the end of this article.

Add a Z-Wave device

Once you see that your Z-Wave network has started, adding a device is a piece of cake. First click the Add Node button. When you click the button, nothing will happen, but go ahead and put your device in inclusion mode. Once the device is in inclusion mode, Home Assistant should automatically add the device.

At this point, if you navigate back to Configuration (Gear icon) and select Devices

You should see your newly added Z-Wave device!

At this point, you can select the Device to give it a friendly name or start to work on building your own home automation actions.

Hope this helped! If you have any comments or suggestions on how to improve this guide, please drop it below.

Troubleshooting Missing Z-Wave Configuration

The first time I ran through this, I noticed I was missing the Z-Wave configuration tile after making changes to the configuration.yaml file. It turned out I specified the wrong device path in the configuration file. To verify, you can check the logs from your Docker container by executing the following command in your Terminal or via SSH. (Replace home-assistant with the name of your container if you specified something else)

sudo docker logs home-assistant

In my case, I had the following error:

2020-02-16 21:08:01 INFO (MainThread) [homeassistant.components.scene] Setting up scene.homeassistant
2020-02-16 21:08:02 INFO (MainThread) [homeassistant.components.zwave] Z-Wave USB path is /dev/ttyACM01
2020-02-16 21:08:02 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Z-Wave (import from configuration.yaml) for zwave
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 78, in __init__
    raise ZWaveException(u"Can't find device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))
openzwave.object.ZWaveException: "Zwave Generic Exception : Can't find device /dev/ttyACM01 : ['NoneType: None\\n']"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 215, in async_setup
    hass, self
  File "/usr/src/homeassistant/homeassistant/components/zwave/__init__.py", line 369, in async_setup_entry
    config_path=config.get(CONF_CONFIG_PATH),
  File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 81, in __init__
    raise ZWaveException(u"Error when retrieving device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))
openzwave.object.ZWaveException: 'Zwave Generic Exception : Error when retrieving device /dev/ttyACM01 : [\'Traceback (most recent call last):\\n\', \'  File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 78, in __init__\\n    raise ZWaveException(u"Can\\\'t find device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))\\n\', \'openzwave.object.ZWaveException: "Zwave Generic Exception : Can\\\'t find device /dev/ttyACM01 : [\\\'NoneType: None\\\\\\\\n\\\']"\\n\']'

Here you can see I accidentally specified /dev/ttyACM01 vs /dev/ttyACM0. Simply updating the configuration.yaml file with the correct device path solved the issue.

Setting up a new Raspberry Pi via SSH

This is my super subpar tutorial on how to quickly setup a new Raspberry Pi via SSH (no mouse/keyboard/monitor directly attached to the device).

  1. Download the latest copy of the operating system (I personally prefer Raspbian Stretch Lite for the most minimal setup): https://www.raspberrypi.org/downloads/raspbian/
  2. Extract the download so you have a copy of the ****-**-**-raspbian-stretch-lite.img file
  3. Download Etcher to burn the image to an SD Card: https://etcher.io/
  4. Download a copy of Putty if you don’t have a way to ssh: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
  5. Open the SD card you just flashed and you should see the “boot” partition.  Create a file called ssh (no file extension or data needs to be written to the file)
    1. Note: ssh is disabled on all OS builds starting November 16 forward — see here: https://www.raspberrypi.org/documentation/remote-access/ssh/
  6. Default credentials:
    1. Username: pi
    2. Password: raspberry
  7. Quick commands
    1. Configure Raspberry PI specific settings: sudo raspi-config
    2. Proper Shutdown (-h) / Restart (-r): sudo shutdown -h now

Unlike most laptops/desktops, the Raspberry Pi doesn’t have a shutdown button, so always use the commands above to prevent SD Card corruption!