First, validate what version of Z-Wave JS you are running. To do this, navigate to the Z-Wave JS webpage and hover over the i icon to validate what versions of the software you are running. The Z-Wave JS webpage can typically be accessed at http://yourip:8091.
Get the current name of your container and version
sudo docker ps
In running this command, note the NAME of your container as well as the IMAGE.
Stop and delete the container
Replace the name of the container in the command below with the value you had.
sudo docker stop zwave-js
sudo docker rm zwave-js
Update packages
Some versions of HA require newer versions of Python, Docker, etc. I may consider updating to latest package versions first.
sudo apt-get update
sudo apt-get upgrade
Pull the latest container from Docker Hub
Replace the value below with your IMAGE value you documented in the previous steps.
sudo docker pull zwavejs/zwavejs2mqtt:latest
Deploy the container
Make sure your replace the name and value of the image with the values in the previous step. In addition, ensure you specify the correct path to where you existing configuration files exist to have the container load your existing configurations.
Debian has a really good write up here on what backports are. Copying directly from their introduction paragraph:
You are running Debian stable, because you prefer the Debian stable tree. It runs great, there is just one problem: the software is a little bit outdated compared to other distributions. This is where backports come in.
Backports are packages taken from the next Debian release (called "testing"), adjusted and recompiled for usage on Debian stable. Because the package is also present in the next Debian release, you can easily upgrade your stable+backports system once the next Debian release comes out. (In a few cases, usually for security updates, backports are also created from the Debian unstable distribution.)
Backports cannot be tested as extensively as Debian stable, and backports are provided on an as-is basis, with risk of incompatibilities with other components in Debian stable. Use with care!
It is therefore recommended to only select single backported packages that fit your needs, and not use all available backports.
Once I enable backports will all packages use them?
No! Any new packages and updates to existing stable packages will prefer the stable releases. The only time you will leverage a new backport package is if you explicitly specify to pull from them.
How do I enable backports?
First you need to add the new backport source to your sources.list file. Edit the file in vi:
sudo vi /etc/apt/sources.list
Arrow down to the last row, press o to create a new line and then enter the following:
deb http://deb.debian.org/debian buster-backports main
Press escape and then type :wq to save the changes and exit via.
Next, we need to specify a keyserver to verify the authenticity of these packages. Note we use Ubuntu's key servers to validate the packages. Interestingly, Debian has a keyring to validate the packages, however the keyring doesn't contain the backports for buster on the raspberry pi at time of writing this. Ubuntu's servers will work fine to validate the authenticity of these packages and you will ultimately pull the packages from Debian rather than Ubuntu.
Continuing from my previous guide on how to setup Home Assistant + Docker + Z-Wave + Raspberry Pi, this tutorial will show you how to update Home Assistant to the latest version. Updating Home Assistant to the latest version is critical to ensure you have the latest bug fixes, integrations, and security patches.
Note: during the update your devices will continue to work fine, but please note any automations or access to the application will not be available, so it's recommended to do this during a time that you know no automations will be running.
Validate your current version
Navigate to the Developer Tools section of Home Assistant. Here you can validate the latest version you currently have deployed.
Get the current name of your container and version
sudo docker ps
In running this command, note the NAME of your container as well as the IMAGE.
Stop and delete the container
Replace the name of the container in the command below with the value you had.
Make sure your replace the name and value of the image with the values in the previous step. In addition, ensure you specify the correct path to where you existing configuration files exist to have the container load your existing configurations.
Note: If you are now using the Z-Wave JS docker container, you will not want to attach the Z-Wave stick to the Home-Assistant container. In this case, run the following command:
In newer versions of the docker container --init should not be specified in the docker run command. Specifying --init will result with the following error: "s6-overlay-suexec: fatal: can only run as pid 1". This was mentioned as a breaking change in: 2022-06-01 update: 2022.6: Gaining new insights! - Home Assistant (home-assistant.io)
Notice: Home Assistant has released a new integration called Z-Wave JS. You should be using that integration vs the older Z-Wave integration that this article covers. I will be updating this guide soon.
A few years back I had a SmartThings Hub and for the most part it worked great. It was simple to setup, can be accessed anywhere, and for the most part automatically updated itself. Unfortunately, with the acquisition of it by Samsung, it seems to have turned into bloatware with poor responsiveness, the mobile application's UI is horrific, and they have a less than desirable security/privacy policy.
Luckily, the open source community has thrown together Home Assistant, an open source home automation project backed by hundreds/thousands of individuals. Over the years, they have now brought native support for mobile devices, at time of writing this there are 1500+ integrations for dang near any device, and the software puts you in control of who has access to and where your data is accessible.
The one trade-off though is while Home Assistant works well and is very extensible, the documentation and usability of the application can be overwhelming to understand for someone new to home automation, unfamiliar with Linux/Open Source technologies, or new to debugging/command line interfaces.
In this case, I've tried to document a crash course in getting Home Assistant up and running as quickly as possible for those that want to get started with Z-Wave devices and Home Assistant.
Home Assistant will run on any version of Raspberry Pi, but it is recommended to use version 3 or 4 for best performance. In this guide, I use a Raspberry Pi 4 for reference. Below is a link to the Raspberry Pi kit, which contains everything you need to get started.
First things first, update your Raspberry Pi with the latest updates. Open up Terminal or SSH to your Raspberry Pi and execute the following command:
sudo apt-get update && sudo apt-get upgrade
Prepare your Z-Wave USB Stick
Plug in your Z-Wave USB stick. Once plugged in, we need to find the device path so that we can reference it for Home Assistant. Execute the lsusb command to find your device ID. In this case, you can see my device ID begins with 0658.
root@raspberrypi:/dev# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Next, let's find what the device path is for the USB stick. You can do this by executing the following command: dmesg | egrep '0658|acm' Please note, if you purchased a difference device, 0658 may be a different number. In this case, you can see my device is presented on ttyACM0.
root@raspberrypi:/dev# dmesg | egrep '0658|acm'
[ 1.405327] usb 1-1.2: New USB device found, idVendor=0658, idProduct=0200, bcdDevice= 0.00
[ 3.468875] cdc_acm 1-1.2:1.0: ttyACM0: USB ACM device
[ 3.471348] usbcore: registered new interface driver cdc_acm
[ 3.471359] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
Install Docker
Home Assistant doesn't require Docker, but by leveraging Docker you can easily copy/backup your configuration and simply redeploy the container if something goes wrong. As updates are made, you can simply remove your container and redeploy. To install Docker, execute the following command:
curl -sSL https://get.docker.com | sh
Deploy Home Assistant Docker Container
Once Docker is installed, you can deploy the container from Docker Hub. Docker Hub is a public repository that has tons of different prebuilt containers to deploy. Here you can find the official homeassistant containers: https://hub.docker.com/u/homeassistant
To deploy the container, execute the following line, replacing the following variables with your desired configuration:
This allows the container to leverage the Z-Wave USB device. Make sure you specify the path to your device found in the previous step
-v /home/docker/home-assistant:/config
This is the path that the home assistant configuration files should be stored to. You can specify a fileshare or other path to place your configuration files.
The first half of this is the container you wish to deploy and the second half is the version. You can find all of Home Assistant's official containers here: https://hub.docker.com/u/homeassistant
Note: In newer versions of the docker container --init should not be specified in the docker run command. Specifying --init will result with the following error: "s6-overlay-suexec: fatal: can only run as pid 1". This was mentioned as a breaking change in: 2022-06-01 update: 2022.6: Gaining new insights! - Home Assistant (home-assistant.io)
Setup Home Assistant
Give the container a few minutes to deploy and configure itself for the first time. After a few minutes, try opening your web browser and navigating to the IP address assigned to your machine, using port number 8123: http://192.168.1.2:8123/
When the page loads, it should first ask for your Name, Username, and Password. This is the username and password you will use to login to Home Assistant.
Next, specify the location of where your Home Assistant deployment is located. Oddly enough, you cannot type in a location, but you can place the pin near your location by dragging the map around and clicking once to set the pin.
Once you click Next, Home Assistant may have already found a few devices connected to your network. You can add them now or skip and add them later.
Tell Home Assistant to use your Z-Wave USB Stick
Although we granted access to the container to use the Z-Wave USB Stick, you need to tell Home Assistant how to leverage the device. To do so, you will need to open up Terminal or SSH to your machine and edit the configuration.yaml file to point to the device. Before we get into modifying the configuration.yaml file, first execute the following command to generate a Z-Wave Security Key. This key may be required by Z-Wave security devices (Door Locks, Keypads, etc), as an extra layer of security. More information on this can be found here: https://www.home-assistant.io/docs/z-wave/adding#network-key
Execute the following command via Terminal or SSH:
Next, we need to edit the configuration.yaml file, which can be found in the path specified when the Docker container was deployed (using the -v parameter). For the purpose of this article, /home/docker/home-assistant/configuration.yaml is where the file is located. Using your favorite text editor, add the following lines of code:
Once saved, go back to Home Assistant and click the Gear icon and then select Server Controls
Select the Restart button to restart Home Assistant. Any time you make a change to the configuration.yaml file, you will need to restart Home Assistant to pickup the configuration changes.
Click OK to Restart
Upon restart, navigate back to the Gear icon and you should see a new entry in the Config portal for Z-Wave. If you do not see the "Z-Wave" section, scroll down to the troubleshooting step at the end of this article.
Add a Z-Wave device
Once you see that your Z-Wave network has started, adding a device is a piece of cake. First click the Add Node button. When you click the button, nothing will happen, but go ahead and put your device in inclusion mode. Once the device is in inclusion mode, Home Assistant should automatically add the device.
At this point, if you navigate back to Configuration (Gear icon) and select Devices
You should see your newly added Z-Wave device!
At this point, you can select the Device to give it a friendly name or start to work on building your own home automation actions.
Hope this helped! If you have any comments or suggestions on how to improve this guide, please drop it below.
Troubleshooting Missing Z-Wave Configuration
The first time I ran through this, I noticed I was missing the Z-Wave configuration tile after making changes to the configuration.yaml file. It turned out I specified the wrong device path in the configuration file. To verify, you can check the logs from your Docker container by executing the following command in your Terminal or via SSH. (Replace home-assistant with the name of your container if you specified something else)
sudo docker logs home-assistant
In my case, I had the following error:
2020-02-16 21:08:01 INFO (MainThread) [homeassistant.components.scene] Setting up scene.homeassistant
2020-02-16 21:08:02 INFO (MainThread) [homeassistant.components.zwave] Z-Wave USB path is /dev/ttyACM01
2020-02-16 21:08:02 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Z-Wave (import from configuration.yaml) for zwave
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 78, in __init__
raise ZWaveException(u"Can't find device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))
openzwave.object.ZWaveException: "Zwave Generic Exception : Can't find device /dev/ttyACM01 : ['NoneType: None\\n']"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 215, in async_setup
hass, self
File "/usr/src/homeassistant/homeassistant/components/zwave/__init__.py", line 369, in async_setup_entry
config_path=config.get(CONF_CONFIG_PATH),
File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 81, in __init__
raise ZWaveException(u"Error when retrieving device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))
openzwave.object.ZWaveException: 'Zwave Generic Exception : Error when retrieving device /dev/ttyACM01 : [\'Traceback (most recent call last):\\n\', \' File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 78, in __init__\\n raise ZWaveException(u"Can\\\'t find device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))\\n\', \'openzwave.object.ZWaveException: "Zwave Generic Exception : Can\\\'t find device /dev/ttyACM01 : [\\\'NoneType: None\\\\\\\\n\\\']"\\n\']'
Here you can see I accidentally specified /dev/ttyACM01 vs /dev/ttyACM0. Simply updating the configuration.yaml file with the correct device path solved the issue.
In my career in doing IT, handling email is one of the most tedious tasks to setup/maintain due to so many moving pieces; many of which may be out of your control. Dealing with spam, blacklisting, having emails non-deliverable for several reasons, handling dns records, certificates, etc.... it's sometimes worth paying a few extra bucks to have someone else host your email and have peace of mind the message will be delivered. That being said, if you have the extra time on your hands and like the challenge of solving problems, here's a quick way to get started.
Preamble
This guide took me several hours to compile through trial and error. If you have any thoughts, notice any errors/typos, or have ideas on how to further secure/optimize, please leave feedback below to further improve this guide. Thank you and good luck on the deployment of your mail server!
You are running Ubuntu or Debian as per the above guide (you can still follow this guide, you may have to slightly change which commands you use for your distribution -- configuration should remain the same though)
DNS
Let's first start at getting your DNS records configured properly. This guide will talk about configuring MX, SPF, and PTR records. We won't be covering Domain Keys in this article, maybe in a separate article if someone donates to my paypal on the right side of the website 😉
MX Record
Via your nameservers, add a new mx record for your domain name. Here's a list of tutorials for some of the major domain registrars:
Contrary to many websites that say you need to create a "SPF" record type, the SPF record type was never ratified by RFC standards. In this case, the proper way to create a SPF record is via a TXT record with the SPF value (as per RFC 7208).
You can leverage my SPF generator to create a new TXT record in the root of your domain.
PTR Record
To help decrease the odds of your emails being labeled as spam, I'd recommend creating a PTR record that will resolve your IP address to a DNS name (we call this a reverse lookup). For example, if my mail server's domain name was mail.mydomain.com and it resolved to 123.123.123.123, I would create a PTR record for 123.123.123.123 that points to mail.mydomain.com.
In many cases, you will need to either work with your ISP (Internet Service Provider) or domain registrar if you own your own IP block to make changes to the record for your IP address block.
When you are ready, you can leverage the nslookup command on Windows to validate the name from the IP address.
nslookup 123.123.123.123
Or on linux you can leverage the host command to verify the reverse lookup as well:
host 123.123.123.123
Get the OS ready
Download the latest packages and actually perform any updates.
sudo sh -c 'apt update && apt upgrade'
Prepare MariaDB for virtual users/aliases
One of the primary reasons we need to configure a database is it is what will contain the information about all of our users and their corresponding email addresses (aliases). To do so, we need to create 3 new tables inside of a new database.
Login to the database
sudo mariadb -u root -p
Create the database, database user, and tables
Create a new database for our users (in this case, I'm calling the database mailserver). Note: This command must be run in the context of mariadb, this is not a bash command.
create database mailserver;
Create a new user called mailuser, grant them access to the entire database, require the user to only create connections from 127.0.0.1 (localhost), and specify a password for the user.
GRANT SELECT ON mailserver.* TO 'mailuser'@'127.0.0.1' IDENTIFIED BY 'mysupersecretpassword';
Execute the following command to apply the changes
FLUSH PRIVILEGES;
Create a table for each of the domain names we will leverage for our email addresses.
CREATE TABLE `mailserver`.`virtual_domains` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(50) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Create a table that will hold each of the users that will need mailboxes.
CREATE TABLE `mailserver`.`virtual_users` (
`id` int(11) NOT NULL auto_increment,
`domain_id` int(11) NOT NULL,
`password` varchar(106) NOT NULL,
`email` varchar(100) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Create a table that will hold aliases (additional email addresses) for a particular user.
CREATE TABLE `mailserver`.`virtual_aliases` (
`id` int(11) NOT NULL auto_increment,
`domain_id` int(11) NOT NULL,
`source` varchar(100) NOT NULL,
`destination` varchar(100) NOT NULL,
PRIMARY KEY (`id`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Insert a new user into the database
First, we need to add our first domain name into the domains table
INSERT INTO `mailserver`.`virtual_domains`
(`name`)
VALUES
('mydomain.com');
Second, we need to create the user. Replace mysupersecretpassword with your password.
Type exit once you are done to leave the context of MariaDB.
Install Packages for Postfix and Dovecot
Postfix is what we call a Mail Transport Agent (MTA) and is responsible for actually sending/receive the messages from the internet. Later, we will talk about Dovecot which will be our MDA (Mail Delivery Agent) (what actually interacts with the mailbox).
The following command will install postfix, dovecot, and pull the packages to interact with MySQL. Although these are labeled MySQL, they should interact fine with MariaDB.
During the installation of Postfix, you will be prompted to configure the connection type to the mail server. In this case, select Internet Site for the mail configuration.
On the second installation prompt, it will ask for the domain name used in receiving email. In this prompt, specify one of the domain names you will be using for your users. For example, if your email addresses are going to be [email protected] you would specify mydomain.com for this prompt. Don't worry if you have multiple email addresses, we will cover that later on.
Configure Postfix to leverage MariaDB
First, let's create a backup of the Postfix configuration, so we have a baseline to refer back to.
Copy the following configuration and replace the domain name example.com with yours. Credit to linode for sharing their configuration as it not only defines integration into a database, but also hardens the Postfix deployment.
# See /usr/share/postfix/main.cf.dist for a commented, more complete version
# Debian specific: Specifying a file name will cause the first
# line of that file to be used as the name. The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname
smtpd_banner = $myhostname ESMTP $mail_name
biff = no
# appending .domain is the MUA's job.
append_dot_mydomain = no
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
readme_directory = no
# TLS parameters
smtpd_tls_cert_file=/etc/letsencrypt/live/mydomain.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mydomain.com/privkey.pem
smtpd_use_tls=yes
smtpd_tls_auth_only = yes
smtp_tls_security_level = may
smtpd_tls_security_level = may
smtpd_sasl_security_options = noanonymous, noplaintext
smtpd_sasl_tls_security_options = noanonymous
# Authentication
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.
# Restrictions
smtpd_helo_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_invalid_helo_hostname,
reject_non_fqdn_helo_hostname
smtpd_recipient_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
reject_unlisted_recipient,
reject_unauth_destination
smtpd_sender_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_non_fqdn_sender,
reject_unknown_sender_domain
smtpd_relay_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
defer_unauth_destination
# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.
myhostname = example.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydomain = mydomain.com
myorigin = $mydomain
mydestination = localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
# Handing off local delivery to Dovecot's LMTP, and telling it where to store mail
virtual_transport = lmtp:unix:private/dovecot-lmtp
# Virtual domains, users, and aliases
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-users.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-mailbox-aliases.cf,
mysql:/etc/postfix/mysql-virtual-mailbox-users.cf
# Even more Restrictions and MTA params
disable_vrfy_command = yes
strict_rfc821_envelopes = yes
#smtpd_etrn_restrictions = reject
#smtpd_reject_unlisted_sender = yes
#smtpd_reject_unlisted_recipient = yes
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtp_always_send_ehlo = yes
#smtpd_hard_error_limit = 1
smtpd_timeout = 30s
smtp_helo_timeout = 15s
smtp_rcpt_timeout = 15s
smtpd_recipient_limit = 40
minimal_backoff_time = 180s
maximal_backoff_time = 3h
# Reply Rejection Codes
invalid_hostname_reject_code = 550
non_fqdn_reject_code = 550
unknown_address_reject_code = 550
unknown_client_reject_code = 550
unknown_hostname_reject_code = 550
unverified_recipient_reject_code = 550
unverified_sender_reject_code = 550
Next, we need to create the mappings of domain names, users, and aliases. In the same directory as the main.cf (/etc/postfix) we need to first create a file that will tell postfix how to lookup what domain names exist. You can open the documents with your favorite text editor; I use vi since it's universally installed.
sudo vi /etc/postfix/mysql-virtual-mailbox-domains.cf
Press i to get vi into insert mode and paste the following, replacing the password with the mailuser we specified earlier in this tutorial.
user = mailuser
password = mysupersecretpassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_domains WHERE name='%s'
Press : and then type wq and press enter to write the changes to the file and quit in vi.
Next, we will create another file that is used to lookup each user's mailbox.
sudo vi /etc/postfix/mysql-virtual-mailbox-users.cf
Press i to get vi into insert mode and paste the following, replacing the password with the mailuser we specified earlier in this tutorial.
user = mailuser
password = mysupersecretpassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM virtual_users WHERE email='%s'
Press : and then type wq and press enter to write the changes to the file and quit in vi.
Last, we will create another file that is used to map an alias to a user's mailbox.
sudo vi /etc/postfix/mysql-virtual-mailbox-aliases.cf
Press i to get vi into insert mode and paste the following, replacing the password with the mailuser we specified earlier in this tutorial.
user = mailuser
password = mysupersecretpassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT destination FROM virtual_aliases WHERE source='%s'
Press : and then type wq and press enter to write the changes to the file and quit in vi.
Restart the Postfix service for the changes to take effect
sudo service postfix restart
Next, to enable port 587 and 465 to connect securely with email clients, we need to modify /etc/postfix/master.cf. First, let's create a backup of the master.cf file.
Press : and then type wq and press enter to write the changes to the file and quit in vi.
Restart the Postfix service for the changes to take effect
sudo service postfix restart
Configure Dovecot
Now that we have our MTA configured, we now need to configure our MDA. You can think of Postfix as a shipping center and Dovecot as the courier, who interfaces directly with your mailbox. Roundcube will be our MUA (mail user agent) that interfaces with Dovecot to display your mail. The goal for this section is to ensure Dovecot requires SSL.
First, we'll create backups of each of the Dovecot configuration files
Next we need to edit the user authentication file (/etc/dovecot/conf.d/10-auth.conf) to tell Dovecat to leverage MariaDB for our users. Execute the following commands:
sudo sed -i 's/^#disable_plaintext_auth = yes/disable_plaintext_auth = yes/g' /etc/dovecot/conf.d/10-auth.conf
sudo sed -i 's/^#auth_mechanisms = plain login/auth_mechanisms = plain login/g' /etc/dovecot/conf.d/10-auth.conf
sudo sed -i 's/^!include auth-system.conf.ext/#!include auth-system.conf.ext/g' /etc/dovecot/conf.d/10-auth.conf
sudo sed -i 's/^#!include auth-sql.conf.ext/!include auth-sql.conf.ext/g' /etc/dovecot/conf.d/10-auth.conf
Once we have the authentication file configured, we need to update the sql driver (/etc/dovecot/conf.d/auth-sql.conf.ext) to point to our mailboxes. You will need to uncomment the passdb section and uncomment the userdb driver that is static.
sudo vi /etc/dovecot/conf.d/auth-sql.conf.ext
Press i to get vi into insert mode and paste the following configuration
# Authentication for SQL users. Included from 10-auth.conf.
#
# <doc/wiki/AuthDatabase.SQL.txt>
passdb {
driver = sql
# Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
args = /etc/dovecot/dovecot-sql.conf.ext
}
# "prefetch" user database means that the passdb already provided the
# needed information and there's no need to do a separate userdb lookup.
# <doc/wiki/UserDatabase.Prefetch.txt>
#userdb {
# driver = prefetch
#}
#userdb {
# driver = sql
# args = /etc/dovecot/dovecot-sql.conf.ext
#}
# If you don't have any user-specific settings, you can avoid the user_query
# by using userdb static instead of userdb sql, for example:
# <doc/wiki/UserDatabase.Static.txt>
userdb {
driver = static
args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
}
Press : and then type wq and press enter to write the changes to the file and quit in vi.
The final Dovecot file we need to modify will set our database settings (/etc/dovecot/dovecot-sql.conf.ext). Execute the following commands to uncomment the correct settings. Note: be sure to replace the password with the database password we configured earlier.
sudo sed -i 's/^#driver = /driver = mysql/g' /etc/dovecot/dovecot-sql.conf.ext
sudo sed -i 's/^#connect =/connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mysupersecretpassword/g' /etc/dovecot/dovecot-sql.conf.ext
sudo sed -i 's/^#default_pass_scheme = MD5/default_pass_scheme = SHA512-CRYPT/g' /etc/dovecot/dovecot-sql.conf.ext
sudo sed -i '/^#password_query = \\/i password_query = SELECT email as user, password FROM virtual_users WHERE email=\x27%u\x27;' /etc/dovecot/dovecot-sql.conf.ext
After making the changes to the dovecot-sql.conf.ext file, next we need to change the owner and the group of the dovecot folder to the vmail user:
Next, we need to disable the unencrypted versions of IMAP and SMTP.
sudo vi /etc/dovecot/conf.d/10-master.conf
We need to edit the /etc/dovecot/conf.d/10-master.conf file and set ports to 0 to disable non-encrypted imap/pop3. Find service imap-login { and make it look like the following.
service imap-login {
inet_listener imap {
port = 0
}
inet_listener imaps {
port = 993
ssl = yes
}
# Number of connections to handle before starting a new process. Typically
# the only useful values are 0 (unlimited) or 1. 1 is more secure, but 0
# is faster. <doc/wiki/LoginProcess.txt>
#service_count = 1
# Number of processes to always keep waiting for more connections.
#process_min_avail = 0
# If you set service_count=0, you probably need to grow this.
#vsz_limit = $default_vsz_limit
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
In the same file, find service lmtp { and replace the whole block down to the third } with the following:
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
mode = 0600
user = postfix
group = postfix
}
# Create inet listener only if you can't use the above UNIX socket
#inet_listener lmtp {
# Avoid making LMTP visible for the entire internet
#address =
#port =
#}
}
In the same file, find service auth { and replace the whole block down to the third } with the following:
service auth {
# auth_socket_path points to this userdb socket by default. It's typically
# used by dovecot-lda, doveadm, possibly imap process, etc. Users that have
# full permissions to this socket are able to get a list of all usernames and
# get the results of everyone's userdb lookups.
#
# The default 0666 mode allows anyone to connect to the socket, but the
# userdb lookups will succeed only if the userdb returns an "uid" field that
# matches the caller process's UID. Also if caller's uid or gid matches the
# socket's uid or gid the lookup succeeds. Anything else causes a failure.
#
# To give the caller full permissions to lookup all users, set the mode to
# something else than 0666 and Dovecot lets the kernel enforce the
# permissions (e.g. 0777 allows everyone full permissions).
unix_listener auth-userdb {
mode = 0600
user = vmail
#group =
}
# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
mode = 0600
user = postfix
group = postfix
}
# Auth process is run as this user.
user = dovecot
}
In the same file, find service auth-worker { and replace the whole block down to the } with the following:
service auth-worker {
# Auth worker process is run as root by default, so that it can access
# /etc/shadow. If this isn't necessary, the user should be changed to
# $default_internal_user.
user = vmail
}
Press : and then type wq and press enter to write the changes to the file and quit in vi.
Last, we need to tell dovecot where our SSL certificate is for encryption. We will modify the /etc/dovecot/conf.d/10-ssl.conf file. Make sure to update the directory with the correct path for your certificates.
Execute the following commands, replacing
sudo sed -i 's/^ssl = yes/ssl = required/g' /etc/dovecot/conf.d/10-ssl.conf
sudo sed -i 's/^ssl_cert = .*/ssl_cert = <\/etc\/letsencrypt\/live\/mydomain.com\/fullchain.pem/g' /etc/dovecot/conf.d/10-ssl.conf
sudo sed -i 's/^ssl_key = .*/ssl_key = <\/etc\/letsencrypt\/live\/mydomain.com\/privkey.pem/g' /etc/dovecot/conf.d/10-ssl.conf
Last, restart devocot to enable all of our changes.
sudo service dovecot restart
Configure Roundcube
Install dependencies for Roundcube
Roundcube requires several PHP PEAR modules. To install the bare minimum featureset, execute the following command:
First, we need to create a new database and user for Roundcube. We can do this by logging into MariaDB and executing the create and grant commands.
sudo mariadb -u myusername -p
CREATE DATABASE roundcubemail CHARACTER SET utf8 COLLATE utf8_general_ci;
GRANT ALL PRIVILEGES ON roundcubemail.* TO roundcube@localhost IDENTIFIED BY 'myreallyreallysecretpassword';
FLUSH PRIVILEGES;
exit
Request SSL Certificates for Roundcube
We will want to ensure all traffic to and from the client is encrypted in transit when trying to access Roundcube. To do this, I leverage Let's Encrypt, which will allow you to request a free SSL certificate. If you have your own SSL certificate, go ahead and copy it to a location on the server so we can reference it later.
We will need to create a directory that will hold Roundcube's files to serve to the web. Let's create a new directory to serve these files and limit permissions to www-data.
We will need to grab the latest copy of Roundcube's code to run the website. Note: please ensure you substitute the correct version for Roundcube when executing the commands below as the version listed in the guide will likely be out of date as time goes on:
cd /tmp
wget https://github.com/roundcube/roundcubemail/releases/download/1.4.1/roundcubemail-1.4.1.tar.gz
tar -xf roundcubemail-1.4.1.tar.gz
mv roundcubemail-1.4.1 /var/www/webmail.mydomain.com
Populate the SQL Database
You will need to execute the following SQL command to populate your Roundcube database with the tables needed to run Roundcube. To do so, execute the following commands.
Roundcube doesn't ship with several javascript dependencies. To ensure the Roundcube pages load properly, you will need to execute the following command to pull down the javascript dependencies.
Let's configure NGINX to point to our web directory for the website. When doing so, it is very important you protect your installation by preventing access to some sensitive files from the web.
First, create a virtual-host file within the nginx sites-available folder:
sudo vi /etc/nginx/sites-available/webmail.mydomain.com
Press i to get vi into insert mode and paste the following. Note: Please replace the values with the path to your SSL Certificate we generated earlier.
##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# https://www.nginx.com/resources/wiki/start/
# https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
# https://wiki.debian.org/Nginx/DirectoryStructure
#
# In most cases, administrators will remove this file from sites-enabled/ and
# leave it as reference inside of sites-available where it will continue to be
# updated by the nginx packaging team.
#
# This file will automatically load configuration files provided by other
# applications, such as Drupal or WordPress. These applications will be made
# available underneath a path with that package name, such as /drupal8.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##
# Default server configuration
#
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name webmail.mydomain.com;
ssl_certificate /etc/letsencrypt/live/webmail.mydomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/webmail.mydomain.com/privkey.pem;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK';
ssl_prefer_server_ciphers on;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/letsencrypt/live/webmail.mydomain.com/chain.pem;
# SSL configuration
#
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
#
# Note: You should disable gzip for SSL traffic.
# See: https://bugs.debian.org/773332
#
# Read up on ssl_ciphers to ensure a secure configuration.
# See: https://bugs.debian.org/765782
#
# Self signed certs generated by the ssl-cert package
# Don't use them in a production server!
#
# include snippets/snakeoil.conf;
root /var/www/webmail.mydomain.com;
# Add index.php to the list if you are using PHP
index index.php index.html index.htm;
# Revoke access to sensitive files and directories
location ~ ^/(README|INSTALL|LICENSE|CHANGELOG|UPGRADING)$ {
deny all;
}
location ~ ^/(config|temp|bin|SQL|logs)/ {
deny all;
}
# pass PHP scripts to FastCGI server
#
location ~ \.php$ {
include snippets/fastcgi-php.conf;
#
# # With php-fpm (or other unix sockets):
fastcgi_pass unix:/run/php/php7.3-fpm.sock;
# # With php-cgi (or other tcp sockets):
# fastcgi_pass 127.0.0.1:9000;
}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
location ~ /\. {
deny all;
access_log off;
log_not_found off;
}
}
Press : and then type wq and press enter to write the changes to the file and quit in vi.
Last, we need to create a link of the virtual host file to /etc/nginx/sites-enabled. You will need to execute the following commands to create the link as well as restart nginx to apply the changes.
sudo ln -s /etc/nginx/sites-available/webmail.mydomain.com /etc/nginx/sites-enabled/webmail.mydomain.com
sudo service nginx restart
Run the Roundcube installer
At this point, if you navigate to https://webmail.mydomain.com/installer, you should see the Roundcube Webmail Installer page. You should see a series of items show OK, NOT AVAILABLE, or NOT OK. You will need to remediate any items that show NOT OK for Roundcube to successfully run.
In this installer, I primarily focused on Step 1 (Checking the environment) and Step 2 (Checking the database). Once both show OK (don't worry about if email is successful or fails (likely it is failing still), move the installer directory to your home drive to secure the environment (IT IS VERY DANGEROUS TO LEAVE THIS PAGE!!! DON'T SKIP THIS STEP).
sudo mv /var/www/webmail.mydomain.com/installer ~
Update Roundcube configuration
I couldn't get Roundcube to actually work during the installation with this setup until I manually specified a few items via the Roundcube configuration file. Within the /var/www/webmail.mydomain.com/config/config.inc.php file, ensure you have the following code snippets to allow Roundcube to properly authenticate to your mailserver.
sudo vi /var/www/webmail.mydomain.com/config/config.inc.php
Ensure you have the following code snippets (typically there is a section under // IMAP that has the config we can start with). To do so, press i to get vi into insert mode and paste the following.
Press : and then type wq and press enter to write the changes to the file and quit in vi.
Verify
At this point, you should be able to login to https://webmail.mydomain.com and send/receive email!
As with all technology, ensure you keep up-to-date with all the latest security patches to keep your environment stable and secure.
If you made it to this point, were able to successfully send/receive mail via Roundcube, pat yourself on the back and grab a fine beverage!
Troubleshooting
Here are some useful commands to help troubleshoot your deployment.
sudo postqueue -p can be used to check if any pending emails are queued.
sudo postmap -q mydomain.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf can be used to validate what domain names are accepted. You should receive the value of 1 if it exists.
sudo postmap -q [email protected] mysql:/etc/postfix/mysql-virtual-mailbox-users.cf will validate if a user account exists with the specified email address. You should receive the value of the email address of the user if it exists.
sudo postmap -q [email protected] mysql:/etc/postfix/mysql-virtual-mailbox-aliases.cf can be used to validate the alias of an email address. You should receive the email address of the user account if it does map back to another user.
tail -f /var/log/mail.log can be useful watching how emails are handled by postfix/dovecot to troubleshoot how messages are being handled
Installing NodeJS on a Raspberry Pi can be a bit tricky. Over the years, the ARM based processor has gone through several versions (ARMv6, ARMv7, and ARMv8), in which there are different flavors of NodeJS to each of these architectures.
Depending on the version you have, you will need to manually install NodeJS vs grabbing the packages via a traditional apt-get install nodejs.
Step 1: Validate what version of the ARM chipset you have
First let's find out what ARM version you have for your Raspberry Pi. To do that, execute the following command:
uname -m
You should receive something like: armv61
Step 2: Find the latest package to download from nodeJS's website
Navigate to https://nodejs.org/en/download/ and scroll down to the latest Linux Binaries for ARM that match your instance. Right click and copy the address to the instance that matches your processor's architecture. For example, if you saw armv61, you'd copy the download for ARMv6
Step 3: Download and install nodeJS
Within your SSH/console session on the Raspberry Pi, change to your local home directory and execute the following command (substituting in the URL you copied in the previous step in what's outlined in redbelow). For example:
cd ~
wget https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-armv6l.tar.xz
Next, extract the tarball (substituting in the name of the tarball you downloaded in the previous step) and change the directory to the extracted files
tar -xvf node-v8.11.3-linux-armv6l.tar.xz
cd node-v8.11.3-linux-armv6l
Next, remove a few files that aren't used and copy the files to /usr/local
Growing up it was always common to spin up a "LAMP" box to host a website. The typical setup was: Linux Apache MySQL PHP
Over the past few years, this model has slightly changed due to new open source technologies bringing new ideas to solve performance and licensing issues at massive scale. In this tutorial, we are going to look at setting up a LEMP box on Debian Stretch (9.1). Linux nginx [engine x] MariaDB PHP
Please note, MariaDB could easily be swapped out with MySQL in this tutorial, however many have opted to jump over to MariaDB as an open source alternative (actually designed by the original developers of MySQL) over fear Oracle may close source MySQL.
Installing Linux
This tutorial assumes you already have either a copy of Ubuntu 14+ or Debian 7+. This probably works on earlier versions as well, but I haven't tested them. On a side note, I typically don't install Linux builds with an interactive desktop environment, so grab yourself a copy of Putty and ssh in or open up Terminal if you have interactive access to the Desktop Environment. Before continuing, go ahead and update apt-get repos and upgrade any packages currently installed:
apt-get update && apt-get upgrade
Installing nginx
Grab a copy of nginx
apt-get install nginx
Installing MariaDB
Grab a copy of MariaDB
apt-get install mariadb-server
Installing PHP
In this case, I want to roll with PHP7. You can specify php5 or php7 depending on your application, but PHP7 has some great performance enhancements, so for new apps, I'd leverage it. The biggest thing here is to make sure you use the FastCGI Process Manager package. If you specify just php or php7, package manager will pull down apache2 as a dependency. That is not what we want in our LEMP stack.
apt-get install php7.3-fpm
Once installed, fire up your favorite text editor (it's ok if it's vi :)) and edit the default site for nginx
vi /etc/nginx/sites-enabled/default
Search for the comment # Add index.php to the list if you are using PHP and add index.php to the line below it. For example:
index index.html index.htm index.php index.nginx-debian.html;
Next, find the comment # pass PHP scripts to FastCGI server and change the block of code to the following to tell nginx to process .PHP files with FastCGI-PHP:
# pass PHP scripts to FastCGI server
#
location ~ \.php$ {
include snippets/fastcgi-php.conf;
#
# # With php-fpm (or other unix sockets):
fastcgi_pass unix:/var/run/php/php7.3-fpm.sock;
# # With php-cgi (or other tcp sockets):
# fastcgi_pass 127.0.0.1:9000;
}
Save the file. If using vi, you can do that by executing :wq
Next, reload the nginx service to pickup the new changes to our configuration:
service nginx reload
Test
At this point, we can create a php file to validate things are working well. Go ahead and create a newfile /var/www/html/info.php and add the following line:
<?php
phpinfo();
If you see a page listing the PHP version and the corresponding environment configuration, congratulations, you have finished setting up your new LEMP stack! 🙂
Here is a quick tutorial on how to add an ingress firewall rule on your CentOS 6 machine. In this example, we will be forwarding port 443 for HTTPS.
Open up terminal if you are on the GUI version of CentOS 6
Execute the following command
system-config-firewall-tui
Use your arrow keys to select Customize and hit enter
Use your arrow keys to select which service you would like to allow. Hit the spacebar to enable or disable the rule and then select Close once you have enabled/disabled the rules you wish.
In this case, I arrowed down to HTTPS and hit the spacebar.
Here is a quick tutorial on how to get VMware Tools up and running on a CentOS 6 Linux machine. Although the instructions are shown with the GUI, we'll use terminal so the guide works with both gui and non-gui based installs.
Mount the VM tools installer to your VM
Open up Terminal
Execute the following command (this will create a mount point for our CD drive)
mkdir /cdrom
Execute the following command (this will map the CD drive to our cdrom mount point)
mount /dev/cdrom /cdrom
Execute the following command to move to your temporary files folder
cd /tmp
Execute the following command to extract the VMware Tools tarball
tar -xvf /cdrom/VMwareTools (tab to autofill the rest of the package)
Execute the following command to run the VMware Tools installer
./vmware-tools-distrib/vmware-install.pl
Press Enter/Return through each of the questions, using their defaults
Verify VMtools is running by looking at the client status in vSphere