Figure 25 – Setting Up a Controller
I use an open-source smart home software, which means I can modify it with code and scripts written by others to enable extended functionality. I don’t have to program my smart thermostat for every hour of the day, every day of the year in order to program it for my peak rate structure. Instead I run an automation that detects when it is a weekend or holiday, and then enables a lower energy mode of operation. That is not something that even the most expensive smart thermostats can do out of the box, so that added-value is the promise of smart technology. Work in the industry long enough and you start to get frustrated with high-end electronics of any sort that are proprietary and don’t provide access to source code.
As far as the material needed, a computer is needed as well as a z-wave antenna. Somewhere near the beginning of hardware programming, the computer needs to be told where on its computer chip the Z-wave antenna is plugged into. So I recommend sticking to hardware solutions that everyone else is using, which makes troubleshooting much easier.
One computer popular amongst home automation users is a $40 computer called a Raspberry Pi. The Rasperry Pi is just hardware that software is installed onto, but because it opens its technical specifications, and the software installed on it is similarly open, it attracts significant attention from hobbyists who collectively contribute knowledge and code, often free of charge, to make the platform better for everyone. That doesn’t mean your Raspberry Pi isn’t password protected, but using open-source hardware and software is one solution to avoid reliance upon manufacturer support.
Even if building a more powerful dedicated computer to host the software, which would improve start-up speeds and add-on capability, sticking to similar computers as what everybody else is doing helps with answers that you might have to otherwise be very knowledgeable in computers to find.
Figure 26 – Open Source Automation Software
The software I use for my hub is open-source as well as free to download and use. Home Assistant is one of the top 10 open source projects on Github (the website for managing open-source projects). So it is a very popular platform and to get it set up, you literally take the home assistant software and install it onto the Raspberry Pi, starting out with a micro SD card as the boot disk.
After inserting the boot disk, the Raspberry Pi starts up. Plug in the Zwave USB anttenae – the RaspberryPi already has an hdmi port for a screen, as well as a wifi signal and ethernet port. Perhaps on a commercial system, a mini pc which lacks a wifi antennae is a more secure option, only using the ethernet port for communication. In fact much of this external stuff, whether it be a zwave antennae or boot disk or whatever gets integrated into the actual computer chip board when you go with higher end consumer smart hubs. And ultimately having the antennae of your smart home network on a USB drive becomes the failure point. The literal connection between the USB stick and the computer could become damaged or loose, causing performance issues.
But that comes later. Let’s now go through the process of building a local energy controller.
Figure 27 – Use The Official Documentation
We’re using the popular home assistant open source platform, which does not require internet access to run. Connect the Pi to the local wifi hub will allow for remote web browser viewing on any connected device function, and connection to the world wide web would allow for remote viewing remote as well as integration with voice services, but it doesn’t have to be integrated
it can run on its own without connection to the outside world.
All this stuff changes fast so don’t use online guides produced by other people, even when they are very good guides, because the software changes can make the guide obsolete so always start with the official documentation.
Figure 28 – Use Discord for Technical Support
One web resource that is a big help is the Home Assistant Discord channel. I moderate a solar Discord community, and I’ve found other Discord groups very helpful for technical support of different kinds of technology. If you like using Facebook for technical discussion groups, you will love Discord and there is more information in the references slide at the end of this class. So between the official documentation and the Discord group, you should be able to get this project out of the ground.
Figure 29 – Downloading Controller Operating System
One of the first steps is to download the home assistant software. There’s different options for various popular hardware, whether it’s the raspberry pi or slightly upgraded mini pcs. The software is downloaded onto a boot disk, something that computer professionals are familiar with. In your computer, there is dedicated disk drive space that is unlike other parts of your hard drive. It contains the primal code that tells a computer how to turn on as energy starts flowing through its circuits, in a process not dissimilar to a needle extracting sound from a record player, except with ones and zeros instead of bumps.
The boot disk for the Raspberry Pi is commonly stored on a data card that then gets plugged directly into the Raspberry Pi. Finding a microSD card as well as a computer that can read it is perhaps the hardest part of the whole process.
The software contained is Home Assistant software written for Raspberry Pi hardware. It might differ slightly for other computers. When it comes time to graduate from the Raspberry Pi to host Home Assistant on a computer, the installation can be slightly different.
Figure 30 – Making a Boot Disk
Anyway, computers are capable of making boot disks because it is a necessary feature for restarting the hardware when your operating system crashes. That same boot disk software can be used to “etch” the boot image of home automation software written for a particular hardware configuration onto any SD card, instead of using the computer to create a boot disk for its own operating system. So your computer may already have this etcher software or boot disk creation software already installed which you could use to create a boot disk for the Raspberry Pi. Or you can download the BalenaEtcher software to put the Home Assistant boot image on the SD data card.
There is an extra step, which is more difficult, to pre-load the wifi network information onto the Raspberry Pi. But it is easier to simply locate the Raspberry Pi wherever your internet router is located, and plug in with an ethernet cable.
Figure 31. – Startup Success
Upon start-up, the energy controller now asks to go through the user creation process. It’s connected to the internet and pulling in information from the outside world. One fascinating input is weather, as the weather forecast information becomes a digital automation input. It is possible to trigger energy automations based on the next day’s weather forecast, such that you could charge the batteries ahead of a storm, or otherwise aggressively discharge the batteries for normal electric bill optimization. All these really powerful concepts become relatively simple once you get the hang of the software interface.
Figure 32 – Adding Zwave
As a next step, I suggest registering the Zwave USB stick, which allows you to register Zwave devices into the Home Assistant hub. I do not know the USB path of the zwave stick, which refers to how the computer motherboard talks to the virtual registry of devices plugged into the controller’s USB ports. Because I have used commonly used hardware and software with a common configuration, I can cheat and find this information online without using any advanced computer science knowledge.
Figure 33 – Out-of-the-box Functionality
As an optional step, as I say optional because this is a complicated next step which is not necessary for energy automations, but to record this data onto the Raspberry Pi we need to add a data logger.
Right after an inclusion pairing process is successful, Home Assistant will take the data from a zwave energy meter and display it in a 24 hour view. But if you want the data to go back a few days, it is necessary to set up a data logger. Yes energy analytics are really good but it is important to note they are not necessary for energy automation. It is best to have server data stored on a cloud service, such that the device does not fail if the data storage accidently fills up the entire hard drive and degrades system performance. It is possible in fact to restrict the data storage on the local to device to just what is needed rather than to record everything.
In the solar industry, installers get very frustrated with the lack of system data locally available from their solar monitoring systems, but managing this data for every 5-10 second period for years, if not decades, ends up being a lot of data! Whereas a monitoring system without data logging could be a few lines of script.
A Raspberry pi makes a great home automation management system, but if you want to do robust data logging then taking extra steps to start with a mini pc may be in your best interest.
Figure 34 – Adding a database
Fortunately, the Home Assistant software allows this addition to be made within its software. Otherwise, Home Assistant would have to be installed on a computer, running in a container inside a larger computer. Regardless, to store the data the computer needs a database and a popular and therefore well supported one is influxdb. In the past, you would need to know how to run command lines to add the database onto the controller, but Home Assistant can now be easily modified within its own software, like a traditional operating system.
Anyway, storing lots of constantly updating information is not a good practice for an SD memory card, but if you proceed with these steps and do it for anyone on a Raspberry Pi energy controller, it might fill up the card and crash the system. Because there are so many inputs into a smart hub, recording everything possible is not the best approach. Instead, think about what you want to record and log only that. But basically setting up the database is a bunch of username and password creation.
Figure 35 – Visualizing the Database
This is a visual display software that is useful for displaying the database information. Both Influxdumb and Grafana are open source and can be installed directly within the smart hub. Both add-ons are installed and then programmed to talk to each other.
Figure 36 – Setting Up Communication
Here we’re setting up our database with a username and password. If it looks daunting remember this is a common step taken by many, and so the process is well supported if you were to ask for help within Home Assistant’s free to join Discord group.
Figure 37 – Beyond-the-Box Functionality
If you know what you are doing, the entire set up process can be done in under an hour, so makes a good side project. Even if you don’t know much about computers, this process is something you can do, although it may be a bit longer.
Figure 22 – Making a device smart
What makes a device smart? A thermostat is a great example. You can have dumb thermostats that that don’t do anything as well as thermostats on the wall that dial the temperature up and down. These dumb cheap thermostats don’t do much but are great for manual fiddling.
The user interface to a smart thermostat is provided by software, not hardware. This is an open-source thermostat interface which is similar to a nest, except that I still have a clunky programmable thermostat on the wall. But because it is a Z-wave thermostat I can find software for it that gives me the interface of a Nest thermostat, without the additional hardware cost. I can check on the thermostat through any device, cell phone or desktop, provided that smart hub is connected to a network. I can program with the thermostat. All of this advanced functionality does not come with the thermostat out of the box, it’s an added value that comes with connection to a smart hub.
Before getting started on a thermostat swap, be sure to power down not just the thermostat and outdoor unit, but the indoor unit as well. Failure to power down the system can blow a fuse in the indoor unit which will need replacement for the air conditioner to work. The fuse can be purchased at a local hardware store, but it is easily avoided by not rushing into the project.
Figure 23 – Thermostat Types
Programmable thermostats are hated by many. They’re clunky and difficult to program but with the right peak versus off-peak rate structure, programming the thermostat correctly can result in real savings on the electric bill. But not all thermostat adjustments are easy. A proprietary “smart” thermostat may not have all the functions needed for easy programming.The Nest may be programmable for different zones at different times of day, but a regular programmable thermostat can do that too. But is a Nest solar friendly? Does it increase the air conditioner when the electricity is otherwise being sold for cheap back to the grid? Can its programming account for holidays or just days of the week?
Figure 24 – Smart Thermostat Functions
A true smart device to me is one that can communicate with the other smart devices on your smart network, making it easy for this communication to result in computer-driven automations to do useful things. That doesn’t have to be a Nest. It can be a $40 clunky programmable thermostat so long as it has a Z-wave antennae.
After pairing a smart thermostat into the smart hub and enabling voice services, basic voice commands should work immediately. The smart hub provider often provides an automation menu for basic automation programming. But advanced programming features may require access to the software source code, so that you can program an energy automation such as to run the air conditioner harder when the solar array is being over-productive.
Figure 18 – Batteries vs. Hardwired
When I jumped into a smart home project, I was taken in by the nber of battery-powered, wireless devices. I thought hardwiring all the devices was best avoided. But it is not just a matter of battery replacement although battery replacement is a big issue. And not every single device will experience the need for swapping the batteries, typically it is the regularly used devices that conse more power. But here is an example of a sensor that is not just a motion sensor, but also a vibration sensor, as well as a humidity sensor. It’s also measuring how much light is in the room, which could drive an automation to dim the light according to natural sunlight.
That’s a lot of sensing stuff going on. The vibration sensor is an anti-theft device so is it needed in all cases? It might be better for energy consumption if a motion sensor is just a motion sensor. A multisensor may not be the right tool for the job, even if it only costs a little more than a more simple sensor. All those additional things take up more energy and that means you’re swapping out batteries more frequently.
Although, if you hardwire these sorts of devices, they’re great. Many battery-powered smart devices come with usb ports for hardwired power. Only detached sensors like window
or door sensors really need to be battery powered. Or perhaps there is some flexibility here, as the USB port would allow for a very large external battery pack if a sensor to reduce the need for frequent battery swaps. The user will simply have a better experience with hard wired powered devices.
Figure 19 – Zwave and Zigbee
Another problem with some wireless smart home devices, such as lower than wifi frequencies like Zwave or Zigbee, is that the wireless devices are mesh rather than point-to-point. This means the devices rely on neighboring devices to relay their messages throughout the network, but the important thing to know about that, is the repeater stations need to be hardwired.
You can’t have a 100% wireless zwave or zigbee network, some of the sensors need to be hard wired otherwise they won’t be able to relay their messages back to the hub. So if you jump into a smart home with a bunch of battery-powered, wireless devices you’re going to have a bad time. You need to throw at least some hardwired devices into the mix.
Non-wifi wireless platforms like Zwave and Zigbee aren’t necessary if you trust your wifi security system, and can have worse performance than their higher energy wifi counterparts. But ultimately I choose Zwave or Zigbee devices when I need a generic product that can be integrated into a smart home system without dragging me into a cloud service provider. So at the very least, these non-wifi platforms make a good starting point for a smart home build.
Figure 20 – Wifi vs. Zwave vs Zigbee
Wi-fi operates at a 2400 megahertz or 2.4 gigahertz frequency, with new wifi going in at 5Ghz. Both Zwave and Zigbee are around 900 Megahertz. What does that really mean? Those old enough to remember hard-wired telephones as they transitioned to wireless corded handsets (before cell phones) might remember first generation product that didn’t have 20 or 30 feet of
range, replaced by a better phone later that had substantially more range allowing you to walk anywhere in the house. Z-wave and zigbee use that same frequency range.
So it’s not a new frequency and it’s not necessarily a better frequency than wifi. Wifi uses a higher frequency, which requires more energy for the same range. But wifi has a longer range than Z Wave overall, even if it means higher energy use. Maybe not all devices need that long of a range, but many do. Wifi can transmit more data, whereas Zwave tops out at around a 56k modem. Transmitting voice over Zwave would take up so much bandwidth it would not allow for any other Zwave devices to be on the network. Even though Zwave networks can hold about 200 devices at once, how chatty the devices are matters as do other considerations.
Real time energy metering requires a lot of data. In the energy controller section to come, I select a Zwave consumption monitor, but I am using a small Zwave network. A commercial facility might opt for a wifi monitor instead. Alternatively, another common solution would be to run multiple Zwave networks. Part of the thrill and frustration of smart home design is like computer programming, there are many solutions to the design.
I don’t think the technical argents of Zwave or Zigbee are enough to justify the technology over wifi. Having fine-tuned sensor settings is more important for energy consumption, and while these settings are adjustable in the open-source software I use, you still have to figure them out and then set them up for each device custom. Later we’ll talk about a security hub that even has a z-wave or zigbee antenna for external device compatibility, but the company’s own sensors are on the 2.4 gigahertz wifi frequency, so what does that tell you about their decision as to which is better?
Z-wave or zigbee make a good starting point, and there aren’t many wifi solutions that don’t draw you into a cloud service, so again, Z-wave and Zigbee make popular smart home starting points and then ultimately a large project begins to look for wifi solutions for a more stable, less limited communication solution.
Energy controls also make a good starting point, because smart technology can save the end user money. National Renewable Energy Labs states that the majority of the population will embrace energy efficiency products which yield a 1-2 year payback. Unlike solar, an energy controller will never eliminate 100% of an electric bill. But in a similar fashion to batteries, it can actively shift a portion of the electrical load from peak to off-peak use, for substantially less cost than batteries.
With an energy controller, the energy data is not simply monitored, but also acted upon by turning the right devices on and off. If everything necessary to do so is on a z-wave
Network, then energy controls can be implemented without requiring access to custom wifi.
This means the systems can be pre-programmed and tested offsite before installation, reducing the technical difficulty in the field. Certain customers may not allow smart devices to access their networks, yet nonetheless have a need for onsite energy controls.
Figure 21 – Wireless Frequency Comparisons
Z-wave or zigbee systems represent less of a threat to a corporate network, especially if they do not have wifi capability. Without the right antennae, it makes it impossible for one smart wireless device to hack the other. Likewise hackers are less interested in these systems as there is less valuable information – wifi devices make for bigger fish.
But although Z-wave or Zigbee can have encrypted communication, that requires more data bandwidth so a large encrypted z-wave or zigbee network is not feasible. Certain devices, such as door locks, require encryption. But encrypted pathways can only be relayed through encrypted relay stations. So for a door lock, is it more secure to be Zwave versus wifi? The answer might depend on user priorities, but I would lean towards higher end wifi. I would only recommend a Zwave door lock when the lock is located within the same room as the hub, so at the very least Zwave door locks may not be well suited for projects where more than one lock is required.
So even though Zwave may have encryption, that doesn’t necessarily mean you are going to use it. Wifi devices are simply more capable than Zwave at crunching the nbers through a data network. A better product may be one that accepts this fact and makes up for the increased energy use with a larger battery. Again, the main reasons I select Zwave or Zigbee devices is that I know the devices will be smart and not attach the system to yet another cloud service.
Figure 12 – Light Switch Locations
But as to the devices themselves, there’s no reason to over do it while getting started. It’s nice to think about living in the Jetson’s home, but National electric code does require at least one switch controlling the light on the wall, and as a practical matter if the smart home network crashes, you want a manual override.
While a smart home can add a wall switch for a light in a convenient place, to make up for an inconvenient place, it’s a little more complicated than meets the eye, so I do not recommend envisioning too many fundamental changes in the way we wire our homes. Light switches will still be on the walls, but perhaps we have fewer of them. Maybe the best solution is to have all the light switches in a room in one central location, as a manual override. With enough box space behind them for any kind of future lighting controller. But even if we swap out most of the light switches in the house with wifi-talking smart buttons, those smart buttons will still benefit from a hardwired power supply. That brings us back to what most would consider obvious, that one way or another we will still have hardwired power in our homes.
Figure 13 – Traveler Circuits
Dumb light switches, in cases where more than one switch controls a single light bulb, are like a double-side railroad line with only one train on the track. The light switch alternates from on to off, and that switches the side of the track (called traveler wires) that the electricity is on, down to the device itself. It is a literal, manual on-off switch.
Figure 14 – Behind-the-switch Lighting Wiring
A smart switch is fundamentally different in that the power goes into the controller and then the power out to the device, skipping over the actual toggle switch on the wall. Not all switches physically regulate actual electricity, some are low voltage relays that trigger an external switch. When retrofitting an existing lighting circuit with smart switches, the power lines are intercepted by the control board, and the light switch itself is rewired as a low voltage relay. Rewiring the lighting circuit is always possible, but the installer might have to cannibalize the traveler wires to use as low voltage relay circuits and it’s not a simple process.
Figure 15 – Different Lighting Styles
While I have found these behind-the-box controllers to be better than the switch replacements, I have also found them too complex to install and recommend starting with a more basic switch replacement, where the controller is built into the switch itself.
We don’t have time to talk about all those nuances in two hours, but the basics is that the actual power output is routed from the controller into the device, with the switch rewired as a low voltage relay. The electricity is no longer coming from the switch itself, but the controller is in fact hardwired to the building power supply.
Figure 16 – Qualified Persons
There is potential for safety improvements in this kind of design, but it is also work that an electrician really should be performing, if the building owner is not comfortable self-performing that kind of work. An electrician is better at doing cable management and placement than a do-it-yourselfer. The controllers are allowed to go inside the lighting junction box, but only if they take up less than 75 percent of the cross-sectional area of the box. On a finer note, most controllers have reset buttons which need to be pressed if the system or device fails. When I install these controllers, I make sure I can access that push button with a push pin of some sort, to prevent having to undo a whole bunch of wiring to access that push button on the controller.
Although the light switch replacements I’ve used have not had reset buttons of any kind, but instead require flicking the switch up and down rapidly to initiate a reset mode, and I find that method much less elegant despite the easier access.
Another option is to have a smart light bulb, where the controller is built into the light bulb itself
Rather than a smart light switch. That’s really interesting because it doesn’t require electricians, and keeps the building wires in place. Smart light bulbs are expensive, but by the time you factor in labor they’re actually worth it.
The major downside to a smart light switch is that like the controller, they need to have an always on power supply to work. So if the dumb light switch kills the power to the light bulb, then lighting automations would no longer work until the analogy switch is changed. The accessible switch on the wall then becomes a hindrance – smart owners resort to taping their lights on to prevent guests from ruining the automation, which reduces the usability of the home. It’s not an easy problem to solve, although smart light bulbs often have more features such as color changing and dimming, some of which can not be delivered by a controller that controls a regular dumb light bulb.
Figure 17 – Adding Push Buttons
I don’t really know which solution is optimal, maybe in the future all the lights in the room will be on one manual override panel, similar to a hotel conference room, with motion detectors and voice commands making up the difference. There are wireless touch plates you can get for the walls, but also, getting one manufacturer’s wireless touch plate to talk to another manufacturer’s lighting controller is not an easy task. At any rate, there are more simple solutions to implement today and we could get stuck trying to figure out the optimal wiring scheme of the future. Swapping out a few lightbulbs or wall switches and getting everything set up on voice commands is a pretty natural fit as a starting point and there are safe ways to do so without being an electrician, although other devices really do fall into the category of electrical work.
Cloud services are a popular model, because the manufacturer can deliver a consumer grade product at a relatively low price, in order to sell you subscription services. This isn’t all nefarious – there are some real time saving advantages to that business model. The convenience of buying a cloud service versus setting up a more technically difficult local service, or the stability or out-of-the-box functionality of a company with profit motive can result in a more finished product than when you build your own smart home network .
But sometimes for smart technology to do what you want it to do, you have to understand the nuts and bolts, so you can take these different parts and put them together into a system that is greater than the some of its parts. This may not mean a local system is more stable than a cloud service, but it can eliminate the need for an internet connection or reliance upon the manufacturer for support.
There are users who embrace the cloud and focus on internet security and that is a perfectly reasonable path to take, but there is also a smaller but healthy contingent of residential and commercial users who under no circumstance want smart devices and their companies spying on the network. While sometimes the local hardware can be cheaper, by the time you put in the programming, the local device will cost a little more and might be even a little less functional than its cloud counterparts.
But there aren’t too many products out there for energy management, and to some extent it is a project that needs to be designed and built custom to the particular jobsite, so it can make sense to build a local energy controller.
Now, a local device can still use wifi for wireless local communication. That can be a completely isolated wifi network on its own hub or antennae, or it can share the existing wifi, or most high end wifi routers can issue a separate virtual network, called a VLAN, for the smart wifi devices. We don’t have time today to talk in depth about network security issues, but it all comes down to password management and inevitably some user is going to want to connect to both the smart home network and the normal building internet and potentially bridge the two networks together. These points of connection can even be intentional – dedicated security gateways can be installed to monitor and control these points where the literal internet cable comes into the building. Well, we will have to talk about that further in another time and place.
But even Amazon and Google are working on getting their devices to store enough data on them locally to enable offline functionality, which means that some cloud devices do have local functionality. That’s not that’s not uniform across the board. But how often do you actually lose the internet anyway?
Whereas backup power management begs for local communication. The internet might have failed because of bad weather. Adding a UPS system to the energy manager is not a bad idea. We’ll get to that later. For now, let’s simply agree there are a few reasons why an energy controller should be a local device.
Turning your home into a living computer creates opportunities for hacking. A cloud connection can be hacked on the other end, which cedes up some user privacy inherently. But to some extent cloud services are inevitable. Voice command services are in demand – these can be local, but the local voice command services are being bought up by smart product developers, such as Sonos with Snips. Stanford has an open-access voice command service called Almond that is privacy oriented but it still requires more computing power than what most tiny computers are capable of running on their own. A local device might be capable of running a few voice commands, but inevitably the smart intelligence is going to be outsourced to a heftier machine. If you want a $40 thermostat to have voice features, it’s going to use a cloud service.
Voice commands are valuable – they promote health safety and welfare. They also promote laziness and comfort. You can get into bed and then turn the lights off rather than actually flicking the switch on the wall. Which to some might be ridiculous but for those of us who can’t keep the TV remote in the same place, being able to talk to your television to turn it on and off is quite wonderful. Voice commands are convenient and then from a disability standpoint voice commands can be liberating. One emerging smart application is enabling the elderly to live in their homes longer, because computers can automate some of the essential tasks such as locks, keys, and even speaking.
Getting voice commands onto devices that don’t have them already built in is what the “internet of things” technology is supposed to do, and it is actually easy to do. You don’t need to do much programming. Basically as long as the cloud service is compatible with the smart hub, then it is a matter of password management. Permission is given both within the cloud service software and the smart hub software, after which google or amazon or whomever sucks in all the smart devices from the hub into their platform (or vice versa), and the end result is the basic commands issued from the hub, like turning on or off a lightswitch or television, now become voice activated commands within Alexa or Google. It almost feels like magic.
One of the cutting edge features of smart homes is facial recognition. People can be disappointed by facial recognition because it requires so much computing processing power. It’s not an instant response that people expect with their sensors, and so facial recognition is likely going to be a cloud service. What I’m saying is that for those comfortable with cameras inside their homes, where facial recognition can be used for really advanced automations based on who is in the room, keeping the smart network local without any kind of cloud interaction is not be practical. And while a simple local smart home service that is custom built doesn’t need to have privacy issues oriented with cloud services, a local service can still benefit from online interconnectivity whether it’s bringing in voice services or facial recognition or even just remote monitoring if you want to check in on your smart home when you’re away from the house. In most circumstances, you’re going to want an internet connection even with a local network. It’s hard to avoid.
[End of Transcript… for now… stay tuned]