i see bots
In the future, you won't think this is so weird
Category Archives: Home Automation
Net Neutrality and the Connected Home? (updated)
Buried in last week’s net neutrality proposal by the FCC was an interesting tidbit about the connected home.
As you may know, the controversial proposal put forth a requirement that all broadband providers allow their customers to access all (legal) online content. You are probably thinking, “I thought they were already required to do that??”
Actually, no. In April of this year, the FCC lost an important court ruling when the United States Court of Appeals for the District of Columbia ruled that the FCC lacked the authority to enforce the notion that providers could not discriminate against content services carried over their pipes. This case arose out of a finding the FCC issued in 2008 in which it ruled that Comcast had secretely blocked or slowed (“shaped”) traffic associated with the BitTorrent Service.
In May, the FCC chairman, Julius Genachowski, indicated in response that he was considering regulating broadband providers using the same regulatory powers the agency uses to regulate telephone networks. The difference is that broadband providers have been regulated as “information services”, not as “telecommunications services”; the latter brings with it much more scrutiny and regulation. This was a controversal proposal; service providers claimed that such a level of regulation would stifle the huge investments required to maintain and expand the country’s broadband networks.
However, in last week’s proposal, Genachowski did not take that significant step, but he did set forth a basic “no blocking” requirement while allowing providers to “shape” traffic as necessary to ensure network health, as long as providers provided notice of their practices. Wireless providers would be given more leeway, given the more limited nature of their networks. There was also a discussion about usage-based pricing, although this is less relevant, given that many major wired and wireless providers have implemented tiered or capped service levels.
(Personally, I found all of this ironic, given that in my neighborhood in Seattle, the best that the local telephone monopoly can offer me is dial-up networking. Comcast’s is the only wired broadband service available to me. So if I find that I don’t like Comcast’s price tiers or “shaping” practices, I really don’t have a choice. I suppose I could turn to 4G wireless services, but from what I can tell, those are still in the infancy stage, combining three or more of these unfortunate characteristics: expensive, relatively slow, limited to fairly low monthly data transfer ceilings, limited to certain geographic areas, and unreliable.)
I found these articles to be excellent summaries of the proposal: PC Magazine and New York Times.
I noticed the following statement in several of the articles on this topic:
“In addition, the proposal would let broadband providers experiment with routing traffic from specialized services such as smart energy grids and home security systems over dedicated networks, as long as the practice doesn’t slow down the public Internet.” (Yahoo!)
“The F.C.C. also will allow companies to experiment with the offering of so-called specialized services, providing separate highways outside the public Internet for specific uses like medical services or home security.” (New York Times)
What does this mean, I wonder? When I think about cloud-based home automation (see “A PIVOT TOWARDS THE CLOUD?” section in this post), or IP-based remote security monitoring, what I worry about isn’t latency but rather reliability. I wonder, then… will Comcast be offering me, say, some kind of uptime guarantee for a premium in their broadband offerings of the future?
What kind of broadband connectivity availability guarantee would be acceptable to me? Let’s say that they offer “three 9s” of availability, or 99.9%, which translates to just under 9 hours of downtime in a given year. (See a handy chart here)
“Three 9s” is the SLA (Service Level Agreement) that Amazon offers for its AWS offerings. If Amazon fails to meet that SLA, the customer is compensated in the form of a credit that appears on a future bill.
Would 9 hours of downtime a year be acceptable for a home automation system? I’d say that if those 9*3600=32,400 seconds of downtime were spread out uniformly across the year (say, 89 seconds each night between the hours of 2am and 3am!), I wouldn’t mind. But if all of those seconds of downtime hit all at once, it would be unacceptable.
To be fair, my current home PC-based home automation system has been down for hours at a time, usually planned but not always. A goal in moving to the cloud (all other things being equal) would be to leverage the actual professionals employed there to keep redundant systems healthy.
What about 99.99% availability? That’s just under a hour (53 minutes) a year. Much better, and probably even acceptable, but from what I can tell, only a few hosters offer that level of service, much less broadband providers.
As far as I know, the only services providers who venture into the rarefied air of “five 9s” - 99.999% - are the POTS guys - the telephone company. The infrastructure investment is impressive. I don’t think that anyone else, outside the corporate data center, bothers, though. Cloud-based architectures are designed to deal with 99.9%.
The possibility of significant disruptions in connectivity is the biggest objection as I think about the characteristics - positive and negative - of a cloud-based home automation or security monitoring service in the cloud. If there’s a possibility that the broadband connection could go down for hours at a time, the functions you’d offload to the cloud would end up being trivial or secondary at best.
Given that it’s unlikely that you could find a 99.99% availability SLA from a broadband provider, and even if you did, it would be very expensive, you might consider that continued thinking about a cloud-based approach for home automation would be a waste of time.
But there is another approach: consider implementing a “diverse” secondary broadband connection. For instance, if your main provider is Comcast, then you’d have a secondary emergency connection via, say, a 3G or 4G wireless provider (supplied by a provider other than Comcast). You would, in effect, be banking on the low probability that both providers would fail at the same time, and paying for that level of availability in the form of a second (wireless) connection that would rarely see service (and which, therefore, you hope would be inexpensive). Your connection to the cloud would be via a fast main pipe, backed up by a (perhaps cheaper) and slimmer backup pipe.
It’s probably too simplistic to compute the combined “SLA” of this primary/secondary connection architecture, but let’s do it anyway. Let’s assume that your primary provider is Comcast… I have anecdotal evidence that they don’t deliver at the 99.9% level… let’s call it it 99.7% (26 hours). Let’s assume that the wireless connection is slightly more reliably, given its roots as a telephone service - say, 99.99%. Therefore, the probability that both services would fail at the same time is something like: (26 hours/8766 hours in a year) * (1 hour/8766 hours in a year) = .3% * .01 % = .003%. Are those acceptable odds?
Let’s assume that those are indeed reasonable odds. So if you’re happy about your bandwidth connection being reliable, what about the service you’re connecting to? If it’s hosted at Amazon AWS, then we’re back to the 99.9% level. But perhaps there are things we can do about that. More on this later.
(There are other critical objections to the concept of a home automation system in the cloud… I’ve found this thread on AVSForum to be interesting for this reason. The signal-to-noise ratio varies greatly over the 10 or so pages of this thread, but I found very interesting comments from a number of folks on that thread… but your mileage may vary.)
UPDATE #1, 6 December 2010: In the original version of this post, I said that “what I worry about isn’t latency but rather reliability”. Well, I found myself thinking about that statement a bit more, and decided to run some simple tests to figure out if that statement was warranted.
So I ran some very simple tests: I fired up an Amazon EC2 instance, telneted to it, and ran “ping” from there to my home gateway (a Linksys appliance), courtesy of Comcast’s home broadband offering (3 mb/s upload, I believe). I used a dynamic DNS service to resolve the address of my gateway. These are the results, via ping:
Amazon EC2 instance running in “us-east-1d” availability zone: ~92 ms
Amazon EC2 instance running in “us-east-1a” availability zone: ~99 ms
This compares with something like 2- to 2.9 ms ping times within my LAN (a PC on my LAN pinging my gateway).
So… the cloud is ~ 30 times ‘farther away’ as compared to my home’s LAN. Is this an issue?
Here’s a thought experiment: Let’s imagine that I’ve got Home Automation functions running on an EC2 instance. Let’s further imagine that a human in my house flips on a switch which has been automated in some way. How long would it take a simple Home Automation gateway on my LAN to notice that state change, communicate that fact to the EC2 instance, and send back a response? It would appear that cloud latency alone would require ~200ms alone for a round-trip. My very informal tests with Z-Wave (using HomeSeer’s “Z-Seer” utility) indicate that communications with a Z-Wave switch takes a small number of milliseconds. And if I assume that the code running in the EC2 instance can turn around a response in a small numver of milliseconds, then it would appear that the cloud latency times will dominate this scenario. Perhaps this cloud-based scenario could execute, end-to-end, in under, say, 300 ms. Would this be “interactive” enough for human-scale Home Automation? I don’t know. It might be.
I think the next test would be to actually code up a simple scenario that actually runs in the cloud and see how well it works… if anyone out there reading this has any ideas about this, would love to hear it…
UPDATE #2, 6 December 2010: Interestingly enough, saw these posts today:
- “Get Ready for $99 Security, Home Automation from Comcast/Xfinity” (link)
- “AT&T Acquires Xanboo, Developer of Home Automation Platform” (link)
100 Devices
Imagine that at some point in the not-too-distant future, you’re the owner of a ‘smart’ house, which, you’re told, contains 100 smart devices. That’s a lot, you think to yourself. What are they all doing?? Here’s an plausible inventory:
- Security devices: one device per window, to detect open/close/breakage, and a number of motion sensors, for a total of 30 devices
- Survellance cameras: one for each entrance, one for each side of the house, for a total of 6 devices
- Thermostats: one for each room, for a total of 10
- Smart light switches: one for each room and hallway, and several more for outside lights, the garage, etc, for a total of 25
- A/V and Control wall panels for most rooms, for a total of 5
- Devices representing main controllers for automation, security, A/V components, VOIP, personal computers, file servers, routers, access points, and broadband modems, for a total of 10
The above adds up to 86 (see Note 1), and while I’m writing this post, I’m sure I’ll think up ideas for 14 more. (see Note 2)
And I didn’t include devices that get around, such as:
- Mobile phones, which aren’t tethered to the house but may spend a good portion of their time in the house.
- Portable music players (iPods, etc) and other portable personal devices
- Your cars or trucks (I’m not sure that it counts as “news” that the new Chevy volt has an IP address, but it’s interesting that it’s part of the headline), and significant components in/on your vehicle, such as: the tire pressure monitoring system (but be careful!), the A/V system, etc)
- Stuff you might wear, such as a smart watch or telemetry-sending exercise shoes
If you ponder this future for a moment, you might arrive at these conclusions and observations:
The number of devices that we’ll rely on, for a wide range of ‘personal scenarios’, will exceed our ability to directly manage them. We’ll know they’re there, working on our behalf, but we’ll likely forget the details about how to manage or configure them outright, reminded of the need only when one stops working or it needs some maintenance (see note 3), or you need it to do something out of the ordinary.
And even if you did try to individually manage a device, it’s likely that you’ll do it remotely, via a web page or specialized application: the device itself will be too small to support direct manipulation (lacking, say, a display and buttons), or the range of options and configurations will be too complex to adequately manage via the simple display and buttons that are on the device (example: thermostats), or the device is unreachable because it’s physically embedded into the house (such as wireless security sensors for windows).
For these reasons, I think we’ll see an architecture where the devices are proxied by a device portal or manager which aggregates basic information such as state and health, enabling a user experience that supports views based on filters over state, alarms triggered on state or health, etc.
The device manager will know, for some classes of devices, such as those which are TCP/IP-enabled - how to directly interrogate them directly or how to subscribe to updates. Other kinds of devices, such as those which aren’t TCP/IP-enabled, may require an intermediate hub or subsystem to broker the communication between the device manager and the devices themselves, such as with a security system (where the window sensors may be simple switches) or a 1-Wire bus, which requires a hub to communicate with.
It may also embody some basic configuration/management, such as “reset”, and then link off to a device-specific management page, served up by the device itself or by a type-specific device hub.
The device manager will represent the devices to the outside world, at least for read operations, isolating them from frequent requests for updates (early Proliphix documentation recommended restricting the API request rate to “a few requests per minute”). You’d want the manager to offer an RSS feed, not the devices themselves. The device manager could also implement some level of security/access permissions.
It’s more than a simply proxy, however… I’d expect it to also know the assigned name, location and type/intended purpose of each device, and provide views/filters around that: “Show me the state of all of the security devices on the second floor”. This implies a device registry or directory. My guess is that you wouldn’t expect or want most devices to handle this metadata on their own.
Finally, you’d also expect that the device manager would implement some level of APIs and scripting, for sophisticated eventing and notifications. Thinking this through, it’s clear that the device manager would need to implement ‘psuedo variables’ representing it’s view of the current state of the devices it’s managing. And, furthermore, what’s just been described here could be approximated by any of several Home Automation systems… depending on the approach, it wouldn’t necessarily be as elegant as you’d like, but it could be done.
USE VS CONFIGURE
In this discussion so far, I’ve not made too much of a distinction between the kinds of access that you’ll need to your devices. Sometimes you want to “configure” a device, and sometimes you want to “use” it. For a thermostat, “configuration” means things like: setting up the kind of heater (single-stage / double-stage, heater only or heater/AC, etc), the setback schedule, and so on. “Using” the device is typically a simple affair: override the current setpoint, for a specified period of time.
You’d generally expect (but I’m not sure I can prove it) that the number of “configuration” options to be at least equal to, or greater than, the number of “use” options, especially as the level of capability of the device increases: more functionality implies more state, and possible actions based on that state, which implies more configuration.
You could imagine entirely separate paths for “configuration” and “use”; in fact, in many cases, it may be very desirable to ensure this. Adhering to the “simpler is better” maxim, you’d want to keep the UI for “use” as simple as possible, and keep “configuration” UI, which might require different paradigms for efficient management, separate.
REMOTE UI
With the Proliphix Thermostats I use, you “configure” via embedded web pages. The configuration experience, delivered via browser, is fairly sophisticated (for a thermostat!). On the other hand, the on-device display and associated 5 buttons are focused almost completely on what the end-user needs: what’s the current temperature, what’s the current setpoint, and how do I get some heat? Some of the UI is dedicated to read-only access to basic configuration info (such as the current IP address), but that info is there to help the user provide good hints to the maintenance team.
In any case, it would be very hard to imagine an on-device user experience for, say, specifying setback schedules for specific days of the month or holidays using the on-device display and buttons. As the designer for the device, you wouldn’t even try. Being able to “express” yourself - as the designer of the “configuration” experience - via web pages means you can expose a lot of useful features in a natural way.
The inverse is probably true, too: don’t build it in the hardware if it can’t be configured by the admin. The embedded web server, while adding $20-$40 to the build cost of the device, means you can offer a lot more functionality and charge for it.
(An aside: what about the “blinking 12:00 AM” VCR clock problem of years past? What this solely due to poor UI design? Or should we just blame the user? Later-model VCRs learned to pick off a time signal from the TV signal, which apparently eliminated the problem for most folks. But if you had this UI challenge in front of you today - not just getting the user to set the correct local time, but also, say, the task of programming the VCR to record a program - would you push it off to a web-based experience, even if it increased the hardware costs of the device by, say, $50?)
So, even though the device in this case is relatively small, the configuration experience is ‘outsized’. Even as devices decrease in size while increasing in capability, they could offer an ‘outsized’ user experience.
If the device (or its hub) supports APIs, you could imagine more than one flavor of UI, in support of various scenarios: beyond the built-in “configuration” and “use” UIs, multiple other “use” UIs could be supported, via APIs, in the form of other web experiences or apps. A device manufacturer might leave the heavy-lifting - the implementation of an elegant end-user UI, for instance - to third-party developers who specialize in that sort of thing.
A colleague of mine imagines a “Facebook for Devices”, a kind of third-party portal where you can see recent updates from your devices and others of interest - and casually pivot on the data in various ways, potentially offering you an easier way to keep track of your 100 devices.
These third-party or extended experiences don’t have to be limited to web sites; imagine a display-oriented device with the sole job of displaying RSS or Twitter-like feeds from multitudes of other devices.
A PIVOT TOWARDS THE CLOUD?
If all of the devices you care about are under one roof, then you could imagine using dedicated device to act as the device manager for all of your devices. It could be implemented via a low-end PC or embedded PC, running headless, with a connection for the local area network, and perhaps dedicated I/O connections for specialized types of devices (such as 1-Wire). You’d access its web page via your LAN. Its device directory, and device data archive, could be based on permanent storage on the device itself or elsewhere on your LAN.
An alternative approach might be for the device manager based in the cloud. You’d likely still need a local agent to connect your home devices, likely stuck behind a firewall/NAT, through to the cloud, and, of course, to ensure that non-TCP/IP-based devices are proxied adequately to the LAN and then on to the cloud.
The resource requirements for a cloud-based manager are likely to be small (just as those of a home-based manager would be), perhaps modest enough that you could use the recently-announced AWS “Free Usage Tier“, which offers a non-trivial level of resources at no cost.
Going to the cloud might mean improved reliability… if your home-based device manager falls over, it’s up to you to detect it, diagnose it, fix it, and get everything running again, while, say, in the AWS cloud, if a disk dies, you’re never supposed to notice it. They have professionals on the case.
Accessing your cloud-based device manager - the UX or the APIs - may be more performant than a home-based manager, especially if you’re outside your LAN, since it’s likely that the cloud-based manager has better peering and connectivity than the slow-ish uplink of your consumer-grade broadband connection. And if it’s possible that your device manager might experience multiple simultaneous requests on a regular basis, putting in the cloud may yield better results.
A cloud-based approach also feels more elegant when the scope of devices goes beyond “the home”. If you include your mobile devices, or devices from your business or other organizations (such as local weather stations or similar services representing virtual devices that you use for more sophisticated eventing, etc), having a cloud-based device agent will make all of this easier, for the reasons already listed: reliability, bandwidth (lower latency, higher thruput, more simultaneous connections), as well as: the availability of an infinite amount of CPU and disk resources, as well as services or capabilities that may be too complicated to contemplate for home hosting. (My same colleague suggests that in-home cameras could serve up images or video that is shipped to the cloud for sophisticated vision processing that couldn’t be done either by the camera or by any software/hardware that’s likely to be installed in the home.)
Of course, a possible downside is that if your home’s broadband connection fails, the Cloud-based portal may miss device updates and therefore may not fire important events. Perhaps a hybrid model is in order, where critical events are handled locally, with the cloud portal handling most other events.
This suggests a future architecture where the cloud is the central management point, with local support - in the form of hardware/software - in the home to handle the vagaries of getting all of your devices connected to the cloud. Again, there are legitimate concerns regarding loss of connectivity, and perhaps security.
Moving some or all of the management of a device mesh to the Cloud is an interesting enough scenario that I’ll be trying it out as soon as I can. More on this later.
A PIVOT TOWARDS “SERVICE AVATARS”?
OK, so we’re now used to the idea that our devices are talking to the cloud, updating their Facebook or Twitter status. In the discussion above, it was the devices that connected up to the cloud, in pursuit of new scenarios that made the device more valuable. But what if the intention involves going the other way? What if cloud services extended down to the devices, in service of new scenarios?
Mike Kuniavsky in his book, “Smart Things: Ubiquitous Computing User Experience Design” (Amazon) describes “Service Avatars”, a term he coined that conveys the added value a focused-purpose device can bring to a service. He holds up the Apple iPod as a prime example of a Service Avator… while it wasn’t the first music player, it was the first to successfully deliver a service-based scenario - a cloud-based music store - down to a device. The device was cool, but it became so much more important when connected to its music service. (We can safely elide certain details, such as the fact that for full functionality, the device requires an intermediary “hub”, in the form of a Mac or PC running the iTunes application).
Notes
#1: I just checked my router. It’s managing DHCP addresses for 15 devices (PCs, fileservers, phones, a VOIP adapter, access point, A/V components, weather station, and a printer). I’ve assigned fixed addresses for another 11 devices (thermostats, Ha7NET, camera server, security system, etc). That adds up to 26 IP addresses, and 26 corresponding devices. If you throw in the other devices of varying smartness…
- 1-Wire temperature sensors (5)
- Security sensors (10 or so)
- Z-wave light swithces (10)
… as of now, I’m up to at least 51 devices. So 100 devices isn’t too far a stretch for a “smart home of the future”.
#2: Wait! Here are 20 more “smart home of the future” devices:
- Connected exercise equipment (1 per house)
- Roving security or health robots (1 per house)
- Connected appliances, which report energy usage, low supplies, or general health statistics (oven, microwave, refrigerator, dishwasher, washer, dryer, furnace, hot water heater) (10 per house)
- Resource-monitoring equipment, such as: water metering, electricity metering, natural gas or oil metering (oil tanks that report when they’re nearly empty, or leaking!) (3 per house)
- Smart sprinklers that only water when absolutely necessary, and never after a rainfall, and only when given the OK by the local water board, augmented up by soil moisture sensors (5 per house),
#3: I’m writing a portion of this post on the eve of that date in the Fall in the US where we “Fall Behind” with our clocks. How many devices in the future home will need resetting every Spring and Fall? You’d hope the answer is “zero”… I take a small perverse pleasure in noting, with each passing Fall and Spring, whether the number of ‘things’ around the house that need to be manually pushed back or forward an house, is decreasing.
1-Wire
“1-Wire” is the name for a device bus protocol designed to be inexpensive, easy to connect, and easy to interface with.
The name “1-Wire” is a marketing take on the fact that data is conducted over a single wire. You still need a ground connection, but “2-Wire” presumably didn’t sound as interesting. In this configuration, devices on the bus operate in “parasitic” mode, storing power accumulated during bus transitions. In reality, I found it more reliable to operate in “3-Wire” mode, where you send some power along a third wire. The specification was originally introduced by Dallas Semiconductor, which was subsumed into Maxim via acquisition.
See Maxim’s list of devices and bus infrastructure. The DS18S20 temperature sensors I’m most familiar with look like your average transistor (image from here):
A given implementation of a 1-Wire bus, which Maxim refers to as a “microlan”, is composed of a bus controller or “master”, the physical wire segments of the network itself, and the “slave” 1-Wire devices attached to those segments. The bus controller can be: a dedicated hardware device, which connects to the 1-Wire bus on end and, say, an RS-232 port on the other, and implements the requisite bus protocol. Or, you can directly connect the bus to a PC’s serial port via a simple circuit, and work the bus in software on the PC.
In theory, any number of 1-Wire devices can be attached to a 1-Wire bus, since each device has a 64-bit unique identifier burned into it, which the controller/master uses to track the slave devices. However, there are important topological considerations to take into account, given the electrical characteristics of the wire segment of the network, which, due to impedence “weights”, can introduce signal reflections. Maxim covers these topics in a design note (here), to help explain which network topologies to avoid (such as “star” networks), and which ones to favor.
I started using 1-Wire nearly 10 years ago, because I found an inexpensive weather station from AAG based on 1-Wire devices. I mounted the weather station on the roof and ran a couple (ok, 3) wires down to the basement, which is where a Midon Design bus controller (at the time, “TEMP05”, but he’s up to “TEMP08” now!), connected via RS-232 to an always-on PC, was located. On the PC, I ran Homeseer: a script would pull ASCII weather data generated by the TEMP05 and log it, and I had other scripts to graph weather trends. The weather station used a 1-Wire temperature sensor for measuring outside temperature, and other 1-Wire devices to transmit wind speed and direction. That setup worked for quite a few years, until our Pacific Northwest weather did it in. I’ve since graduated to a David Instruments weather station - more on that later.
Along the way, I also hooked up a couple of DS18S20 temperature sensors, mostly out of curiousity, and it was through these experiments that I learned a bit about best practices - for my house, anyway - for implementing a reliable 1-Wire bus. I wanted to use, where possible, the existing CAT-5 home-run wiring I had installed in the house… but given that I had a single bus controller - the TEMP05 - I knew that a star configuration would not work. So I had to daisy-chain together the network segments, including the long run to the weather station on the roof. I found that this configuration didn’t work reliably under parasitic power, but fortunately, the TEMP05 device makes it easy to provide the +5V Vdd needed to power the microlan.
This helped, but it still wasn’t rock-solid reliable. In most cases, it didn’t matter, given that I could poll for temperature readings every minute or so and average the results. Adding reflection-dampening resisters (per Midon Design advice) helped some. But I didn’t like the idea that I had to custom-tune the network every time I changed or added devices. A more robust and easy-to-get-right solution seemed to require multiple controllers, perhaps one per physical network leg?
Along that line of thinking, I tried out a relatively inexpensive ($29) bus controller, the “CK0110” four-channel kit from Carl’s Electronics, which seemed to be able to support 4 microlans. That helped.
At some point, I stumbled upon Embedded Systems’ TCP/IP-based bus controller, the HA7Net, which does this: you connect up to three microlans to it, add power, and patch it into your LAN. It looks like this:
You can then point a browser at it and query the devices that it is managing on any of the three microlans. Furthermore, you can connect to it via a telnet session, which is how I integrated it into Homeseer (via the Ultra1Wire plugin, a newer version of which can be found here). I liked this approach because it meant one less RS-232 serial port to deal with (although this turns out to have been a temporary phenomena, given the approach I took to integrating with A/V receiver and monitor - see this post for the details).
The scenarios supported by these 1-Wire temperature sensors had expanded at this point to include:
- Turn on bathroom fans if someone is taking a shower, run for 10 minutes after shower ends (this one took a while to pull off, but now it seems perfectly commonplace)
- Turn on Kitchen fan if high temperature detected over the stovetop
- If temperature in computer rack gets too high, warn
(See this list for a complete list of implemented and planned scenarios)
Those scenarios required 5 or 6 sensors, which means that, with the HA7Net’s support for only three microlans, I would have to daisy-chain a few of the sensors, all of which were hanging off the far ends of CAT-5 homeruns, in the various corners of the house. The approach I took was to wire up a daisy-chain junction box, located in the basement. Here’s a photo:
At each device, one conductor brings the DQ signal line “in” to the device, and another conductor takes it “out” to the next device down the line. All devices are wired to the same power and ground conductors. The junction box down in the basement contains RJ-45 sockets into which patch cables, each one representing a device, are plugged in. One end of the chain goes to the HA7Net.
The “Uplink” cable connects to the HA7Net. The patch cables connected to sockets labeled “1”, “2”, and “3” connect to various 1-Wire temperature sensors, via the house patch panel.
Here’s a schematic:
This is the setup I’m using today. It’s reliable and easy to work with, and the availability of a well-built Homeseer plugin (the Ultra1Wire mentioned above) makes system integration very easy.
If I were starting over now, I would probably end up with a similar arrangement, despite the proliferation of cheap TCP/IP-enabled hacker boards. It’s hard to beat the <$5 price for a sensor-cum-network adapter. And the HA7Net bus master, while not cheap, is reasonably priced given that it’s TCP/IP-enabled and, since it supports three microlans, effectively means that you’d only need one in your house. (You could also substitute an Ardruino, using the 1-Wire library). If I were to try to build a network of temperature sensors that, for instance, is based on TCP/IP instead of 1-Wire, the per-device price would be much higher - for instance, the Arduino ethernet shield goes for $39-$45.
This would be a good point to make a general point about the world of devices or the so-called “Internet of Things” (see overview post)… while TCP/IP will be the dominant protocol at the high-level, since that’s how our top-level controlling computers are connected, down at the leaf nodes, where cost and size factors are important, alternate special-purpose protocols, such as 1-Wire, will be in abundance. Other examples include: the ZigBee wireless protocol, with low-power boards starting at $23, Bluetooth for $45, and the “Nordic” nRF24L01+ series, starting at $34… while some of these prices approach that of, say, the TCP/IP-based WiFi protocol, their physical size and complexity is much less so in comparison.
Even as the cost of TCP/IP-enabling devices drops in price, the lower-boundaries (size, cost) will also continue to drop. I think the thing to focus on is that 1-Wire devices are addressable, given their unique IDs, and ability to connect to a bus that can be bridged (via devices like the HA7Net) to the TCP/IP and therefore the Internet. Thus: not every connected device in the world will have a TCP/IP address, but you’ll still be able to talk to it, learn its state, and issue commands to it. That’s OK, I think.
NOTES
- Carl’s Electronics offers an inexpensive RS-232-based 4-port 1-Wire bus controller kit.
- Quozl (http://quozl.us.netrek.org) is an interesting fellow and offers an interesting site, describing a number of open source projects and hacks. He supplied the code for abovementioned 4-port 1-Wire bus master.
- Arduino support for 1-Wire: background and library: http://www.arduino.cc/playground/Learning/OneWire
- More background on 1-Wire, including links to software for directly driving the 1-Wire bus from a PC: http://www.arunet.co.uk/tkboyd/e1didx.htm.
- Hobby Boards (http://www.hobby-boards.com/catalog/index.php) offers a range of 1-Wire devices and kits, including: temperature, humidity, relays, displays.
- I used - for the first time - “TinyCAD“, an excellent free schematic drawing tool, to draw the 1-Wire schematic above. I found it very easy to use.
Simple
“Simplicity is the ultimate sophistication. “ - Leonardo da Vinci (here and here)
I couldn’t pass up the opportunity to use a quote from L. DaVinci as an introduction to this blog post. But it’s really does set the right tone for a topic that’s been creeping up behind me on pad-feet, creaking the floor boards and occasionally breathing a little loudly. It’s always been there, but now it’s time to write about it.
A main theme for home automation should be, I think, to add value - in terms of convenience, security, energy efficiency, etc - while simplifying. Put another way, if you bring a new scenario to bear, don’t do it in such a way as to add complexity. Just add value, not complexity. Hold the complexity! Or make something simpler than it was.
It’s hard to make something simple, or simpler, while adding cool new functionality. Another quote is due:
To paraphrase Einstein (“simply” because I can):
“Make things as simple as possible, but not simpler” - Einstein (here)
Some questions to ponder when considering adding a new scenario:
- How “natural” does the new scenario seem to the average user?
- How much “training”, if any, is required?
- How much of what the user already knows can be used to leverage the new scenario?
- How robust is the scenario in the face of unexpected user actions or input, perhaps using existing controls or devices? Or other failures (such as power failures, loss of internet connectivity, etc)?
- How are existing scenarios changed?
- What additional “workloads” does the new scenario introduced for the user?
For example, some considerations when installing light switches that can be controlled via Home Automation software:
- Do the switches operate like ordinary switches? Will users just “know” how to use them, because they operate just like other switches around the house?
- What “value” are you introducing with these fancy new switches… is it for energy conservation? Security? What plan will you put in place to avoid confusing or frustrating users, or, worst-case, leave them tripping in the dark, feeling the walls for the switch for the light that just turned off for some reason?
- If the HA software is programmed to turn the lights off under certain conditions, how will users react? Can this automation be over-ridden?
- If the HA software is programmed to turn the lights on under certain conditions, will it also have a plan to turn them off (to conserve power or reduce user workload)?
Consider a (newer) Z-Wave light switch. One of the things I like about Z-Wave is that my controller software (Homeseer) can quickly detect when the user turns the switch on or off (see note 1). This means that I can fire events based on a change in the state of the switch. So I can implement a timer for the switch: if the user turns on the light, I want it to be turned off in, say, 20 minutes, because people in my house don’t know how to turn off lights (or so it seems). So far, so good. But, if someone turns the light on and then off 5 minutes later (that would be me) and THEN turns it back on a minute later, that 20 minute timer needs to be reset. It takes some scripting to make this happen (to clear out any pending “off” events for that switch).
This is an example of using some extra cycles (in the form of additional scripting) to hide the complexity from the user and build in some robustness. It’s also an example of the importance of 2-way communication between devices. It’s hard to create a smart scenario where unpredictable humans are involved if the controls involved aren’t able to communicate bi-directionally. Early Z-Wave devices, and earlier technologies, such as X-10, could not do a good job of keeping the HA software in the loop when the state of the device changed. If the user could turned on a light, the HA software might not know it, or might not know it for several minutes. If you were interested in adding value with these switches, it had to be done carefully to avoid frustrating the user.
Another example: what happens when you “automate” the A/V stack in your living room employing the usual approach of programming a fancy universal remote control (see note 2)? If my household was any indication, you’re introducing a new world of hurt. It seems that the usual approach leaves out the possibility that the user might have the audacity to actually touch the equipment, or perform a step in some order other than what’s been prescribed. This is usually because the typical remote control communicates in 1 direction only: it talks, but does not listen. It has no clue as to whether those commands have been correctly received, or whether it’s own model for the state of the components it thinks it’s controlling is accurate (see note 3). If the hapless user happens to, say, turn on the DVD player by pushing the power button on the player, in order to, say, insert a DVD and then picks up the universal remote to “Play DVD”, confusion results. The hapless universal remote, not knowing that the DVD player is already on will likely send a “power toggle” remote control signal to the DVD player, which will promptly turn if off. The remote is none the wiser. The user, though, is sure that something is screwed up. In an ideal world, the remote and the controlled components would talk to each other. In a slightly-less but still workable world, all component manufacturers would implement discrete remote codes for “power” commands and the like - not toggle commands.
In our house, we’ve moved beyond universal remote controls. At some point, it struck me as to how much of a compromise they represented, in terms of the user experience. Instead of using the remote that came with the component - DVD player, game console, tuner, etc - we were trying to shoe-horn all of the specific functions into a single, large, oddly-shapped remote that also tried to control the components using a 1-way protocol.
It slowly dawned on me that I could take a contrarian approach: if you want to watch a DVD, why not pick up the DVD remote and just use that remote for everything? That remote obviously already has a power button. And, the kicker is, it also has volume up/down buttons - presumably because it can control receivers from the same manufacturer or be pressed into service as a universal remote. So everything one needs to watch a DVD - power, volume, transport, and other DVD-specific functions - are represented as buttons on that one remote. So why not design a DVD scenario around that DVD remote? Similarly, the game console remote had all the buttons one needs to use it as well as a set of volume and mute buttons. Well, this is odd. Let’s go with it.
The approach I took was to reliably and in real-time mirror as software variables in the HA software (virtual devices in Homeseer) the power state of each component. If the DVD player was turned on, I needed the corresponding “DVD Power” virtual device in Homeseer to change instantly to “On”. Ditto the Xbox and any other source components. This was important to get right in order to handle the situation where a pesky human touches something in the A/V stack to, say, switch out a DVD. It took a while to figure out how to do this. For now, I’ll summarize it as follows:
- DVD player: my particular player, a Sony BluRay player, sports a USB port on the front. When the player is powered on, that port is powered up. I built a simple circuit to sense when +5 volt signal is present at that port and change the state of the “DVD Power” virtual device in Homeseer to “On”. When the +5 volt signal is removed, the virtual device state is set to “Off”.
- XBox: took a bit of work (and voided the warranty!). The Xbox also has USB ports, but these remain powered on even when the Xbox is “off”. Foiled! So the trick I used for the DVD player wouldn’t work here. I did, however, find a way to tap into the wires leading to the fans, which are fed a varying level of voltage when the console is on (presumably depending on how hot the console is). I built a simple circuit to detect when any positive voltage is present on the fans, and update a “Xbox Power” virtual device in Homeseer accordingly.
Once I had power state variables I could rely on, I was on the way to implementing this “pick up just the remote for the source you want” scenario. If you want to watch a DVD, pick up that remote and hit the power button. The DVD player will turn on, and Homeseer will take note of it: it will run some additional scripts to turn on the Receiver and the Monitor (which entailed making use of the RS-232-based command sets offered by those two components). Enabling the use of the DVD remote’s volume up/down buttons took more work, involving an PC-based IR receiver/transmitter (the USB UIRT).
I’ll post a more detailed explanation of all this in a follow-up post. But the bottom line is that it’s all working now. The existing controls - the remotes, the buttons on the components themselves - still work as expected, and in fact complement the new scenario. The universal remote control is packed away in the closet, beeping now and then as its battery fades.
A final example involves a scenario inspired by a contractor who was working in our bathroom. He kept saying, “Get it out! Get it all out!!”, when we talked about sizing the exhaust fan. He really (really) believed in the importance of clearly the bathroom quickly of steam from the shower. It not, you run the risk of mildew or rot and the resulting structural problems. Always run the fan while showering, and then for 10-15 minutes afterwards.
In my house, however, few other occupants really took this message to heart. If they remembered to turn the fan on, it was typically after the shower was over and the walls were already dripping with condensation. And no one remembered to turn it off after the required 15 minutes, thus triggering complaints from me about noise and wasted electricity.
I moved the fan to a Z-Wave controlled circuit, and scripted it so that it would turn off automatically after 15 minutes. That’s a good start, but doesn’t solve the problem of getting people to turn the fan on during the shower, not after it’s over and the place is already fogged up.
What would the average user expect when asked to describe a “automatic shower fan”? I’d say this: the fan should turn on automatically when someone starts the shower, and not turn off until 15 minutes after the shower ends. That’s a great goal statement. But, as you can imagine, from an implementation perspective, it seemed like a downright gnarly problem to solve. But it was solved, albeit with some extra hardware (Ha7NET hub), 1-Wire devices, and more complex-than-usual scripting). The “simplest solution” has been working well for a couple of years now. More on that later.
In closing: I like ThingM‘s motto: “Smart devices make things simple”. I think that’s a good criteria to evaluate when deciding whether your ‘bots and automation plans are actually adding value.
Notes:
- If I’ve set it up correctly. Due to various issues with various versions of Homeseer, it’s not a given that this is always the case. More on this later.
- I’ve owned two Logitech Harmony Universal Remotes, but, alas, I can’t recommend them. Both have ended up being disappointed for various reasons… the programming experience is awful, but it pales to the issues caused by the poor product build quality, hardware flakiness, and support. The most recent model I owned is the Harmony 890, which can send signals via IR or RF. If you’re considering one of these, please take a look at the reviews on Amazon first. On paper, it showed promise…
- Too often, component manufacturers take the lazy approach to the power button on the remote, by implementing it was a toggle. Pushing the “power” button on the average remote often sends a single command which the component interprets as this: “if you’re on, turn off; if you’re off, turn on”. This means that the average universal remote must remember if a component is on or off in order to implement a scenario like this: “the user is currently watching a DVD but would now like to play a game on the game console… so, turn off the DVD player and turn on the game console”. If the remote thinks that the game console is already on, it will skip sending a “toggle power” remote code. If, on the other hand, the game console has discrete “power on” and “power off” codes, the remote can confidently send “power on”, which do nothing if the game console is already on. I think that “discrete codes” are one of the marks of a component/device that’s designed to be system-integrated.
‘Bots in the Basement
It’s one thing to write about this stuff, but it’s another thing entirely to get hands-on with it. Since I’m after some credibility here🙂, this post is about some of my own connected experiences, starting in the Home Automation space, in which I started dabbling around 2001.
I try to take a “scenarios-oriented” approach when messing with the house, given that its other occupants may not have a high degree of tolerance for things that don’t work as expected. It’s a goal that any new scenario adds value (in terms of safety, security, energy conservation, comfort, convenience, etc) in a seamless and reliable manner. Sure, sure, sounds good, right?
Here’s a list of scenarios implemented to date:
Lighting
- Turn off lights if rooms aren’t occupied, or after a certain amount of time
- If security system is armed, randomly cycle lights to simulate occupancy
Fans
- Turn off bathroom fans and Kitchen exhaust fan after 10 minutes
- Turn on bathroom fans if someone is taking a shower, run for 10 minutes after shower ends (this one took a while to pull off, but now it seems perfectly commonplace)
- Turn on Kitchen fan if high temperature detected over the stovetop
- Periodically, run fan in the cats’ litter box area
HVAC
- Set temperature back in all zones when no one is home (that is, when the security system is armed) and return to normal schedule when someone comes home (security system is disarmed)
- Periodically, set correct time on thermostats
- Periodically, run the hot water recirculation pump
Security
- Send email notice when security system is armed, disarmed, or when there’s an actually alarm
- Notify if garage door is left open, provide option for remotely opening/closing it.
- When someone comes home (and disarms the security system), turn on certain lights for convenience
- When security system is armed, randomly turn on/off certain lights at night to simulate occupancy
Irrigation
- Run sprinkers on regular schedule (depending recent rainfall, as detected by local weather station)
A/V
- In main zone (living room): If DVD is turned on/off, also turn on/off receiver and monitor, and set receiver source accordingly, for main and 2nd zones. Ditto XBox
- In main zone (living room): If volume up/down is used on any remote, adjust receiver volume
- In Kitchen zone: if kitchen radio is turned on/off, also turn on receiver zone #2 and set source. Monitor volume keypad for key presses, and adjust zone volume accordingly.
Reminders, Warnings, and Notices
- Send email note reminding to take out Trash and Compost bins. Scrape city’s web site to also determine if it’s a Recycling day
- Send nightly reminder note to close apps for better backups
- If the security system is armed at 10pm at night, send an summary note summarizing the status of the automation system
- If temperature in computer rack or over kitchen stove top gets too high, warn
- Send note if temperate has dropped to near-freezing
These scenarios are implemented via these components:
Homeseer Home Automation server (http://homeseer.com): software installed on a small headless box running 7×24 in the basement. Presently, this represents the main controller and (browser-based) user interface for all home automation scenarios.
6Bit TCP/IP-controlled relay and input board (http://www.6bit.com – appears defunct at this time): I can telnet into this board and command any of its 6 relays to open/close or sense when any of 12 inputs go low. Can also download simple macros, such as: “close relay 1 and open in 10 seconds”. This board is used to: sense when the garage door is open (I mounted a switch on the door opener, a bit of a story in itself), and to sense when certain devices turn on (such as the DVD player, Xbox, or kitchen radio) or when buttons are pushed (such as the volume up/down buttons on the kitchen keypad), or when the security system signals an “armed” / “unarmed”, “normal” or “alarm”, or “motion in zone x”. The relays on the board are used to: close the garage door and energize any of the three sprinkler valves. Homeseer and the 6Bit box communicate with each other via a virtual COM port. The 6bit box also offers an HTTP service, so you can use a browser to configure it, etc. It uses a Lantronix XPort device (http://www.lantronix.com/device-networking/embedded-device-servers/?tab=0) to connect the board/sensor logic to the LAN. It’s too bad that this company appears to be defunct/zombied, as the board is well-designed and constructed. My guess is that if I had to replace it, I’d go the Arduino route.
Ha7NET 1-Wire Ethernet Host Adapter (http://www.embeddeddatasystems.com/HA7Net-Ethernet-1-Wire-Host-Adapter_p_22.html): You plug this small box into your LAN. You also plug into it your “1-Wire” networks. “1-Wire” really is “3-Wires”, but that’s OK, it’s only Marketing. 1-Wire devices are small, cheap, nominally intelligent devices. I use the DS18S20 temperature sensor, which looks like a small transistor and costs < $5. You can plop any number of these devices onto the same run of 3 wires: Signal, Ground, Power. Each device has its own unique ID, and implements a basic protocol for manipulating the 1-Wire bus in order to communicate with the host adapter. I have three little networks of these devices in my house (due to topology requires of the 1-Wire bus), which all terminate at the host adapter. On the Homeseer controller application, there’s a plug-in (from Ultra1Wire) that knows how to find and communicate with the Ha7NET box to get regular (every couple of minutes) updates from the sensors. The Ha7NET box also offers an HTTP service that you can hit with a browser to configure the device, get an inventory of devices on the 1-Wire net, etc. Prior to the Ha7NET board, I used a Midon Design “Temp 08” board (http://midondesign.com/TEMP08/TEMP08.html), that connected to the Homeseer app via serial port. But I wanted to move away from serial-based boards…
Davis Scientific Weather Station (http://www.davisnet.com/weather/products/vantagepro.asp): with a TCP/IP-enabled console. The console communicates with the Davis Scientific “http://WeatherLink.com” web site on a regular basis, sending up weather stats like temperature, wind, etc. The site also sends that info over to any of several popular weather sites such as http://weatherunderground.com. I wrote Homeseer scripts that regularly pull down relevant stats – such as recent rainfall – to guide the sprinkler schedule. So, it’s a bit roundabout - weather bytes travel from the weather station on my roof, to the console in our kitchen, and there to the WeatherLink site, and then over to WeatherUnderground, and then back down to the Homeseer app running on a PC in my basement. Presumably they’re out of breath when they arrive. An interesting bit of system engineering, but it’s not always reliable (more on that later).
Z-Wave light switches and modules (http://www.z-wave.com/modules/ZwaveStart): Z-Wave is a specification / industry consortium for RF-based switches and controllers that enable remote control and sensing in the house. I have about 10 Z-Wave light switches installed throughout the house, and a Homeseer-branded Z-Wave controller connected to the Homeseer PC via serial port. As a result, Homeseer knows when someone has turned on a light in, say, the family room, and can be scripted to automatically turn if off after, say, 20 minutes. The Z-Wave light switches look mostly like regular Decora-style light switches, with some quirks and other characteristics. Z-Wave devices communicate via an RF mesh… each device spends some time getting to know its neighbors, forming primary and secondary paths to them based on signal strength. When a device wants to communicate to the controller (connected to the PC), it asks its neighbors to route the message along, via the mesh. The advantage in this approach is that the mesh can dynamically deal with sudden obstructions that might block a primary path, such as if a refrigerator door (a large metal object) is opened, or a car pulls into the garage, or permanent obstructions; in my house, one or two of the Z-Wave devices have a clear (RF) view to the controller, as there are all sorts of heating ducts in the way (ever see the movie “Brazil“??). So the mesh concept comes in handy, but it does have its downsides, in terms of complexity, reliability, and ability to debug, which I’ll relate in a later post. But it’s a heck of a lot better than “X10” (http://www.x10.com), an earlier approach (based on sending signals over the AC powerline) to control and sensing in the home. (Some of you may remember how, in the early days of the web and pop-up ads, it seemed that every other ad was for X10…)
Proliphix web-enabled thermostats (http://proliphix.com): Proliphix offers a line of high-quality thermostats that just happen to sport web interfaces and an API. So, you can hit the home page for one of these thermostats and see the current temperature in that room, the current heat/cool setpoints, the setback schedule, and so on. We have three zones in our little house (one for each of the basement, first floor, and second floor), and there’s a plug-in for Homeseer that surfaces key metrics and commands, so that it’s easy to, say, write a script that sets the heat setpoint back if no one’s in the house.
Security System: I’ve installed a mid-level security system, which, until recently, really didn’t want to play well with the other systems. I managed to arrange its typically archaic programming of zones and sensors so that it closes relays for certain conditions (system armed/disarmed, alarm, activity in certain zones), which are connected to the 6bit board’s input sensors, which Homeseer can track. The end-result is that Homeseer knows about certain security system events, and I can script scenarios such as this: if the security system is armed, set back the Proliphix thermostats to save on heating costs.
USB-UIRT IR receiver/transmitter PC interface (http://www.usbuirt.com): this is a small box that connects via USB to the PC running the Homeseer control application, and is apparently manufactured by a guy in a garage🙂. Homeseer can sense (via a plug-in) when certain IR signals have been received (after training) or transmit IR signals (after training). The result is that I can write scripts like this: “if the volume up button is pressed on the Xbox remote control, then send a signal to the Denon receiver to increment the volume”.
Other elements
I have a Denon receiver with an RS-232 port on the back, which enables communication with a PC or controller. The receiver is connected via serial cable (straight-through! straight-through!!) to the PC running Homeseer. I’ve written scripts for Homeseer which can read responses from, and write commands to, the receiver. So, for instance, someone changes the volume on the receiver – either directly, by turning the knob, or via a remote – the receiver dutifully outputs some characters through the serial port, and my script running on Homeseer can track the changes (updating Homeseer variables which represent the volume level). Similarly, I can, via script, programmatically change the volume on the receiver. Or, I can script Homeseer so that if the DVD player is turned on, the receiver is turned on and its input source set to “DVD”. There’s a similar arrangement with the monitor in the living room. I’ll post more details on how the A/V system is scripted, as I think the resulting level of simplicity it affords to the end-user (no more confusion about remotes, and no more need for universal remotes) is worth sharing.
Some additional, quick notes
- I’ve tried to move away from serial-based connections (RS-232, USB, etc) and towards TCP/IP, since the resulting simplicity and flexibility (in topology, distance, etc) is worthwhile. The 6bit relay/sensor board, Ha7NET 1-Wire host adapter, and the Proliphix thermostats listed above all communicate via Ethernet. However, I found it much easier to program, and reliable, to stick with the antique RS-232 interface for integrating with the A/V components (receiver, monitor, IR controller). More on this later.
- I have this hang-up/obsession around the importance of instant and relevant feedback when dealing with the humans in the house (at least, when it comes to the home automation🙂. If someone presses a button and nothing appears to happen quickly – even if, under the covers, millions of compute cycles are cycled, thousands and thousands of disk seeks are commanded, and umpteen packets are zipped around LAN and WAN, all in service of that human’s simple action – that stinks. The human is left wondering… did I do something wrong? Or did this stupid system fail again? More on this later.
- Another hang-up/obsession: I don’t have (much) patience for systems that fail, especially those that fail silently. I’ll go out of my way – in the form of extra coding, hardware, wiring, etc – to ensure that systems are built to be reliable, and that if they do fail, they don’t fail silently. More on this later.
All together, I think this loose constellation of systems and components, held together with lots of Homeseer scripting, does add value in a mostly seamless or even invisible way. There’s lots of room for improvement (for instance, extended power failures can be tough to recover from…). And if viewed from the perspective of, “Is this ready for the mainstream??”, the answer is a clear “No!”. More on this later.🙂
