i see bots

Imagine that at some point in the not-too-distant future, you’re the owner of a ‘smart’ house, which, you’re told, contains 100 smart devices. That’s a lot, you think to yourself. What are they all doing?? Here’s an plausible inventory:

• Security devices: one device per window, to detect open/close/breakage, and a number of motion sensors, for a total of 30 devices

• Survellance cameras: one for each entrance, one for each side of the house, for a total of 6 devices

• Thermostats: one for each room, for a total of 10

• Smart light switches: one for each room and hallway, and several more for outside lights, the garage, etc, for a total of 25

• A/V and Control wall panels for most rooms, for a total of 5

• Devices representing main controllers for automation, security, A/V components, VOIP, personal computers, file servers, routers, access points, and broadband modems, for a total of 10

The above adds up to 86 (see Note 1), and while I’m writing this post, I’m sure I’ll think up ideas for 14 more. (see Note 2)

And I didn’t include devices that get around, such as:

• Mobile phones, which aren’t tethered to the house but may spend a good portion of their time in the house.
• Portable music players (iPods, etc) and other portable personal devices
• Your cars or trucks (I’m not sure that it counts as “news” that the new Chevy volt has an IP address, but it’s interesting that it’s part of the headline), and significant components in/on your vehicle, such as: the tire pressure monitoring system (but be careful!), the A/V system, etc)
• Stuff you might wear, such as a smart watch or telemetry-sending exercise shoes

If you ponder this future for a moment, you might arrive at these conclusions and observations:

The number of devices that we’ll rely on, for a wide range of ‘personal scenarios’, will exceed our ability to directly manage them. We’ll know they’re there, working on our behalf, but we’ll likely forget the details about how to manage or configure them outright, reminded of the need only when one stops working or it needs some maintenance (see note 3), or you need it to do something out of the ordinary.

And even if you did try to individually manage a device, it’s likely that you’ll do it remotely, via a web page or specialized application: the device itself will be too small to support direct manipulation (lacking, say, a display and buttons), or the range of options and configurations will be too complex to adequately manage via the simple display and buttons that are on the device (example: thermostats), or the device is unreachable because it’s physically embedded into the house (such as wireless security sensors for windows).

For these reasons, I think we’ll see an architecture where the devices are proxied by a device portal or manager which aggregates basic information such as state and health, enabling a user experience that supports views based on filters over state, alarms triggered on state or health, etc.

The device manager will know, for some classes of devices, such as those which are TCP/IP-enabled - how to directly interrogate them directly or how to subscribe to updates. Other kinds of devices, such as those which aren’t TCP/IP-enabled, may require an intermediate hub or subsystem to broker the communication between the device manager and the devices themselves, such as with a security system (where the window sensors may be simple switches) or a 1-Wire bus, which requires a hub to communicate with.

It may also embody some basic configuration/management, such as “reset”, and then link off to a device-specific management page, served up by the device itself or by a type-specific device hub.

The device manager will represent the devices to the outside world, at least for read operations, isolating them from frequent requests for updates (early Proliphix documentation recommended restricting the API request rate to “a few requests per minute”). You’d want the manager to offer an RSS feed, not the devices themselves. The device manager could also implement some level of security/access permissions.

It’s more than a simply proxy, however… I’d expect it to also know the assigned name, location and type/intended purpose of each device, and provide views/filters around that: “Show me the state of all of the security devices on the second floor”. This implies a device registry or directory. My guess is that you wouldn’t expect or want most devices to handle this metadata on their own.

Finally, you’d also expect that the device manager would implement some level of APIs and scripting, for sophisticated eventing and notifications. Thinking this through, it’s clear that the device manager would need to implement ‘psuedo variables’ representing it’s view of the current state of the devices it’s managing. And, furthermore, what’s just been described here could be approximated by any of several Home Automation systems… depending on the approach, it wouldn’t necessarily be as elegant as you’d like, but it could be done.

USE VS CONFIGURE

In this discussion so far, I’ve not made too much of a distinction between the kinds of access that you’ll need to your devices. Sometimes you want to “configure” a device, and sometimes you want to “use” it. For a thermostat, “configuration” means things like: setting up the kind of heater (single-stage / double-stage, heater only or heater/AC, etc), the setback schedule, and so on. “Using” the device is typically a simple affair: override the current setpoint, for a specified period of time.

You’d generally expect (but I’m not sure I can prove it) that the number of “configuration” options to be at least equal to, or greater than, the number of “use” options, especially as the level of capability of the device increases: more functionality implies more state, and possible actions based on that state, which implies more configuration.

You could imagine entirely separate paths for “configuration” and “use”; in fact, in many cases, it may be very desirable to ensure this. Adhering to the “simpler is better” maxim, you’d want to keep the UI for “use” as simple as possible, and keep “configuration” UI, which might require different paradigms for efficient management, separate.

REMOTE UI

With the Proliphix Thermostats I use, you “configure” via embedded web pages. The configuration experience, delivered via browser, is fairly sophisticated (for a thermostat!). On the other hand, the on-device display and associated 5 buttons are focused almost completely on what the end-user needs: what’s the current temperature, what’s the current setpoint, and how do I get some heat? Some of the UI is dedicated to read-only access to basic configuration info (such as the current IP address), but that info is there to help the user provide good hints to the maintenance team.

In any case, it would be very hard to imagine an on-device user experience for, say, specifying setback schedules for specific days of the month or holidays using the on-device display and buttons. As the designer for the device, you wouldn’t even try. Being able to “express” yourself - as the designer of the “configuration” experience - via web pages means you can expose a lot of useful features in a natural way.

The inverse is probably true, too: don’t build it in the hardware if it can’t be configured by the admin. The embedded web server, while adding $20-$40 to the build cost of the device, means you can offer a lot more functionality and charge for it.

(An aside: what about the “blinking 12:00 AM” VCR clock problem of years past? What this solely due to poor UI design? Or should we just blame the user? Later-model VCRs learned to pick off a time signal from the TV signal, which apparently eliminated the problem for most folks. But if you had this UI challenge in front of you today - not just getting the user to set the correct local time, but also, say, the task of programming the VCR to record a program - would you push it off to a web-based experience, even if it increased the hardware costs of the device by, say, $50?)

So, even though the device in this case is relatively small, the configuration experience is ‘outsized’. Even as devices decrease in size while increasing in capability, they could offer an ‘outsized’ user experience.

If the device (or its hub) supports APIs, you could imagine more than one flavor of UI, in support of various scenarios: beyond the built-in “configuration” and “use” UIs, multiple other “use” UIs could be supported, via APIs, in the form of other web experiences or apps. A device manufacturer might leave the heavy-lifting - the implementation of an elegant end-user UI, for instance - to third-party developers who specialize in that sort of thing.

A colleague of mine imagines a “Facebook for Devices”, a kind of third-party portal where you can see recent updates from your devices and others of interest - and casually pivot on the data in various ways, potentially offering you an easier way to keep track of your 100 devices.

These third-party or extended experiences don’t have to be limited to web sites; imagine a display-oriented device with the sole job of displaying RSS or Twitter-like feeds from multitudes of other devices.

A PIVOT TOWARDS THE CLOUD?

If all of the devices you care about are under one roof, then you could imagine using dedicated device to act as the device manager for all of your devices. It could be implemented via a low-end PC or embedded PC, running headless, with a connection for the local area network, and perhaps dedicated I/O connections for specialized types of devices (such as 1-Wire). You’d access its web page via your LAN. Its device directory, and device data archive, could be based on permanent storage on the device itself or elsewhere on your LAN.

An alternative approach might be for the device manager based in the cloud. You’d likely still need a local agent to connect your home devices, likely stuck behind a firewall/NAT, through to the cloud, and, of course, to ensure that non-TCP/IP-based devices are proxied adequately to the LAN and then on to the cloud.

The resource requirements for a cloud-based manager are likely to be small (just as those of a home-based manager would be), perhaps modest enough that you could use the recently-announced AWS “Free Usage Tier“, which offers a non-trivial level of resources at no cost.

Going to the cloud might mean improved reliability… if your home-based device manager falls over, it’s up to you to detect it, diagnose it, fix it, and get everything running again, while, say, in the AWS cloud, if a disk dies, you’re never supposed to notice it. They have professionals on the case.

Accessing your cloud-based device manager - the UX or the APIs - may be more performant than a home-based manager, especially if you’re outside your LAN, since it’s likely that the cloud-based manager has better peering and connectivity than the slow-ish uplink of your consumer-grade broadband connection. And if it’s possible that your device manager might experience multiple simultaneous requests on a regular basis, putting in the cloud may yield better results.

A cloud-based approach also feels more elegant when the scope of devices goes beyond “the home”. If you include your mobile devices, or devices from your business or other organizations (such as local weather stations or similar services representing virtual devices that you use for more sophisticated eventing, etc), having a cloud-based device agent will make all of this easier, for the reasons already listed: reliability, bandwidth (lower latency, higher thruput, more simultaneous connections), as well as: the availability of an infinite amount of CPU and disk resources, as well as services or capabilities that may be too complicated to contemplate for home hosting. (My same colleague suggests that in-home cameras could serve up images or video that is shipped to the cloud for sophisticated vision processing that couldn’t be done either by the camera or by any software/hardware that’s likely to be installed in the home.)

Of course, a possible downside is that if your home’s broadband connection fails, the Cloud-based portal may miss device updates and therefore may not fire important events. Perhaps a hybrid model is in order, where critical events are handled locally, with the cloud portal handling most other events.

This suggests a future architecture where the cloud is the central management point, with local support - in the form of hardware/software - in the home to handle the vagaries of getting all of your devices connected to the cloud. Again, there are legitimate concerns regarding loss of connectivity, and perhaps security.

Moving some or all of the management of a device mesh to the Cloud is an interesting enough scenario that I’ll be trying it out as soon as I can. More on this later.

A PIVOT TOWARDS “SERVICE AVATARS”?

OK, so we’re now used to the idea that our devices are talking to the cloud, updating their Facebook or Twitter status. In the discussion above, it was the devices that connected up to the cloud, in pursuit of new scenarios that made the device more valuable. But what if the intention involves going the other way? What if cloud services extended down to the devices, in service of new scenarios?

Mike Kuniavsky in his book, “Smart Things: Ubiquitous Computing User Experience Design” (Amazon) describes “Service Avatars”, a term he coined that conveys the added value a focused-purpose device can bring to a service. He holds up the Apple iPod as a prime example of a Service Avator… while it wasn’t the first music player, it was the first to successfully deliver a service-based scenario - a cloud-based music store - down to a device. The device was cool, but it became so much more important when connected to its music service. (We can safely elide certain details, such as the fact that for full functionality, the device requires an intermediary “hub”, in the form of a Mac or PC running the iTunes application).

Notes

#1: I just checked my router. It’s managing DHCP addresses for 15 devices (PCs, fileservers, phones, a VOIP adapter, access point, A/V components, weather station, and a printer). I’ve assigned fixed addresses for another 11 devices (thermostats, Ha7NET, camera server, security system, etc). That adds up to 26 IP addresses, and 26 corresponding devices. If you throw in the other devices of varying smartness…

• 1-Wire temperature sensors (5)
• Security sensors (10 or so)
• Z-wave light swithces (10)

… as of now, I’m up to at least 51 devices. So 100 devices isn’t too far a stretch for a “smart home of the future”.

#2: Wait! Here are 20 more “smart home of the future” devices:

• Connected exercise equipment (1 per house)
• Roving security or health robots (1 per house)
• Connected appliances, which report energy usage, low supplies, or general health statistics (oven, microwave, refrigerator, dishwasher, washer, dryer, furnace, hot water heater) (10 per house)
• Resource-monitoring equipment, such as: water metering, electricity metering, natural gas or oil metering (oil tanks that report when they’re nearly empty, or leaking!) (3 per house)
• Smart sprinklers that only water when absolutely necessary, and never after a rainfall, and only when given the OK by the local water board, augmented up by soil moisture sensors (5 per house),

#3: I’m writing a portion of this post on the eve of that date in the Fall in the US where we “Fall Behind” with our clocks. How many devices in the future home will need resetting every Spring and Fall? You’d hope the answer is “zero”… I take a small perverse pleasure in noting, with each passing Fall and Spring, whether the number of ‘things’ around the house that need to be manually pushed back or forward an house, is decreasing.