i see bots

In the future, you won't think this is so weird

Monthly Archives: November 2010

More “100”

In my post on “100 bots”, it appears that I unwittingly tapped into a “100” theme. ReadWriteWeb reports that “ThingMagic”, a company that seems to focus on all things RFID, has been building a list of 100 things you can do with RFID. Each entry in the list is an informal blog-like post with a short description of the application, with links to more information and exploratory questions for the reader.

It’s an interesting list, if you keep in mind the fact that it’s coming from a company that sells RFID technology. It is a bit uneven; not all of the “100 uses” are at the same level or carry the same weight. Some of the applications described actually exist, while others are hypothetical.

As I toured the list, I noticed a common thread in several articles describing human-tracking applications… these articles ended with an open-ended question along these lines: do you, dear reader, find this application promising enough to outweigh the privacy concerns that often arise when RFID? Some examples:

  • “Can You See Mi Now?” describes a bicycle-safey application implemented in the Danish city of Grenå: “the city implemented battery-powered RFID readers at busy intersections designed to read RFID tags placed in the steering columns of bikes.  When a cyclist approaches and stops at an intersection, the RFID reader sends a notice to an electronic sign mounted on the traffic light pole.  This notice triggers the display of a flashing ‘cyclist’ image, indicating that a rider is near and drivers should look before making a turn.” The idea is that a motorist would notice the flashing warning and take extra care when turning. The article ends with this question: “Does addressing a real safety issue - like reducing bicycle related deaths and injuries - move you past privacy concerns you may have with RFID?”
  • “India’s National ID Card Program” entry outlines aspects of India’s initiative to store fingerprints and iris scans for all of its citizens, with the goal of, among other things, delivering better services while reducing fraud. This initiative is apparently accompanied by RFID-equipped national identity cards. This article ends with the question: “What are your thoughts about the growing use of RFID and biometric-enabled national ID cards?  Do the proposed benefits of modernization, reduced fraud, and security outweigh the potential risks?”
  • “RFID-Enabled Smart Displays” describes a new kind of synthetic vision-equipped public area display that are smart enough to tailor advertisements and other information based on what clues they can discern about of the person standing in front of it … such as gender, age range, and height. The article teases out how that customization could be ever more interesting if the person was wearing an RFID tag which allowed the display to access more “personal preferences”? The article ends with this question: “Share your thoughts about the evolution of smart signs.  Where will they work?  Where won’t they work?  How are personal data security issues best addressed?”

It’s commendable that these articles point out the potential for privacy concerns when it comes to tracking people via RFID technology, even if solutions are not proposed. Elsewhere on the ThingMagic site, privacy is described as a future topic that needs to be addressed:

“New technical and policy approaches will have to solve the real privacy and security concerns identified by industry analysts, technologists, and public watchdogs. If not, restrictive legislation or public backlash could thwart widespread acceptance—and limit the powerful benefits that RFID offers businesses and consumers.”

There’s also a “Dead Tags Don’t Talk” discussion, about how retailed-purposed tags - such as those embedded in clothes, for inventory purposes - can be disabled at the point of sale, so that their owners aren’t trackable afterwards.

I did wonder, however, why “privacy” wasn’t one of the 136 tags with which these articles were tagged.

My take on today’s RFID is that it’s great for things but not yet ready for people, outside of job/organizational security applications.

Robots in the News

A quick recap of a few recent mainstream articles concerning “robots” and robot platforms…

  • Protecting Your Home From Afar With a Robot” (New York Times, Nov 3, 2010): An informal tour of available low-cost robots and discussions with their owners, who seem to have found an inexpensive and fun way to experiment with “telepresence”, especially for surveillance of the home. Some of these robots are designed to be hacked.
  • Drones Get Ready to Fly, Unseen, Into Everyday Life” (Wall Street Journal, Nov 2, 2010): A focused look into progress on the consumer and military fronts to create small, autonomous flying drones. A quote from the article: a “goal is to develop a drone the size of a pizza box with small propellers that can watch a soldier’s back on the battlefield.”

(UK Daily Mail treatment of the same topic/article, here).

Some observations

There’s an explosion of innovation happening at the intersection of amateur aircraft building and software/hardware hacking, spurred on, I’d guess, by advances in powerful tiny electric motors (made possible by really powerful magnets), battery technology, and the availability of cheap and easy-to-integrate subsystems for controlling motors, sensors (GPS, accelerometers, gyroscopes, cameras, ultrasound), and connectivity. Throw in the availability of carbon fiber for strong, lightweight chassis.

Hobbyists seem to be genuinely pushing the envelope with cheap and capable designs integrated via powerful processing units. I challenge you to spend 5 minutes on the DIY Drones site and not be impressed with the energy and ingenuity of the community there. (For one thing, it makes it easy to impress when your projects are “self-documenting”… that is, drones with on-board video cameras).

And on the ready-to-fly/consumer front, I’m impressed with the level of technology offered. For instance, this is the list of coolness that AR.Drone brings to the table (as written on the site):

  • “A quadricopter made in carbon fiber and high resistance PA66 plastic
  • MEMS (Micro-Electro-Mechanical Systems) and video processing to ensure a very intuitive piloting of a radio controlled object
  • Wi-Fi and video streaming for a modern interface with an iPhone™ or iPod touch®
  • Images processing software for augmented reality”

Of the four consumer robots that I found (list below) via the articles note above, or via other casual searches, all four are designed to be hacked. I was able to determine that three of the four are based on Linux, and my guess is that the fourth one is as well.

It’s cool that manufacturers have jumped onto hacker bandwagon so strongly. This would be a great way to teach kids about programming for the real world, especially if they’ve outgrown MindStorms.

I’m wondering whether Google’s Android operating system (a variant of Linux) will see any adoption in this space? Something’s happening on the Android phone hardware front, for sure: a number of folks are building robots using Android-based cell phones as the controllers, system integrators, or remotes (here’s an example.) Given that your average smart phone has significant CPU, memory, and sensor resources, all that’s needed to make a robot is a chassis, motors and controllers, etc. For instance, the Google Nexus One has a 1Ghz processor and 512MB of memory… the robots listed below typically sport processors running at half that clock rate.

Beyond surveillance and getting kids interested in hacking, how are these cheap robots being used? For helping people living at home who might need extra attention, for one thing, according to Hoaloha Robotics.

(Another tip of the hat to Charles, for forwarding a number of these links.)

An initial set of related links, more to come:

  • New York Times’ list of Robot articles: “News about robots, including commentary and archival articles published in The New York Times”

Robots

  • Rovio (from WowWee, ~$179US): “the groundbreaking new Wi-Fi enabled mobile webcam that lets you view and interact with its environment through streaming video and audio.” Includes API documentation, apparently for client web apps that access its built-in web server. Couldn’t easily figure out what software/hardware it’s based on. Loved their tagline: “Rovio - now you can finally be in two places at once!”
  • Spykee (from Meccano: http://spykeeworld.com, $329US): “WiFi spy robot”. Firmware is provided in source form. Appears to be Linux-based, with an ARM processor.
  • AR.Drone (http://ardrone.parrot.com/parrot-ar-drone/usa, $299): “The flying video game”. “First quadricopter that can be controlled by an iPhone/iPod Touch/iPad.” Includes an on-board video camera. Updatable firmware. SDK available for game developers. ARM processor running a Linux OS.
  • Spy Gear Spy Video TRAKR (http://www.spygear.net, $129). Can download apps from a catalog and also build apps on your own with a web-based “IDE”. ARM processor running a Linux OS (on both the robot and the remote!).

Software

  • robodance.com (http://robodance.com): “Robodance is the ultimate software program for your WowWee Rovio”; other robots also supported.

Communities / Blogs

Bonus links: Robots and How we view them (these will be funny right up until Skynet becomes self-aware):

  • “Television’s Greatest Robots: A Video Timeline” (Gawker)
  • Baca Robo (“Stupid Robot”) Contest, Budapest, Hungary (via io0.com): Not in English, but fun to watch even with the sound turned down. The lab coats are a nice touch.

100 Devices

Imagine that at some point in the not-too-distant future, you’re the owner of a ‘smart’ house, which, you’re told, contains 100 smart devices. That’s a lot, you think to yourself. What are they all doing?? Here’s an plausible inventory:

  • Security devices: one device per window, to detect open/close/breakage, and a number of motion sensors, for a total of 30 devices
  • Survellance cameras: one for each entrance, one for each side of the house, for a total of 6 devices
  • Thermostats: one for each room, for a total of 10
  • Smart light switches: one for each room and hallway, and several more for outside lights, the garage, etc, for a total of 25
  • A/V and Control wall panels for most rooms, for a total of 5
  • Devices representing main controllers for automation, security, A/V components, VOIP, personal computers, file servers, routers, access points, and broadband modems, for a total of 10

The above adds up to 86 (see Note 1), and while I’m writing this post, I’m sure I’ll think up ideas for 14 more. (see Note 2)

And I didn’t include devices that get around, such as:

  • Mobile phones, which aren’t tethered to the house but may spend a good portion of their time in the house.
  • Portable music players (iPods, etc) and other portable personal devices
  • Your cars or trucks (I’m not sure that it counts as “news” that the new Chevy volt has an IP address, but it’s interesting that it’s part of the headline), and significant components in/on your vehicle, such as: the tire pressure monitoring system (but be careful!), the A/V system, etc)
  • Stuff you might wear, such as a smart watch or telemetry-sending exercise shoes

If you ponder this future for a moment, you might arrive at these conclusions and observations:

The number of devices that we’ll rely on, for a wide range of ‘personal scenarios’, will exceed our ability to directly manage them. We’ll know they’re there, working on our behalf, but we’ll likely forget the details about how to manage or configure them outright, reminded of the need only when one stops working or it needs some maintenance (see note 3), or you need it to do something out of the ordinary.

And even if you did try to individually manage a device, it’s likely that you’ll do it remotely, via a web page or specialized application: the device itself will be too small to support direct manipulation (lacking, say, a display and buttons), or the range of options and configurations will be too complex to adequately manage via the simple display and buttons that are on the device (example: thermostats), or the device is unreachable because it’s physically embedded into the house (such as wireless security sensors for windows).

For these reasons, I think we’ll see an architecture where the devices are proxied by a device portal or manager which aggregates basic information such as state and health, enabling a user experience that supports views based on filters over state, alarms triggered on state or health, etc.

The device manager will know, for some classes of devices, such as those which are TCP/IP-enabled - how to directly interrogate them directly or how to subscribe to updates. Other kinds of devices, such as those which aren’t TCP/IP-enabled, may require an intermediate hub or subsystem to broker the communication between the device manager and the devices themselves, such as with a security system (where the window sensors may be simple switches) or a 1-Wire bus, which requires a hub to communicate with.

It may also embody some basic configuration/management, such as “reset”, and then link off to a device-specific management page, served up by the device itself or by a type-specific device hub.

The device manager will represent the devices to the outside world, at least for read operations, isolating them from frequent requests for updates (early Proliphix documentation recommended restricting the API request rate to “a few requests per minute”). You’d want the manager to offer an RSS feed, not the devices themselves. The device manager could also implement some level of security/access permissions.

It’s more than a simply proxy, however… I’d expect it to also know the assigned name, location and type/intended purpose of each device, and provide views/filters around that: “Show me the state of all of the security devices on the second floor”. This implies a device registry or directory. My guess is that you wouldn’t expect or want most devices to handle this metadata on their own.

Finally, you’d also expect that the device manager would implement some level of APIs and scripting, for sophisticated eventing and notifications. Thinking this through, it’s clear that the device manager would need to implement ‘psuedo variables’ representing it’s view of the current state of the devices it’s managing. And, furthermore, what’s just been described here could be approximated by any of several Home Automation systems… depending on the approach, it wouldn’t necessarily be as elegant as you’d like, but it could be done.

USE VS CONFIGURE

In this discussion so far, I’ve not made too much of a distinction between the kinds of access that you’ll need to your devices. Sometimes you want to “configure” a device, and sometimes you want to “use” it. For a thermostat, “configuration” means things like: setting up the kind of heater (single-stage / double-stage, heater only or heater/AC, etc), the setback schedule, and so on. “Using” the device is typically a simple affair: override the current setpoint, for a specified period of time.

You’d generally expect (but I’m not sure I can prove it) that the number of “configuration” options to be at least equal to, or greater than, the number of “use” options, especially as the level of capability of the device increases: more functionality implies more state, and possible actions based on that state, which implies more configuration.

You could imagine entirely separate paths for “configuration” and “use”; in fact, in many cases, it may be very desirable to ensure this. Adhering to the “simpler is better” maxim, you’d want to keep the UI for “use” as simple as possible, and keep “configuration” UI, which might require different paradigms for efficient management, separate.

REMOTE UI

With the Proliphix Thermostats I use, you “configure” via embedded web pages. The configuration experience, delivered via browser, is fairly sophisticated (for a thermostat!). On the other hand, the on-device display and associated 5 buttons are focused almost completely on what the end-user needs: what’s the current temperature, what’s the current setpoint, and how do I get some heat? Some of the UI is dedicated to read-only access to basic configuration info (such as the current IP address), but that info is there to help the user provide good hints to the maintenance team.

In any case, it would be very hard to imagine an on-device user experience for, say, specifying setback schedules for specific days of the month or holidays using the on-device display and buttons. As the designer for the device, you wouldn’t even try. Being able to “express” yourself - as the designer of the “configuration” experience - via web pages means you can expose a lot of useful features in a natural way.

The inverse is probably true, too: don’t build it in the hardware if it can’t be configured by the admin. The embedded web server, while adding $20-$40 to the build cost of the device, means you can offer a lot more functionality and charge for it.

(An aside: what about the “blinking 12:00 AM” VCR clock problem of years past? What this solely due to poor UI design? Or should we just blame the user? Later-model VCRs learned to pick off a time signal from the TV signal, which apparently eliminated the problem for most folks. But if you had this UI challenge in front of you today - not just getting the user to set the correct local time, but also, say, the task of programming the VCR to record a program - would you push it off to a web-based experience, even if it increased the hardware costs of the device by, say, $50?)

So, even though the device in this case is relatively small, the configuration experience is ‘outsized’. Even as devices decrease in size while increasing in capability, they could offer an ‘outsized’ user experience.

If the device (or its hub) supports APIs, you could imagine more than one flavor of UI, in support of various scenarios: beyond the built-in “configuration” and “use” UIs, multiple other “use” UIs could be supported, via APIs, in the form of other web experiences or apps. A device manufacturer might leave the heavy-lifting - the implementation of an elegant end-user UI, for instance - to third-party developers who specialize in that sort of thing.

A colleague of mine imagines a “Facebook for Devices”, a kind of third-party portal where you can see recent updates from your devices and others of interest - and casually pivot on the data in various ways, potentially offering you an easier way to keep track of your 100 devices.

These third-party or extended experiences don’t have to be limited to web sites; imagine a display-oriented device with the sole job of displaying RSS or Twitter-like feeds from multitudes of other devices.

A PIVOT TOWARDS THE CLOUD?

If all of the devices you care about are under one roof, then you could imagine using dedicated device to act as the device manager for all of your devices. It could be implemented via a low-end PC or embedded PC, running headless, with a connection for the local area network, and perhaps dedicated I/O connections for specialized types of devices (such as 1-Wire). You’d access its web page via your LAN. Its device directory, and device data archive, could be based on permanent storage on the device itself or elsewhere on your LAN.

An alternative approach might be for the device manager based in the cloud. You’d likely still need a local agent to connect your home devices, likely stuck behind a firewall/NAT, through to the cloud, and, of course, to ensure that non-TCP/IP-based devices are proxied adequately to the LAN and then on to the cloud.

The resource requirements for a cloud-based manager are likely to be small (just as those of a home-based manager would be), perhaps modest enough that you could use the recently-announced AWS “Free Usage Tier“, which offers a non-trivial level of resources at no cost.

Going to the cloud might mean improved reliability… if your home-based device manager falls over, it’s up to you to detect it, diagnose it, fix it, and get everything running again, while, say, in the AWS cloud, if a disk dies, you’re never supposed to notice it. They have professionals on the case.

Accessing your cloud-based device manager - the UX or the APIs - may be more performant than a home-based manager, especially if you’re outside your LAN, since it’s likely that the cloud-based manager has better peering and connectivity than the slow-ish uplink of your consumer-grade broadband connection. And if it’s possible that your device manager might experience multiple simultaneous requests on a regular basis, putting in the cloud may yield better results.

A cloud-based approach also feels more elegant when the scope of devices goes beyond “the home”. If you include your mobile devices, or devices from your business or other organizations (such as local weather stations or similar services representing virtual devices that you use for more sophisticated eventing, etc), having a cloud-based device agent will make all of this easier, for the reasons already listed: reliability, bandwidth (lower latency, higher thruput, more simultaneous connections), as well as: the availability of an infinite amount of CPU and disk resources, as well as services or capabilities that may be too complicated to contemplate for home hosting. (My same colleague suggests that in-home cameras could serve up images or video that is shipped to the cloud for sophisticated vision processing that couldn’t be done either by the camera or by any software/hardware that’s likely to be installed in the home.)

Of course, a possible downside is that if your home’s broadband connection fails, the Cloud-based portal may miss device updates and therefore may not fire important events. Perhaps a hybrid model is in order, where critical events are handled locally, with the cloud portal handling most other events.

This suggests a future architecture where the cloud is the central management point, with local support - in the form of hardware/software - in the home to handle the vagaries of getting all of your devices connected to the cloud. Again, there are legitimate concerns regarding loss of connectivity, and perhaps security.

Moving some or all of the management of a device mesh to the Cloud is an interesting enough scenario that I’ll be trying it out as soon as I can. More on this later.

A PIVOT TOWARDS “SERVICE AVATARS”?

OK, so we’re now used to the idea that our devices are talking to the cloud, updating their Facebook or Twitter status. In the discussion above, it was the devices that connected up to the cloud, in pursuit of new scenarios that made the device more valuable. But what if the intention involves going the other way? What if cloud services extended down to the devices, in service of new scenarios?

Mike Kuniavsky in his book, “Smart Things: Ubiquitous Computing User Experience Design” (Amazon) describes “Service Avatars”, a term he coined that conveys the added value a focused-purpose device can bring to a service. He holds up the Apple iPod as a prime example of a Service Avator… while it wasn’t the first music player, it was the first to successfully deliver a service-based scenario - a cloud-based music store - down to a device. The device was cool, but it became so much more important when connected to its music service. (We can safely elide certain details, such as the fact that for full functionality, the device requires an intermediary “hub”, in the form of a Mac or PC running the iTunes application).

Notes

#1: I just checked my router. It’s managing DHCP addresses for 15 devices (PCs, fileservers, phones, a VOIP adapter, access point, A/V components, weather station, and a printer). I’ve assigned fixed addresses for another 11 devices (thermostats, Ha7NET, camera server, security system, etc). That adds up to 26 IP addresses, and 26 corresponding devices. If you throw in the other devices of varying smartness…

  • 1-Wire temperature sensors (5)
  • Security sensors (10 or so)
  • Z-wave light swithces (10)

… as of now, I’m up to at least 51 devices. So 100 devices isn’t too far a stretch for a “smart home of the future”.

#2: Wait! Here are 20 more “smart home of the future” devices:

  • Connected exercise equipment (1 per house)
  • Roving security or health robots (1 per house)
  • Connected appliances, which report energy usage, low supplies, or general health statistics (oven, microwave, refrigerator, dishwasher, washer, dryer, furnace, hot water heater) (10 per house)
  • Resource-monitoring equipment, such as: water metering, electricity metering, natural gas or oil metering (oil tanks that report when they’re nearly empty, or leaking!) (3 per house)
  • Smart sprinklers that only water when absolutely necessary, and never after a rainfall, and only when given the OK by the local water board, augmented up by soil moisture sensors (5 per house),

#3: I’m writing a portion of this post on the eve of that date in the Fall in the US where we “Fall Behind” with our clocks. How many devices in the future home will need resetting every Spring and Fall? You’d hope the answer is “zero”… I take a small perverse pleasure in noting, with each passing Fall and Spring, whether the number of ‘things’ around the house that need to be manually pushed back or forward an house, is decreasing.