Design Article
Comment
KerryM
Troubleshoot1
(*) 1980 A multi-story Bank with remote facilities addressed and installed a ...
What Internet of Things needs to become a reality
Kaivan Karimi, Freescale; Gary Atkinson, ARM
10/30/2012 10:57 AM EDT
Internet of Things use cases
When devices can sense and communicate via the Internet, they can go beyond local embedded processing to access and take advantage of remote super-computing nodes. This allows a device to run more sophisticated analyses, make complex decisions and respond to local needs quickly, often with no human intervention required.
Let’s take a look at the most common use cases for the Internet of Things.
Pervasive Remote Tracking/Monitoring and (if needed) Control & Routing (TCC&R): This refers to remote tracking/monitoring and, if needed, command, control and routing functions for tasks and processes today usually done manually, or, if done remotely, that require additional infrastructure. For example, in most homes today, it’s a manual process to turn on and off certain lights, set temperature zones and turn on and off a washing machine. In the future, doors, windows, electrical outlets, appliances and many other types of standalone equipment will become “smart” with a unique ID. Those smart devices can then be connected via wired or wireless communication, allowing a user to monitor his or her house remotely, change settings on a refrigerator or washing machine and control household tasks through a laptop or mobile phone. In fact, there are some services offered today by security or Internet service providers to do exactly that, but on a much smaller scale and with fewer capabilities than we’ll see in the future.
Asset Tracking: An extension of these kinds of services is asset tracking, which today is done via barcode and a variety of manual steps, but in the future will leverage smart tags, near-field communication (NFC) and RFID to globally track all kinds of objects, interactively. The word geo-tagged is now being used by some companies to refer to this class of applications. In a future scenario, a user would be able to use Google Earth to track anything with an RFID tag. Alternatively, your refrigerator could keep track of your smart-tagged groceries and tell your cell phone app you’re low on a certain item. If your bag of frozen vegetables can have a smart tag, other objects such as valuable cars, jewelry and handbags could too, and they could be tracked via the Internet and also take advantage of a variety of available web-based applications. Some telehealth-related services also belong in this category. The graphic shown below gives an example of how even a pill’s progress through the human body could be remotely monitored.
Process Control and Optimization: This is when various classes of sensors (with or without actuation capabilities) are used for monitoring and to provide data so a process can be controlled remotely. This could be as simple as the use of cameras to position boxes of various sizes on a conveyer belt so a label machine can properly apply labels to them. This task can be done in real time by sending the data to a remote computer, analyzing it and bringing a “command” back to the line so various “control” actions can be taken to improve the process … without any human intervention.
Resource Allocation and Optimization: The smart energy market provides an ideal example of this use case. The term smart energy has been used in many ways, but it basically refers to accessing information about energy consumption and reacting to the information to optimize the allocation of resources (energy use). In the case of a household, for example, once the residents know they’ve been using their washing machine during peak hours when the grid is most constrained and the cost of electricity is at premium, they could adjust their behavior and wash their laundry during non-peak hours, saving money and helping the utility company cope with peak demand.
Context-aware Automation and Decision Optimization: This category is the most fascinating, as it refers to monitoring unknown factors (environmental, interaction between machines and infrastructures, etc.) and having machines make decisions that are as “human-like” as possible … only better!
Here’s a personal example from Kaivan’s past that can help illustrate this: “When I was a young engineer, I worked on a traffic collision avoidance system (TCAS). In that system, when two airplanes were approaching each other on a collision path, the ‘machines’ in the two airplanes would take over. The system first would send an audible warning to the pilots about the danger ahead, while at the same time communicating between the two planes and deciding how each plane should move to avoid a collision. The assumption was that if the two pilots were warned and were in control to make quick decisions, they could both decide to make turns that would still cause a crash.”
There are a whole host of new technologies available today and in development that could allow vehicles to communicate with each other as well as with a central control unit. These smart vehicles also could sense the road, traffic signs and lane markers and, using GPS and a communication link, avoid incoming traffic, avoid accidents around a curve or, in conjunction with the central control unit, avoid going over a distressed bridge on the verge of collapse.
Remote patient monitoring is another example relevant to this use case. For instance, imagine an implantable sensing node that tracks biometrics and sends a signal regarding an abnormal readout for an elderly patient. If the patient doesn’t respond by taking a medication, the node could place an emergency call to a contact from a list, and if there’s no answer, call a second contact, and finally, if no answer, contact a monitoring clinic or quickly provide other emergency assistance. Another example is continuous monitoring of chronic diseases to help doctors determine best treatments, with minimal human intervention.
Requirements common to all of the use cases above include:
In the factory automation example mentioned above (applying labels to boxes), a camera detects information using a charge-coupled device (CCD) sensor (sensing node), the collected data is then communicated to an embedded processor/controller (embedded processing node) using wired or wireless communication technology (connectivity node), a decision is made by the remote server (remote embedded processing node) and communicated (connectivity node), which causes a mechanical action to take place that corrects the situation.
For a Context-aware Automation and Decision Optimization example, we can use the example of a smart car, using its active safety radar system (sensing node), in conjunction with image processing cameras (sensing nodes), it communicates with an embedded processor (embedded processing node) in the center stack of the car to make an appropriate decision regarding danger ahead. Or, the vehicle could leverage its built-in GPS and wide-area-network (WAN) wireless communication capability (connectivity node) to pass along information to a central processing server on the network (in the cloud) (remote embedded processing node) that could then make the car aware of the information it had just received, from the sensors of an unstable bridge (sensing node) on the road ahead, that was being pounded by rain flood and loosing its structural integrity, and hence guide the car to a different route to avoid danger.
When devices can sense and communicate via the Internet, they can go beyond local embedded processing to access and take advantage of remote super-computing nodes. This allows a device to run more sophisticated analyses, make complex decisions and respond to local needs quickly, often with no human intervention required.
Let’s take a look at the most common use cases for the Internet of Things.
Pervasive Remote Tracking/Monitoring and (if needed) Control & Routing (TCC&R): This refers to remote tracking/monitoring and, if needed, command, control and routing functions for tasks and processes today usually done manually, or, if done remotely, that require additional infrastructure. For example, in most homes today, it’s a manual process to turn on and off certain lights, set temperature zones and turn on and off a washing machine. In the future, doors, windows, electrical outlets, appliances and many other types of standalone equipment will become “smart” with a unique ID. Those smart devices can then be connected via wired or wireless communication, allowing a user to monitor his or her house remotely, change settings on a refrigerator or washing machine and control household tasks through a laptop or mobile phone. In fact, there are some services offered today by security or Internet service providers to do exactly that, but on a much smaller scale and with fewer capabilities than we’ll see in the future.
Asset Tracking: An extension of these kinds of services is asset tracking, which today is done via barcode and a variety of manual steps, but in the future will leverage smart tags, near-field communication (NFC) and RFID to globally track all kinds of objects, interactively. The word geo-tagged is now being used by some companies to refer to this class of applications. In a future scenario, a user would be able to use Google Earth to track anything with an RFID tag. Alternatively, your refrigerator could keep track of your smart-tagged groceries and tell your cell phone app you’re low on a certain item. If your bag of frozen vegetables can have a smart tag, other objects such as valuable cars, jewelry and handbags could too, and they could be tracked via the Internet and also take advantage of a variety of available web-based applications. Some telehealth-related services also belong in this category. The graphic shown below gives an example of how even a pill’s progress through the human body could be remotely monitored.
Process Control and Optimization: This is when various classes of sensors (with or without actuation capabilities) are used for monitoring and to provide data so a process can be controlled remotely. This could be as simple as the use of cameras to position boxes of various sizes on a conveyer belt so a label machine can properly apply labels to them. This task can be done in real time by sending the data to a remote computer, analyzing it and bringing a “command” back to the line so various “control” actions can be taken to improve the process … without any human intervention.
Resource Allocation and Optimization: The smart energy market provides an ideal example of this use case. The term smart energy has been used in many ways, but it basically refers to accessing information about energy consumption and reacting to the information to optimize the allocation of resources (energy use). In the case of a household, for example, once the residents know they’ve been using their washing machine during peak hours when the grid is most constrained and the cost of electricity is at premium, they could adjust their behavior and wash their laundry during non-peak hours, saving money and helping the utility company cope with peak demand.
Context-aware Automation and Decision Optimization: This category is the most fascinating, as it refers to monitoring unknown factors (environmental, interaction between machines and infrastructures, etc.) and having machines make decisions that are as “human-like” as possible … only better!
Here’s a personal example from Kaivan’s past that can help illustrate this: “When I was a young engineer, I worked on a traffic collision avoidance system (TCAS). In that system, when two airplanes were approaching each other on a collision path, the ‘machines’ in the two airplanes would take over. The system first would send an audible warning to the pilots about the danger ahead, while at the same time communicating between the two planes and deciding how each plane should move to avoid a collision. The assumption was that if the two pilots were warned and were in control to make quick decisions, they could both decide to make turns that would still cause a crash.”
There are a whole host of new technologies available today and in development that could allow vehicles to communicate with each other as well as with a central control unit. These smart vehicles also could sense the road, traffic signs and lane markers and, using GPS and a communication link, avoid incoming traffic, avoid accidents around a curve or, in conjunction with the central control unit, avoid going over a distressed bridge on the verge of collapse.
Remote patient monitoring is another example relevant to this use case. For instance, imagine an implantable sensing node that tracks biometrics and sends a signal regarding an abnormal readout for an elderly patient. If the patient doesn’t respond by taking a medication, the node could place an emergency call to a contact from a list, and if there’s no answer, call a second contact, and finally, if no answer, contact a monitoring clinic or quickly provide other emergency assistance. Another example is continuous monitoring of chronic diseases to help doctors determine best treatments, with minimal human intervention.
Requirements common to all of the use cases above include:
- Sensing and data collection capability (sensing nodes)
- Layers of local embedded processing capability (local embedded processing nodes)
- Wired and/or wireless communication capability (connectivity node)
- Software to automate tasks, and enable new class of services
- Remote network/cloud-based embedded processing capability (remote embedded processing node )
- Full security across the signal path
In the factory automation example mentioned above (applying labels to boxes), a camera detects information using a charge-coupled device (CCD) sensor (sensing node), the collected data is then communicated to an embedded processor/controller (embedded processing node) using wired or wireless communication technology (connectivity node), a decision is made by the remote server (remote embedded processing node) and communicated (connectivity node), which causes a mechanical action to take place that corrects the situation.
For a Context-aware Automation and Decision Optimization example, we can use the example of a smart car, using its active safety radar system (sensing node), in conjunction with image processing cameras (sensing nodes), it communicates with an embedded processor (embedded processing node) in the center stack of the car to make an appropriate decision regarding danger ahead. Or, the vehicle could leverage its built-in GPS and wide-area-network (WAN) wireless communication capability (connectivity node) to pass along information to a central processing server on the network (in the cloud) (remote embedded processing node) that could then make the car aware of the information it had just received, from the sensors of an unstable bridge (sensing node) on the road ahead, that was being pounded by rain flood and loosing its structural integrity, and hence guide the car to a different route to avoid danger.
|
|
|
|
|
Navigate to related information
SylvieBarak
10/31/2012 2:45 PM EDT
You know what the Internet of Things really needs to take off? Assurance that the security is in place to make it a good thing to use...
Sign in to Reply
lcovey
11/1/2012 12:10 PM EDT
Not so sure about the security angle, Sylvie. Like Helen Keller said "Security is an illusion." It will never be absolute unless humanity suddenly becomes universally sinless.
The real issue of the IoT is the management of data. Current search is based on words and phrases, primarily, but it is constantly being gamed by the SEO industry. We need a software approach that institutes an automated omniscience, more than just artificial intelligence, that brings the right information to the devices when requested. There's a lot of stuff we need to know but it comes to us in semi-truck loads, rather than in appropriate amounts. when someone figures that out, the IoT will take off.
Sign in to Reply
KerryM
5/7/2013 2:29 PM EDT
Sylvie,
I agree that you hit the nail on the head.
lcovey is right that managing the data is a 'big deal'. But unless you can be sure that the data is authentic/accurate/reliable, it's not clear that the data will be of any value. And if I can spoof data to make it look like it came from you instead of me there are all sorts of unpleasant options.
The technology is there - cryptography, hardware security devices, protected protocols - we just need to see them get integrated in the solutions.
Sign in to Reply
kaivan.karimi
10/31/2012 3:34 PM EDT
You are absolutely right Sylvie. With Millions/Billions of devices hanging off the "universal neural net", the potential security issues can be disastrous….…. No one likes to see something benign like their home electricity hacked. Imagine what can happen to larger mission critical infrastructures if they become the target. Security for IoT is perhaps the most aspect of the whole thing.
Sign in to Reply
Max the Magnificent
10/31/2012 3:47 PM EDT
See also my blog: IPv4, IPv6, The Internet of Things, 6LoWPAN, and lots of other “Stuff” ( http://bit.ly/VE4fMH )
Sign in to Reply
iniewski
11/1/2012 3:01 PM EDT
Pretty interesting stuff Max, would you be interested in giving a talk on this topic at emerging technologies event in Whistler in 2013? www.cmoset.com, Kris
Sign in to Reply
Max the Magnificent
11/1/2012 3:12 PM EDT
I didn't write this article -- but I can put you in touch with the authors if you wish
Sign in to Reply
expendable crewman #1
10/31/2012 4:44 PM EDT
How long will it be before “We” become part of the “Internet of Things”? As soon as everything about you is able to be tracked, you will not be allowed to be without your connected device. To make sure that doesn’t happen the device will be embedded in you at birth and constantly transmit data about you. Not long after that the data will become bidirectional instructing you how to best serve the collective. Resistance is futile.
Sign in to Reply
Max the Magnificent
10/31/2012 4:45 PM EDT
That's why I never go outside without wearing my aluminum hat
Sign in to Reply
expendable crewman #1
10/31/2012 4:55 PM EDT
Is that the one with the propeller on top?
Sign in to Reply
Max the Magnificent
10/31/2012 4:59 PM EDT
No -- the Propeller Beanie is actually used to hide my aluminum skull-cap (I don;t want people to laugh at me :-)
Sign in to Reply
docdivakar
11/1/2012 12:04 PM EDT
Good article... one thing the authors do not mention is the value from prognostics capability that can be in the embedded processing layer of sensor networks.
MP Divakar
Sign in to Reply
kaivan.karimi
11/2/2012 5:31 PM EDT
Hi MP,
Hope all is well. could you please elaborate more?
Thanks,
Kaivan
Sign in to Reply
docdivakar
11/21/2012 5:30 PM EST
Kaivan, didn't get to this message for a while, sorry for the late reply.
My comment was rather high-level, addressing the local embedded processing layer most often done by MCU/Micro's you were referring to. I think there are many software+hardware solution opportunities, both hierarchical or single stack, that can provide intelligent and correlated information from IoT. I will stop right there (since that is what I am working to protect as IP!). But you were right on track in the article!
MP Divakar
Sign in to Reply
mac_droz
11/2/2012 6:16 AM EDT
Apart from being interesting concept for all the geeks around, why would you need your home to be smart? One of the arguments I hear is that it will save the energy.... what a lot of bollocks. Unless we start building houses that are truly passive (that's the technology to follow and it exists for more than 20 years) nothing will change. If your house is not insulated than the super-duper thermostat with weather prediction and all the IP addressable light bulbs will not make it warmer. What is wrong with standard light switch? Are we too retarded to use it? And remember to switch it off when we leave the room? Or do you guys have 1000W lamps in every room?
Sorry for my rant but every now and then I see those articles about super benefits of smart homes but they are not addressing real problems.
The real problem with houses is to keep them warm/cold depending on the season with as little external energy as possible.
Sign in to Reply
iniewski
11/2/2012 10:57 AM EDT
Good point @mac_droz, the benefits of smart homes were never clear to me...and exactly to your point: I have moved to a brand new house from 15-year old house. The old house was properly build with extra wall insulation. The new house was build to some green standards, have high tech windows, efficient lighting etc. And guess what, walls are thin and my heating being is larger although the new house is less than half the size of the old ones. I can install as much electronics as I want the heat is escaping thru the walls and holes in all windows and doors. I guess I ended up getting a stupid house ;-)...Kris
Sign in to Reply
Bert22306
11/2/2012 4:27 PM EDT
Okay, another "excuse" to utter my mantra: the IoT is nothing more than "more of the same." Like everything else touted as being something brand new, yes, even this "cloud" business, in reality it is an evolution of what just about everyone is already familar with.
The Internet, to get down to the fundamentals, has ALWAYS been about "things." Internet addresses have ALWAYS been the address of an interface to a "thing." Not to a person. To a computer, to a peripheral device such as a printer, to a sensor, to a router, to a server at home or at work or in "the cloud." All things. Even your e-mail "address" becomes translated into the actual IP address of a "thing."
It's only cost and available Internet bandwidth that has kept things more or less under control. But already, you can contact your home PVR over the Internet, you can contact your car over 3G (and get remote diagnostics), easy enough to think of controlling your home heating system remotely over the Internet, and surely everyone knows that your home heating systems are also auto-adjusting (for too many decades to bother mentioning).
Remote and automated factory controls have been available for many years. Remote as well as automated control systems for cars, airplanes, ships, public transportation systems such as Metro, airport ground transportation systems, you name it, have also been with us for a whole lot of time now. Connecting each of these to the Internet, assuming they aren't already connected (many are!), is not a huge leap, is it?
So, if you want to make a few more of your own personal devices ALSO available over the Internet, surely this shouldn't be seen as ground-breaking innovation?
Sign in to Reply
kaivan.karimi
11/2/2012 5:21 PM EDT
Hi Bert22306,
You are absolutely right that all of this can be done today from remote comand and control basis, and IPv6 will make it more pervasive. This said, there are a ton of services being worked on that leverages these capabilities today, that VCs weren't spending money on before, and now they are. IPv6 along with a set of security standards, maybe the tipping point that leverages most of what is there today, and come up with a whole new set of services.
all the best,
Kaivan
Sign in to Reply
Bert22306
11/2/2012 5:38 PM EDT
Kaivan, I agree that IPv6 can make these Internet-connected devices "more pervasive," simply because it has so much more address space. But a problem might be that people will misconstrue that comment.
You don't HAVE to use IPv6 to achieve most of these capabilities. And the security measures available in IPv6 are also available in IPv4. The very same security protocols.
So as far as I'm concerned, people should not go off assuming that this IoT was practically impossible until now.
One simple technique, which has worked for decades, is that previously isolated control system networks are connected to the Internet via a gateway device. The gateway device is easily capable of accepting messages from the Internet at an interface with an IP address, and then translating that address into the scheme used by the previously isolated control system net. And vice versa, for messages originating from the control system net.
With that technique, as well as more standardized techniques such as NAPT, you can greatly expand the usefulness of even the 32-bit IPv4 address space.
It' true that refrigerators and toasters have not typically been connected to the Internet in the past, so perhaps the average joe didn't know this could be done. However EET readers should not be bowled over by any of this.
Sign in to Reply
kaivan.karimi
11/9/2012 5:04 PM EST
Hi Bert,
IoT is nothing but a collection of evolutionary steps that most of the technologies are available today, however collectively it enbales new classes of services....the "whole" at that point will be much greater than the sum of the pieces.
BR,
Kaivan
Sign in to Reply
iniewski
11/2/2012 4:32 PM EDT
I agree @Bert...IoT is just a new buzzword, along the lines of many other ones (like cloud computing which was done years earlier without using the term)...marketing news buzzwords though, otherwise we would be lacking any exciting topics to talk about ;-)
Sign in to Reply
kaivan.karimi
11/2/2012 5:16 PM EDT
Hello @mac-droz and iniewski,
Hope all is well. @mac_droz has a great point about passive houses with the right insulation, as indeed heating and cooling are the largest expense for most home, and account for over 50% of the energy use in a typical U.S. home. You don’t need a “smart home” to keep cost down on your heating, cooling or lighting for that matter. There are a couple of cases, that a “connected” home with visibility into their appliance usage can help the grid, hence the concept of energy savings. During the peak hour for cooling or heating, e.g. 2-5 pm mid July in Austin, Texas, when the air conditioner is blasting non-stop, and the grid is experience a peak capacity….at the margins, the cost of producing electricity is a lot more expensive for the utility companies, than what they are selling the electricity to you. If they can track your appliances usage and make sure that during these peak hours you are not also as an example running your washing machine, and incentivize you to run it late at nights, then it’s a win-win for everyone, and cost of energy consumption can be reduced by a reasonable amount. This can only happen when the energy usage can be tracked at a major appliance level at the time of usage, as oppose to aggregate at the meter.
Most other services related to a “smart home” are convenience based, for remote tracking and monitoring and command and control….all of this can be done today, with or without the IoT tag on it… the only thing that makes all of this happen is IPv6, so more unique addresses for more uniquely identifiable devices on the internet.
Iniewski,
I am working on a paper on the role of sensor fusion in IoT that may be suitable for your cmoset. Please send me an email and we can discuss (kaivan.karimi@freescale.com).
All the best,
Kaivan
Sign in to Reply
iniewski
11/2/2012 5:42 PM EDT
thank you Kaivan, I will be happy to discuss, will send you an email in a moment
Sign in to Reply
Chris_999
11/8/2012 4:21 PM EST
Is there a chance to get the images in higher resolution
Sign in to Reply
Max the Magnificent
11/8/2012 4:43 PM EST
Let me see what I can do -- my Internet went down in the office, so I'm working from home -- I'll have a look around tomorrow -- Max
Sign in to Reply
Max the Magnificent
11/9/2012 9:45 AM EST
I just uploaded them as a PDF http://bit.ly/RJnB4J
Sign in to Reply
iniewski
11/9/2012 11:34 AM EST
Max, interesting pictures...I will be editing a book on IoT in 2013, would you be interested in co-editing? kris.iniewski@gmail.com
Sign in to Reply
kaivan.karimi
11/9/2012 5:00 PM EST
Try
http://www.freescale.com/files/32bit/doc/white_paper/INTOTHNGSWP.pdf
I can also send you the ppt version of each graphic. please send me an email kaivan.karimi@freescale.com
and let me know which one you want.
BR,
Kaivan
Sign in to Reply
Chris_999
11/9/2012 5:15 PM EST
Many thanks for following up so quickly with the images!
Sign in to Reply
Max the Magnificent
11/9/2012 6:01 PM EST
I live to serve :-)
Sign in to Reply
Troubleshoot1
3/11/2013 1:14 PM EDT
(*) 1980 A multi-story Bank with remote facilities addressed and installed a 100% Computer Controlled “complex”. It was operational on Day One. Within the next (7) Years additional technology was adapted in other projects that would be very compatible to enhance the First Project. (Interactive Tele-video with data – proven 1966, Holography Proven 1986, To the Penny monetary flows same day as start of SWIFT. Yet another example is the Chicago Mercantile exchange.) All other areas were already functional. Real World Functional operations exist to prove it! Newer devices such as the Apple I-5 and similar have addressed total communications with all these functions at this time. The ONLY limitations are the “VISION” of those involved and the KNOWLEDGE they share!!!
Sign in to Reply