Machine Data for End-to-End IoT System Monitoring

Last updated at Mon, 06 Nov 2017 18:30:33 GMT

The Internet of Things (IoT) is really the morphing of physical devices into a software world. This means a lot of the same things you would do with applications and servers, you can now do with devices, including leveraging data to respond faster to issues, track trends, and improve your roadmap.

The Internet of Things, Internet of Everything, Machine-to-Machine (M2M), Smart Systems etc. all refer to devices or gadgets that are intelligently connected to their users and the web, in order to provide additional value. And these devices are data hogs. They leverage an array of data gathering components: embedded sensors, actuators, etc. that generate mounds of useful information. Which means there is a huge opportunity to use this data to improve future versions, and respond to the experience and needs of users.

IoT devices are all around us. As the ‘Connected life’ evolves, it is estimated that by 2020, there will be around 50 billion devices connected to the web. A few early adopter examples are devices that control house lighting (Philip’s hue), locate missing keys (Cobra Tag), sense air quality (air quality egg), and auto-diagnose vehicles (Delphi Connect).

And, when you bring these devices together with a log analysis platform, organizations can easily collect, analyze and understand device activity and performance via the machine data.

Leveraging Machine Data for End-to-End IoT System Monitoring

Logs play an important role in troubleshooting, debugging, monitoring, maintaining, and securing systems. When it comes to IoT, the connected devices are expected to work with little human intervention. But this means that there is very little opportunity to learn about the device adoption, performance and user experience without automatically collecting it.

The machine data in the form of application logs, configuration files, message queues, logging API, SCADA data, audit logs, syslog etc. can be collected in a very similar way to how it is done with server logs because most IoT devices are running a form of Linux. Common log analysis agents, such as  the Logentries agent, can be installed directly on the device, and ship with each unit. If there is a special and/or limited embedded OS, it is very easy to create separate application logging processes, which store logs in local ram until they can be transmitted to the log platform via REST API.

Use Cases:  How IoT Data is Changing The World

There is no need to wait for the future. Examples of devices, and the ways their log data are providing better solutions for the market, are everywhere.

Healthcare

Patient health can be monitored using a simple wearable device. An example are devices designed to maintain elderly user freedom and independence, while providing peace of mind to family members and caregivers. The device sends individual eating and medicine habit data to a cloud analytics platform. The system unobtrusively learns a person’s routine over time, and alerts family and friends of changes that may indicate a problem. It also allows the cloud platform to be aware of the device availability and function, making sure it continues to work as expected, and alerts users and the manufacturer when it is not.

Smart Cities

Solar-powered and WiFi connected trash compactors are used for smart detecting of rubbish levels. They provide great value to cities when planning for future needs, and improving waste collection efficiencies. Waste management facilities use historical data to plan collection activities and adjust accordingly. They also want to make sure that all the devices are responsive, as a missed reading could mean missed pickup, or a neglected bin altogether.

Home Automation

The popular Nest thermostat helps monitor and control home temperature. The sensors sense the presence of an individuals home temperature, humidity, current weather conditions, etc. This data is collected and used to learn the daily routine of users and their temperature preferences. Nest uses the data to enhance the accuracy of the training algorithms. They also leverage AC and Furnace details to help users troubleshoot complex problems over the phone with trained HVAC support personnel.

Logistics

There are a variety of in-vehicle technologies for monitoring, navigation and remote diagnostics. These tools can be used to estimate destination arrivals which help optimize routes. They can also report on weather conditions and other factors such as engine trouble, that might delay a shipment. The data is critical not only for optimization, but also important for litigation such as in accidents, or to make sure that delivery times are within promised windows. We are developing the logistics market’s version of the SLA.

At the very least, every device is going to be connected via bluetooth with your phone. But more likely connect to the web directly via their own cellular and WiFi modems. And the users of these devices are going to expect that it is responsive to their needs, any issues are addressed quickly, and updated when there are new fixes and functionality available. The only way to give these users what they want, and get a clear picture on how devices are being used is strong log data, and strong analysis behind that data.

Any more physical devices are just software applications. The devices themselves have robust components all controlled by a complete suite of operating system, applications, cloud synchronization etc. And this is where the data from the devices can be utilized in a similar way you would any other web application.

By looking at system logs, configuration files, or even analyzing battery degradation over time, you have a clear picture of what the devices are doing out in the wild. You can keep users current with updates, and bug fixes. And finally you can enhance your roadmap with real-time data.