Sensor Fusion Comes old

Over time, sensors have morphed from simple analog and mechanical constructs to chip-based digital devices that connect with a piece of equipment to monitor the machine's health as well as environmental conditions. Similarly, sensor fusion – multiple kinds of sensors working together to solve a problem – has combined the threads of many other technologies to produce something very new and exciting.

The concept of using a computational device to sort out the information from multiple sensors and combine information to draw a conclusion has been around since a minimum of the 1950s. However it was exceptionally difficult to do. Around 1960, several mathematicians developed sets of algorithms in order to have a machine draw a conclusion based on input from multiple sensors. These filters also removed meaningless data from noise or any other sources. Obviously, it wasn't well before the military decided this technology would be beneficial in their applications. Being able to process inputs from multiple sources and compare them with stored data would allow the military to higher track and identify potential airborne targets and even compute the certainty from the results. With better computers and sensors, we've got the technology was advancing, but there were still complex and dear problems to resolve.

Potential Applications

When the microprocessor first became available, people described it as an answer in search of problems. Exactly the same case can be made for sensor fusion. If you have the power and intelligence to monitor multiple sensors, analyze the information in real-time, and only provide a simple direction or control an action, heretofore-unthought-of applications could be almost limitless. The following examples just scratch the surface:

• Health monitoring – including healthy athletics, patient monitoring, and research
• Monitoring seniors – wellness monitoring to lessen the responsibility of staffing
• Automotive, transportation systems – monitoring and controlling efficiency and safety functions
• Public safety – identification of potentially hazardous conditions with much greater accuracy than simple fire and security systems
• Entertainment – gaming, including controllers and virtual reality headsets
• Weather – intelligent weather forecasting stations, that does not only warn of fixing conditions, but control systems to organize for any storm (for instance, closing storm shutters, closing valves, etc.)
• HVAC/Air Quality – intelligent control of room temperature, humidity, quality of air, system maintenance, etc.

While many of these kinds of capabilities have existed in certain form for several years, the ability of a method to see multiple sensors and are available to an intelligent conclusion, and even initiate action, is revolutionary.

A Convergence of Technologies Fortunately, as they have done in other parts of electronics, numerous IC manufacturers have taken on the task to do the heavy lifting. With off-the-shelf sensor fusion and sensor hub chips, it is now easy to efficiently interface to a variety of digital sensors, along with other pathways. The responsibility of creating your personal algorithms continues to be eliminated.

While their terminology does vary a bit, a number of IC manufacturers have either adapted existing lines of products or created entirely new ones to tackle sensor fusion tasks. The processing is performed with a specialized controller chip, which may be identified as an MCU, a sensor hub or a sensor fusion processor. We're already seeing this technology used in the customer market in smartphones, activity monitors along with other devices.

The latest generation of smartphones from Apple, Samsung, yet others contain powerful and diverse sensing capabilities, without need for external interfacing. Included in this are a three-axis magnetometer, a three-axis accelerometer, along with a three-axis gyroscope. This combined capability is often referred to as 9-DoF, nine examples of freedom.

For probably the most part, these functions are \”always on\” in a cell phone. If the processing of data from these sensors was managed by the phone's central microcontroller, life of the battery would be significantly shortened. Instead, highly efficient dedicated MCU chips process the information as sensor hubs, using a fraction of the power. The NXP ARM M3 series of MCUs is one example. Based on Chipworks, a product teardown specialist, as reported by EETimes, Apple uses a customized form of the NXP chip to watch its sensors in the iPhone 5S. \”The M7 controls functions from the variety of discrete sensors including a gyroscope, an accelerometer, and a compass.\” Samsung takes on the same task with a microcontroller from Atmel, the Core 8-bit AVR MCU.

With such powerful on-board sensing technology, apps are appearing that take advantage of the 9-DoF mobile phone hardware to supply health insurance and activity monitoring in order to function in concert with GPS and external data to provide much more information for the user. Now, increase those already-diverse sensor inputs data from an external device that communicates via Bluetooth and the capabilities seem limitless. The goal of the chip manufacturers would be to make it practical for engineers to design systems that provide real-time sensor data which may be accustomed to supply the desired contextual awareness with minimal power consumption and maximum battery life. Beyond smartphones, highly optimized solutions can address such applications as tablets, Ultrabooks, IoT-enabled devices, gaming, healthcare, environmental monitoring, and wearable computing.

Development boards are available that permit design engineers to simply obtain feet wet within this technology. One such example may be the ATAVRSBIN2 by Atmel. Atmel has embraced sensor fusion with a wide variety of products, which they call \”the Complete Sensor Ecosystem.\” Atmel identified the simultaneous analysis and fusion of information from different sensors and sensor types was not an activity it might handle solo. To obtain past these complexities, the organization partnered having a number of leading sensor manufacturers and sensor fusion specialists to provide a complete, easy-to-implement Sensor Hub Solution.

A current trend combines an MCU with three or more MEMS sensors in a single package. One example is STMicroelectronics' LIS331EB, which combines a high-precision three-axis, digital accelerometer with a microcontroller in a single Thrice 3 x 1 mm package. The microcontroller is definitely an ultra-low-power ARM Cortex-M0, with 64-Kbyte Flash, 128-Kbyte RAM, embedded timers, 2x IC (master/slave) and SPI (master/slave). The LIS331EB may also internally process data sensed by external sensors (for a total of nine), for example gyroscope, magnetometer, and pressure sensors. Functioning as a sensor hub, it fuses together all inputs with the iNEMO Engine software. STMicroelectronics' iNEMO engine sensor fusion software suite applies some adaptive prediction and filtering algorithms to create feeling of (or fuse) the complex information coming from multiple sensors.

Freescale also provides an item type of devices that combine MCUs and sensors in one package. Their FXLC95000 Xtrinsic Motion-Sensing Platform integrates a MEMS accelerometer and a 32-bit ColdFire MCU. Similar to the STMicroelectronics device, the FXLC95000 can simultaneously manage data from external and internal sensors. Freescale was the very first company to market an MCU with a sensing hub embedded that is also programmable for customer-specific applications and algorithms. As much as 16 sensor inputs could be managed by a single device, allowing calibration, compensation and sensor functions to be offloaded from the application processor. It functions with either Freescale or third-party drivers.

Fusion Meets the Cloud

While quite a lot of functionality can be achieved at a local level, interaction using the Cloud is how the enjoyment really begins. Remote sensor data can be processed with a sensor fusion tool and delivered to the Cloud for recording, further analysis, or perhaps to order action.

For example, an unattended pump operating in a remote location is always at a hazard of failing. A few years ago, an online sensor might have been in place to identify whether it was running hot or had even failed. Now, the same pump may also be monitored for vibration, exhaust chemistry, bearing noise, and the external conditions around it. An established program could empower the sensor fusion controller to shut on the pump or even cycle its operation until a technician can arrive. The system would also know in advance whether it is likely the entire pump must be replaced or just an element. Here, a sensor fusion solution could eliminate downtime as well as costly emergency service calls, and even collect data to analyze how well the pump is working overtime. The same general idea pertains to monitoring an aircraft engine flying, a building elevator, or just about anything mechanical.

Another use of the Cloud is perfect for the sensor fusion to take place there, rather than on-site. With open-source sensor fusion software available, individual sensor data can be transmitted to a server, in which the processing would take place.

Conclusion

Sensor fusion is a technology that has come old, and at just the right time to take advantage of developments in sensors, wireless communication, along with other technologies. Once out of reach of basically the most advanced government labs, the technology is now available off-the-shelf, at prices that even fit into the BOM budget for many consumer products.

Now closely associated with mobile technology and the rapid growth and development of lower-cost digital sensors, sensor fusion is poised for explosive growth. For the design engineer, it is a great time to use some creativity and also to start experimenting!

Article courtesy: Mouser Electronics