Applying edge AI to predictive maintenance in robotics
Production line downtime can cost tens of thousands of dollars per minute. So how do we better anticipate robot failures to optimise output? In this article we demonstrate a method to rapidly acquire industrial robot data for training and testing lightweight ML models, which we then apply to a predictive-maintenance use case.
Previously, capturing robot data was a complex and fragmented task. However, with the recent announcement of Forge Edge by READY Robotics, it's now possible to access robot data streams using a single standardised interface for hundreds of models of industrial robots and cobots (from ABB, EPSON, FANUC, Kawasaki, Stäubli, Yaskawa, Universal Robots, and more). Using the Edge Impulse edge AI platform, we were able to ingest all the sensor data, and train and test an accurate model in under one hour. Deploying inferencing at the edge provides latency, privacy, and reliability advantages for our application.
We also included Balluff Condition Monitoring sensors in our system to provide accelerometer data and other insights into the workcell.
#"I'll be back" (with some sensor data)
We're using a 6-axis robot, which has six individual motors driving each of its joints. Each of these motors has position and torque sensors providing feedback to the joint controller. Increased torque is needed to compensate for worn mechanical gears, so our hypothesis was that this should be detectable by machine learning using torque sensor data from the robot joints. We were able to use Forge Task Canvas to associate the program state with the sensor readings, giving us greater insight into the workcell condition.
We obtained data streams for position and torque for each of the six rotational joints, and the Balluff sensor, using READY Robotics' ForgeOS 5. We found a sample rate of 10Hz to be sufficient, although higher rates are possible. ForgeOS uses protobuf for efficiency internally, but the recently announced Forge Edge provides the same data as a socket.io stream.
#Sending training data to Edge Impulse
We relayed the live robot data to Edge Impulse using a small Node.js client based on their Ingestion API code example. We also integrated with Edge Impulse Studio using the very nice Remote Management Protocol. Now, when a user presses the capture button in an Edge Impulse project, live robot data appears. Magic!
#A tale of two robots
We sourced two identical robots, one new and one old with known joint issues. Ideally, we would do a longitudinal study with just one robot degrading over time to eliminate the possibility of unanticipated variables, but we'll save that for another blog.
We sampled sensor data from two robots running an identical program, with samples taken on identical movements. These were labelled "good" and "bad" according to which robot they were acquired from. On visual inspection, the two sample sets did look very, very similar. Small variations aside, your humble author had to double-check that they weren't collecting data from the same robot twice.
#Feature Explorer
Edge Impulse's Feature Explorer automatically detected that Torque RMS (Root Mean Squared) was distinctly higher for the bad joints on our old robot. This reflects the fact that a greater amount of torque was applied over the entire sample period, even though there were no obvious peaks in the raw data. This is easily seen in the Feature Explorer graph, giving us confidence that our algorithms will be able to distinguish between the two cases.
#Anomaly detection
Next, we passed the features into the Anomaly Detection (K-means) algorithm. The idea is that we train the algorithm with data from a known good system; it then monitors the system, alerting us if any new data looks unusual. This method uses unsupervised learning, so the labels we applied to the samples are not needed by the algorithm. Instead, we ensured the training set only contained "good" samples, and the algorithm learned that the entire training set was expected behaviour. The anomaly score is based on how far a new item is from the centre of clusters of that training data.
Testing the anomaly detection showed 98.75% accuracy. This is a great start, and with more refinement it could form the basis of automated anomaly detection for our robot in deployment.
#Training a neural network classifier
Next, we tried deep learning. The NN Classifier uses supervised learning, which means it takes labelled training data as an input, so we moved a mix of both "good" and "bad" labelled samples back into our training set. After a minute or so, the Edge Impulse platform had trained a TensorFlow Lite classifier based on our robot data.
The confusion matrix showed that, against the validation set, our initial model had an accuracy of 96.6%. In fact, the model always succeeded in identifying the "bad" robot. Where it fell down was two false positives, where it mistook the "good" robot for "bad". We could add more labels given the context of the robot's operation known by ForgeOS (the program stages, payload, robot state, error codes, and so on), and the classifier could be trained to identify even more robot conditions.
#Conclusion
With Forge Edge, we were able to quickly stream live robot joint and torque data in a standardised format. Combined with Edge Impulse's AI platform, we were able to generate valuable insights from that data in under an hour. At around 20 kilobytes in size, the model was extremely lightweight and could easily be deployed to the same machine running ForgeOS for reliable, low-latency predictive maintenance without requiring an internet connection.
We're excited to be enabling a future where data from industrial robots used in production is finally accessible to modern, high-productivity analysis tools like Edge Impulse. There are a ton of possibilities, and this is just the start.
Also published on LinkedIn.