Development of a real-time activity recognition algorithm for the Kinetyx sensory insole system

Authors: Hoitz F.¹, Subramanium A.¹, Nigg S.², Nigg B.¹, Suleman O.³, Blades S.²

1 University of Calgary
2 Biomechanigg Sport and Health Research Inc.
3 Kinetyx Sciences Inc.

Highlight 

Kinetyx has developed an algorithm that can detect and classify your activity in real-time.

Background 

The introduction of smart devices, such as smart phones and smart watches, has made wearable technology ubiquitous in our daily lives. Inertial measurement units (IMUs) [1], a combination of multiple sensors (e.g., accelerometers and gyroscopes), are a key component to smart devices, and can provide valuable data for monitoring human activity [2; 3].

For sport-specific activities such as running, wearable technology can provide athletes with important training data (e.g., mileage, step counts, etc.), potentially mitigating overuse injuries and enhancing performance [4].

Most wearable technology solutions, however, are developed in a controlled laboratory environment and are specifically developed for a single sport and / or exercise [5; 6]. For instance, IMU-based algorithms have been specifically developed to identify foot impacts during running, and to estimate the associated ‘shock’ on the human body [7].

When adopted outside the laboratory setting, these algorithms often fail due to their immutable nature. As a result, wearable technology struggles to adjust to the variability of real-world scenarios and cannot account for multiple activities [8].

Recently, machine learning algorithms have been explored to expand the capabilities of wearable technology, and have shown potential in identifying many different sport-specific movements (e.g., running, jumping, etc.), primarily by utilizing the integrated IMUs [9; 10]. These algorithms may be further enhanced through additional data from ground reaction forces (GRFs), as this data encodes information that is crucial for subject-specific movement characteristics [11]. Such data can be collected through wearable plantar pressure sensors [12; 13].

The newly developed Kinetyx sensory insole is a sensorized system that combines IMUs and plantar pressure sensors (Figure 1).

Thus, the Kinetyx sensory insole system is a unique wearable solution that could provide data for an effective and accurate activity recognition. 

The purpose of this work was to develop a state-of-the-art machine learning algorithm that excels at recognizing a multitude of sport-specific activities using data provided by the Kinetyx sensory insole system.

Methods 

This project was executed in two distinct phases. Phase 1 focused on the development of an activity recognition algorithm. Phase 2 validated a variation of the developed algorithm that was implemented by Kinetyx. For both phases, unique data sets were collected and analyzed.

Participants and protocol 

For Phase 1, the collected data set consisted of 18 participants (5 males and 13 females). For Phase 2, data from twenty participants were collected (10 males and 10 females). All participants were asked to perform a multitude of activities (listed below) while wearing the Kinetyx sensory insole system.

• Running

• Standing

• Squats

• Sit ups

• Burpees

• Cycling

• Push ups

• Walking

• Lunges

The collected activities included activities of daily living (e.g., walking) and various sport-specific activities (e.g., running, cycling, burpees). Participants performed all activities for either 5 minutes (i.e., walking, cycling, etc.) or for a total count of 20 repetitions (i.e., burpees, etc.). Participants were allowed to rest between activities to prevent exhaustion. The order of activities was randomized for Phase 2.

Approval for this protocol was obtained from the University of Calgary’s Conjoint Health Research Ethics Board (REB20-1734), and participants provided written and informed consent prior to testing. The inclusion criteria for this study were: >18 years old and no musculoskeletal injuries in the past six weeks.

Algorithm development

Conceptually, the recognition algorithm was based on a real-time neural network classification that distinguished between the various activities. The algorithm was designed and tested to operate with real-time data. Each trial was divided into time windows over which the classification was applied.

The algorithm’s performance was evaluated in terms of accuracy, which expressed the percentage of correctly classified time windows with respect to the total number of time windows.

Validation

To validate Kinetyx’s implementation of the proposed algorithm, the predicted activity labels were compared to the true labels that were recorded manually during testing. Again, the algorithm’s performance was expressed in terms of accuracy.

Results

Figure 2 highlights the accuracy of the recognition algorithm during the development stage. The recognition algorithm was trained and tested on 16 participants (2 participants were excluded due to bad data) in a leave-one-out fashion and the subject-specific accuracies were averaged. The algorithm's accuracy peaked at 100% for Running and push-ups. Accuracies above 90% were recorded for burpees, cycling, lunges, standing, and walking, while the algorithm performed worst on sit-ups and squats (< 80%).

Figure 3 highlights the accuracy of the algorithm variation that was implemented by Kinetyx during Phase 2. The recorded accuracies were very good (≥ 97%) for all activities except squats and standing, that were recognized with an 81% and 90% accuracy, respectively.

Discussion

This work aimed to develop a state-of-the-art machine learning algorithm that excels at recognizing a multitude of sport-specific activities using data provided by the Kinetyx sensory insole system. The project was executed in two phases, focusing on the development of the algorithm and a real-time implementation, respectively. For both phases, the algorithm’s recognition rates were exceptional for most activities.

However, squats were difficult to identify in both evaluations. It is likely that the decision to focus only on inertial data influenced this outcome. This decision was made because at the time of the study, the pressure-based data collection was still in development. While performing squats, however, the recorded inertial data is comparable to that recorded during standing. As the feet are stationary during both activities, the developed algorithm must have struggled to discern the two activities. Future implementations of the recognition algorithm may address this by including normalized plantar pressure data.

The evaluation of Phase 1 showed that sit ups were also difficult to discern. The evaluation of Phase 2, however, did not show this. Small differences between the two compared implementations of the algorithm may be responsible for these outcomes.

Finally, both phases evaluated data collected in a controlled indoor environment. Future developments should, therefore, consider collecting data in an unsupervised and more natural environment to ensure the algorithm’s robustness.

References

[1] O. J. Woodman, “An introduction to inertial navigation,” University of Cambridge, Computer Laboratory, Tech. Rep., 2007.

[2] L. Pei, R. Guinness, R. Chen, J. Liu, H. Kuusniemi, Y. Chen, L. Chen, and J. Kaistinen, “Human behavior cognition using smartphone sensors,” Sensors, vol. 13, no. 2, pp. 1402–1424, 2013.

[3] G. M. Weiss, J. L. Timko, C. M. Gallagher, K. Yoneda, and A. J. Schreiber, “Smartwatch-based activity recognition: A machine learning approach,” in 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI). IEEE, 2016, pp. 426–429.

[4] Y. Adesida, E. Papi, and A. H. McGregor, “Exploring the role of wearable technology in sport kinematics and kinetics: A systematic review,” Sensors, vol. 19, no. 7, p. 1597, 2019.

[5] V. Camomilla, E. Bergamini, S. Fantozzi, and G. Vannozzi, “Trends supporting the in-field use of wearable inertial sensors for sport performance evaluation: A systematic review,” Sensors, vol. 18, no. 3, p. 873, 2018. 

[6] R. Chambers, T. J. Gabbett, M. H. Cole, and A. Beard, “The use of wearable microsensors to quantify sport-specific movements,” Sports medicine, vol. 45, no. 7, pp. 1065–1081, 2015. 

[7] A. G. Lucas-Cuevas, A. EncarnaciÅLon-MartÅLınez, A. Camacho-GarcÅLıa, S. Llana-Belloch, and P. PÅLerez-Soriano, “The location of the tibial accelerometer does influence impact acceleration parameters during running,” Journal of Sports Sciences, vol. 35, no. 17, pp. 1734–1738, 2017. 

[8] S. Ramasamy Ramamurthy and N. Roy, “Recent trends in machine learning for human activity recognition—a survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 8, no. 4, p. e1254, 2018.

[9] Y.-L. Hsu, S.-C. Yang, H.-C. Chang, and H.-C. Lai, “Human daily and sport activity recognition using a wearable inertial sensor network,” IEEE Access, vol. 6, pp. 31 715–31 728, 2018.

[10] J. Margarito, R. Helaoui, A. M. Bianchi, F. Sartor, and A. G. Bonomi, “User-independent recognition of sports activities from a single wrist-worn accelerometer: A template-matching-based approach,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 4, pp. 788–796, 2015.

[11] T. C. Pataky, T. Mu, K. Bosch, D. Rosenbaum, and J. Y. Goulermas, “Gait recognition: highly unique dynamic plantar pressure patterns among 104 individuals,” Journal of The Royal Society Interface, vol. 9, no. 69, pp. 790–800, 2012.

[12] L. C. Benson, C. A. Clermont, E. Boˇsnjak, and R. Ferber, “The use of wearable devices for walking and running gait analysis outside of the lab: A systematic review,” Gait & posture, vol. 63, pp. 124–138, 2018.

[13] A. Sunde, Ø Støren, M. Bjerkaas, M. H. Larsen, J. Hoff, and J. Helgerud, “Maximal strength training improves cycling economy in competitive cyclists,” The Journal of Strength & Conditioning Research, vol. 24, no. 8, pp. 2157–2165, 2010.