Developing With Kinect Sensors For Fitness And Health

The Kinect sensor has been widely used (hacked/developed/applied) by many ever since the Xbox 360 was first released. A couple of years ago, a fellow sports engineer from SHU studied the feasibility of using the Kinect sensor as a biomechanical analysis tool. He concluded that although the Kinect was fairly accurate, it wasn’t good enough for serious analysis (You can read more in his blog post here). The main advantage of the Kinect was (and still is) it’s price compared to professional motion sensors, and the Microsoft SDK which allows developers to come up with interesting applications (Check out various kinect hacks here).

I recently started working on a project that utilises the Kinect sensor. The project is basically developing a fitness product/system that combines the use of various sensors for assessing gym exercises. It is a rather interesting and novel concept because not only does the product quantify different gym workouts, it has a gamification portion where each user is competing with another gym user at the same time. No, it’s not like online gaming. In fact, this system is not designed to be used at home, but rather in a gym setting where participants perform the workouts together and get scored at the end of each session. Think Nike+ Kinect Training but for many people physically at the same place and with smart gym equipment (Equipment with sensors and smart algorithms). I probably should not go into too many details to avoid spoilers, but do look out for its launch sometime this year!

Anyway, I had the opportunity to test out the Nike+ Kinect Training (NKT) and found that it has quite a well-designed interface that helps the user perform workouts with proper techniques. For example, the Kinect (ver 1) sensor is not the most accurate in measuring depth, so for exercises like push-ups, burpees, and core exercises like the bird-dog, the NKT gets users to turn to the side instead of face the TV/Kinect sensor; that way, the user’s movements are tracked more accurately. The concept of the NKT program is also pretty good because it starts with putting the user through an assessment – a series of movement tests and exercises then rates the user in terms of strength, flexibility and stamina. Following that, it recommends a scheduled training program with a combination of exercises that can help you reach your goal (either to build power, become toned or lean). The feedback given by the on-screen personal trainer are usually quite spot on, usually correcting my posture, asking me to slow down (for exercises that are meant to be controlled) or speed up (for endurance type exercises), or just encouraging me to push on for the last few reps. There are instances where the Kinect sensor was unable to track some of my joints accurately and failed to count my reps, especially in a few of the floor exercises. But all in all, it is a pretty good program based on some sports science fundamentals and it could be an effective training tool for people who like to workout alone. I also got some good ideas off it that might be useful in the project I am working on.

{On a separate note, there has been some interesting devices/gadgets developed for the fitness and strength training folks in the last few months. Here’s a few that caught my attention:

1. PUSH – a wearable arm band (possibly built with inertia sensors) that is able to determine force, velocity and power of each strength training rep
2. Hexoskin – another wearable smart apparel that not only measures movement (activity level, steps, cadence), but also the users physiology (heart rate and breathing rate).
3. Athos – similar to the Hexoskin, it is a wearable smart apparel with the addition of electromyography (EMG) capabilities embedded in the apparel.
4. Skulpt Aim – a mobile device that measures the user’s body fat percentage and muscle quality in individual muscles.

These devices (and other smart devices) could potentially become a common sight in gyms in the near future, allowing users to track more about their workout sessions and gain more understanding of what’s happening. A common trait among these gadgets is that they all have (or are developing) iPhone apps, which means users will have access to their workout history on their fingertips and probably be able to brag about it on social media.}

Going back to the Kinect sensor, apart from sports and fitness applications, developers have also come up with practical solutions for the medical and health industry. One such application is the Teki system developed by technology services company Accenture, and a few other partners including Microsoft. The main purpose of the Teki system is to reduce the need for elderly patients to travel to the hospital for routine consultations and check-ups, saving time and money. Using a Kinect sensor, set up at the elderly patient’s home, together with a few other wireless medical devices like a pulse oximeter and a spirometer, the doctor is able to do a remote consultation using a webcam in the hospital/clinic. The Kinect sensor comes in when the doctor needs to evaluate the patient’s range of motion; or when there are prescribed rehabilitative exercises that the patient need to perform and the Kinect sensor is able to assess and provide feedback to assist the patient.

It was mentioned earlier that the Kinect sensor isn’t the most precise in measuring movements, especially in terms of depth and also higher speed motions. Although the specification says that it could measure up to 30 fps, but after testing it myself, I found that it is usually around 15-16 fps (depending on your program). Lighting and certain background objects could also affect the detection of a full skeleton. But all these little ‘glitches’ will no longer be there with the release of the new Kinect 2 sensor which features improved performance over the original Kinect. Those improvements include: a wide-angle time-of-flight (ToF) camera allowing better range (or depth) measurements; capturing 1080p video, and ability to ‘see’ in the dark with its new active IR sensor; it can detect more joints on the body (5 more than the previous) with much higher accuracy, and it can track up to 6 skeletons at one time. Also, it is capable of measuring the users’ heart rate via a change in the user’s skin tone and even detecting mood from the user’s facial expression. {Just watch this video that basically demos all the improvements.}

With this newer Kinect sensor, it will be a lot more exciting for hackers/developers and who knows what interesting applications could be invented. But as of now, there is still no news of when Microsoft will officially release the windows version of Kinect 2 for developers; for those who are really keen, there is a preview program with limited spots that you can apply for here!

If you know any other Kinect applications in sports and health, feel free to comment below. Thanks for reading and here’s wishing everyone a happy new year!