Sword Health's thrive program review (ongoing)

My insurance added what sounded like a great program, Thrive, which lets you do physical therapy at home instead, as in my case, needing to drive an hour plus each way for a less than 15-minute session. The tablet uses the camera to watch your movements. It appears to simulate a Kinect sensor with a single camera, which is probably the main issue with the system, since the patient needs to get the angle just right for the AI to line you up. While the tablet does show you the "stick figure" (you might have seen similar ones on ghost-hunter shows using the Kinect) as an overlay on your video to help, the only indicator is that it turns green when it sees you are in the right position. Leaving it to you to sort out how to get there,

Initial consultation

The first red flag was that, during the initial consultation, I learned they focus on only one body part. So instead of working on my osteoarthritis, we set the focus on my left wrist, which I broke twice last year and never got full movement back in. Each session is supposed to be about 5 minutes.

Day one

After getting the tablet, I started my first session seated at my desk since the exercises were just for my wrist. As the session started, I first found the AI needed to see most of me, and in a standing position. So now, while the tablet kept telling me to get into position, I needed to find a place where I could stand in front of it at the right height to be in the right position for its camera. Note, they say they do not save the video, and I'm not going to worry about being seen with my shirt off, so I did not bother putting one on. And again, we are supposed to be focused on my wrist.

First exercise

Turning my hand palm up and down. This is really more of an elbow motion, so it was not that hard, but the AI seemed to get confused, saying "down" several times. Possibly since I did not have a shirt on, and the position had my hand in front of my belly. Somehow, I received a 98% score, though.

Second exercise

Arm straight out to my side, and for some reason, head bent to the right. Thanks to the aforementioned osteoarthritis, I could not bend my neck enough to make the AI happy, which kept telling me to bend it until I shut the tablet off, despite my comments that it would not bend any further. I tried to text the "team" back, but noticed the number did not accept replies. So I opened the tablet again, which wanted to start the "second session". The tablet seems to accept voice input only after the initial setup, and was focused on starting the next session. So after a few choice comments, off it went again. I expect a response to my choice words left with the AI for feedback tomorrow.

Day two

Started off with it, thinking this was session three. At the start of the session, I got it to confirm that it would skip the second exercise. A couple of times, in fact. It also asked for more light this time, so I flipped on my showroom-level lighting.

First exercise

Went well as I had put on a shirt and knew what to expect.

Second exercise

Again, it popped up and got stuck on me, not being able to bend my head enough to the right. After trying every response I could think of, I hit pause and finally saw the option to skip.

Third exercise

From the side, no issues

Sit-down exercises

It now wanted me to sit down, like I expected from the start. So now I needed to set up in a new location where I could hold my hand steady in front of the camera. I was almost ready to pass again on some of these because my little finger has about half the range of motion they expect.  

Optional "cool down" stretches

First, I assumed I had a squeeze ball, which I fortunately had to hand. Next, I needed to stand, place my palm flat on the table, and lift each finger off the table. This did not go well, again for arthritis reasons.

Follow up

Lastly, it asked how I felt and asked for feedback, and I again mentioned that they are not taking my arthritis into account.

Day Three

Had to skip a few exercises because the app would keep telling me to do something I was already doing. For instance, in the touching thumb to fingertips exercise, I'd be touching my little finger, but it just stopped seeing I was after about three goes. Same with the thumb-to-index-finger base exercise.

Day Four

Another day of "beta testing" their app. That's how I'm seeing it anyway, despite no response from them other than removing the standing exercise that wanted me to bend my neck in ways it will not. I'm not seeing any improvement in my range of motion, but I am getting better at finding the right angles so their AI can properly see the position of my fingers in the sit-down exercises. For instance, the thumb-to-little-finger exercise appears to have failed, not because it did not see them touching, but because my ring finger was not straight enough. For the fist-making exercises, I found that getting the angle right so the AI indicated it was correct, and then keeping my wrist at that angle, improved my score significantly.

For the standing exercise, where I keep my elbow at my side and tilt my wrist up and down, I'm still trying to sort out whether it fails because of the AI or because it just expects it to bend way more than it can. It appeared to line up if I bent my elbow a bit, even though I was not supposed to bend my elbow. 

I provided extensive feedback on how to improve their app and, hopefully, their hardware for future clients. I especially pointed out that starting with a fist rather than an open hand would be a simple fix that would greatly improve their app.

Six weeks in

A few lessons learned:

• Reboot the tablet every few weeks. It gets slower over time. About 5 weeks in, it slowed down so much that it was dropping frames, making it almost impossible to know when you were in position for the "hold" exercises, as you would already be past the correct position by the time it said you were there.
• The video of a person doing the exercise is just a starting point. Angles are everything here. Watching the "Kinect style" stick figure is more important. Especially with the standing exercises, I've found that "parallel to the floor" often meant tilted down a fair bit instead of parallel to the tablet; an angle of up to 20 degrees away from the tablet worked better. Even when the stick figure is clearly not "parallel to the floor". Same with the thumb-to-index-finger exercises. In the thumb-to-index-finger exercises, for example, the small finger is the one that takes the tightest alignment. Get that one set and leave your wrist angle there for the rest of the fingers.
• Be sure to say when an exercise is not doable. Despite my age and arthritis being known, they suggested some yoga poses for "cool down exercises". I told the app to skip them the first time. The second time, I made a point of telling the AI coach that getting down on the floor is not going to happen. After that, they seemed to get that I might have limitations.
• Lighting and background are very important. For the standing exercises, you really want a blank wall behind you and wear a plain short-sleeved shirt. The best lighting appears to be one aimed straight at you, casting the only shadows behind you. You want similar lighting for hand exercises. Even whitening out your palm a bit helps the AI see your hand against the background. Though too much light makes the white stick figure of your hand hard to see.

Things that need improvement:

• Indicators of where the AI thinks you should be. The "Kinect style" display shows you the points the AI uses to reduce you to a stick figure, making it easier to check positioning. The "figure" is white when you are in the start position and until you reach the goal position, then it turns green or red if you get too far off course. What is needed is an indicator, say in yellow, of where the AI goal position is. Even better if it could highlight which part is out of alignment. As mentioned above, changing the angle of my hand to the tablet was the magic needed. Knowing that, instead of the vertical tilt of my wrist being the issue, I needed to rotate my wrist toward the tablet so it could better see my knuckles would have saved me so much time figuring this out by trial and error.
• The tablet really needs at least a second camera. Better yet, a real Kinect sensor. They use multiple cameras and IR to really see you in 3D. People are using these fairly cheap sensors to DIY ghost sensors, and they are already simulating one, so adding one to this setup should be easily doable.