Deep dive: How exactly the Apple Watch tracks swimming

The onboard accelerometer, gyro, and GPS chip help the Apple Watch know how you're moving in the water.

Apple

Last week I splashed into an underground university pool with an Apple Watch Series 3. As the company’s wearable has matured, Apple has marketed it more and more as a fitness device, one that's, thanks to a partnership with Nike, particularly well-suited as a running companion. But the Apple Watch also tackles something more dynamic and varied than your morning jog: exercise in the water.
The device has functioned as a swim tracker since it became water-resistant in 2016, but with its latest operating system, it presents a more granular metric: set detection. It knows when you rest at the pool’s edge and then uses that information to divide the workout into sets of laps, showing you how far and long you swam in each, what stroke, and your rest time.
You don’t need the latest model Apple Watch to get this information. A Series 2 running the 4th-generation operating system will do just fine. But I swam with the company’s newest version, the one with a cellular connection. I could make a call on my way to the pool without bringing my phone, but that was the biggest advantage the latest and greatest offered.
In the pool, the Apple Watch becomes a tool to quantify the complex motions of a watery workout, and as a longtime amateur swimmer, I’ve been interested in how that works.
Apple built its own algorithms to discern the stroke of swimmers of all skill levels, as well the calories they’re burning. The process involved gathering data from more than 700 swimmers and over 1,500 swim sessions. Part of that data-gathering involved swimming in place (in an "endless" pool) while wearing a mask that descended from the ceiling.
That's a lot of data. And all that data is required, because skill level affects form. “If you are Michael Phelps, you’re distinctively making very good differences between those four strokes—the backstroke, freestyle, breaststroke, and butterfly,” says Jay Blahnik, who directs fitness and health technologies at Apple. “If you are me, it turns out that once you look at the signals from the gyroscope and accelerometer, they’re not as strong and as clear.”

From wrist to satellite

The watch uses the gyroscope and accelerometer to track the motion of your strokes, but in open water, it can leverage another sensor: the GPS chip. If you’re swimming in the ocean or a lake, your watch uses that to determine how fast and how far you go. But GPS signals don’t travel through H2O. Luckily, people are likely to do freestyle in open water and for that stroke, your arms regularly break the surface. Apple sets the GPS chip in acquisition mode for the whole swim. It looks for the satellite signal each time your hand rises from the water. “We’re trying to catch it every single time,” says Ron Huang, Apple’s director of engineering for location and motion services.
The accelerometer measures motion, and the gyroscope determines how many degrees the watch is rotating per second. Together, those sensors help Apple figure out the stroke type.
“What we’re looking at is literally the trajectory of your wrist as you’re doing each stroke,” Huang says. Butterfly and freestyle look most similar to the watch’s sensors, which presents their biggest “confusion matrix.” But it’s important to get it right, as stroke type affects how many calories you burn. Butterfly, for example, can burn about 40 percent more calories than breaststroke.
To complicate matters, the gyro has a tendency to indicate that the watch is rotating, even when it’s not—a phenomenon called gyro drift. For example, your wrist could be still, but the gyro might say it’s rotating by 10 degrees per second. In this instance, Apple relies on the accelerometer to find the ground truth. If it sees from the accelerometer that you’re not moving, but the gyro still shows rotation, it can learn by what amount to discount the gyro’s reading.

Turn, turn, turn

Skilled swimmers do flip turns, which is like a watery half somersault and twist used to reverse direction, when they get to the edge of the pool. But not me, and not Blahnik, the Apple health executive. “If you’ve ever seen my flip turn, it’s not so good—and in fact it’s so bad that I don’t flip turn at all,” Blahnik admits. “We needed to be really good at determining the turn, whether you are a great swimmer or whether you are a less experienced swimmer.” In other words, no matter how you turn, the watch still needs to notice it.
This is where the gyro really helps. It measures rotation in three separate planes in space: around x, y, and, z axes. The x-axis goes horizontally across the display; the y-axis, vertically; and the z shoots straight out from the screen. The watch’s software interprets the data from the gyro to figure out when you’ve reached a wall and turned 180-degrees in a new direction, like the way an airplane’s yaw is a measure of the direction of its nose. For example, hold your wrist flat in front of you and parallel to the ground, and imagine that z-axis shooting straight up to the sky: A rotation around that z-axis reveals a change in your direction.
By learning how many strokes it typically takes you to cross the pool, Apple knows when to expect you to make a turn. Suppose you normally need about 25 strokes, but later in your workout, Apple notices you’ve taken just 10. It can then deduce that’s likely because you’re just hanging out in the center of the pool, and it should disregard whatever yaw change it sees. You may have just turned 90 degrees to hang out on the lane-dividing buoys.

Trust your senses

Perhaps the most interesting decision Apple made is to trust its onboard sensors more than one specific piece of user input. Before starting a swim, the watch prompts you to tell it how long the pool is, in meters or yards. But Apple knows there’s a good chance you might get that measurement wrong. Maybe the pool is 25 meters long, but you say it’s 25 yards. “It’s all over the map,” Huang says. At the end of your swim, the watch tells you how far you swam in total, but that’s based simply on how long you said the pool is; swim 10 laps in a pool that you said is 25 meters long, and the watch will inform you that you swam 250 meters. Apple just goes with what you said for that metric. Anyway, the watch doesn’t need to know the length of the pool from you to accurately count your laps, because it’s doing that independently by marking the turns.
But, importantly, that total-distance swam measurement doesn’t really matter, at least not when it comes to figuring out how many calories you burned. That’s because Apple is using its own sensors to determine that metric. “One thing we noticed is that the number of laps you swam is not a very good indicator of the actual calories that you’ve burnt,” Huang says.
Think about a great swimmer who crosses the pool easily with few strokes, he says. They’re not burning as many calories as someone who is laboring to make it to the wall, making many strokes as they do. So the watch uses its sensors to look for the yaw changes to mark when you’ve finished a lap and turned, and it counts your strokes; it uses that information to help estimate caloric burn. “We didn’t want to just go by lap counting and distance to give you calories; we wanted to actually measure your swim efficiency level, and the number of strokes you took,” Huang says.