How Apple’s New Find My Service Locates Missing Hardware That’s Offline

posted in: July 2019 | 0
Craig Federighi, Apple’s senior vice president of Software Engineering, introduces the new Find My at WWDC 2019.

Image by Apple


How Apple’s New Find My Service Locates Missing Hardware That’s Offline

If you’ve ever lost a Mac, iPhone, or iPad, or had one stolen, you may know the frustration of having Find My Whatever enabled, but never getting a ping that it’s back on the network or never receiving confirmation the device was erased after you issued that command.

Apple aims to improve that situation later this year with revisions to its Find My iPhone service that turns nearby Apple hardware into relay beacons. If you mark a Find My-tracked device as lost, but it’s not connected to Wi-Fi or a cellular network, Apple may be able to determine its location anyway with the passive help of your fellow Apple product owners.

The trick is that any Internet-connected Apple device running iOS 13 or macOS 10.15 Catalina can identify broadcasts from the Bluetooth adapter in other Internet-offline Apple devices nearby and pass that information back to Apple. This reporting works even when the missing Mac, iPhone, or iPad is on standby or sleeping, though it can’t work for a device that’s powered down, or if you have disabled Bluetooth or put your device into Airplane Mode.

This technique solves the problem of how to find a device that isn’t connected to the Internet: by relying on other Internet-connected devices in close physical proximity! (To be fair, Apple didn’t invent this approach, and Bluetooth-enabled location trackers like Tile have used similar crowdsourced approaches for some time.)

In line with Apple’s commitment to privacy, the company’s description of the feature promises that it won’t reveal to anyone but you that the lost device is being tracked and where it’s located. Whether or not it actually helps users recover that many more devices from under a car seat or from thieves, Apple has chosen a nifty set of interlocking encryption algorithms and privacy-preserving policies.

Apple never had a unified name for this device-locating feature previously, at best referring to the app as Find My iPhone, and customizing the name on whatever device it appeared, like Find My Mac and Find My iPad. In iOS 13 and Catalina, Apple is combining Find My iPhone with the active, intentional location-sharing service Find My Friends. The new app and service will simply be called Find My.

How the New Find My Service Works

Apple introduced Find My iPhone in 2010. Over the next year, the company gradually extended the service to more devices and subsequently improved how it located, tracked, and wiped remote hardware. The service works via an app or iCloud’s Web site, and it can find iOS devices, Macs, Apple Watches, and AirPods. But not Apple TVs or HomePods, neither of which is easily misplaced or likely to be stolen.

You can use the Find My iPhone iOS app or the iCloud Web app to pinpoint hardware and activate various features on lost devices. Depending on the kind of device, you can erase its contents, lock it, display a recovery message, play a loud sound, or track it.

But Find My iPhone has always relied on the device being connected to the Internet to carry out your commands—a reasonable requirement! Depending on the hardware, that means accessing a cellular data network or a Wi-Fi network.

Wi-Fi is particularly tricky as a connection type, because most hotspots require some kind of authentication or acceptance of terms of service, even if you’ve connected before. A recent medical appointment took me across several floors of a clinic’s building, and each time I moved, I was asked to “click Accept,” even though it was ostensibly a single network. Plus, by itself, a Mac or iOS device won’t connect to new Wi-Fi networks, and may have difficulty re-associating with previously visited ones.

Apple’s trick in the new Find My service is to combine always-available Bluetooth networking with the near ubiquity of other people carrying Apple gear. The company adds a careful privacy formulation on top of this so that only the owner of a lost device can figure out where it is. Even Apple won’t be able to decode where a specific device is located.

Craig Federighi in front of a screen showing macOS Find My app.

Security researcher Matthew Green, who has documented weaknesses and encryption failures in tech products for years, has a generally positive take based on Apple’s briefings and comments. He has identified some key problems and ways in which he believes Apple might solve them. The devil is, as always, in the details.

Apple hasn’t yet released technical details of how the revised Find My service works and the company didn’t respond to my request for a briefing. However, the general outline so far is this:

  • You need at least two Apple devices logged into the same iCloud account.
  • On activating Find My, your devices exchange encryption information.
  • Apple facilitates this exchange in a zero-knowledge manner, so it can’t access encryption keys.
  • All iOS, iPadOS, and macOS devices running operating system updates released later this year will recognize Bluetooth messages from offline devices, and continuously pass those on to Apple along with the detecting devices’ current coordinates.
  • Apple promises these messages will consume negligible bandwidth and battery power.
  • After you mark a device as lost, you will be able to send a query to Apple from one of your other Find My-registered devices and retrieve encrypted location information related to the lost device.

We don’t yet know how the user side of this will present itself outside of limited screen captures shown during the WWDC keynote. Location and tracking information might be identical to current Find My iPhone apps, or it could show pushpins at every place another Apple device has spotted the missing one.

But how does Apple both capture all this crowdsourced information and keep it fully anonymous from other users and itself? Apple already has some experience on that front.

Keeping Secrets Even from Itself

Apple says the updated Find My service will be “completely anonymous and encrypted end to end, so everyone’s privacy is protected.” This seems plausible because Apple has already built several services that work in a similar fashion, with end-to-end encryption after initial setup.

For instance, iMessage uses iCloud for login, but once your device is connected, Messages relies on information stored only in your devices (that’s never accessible by Apple) to encrypt outgoing messages and decrypt incoming messages. The same is true with FaceTime audio and video calls. Apple uses similar techniques for Health data, payment information, Screen Time monitoring, Siri, and Wi-Fi network passwords and connections.

Apple also reportedly uses end-to-end encryption to sync information about photos for which you’ve identified people’s faces. The company doesn’t document this fully, but Craig Federighi, Apple’s senior vice president of Software Engineering, offered some detail to John Gruber in a live interview in 2017. Each of your devices analyzes stored photos locally, makes its own guesses about which faces are the same, and stores your confirmation or rejection of those matches. Only your identification and association of faces is synced across your devices using end-to-end encryption. This approach prevents Apple from knowing which face you’ve labeled with which name and seeing any facial-recognition results whatsoever, unlike techniques used by some other big tech companies.

iCloud Keychain, however, most closely parallels how the new Find My service works. If you’ve set up iCloud Keychain, you may recall that when you start syncing iCloud Keychain to a new device, you have to approve it from a device that’s already set up with the sync service. Those devices then securely exchange encryption key information in a way that Apple can’t access. (You can also set a special iCloud Security Code that adds another layer of protection beyond access to approved and unlocked iOS and macOS devices.)

Without getting too far into the encryption weeds, the Bluetooth broadcast will be a public key, Apple told Wired magazine. Public-key encryption relies on paired public and private keys: you can freely and safely distribute the public key so others can use it to encrypt messages that only you can decrypt with your associated private key.

In a world tainted by the egregious behavior of Facebook and ad-tracking companies, you’d be excused for worrying that your public key could become another way for you to be tracked by marketing firms or government agencies. But Apple said it would change the public key at some undisclosed interval, which prevents tracking over time. And according to its statements, the public key is broadcast over Bluetooth only when a device can’t reach the Internet.

Any Apple device running iOS 13 or Catalina will encrypt and report to Apple its own location paired with a common one-way cryptographic conversion (a “hash”) of the Bluetooth-transmitted public key for every device in its vicinity. That hash can’t be reversed, so Apple won’t know which public key was recorded, but any device with the original public keys can perform the same one-way hash and create a match.

Craig Federighi in front of a screen showing an illustration of a Mac broadcasting Bluetooth keys to iOS devices.

As a result, Apple could amass up to billions of data points a day, none of which it could use to connect devices and locations. It will obviously also retain that data for only a finite period of time, both because of the sensitivity of the information (even in encrypted form) and the sheer amount of data involved.

If you use your iPad to mark your iPhone as lost, for instance, the iPad will send a query to Apple’s database to retrieve matching relevant records. It can then decrypt those records locally to determine the locations at which the iPhone was found. The only point of weakness in this system is that Apple will seemingly know which device or iCloud account that makes the query for particular hash/location data. I presume Apple will provide a privacy disclosure about how it records or deletes that data, too.

Some of the coverage of this feature seems to suggest that only devices marked as lost will have their Bluetooth key and the finding device’s location uploaded. However, that’s an unlikely scenario, because it would require a detecting device to consult Apple database’s to figure out if detected hardware were stolen, which could lead to privacy violations. Instead, I think there’s a confusion in some articles between how a Find My-enabled device will start broadcasting its key whenever it’s offline, rather than detecting devices having to make a determination about whether to upload it.

For instance, if you walk into a cafe with 100 Apple devices, most will be connected to the Internet. Of those that aren’t, as I read Apple’s descriptions, your device will pick up and transmit their Bluetooth key and your location, as will any other Internet-connected Mac, iPad, or iPhone running the latest software. I expect Apple will throttle the communication in some way so that this information is sent infrequently for each public key and location.

How Worthwhile Is This, Anyway?

We don’t know how many people have benefitted from the current Find My iPhone service. While I’m sure that we all have or have heard stories about lost hardware, Apple has never quantified it. Is Find My iPhone used 1 million times a year or 10 million? Has it helped locate hundreds of thousands of devices underneath couch cushions or millions? How many stolen devices has it helped locate and recover? We don’t have those answers and Apple hasn’t said.

Craig Federighi in front of an iPhone showing the revised Find My app.

With that proviso, Apple’s planned improvements are certainly useful if you misplace a device, particularly those that rely solely on Wi-Fi. One of my kids lost an old iPhone they were using with a very limited T-Mobile pay-as-you-go plan that costs about $4 per month without data. They think they lost it on a bus, and Seattle-area buses often have Wi-Fi, but apparently it wasn’t connected. Had Apple’s new Find My service been in place, we might have recovered it.

I have more questions about whether the service will help recover stolen items. Thieves ostensibly already know that iPhones and some iPads can transmit their locations over the cell network and power them down immediately or stick them in a cheap wire-mesh bag that blocks signals. Of course, criminals aren’t always that bright, and you can find plenty of stories about Find My iPhone leading police right to the thief’s front door. But I don’t think the new Find My service will pose new problems for any savvy thief.

It’s likely more significant for Wi-Fi-only iPads and Macs, where Bluetooth signals would continue to transmit as long as a device is in standby and thieves might not know to power down such devices. That might let, for instance, the police grab surveillance video associated with a location or even find a device in real time as it’s moving around. That said, from most reports, law enforcement mostly seems to care about such thefts when they’re associated with a crime ring or tied to violence.

We have to assume Apple believes that the significant investment into Find My’s new approach is worthwhile, either based on requests from customers or as a marketing point to encourage future sales. Regardless of why, it’s coming, and we might even eventually find out how useful it is.