Digitize Any Text with Live Text in iOS 15 and iPadOS 15

One of the coolest features in iOS 15 and iPadOS 15 is Live Text. Once you’re running these operating systems, the Camera, Photos, Notes, and Safari apps—and undoubtedly more soon—can recognize text in photos and let you select it just as you would any other text. You can then copy it and paste it into any other app. The system-wide search finds text in photos, too. (And if you can’t upgrade or haven’t yet upgraded to iOS 15 or iPadOS 15, or want similar capabilities on the Mac before macOS 12 Monterey ships, see “Work with Text in Images with TextSniper and Photos Search,” 23 August 2021. Even after Live Text is available everywhere, I suspect the developers of these and similar utilities will look for ways of going beyond what Apple provides.)

Live Text is immediately useful. Whenever I need a spare part for some appliance, I take a picture of the sticker inside it to figure out its model number. I can now directly copy the model number and paste it into a search engine without manually transcribing it. Took a photo of a business card but can’t find it? You can search for it. Read a passage in a book that you’d like to research? Take a picture and then paste the text into Notes to refer to later.

Our own Michael Cohen figured out a clever use for this feature. If you’re reading something in Libby or another app that doesn’t let you copy text, you can take a screenshot and then copy the text from a screenshot.

To use Live Text, you need an iPhone or iPad with an A12 Bionic processor or better. That means you need:

• iPhone XR, XS, XS Max, and later, including the iPhone SE (2nd generation)
• iPad Pro 12.9-inch (3rd generation) and later
• iPad Pro 11-inch
• iPad Air (3rd generation) and later
• iPad (8th generation)
• iPad mini (5th generation)

See “The Real System Requirements for Apple’s 2021 Operating Systems” (11 June 2021) for more specifics on iOS 15 and iPadOS 15’s requirements.

Most Live Text use happens in the Photos and Camera apps, Safari, and within the system-wide search. You can also use Live Text with the device’s camera to import real-world text directly into any app that accepts text input. Here’s how to use each. See my book, Take Control of iOS 15 and iPadOS 15, for more tips on new features in those operating systems.

Live Text in Photos

You select text in a photo like any other text: touch and hold the text until it highlights in blue. Then you can move the blue drag handles to adjust the selection. After you select text, the familiar popover appears with options to copy text, select all text, or look up a term.

If you’re having trouble, tap the Detect Text button in the lower-right corner of the photo, which darkens the image apart from the detected text, which is highlighted.

Detect Text usually works pretty well, but if the text is angled, odd things can happen, like being forced to select text you’re trying to exclude or preventing you from adjusting the text selection.

You may wonder how Live Text interacts with Live Photos that contain text since you touch and hold to animate a Live Photo, and you also touch and hold to select text. The trick is to touch and hold somewhere away from text to animate the Live Photo.

Live Text in Camera

If you don’t want to clutter up your camera roll, you can copy text directly in the Camera app. Aim your viewfinder over the text. A yellow outline appears around any text the Camera app detects.

Tap the Detect Text button to zoom in on the text, which is frozen so you can easily copy it.

I find this more awkward than snapping a photo and copying text later, so I don’t use it often.

Search for Live Text

iOS 15 and iPadOS 15 index all of the text they recognize in photos so you can search for it. Bizarrely, you can’t search for Live Text in the Photos app. Instead, you have to swipe down from the middle of the Home screen to activate the system-wide search (Apple doesn’t call it Spotlight in iOS and iPadOS anymore). Scroll down to the Text Found in Photos heading.

Live Text in Safari (And Other Web Browsers)

Just as in Photos, you can touch and hold an image in Safari to select text. As you might expect, given Apple’s requirement that all Web browsers rely on WebKit in iOS and iPadOS, it also works in Brave and Firefox and likely in other browsers, too.

Also, as in Photos, if you need to touch and hold an image to invoke another action (like seeing a preview of the page the image links to), touch and hold away from any text.

Import Live Text Directly into Apps

It may take some time before we become accustomed to this capability, but you can use Live Text with the device’s camera to import real-world text directly into any app that accepts text input. Tap in a text field to reveal the popover and tap the Scan Text icon.

A viewfinder appears on the bottom half of the screen, with a yellow outline around the text. Detected text floods into the text field, but only as a preview. If it’s what you want, tap the “insert” button. (We have no idea why its name is lowercase.)

As you can see, text input via Live Text works pretty darn well, even if using it may take some getting used to.

VoiceOver for Live Text

Live Text is also a great feature for accessibility because it enables VoiceOver to read the text in images. Go to Settings > Accessibility > VoiceOver > VoiceOver Recognition and turn on Text Recognition to enable it.

When viewing a photo with text while VoiceOver is enabled, tap the text in the photo to have VoiceOver read it to you.

Translating Live Text

One last tip. The Translate app is now a system service, so you can translate any text you select, including text in photos or picked up by the camera. It’s one of the options in the popover that appears when you select text.

What are your favorite uses for Live Text so far? Let us know in the comments!