The Ins and Outs of Non-destructive Editing in Photos for Mac and iOS

posted in: July 2019 | 0
Taking a snowy selfie with a Nikon camera

Photo by Jakob Owens

13 comments

The Ins and Outs of Non-destructive Editing in Photos for Mac and iOS

 

One of the near-magical features of Photos, and iPhoto before it, is the way it allows you to edit a photo with the assurance that those edits are non-destructive—you can always revert to the original version. That said, non-destructive editing brings with it some confusion, particularly for those working with RAW images or using Photos extensions to do the editing.

To shed light on the situation, I’ll first explain how non-destructive editing works in photo applications in general, using Photos as an example. Then I’ll go into detail about how non-destructive editing in Photos works with RAW image files, Photos extensions, and more. I will also identify some trouble spots.

To give you some background, I was the lead developer for Aperture, and I later led the team that developed the editing engine for the Mac version of Photos. Now I’m on my own, developing a Photos extension called RAW Power that uses the same RAW engine underlying Photos and Aperture. In this article, I will often use RAW Power as an example, though other Photos extensions work similarly. If you are unfamiliar with RAW, you may wish to read a separate article of mine that discusses the benefits of RAW as well as touching on Apple’s new HEIF image format.

Basics of Non-destructive Editing

Most applications directly modify your documents. For example, when you choose Save in Microsoft Word, it saves your changes directly to the document’s file, replacing the previous version of the file in the process. Some image-editing applications also modify image files directly, notably Adobe Photoshop and Apple’s Preview.

In contrast, most photo-editing applications work non-destructively and never modify the original photo. The application applies edits to an in-memory copy of the original to produce a real-time preview of the adjusted image.

A basic example of non-destructive editing.The application may automatically save the edit instructions, or it may wait for you to hit Done or choose Save. From your standpoint, the image appears to be modified, but the original file remains untouched. While extremely useful, this non-destructive model can be confusing to users because it clashes with their expectations based on other applications.

Photos saves the edit instructions in a location separate from the original file and generates a full-size JPEG (and smaller thumbnail images) that contain the results of the edits. (Photos makes JPEGs for all file types, including RAWs and Apple’s new HEIC files. You cannot rely on the Info pane in the Mac version of Photos to tell you the file type you are editing because it always shows you the file type of the original.) After the edit is completed, this is what is stored on disk:

What's stored in non-destructive editing.

The full-size JPEG is an individual file on disk. The smaller thumbnails are stored in “container” files that hold multiple thumbnails at once. The edit instructions are stored in the photo library’s database, along with other information about the image.

If you want to continue editing the image at a later date, Photos reloads both the original and the edit instructions and then combines them to display the adjusted image.

Outside of editing mode, Photos hides the original photo from view, instead showing the full-size JPEG or smaller thumbnails at every point in the application. While this is clearly correct, hiding the original reinforces the perception that the original has been modified. This is the main point of confusion for people. Many users ask me how to copy or duplicate images before they edit them—they are convinced by Photos’ sleight of hand and believe they need to preserve their original before making any adjustments. (Some people duplicate images because they want to have both unadjusted and adjusted copies visible in the grid, but this is less common.)

Ironically, one reason non-destructive editing exists is because RAWs cannot be edited in place. It is not possible to make arbitrary edits to RAW sensor data, and from those edits, create RAW sensor data (it is possible to make a linearized DNG, but linearized DNGs are not RAW images). Photos never modifies the original RAW images.

Why go through all this trouble? Because non-destructive editors offer three important features:

  1. Instantly show the original image to allow for A/B comparisons
  2. Discard all the edits and thumbnails for an image, revealing the original
  3. Revise or remove individual edits without loss of quality

Photos always stores its editing instructions in the library’s database. Other applications may use a database or store them in separate “sidecar files,” typically using the same name with a different file extension. For example, for an image named IMG_0005.JPG, an application may store a sidecar titled IMG_0005.xmp or IMG_0005.dat in the same directory. Sandboxed apps may store the sidecar elsewhere.

Sidecars are general-purpose files, so they can also hold thumbnails and miscellaneous metadata such as ratings. Applications sometimes store the sidecar information inside the original file because most image formats can store miscellaneous data without modifying the pixels or impairing the ability of other applications to read the original pixel data. This ensures the editing information stays with the original file data but can lead to corruption or data loss if the application fails to insert the data into the original file properly. In addition, not all applications correctly retain sidecar data from other applications when they write their sidecar data into the original file. For these reasons, many photographers prefer the sidecars to be stored separately from originals.

Photos Extensions

Although we mostly think of non-destructive editing in the context of the native editing tools in Photos, that’s not the only place it’s used. Developers can create a type of plug-in called a “Photos extension” that gives Photos additional editing features. To access Photos extensions, start editing a photo and then click the Extensions (···) button at the top of the window.

Accessing Photos extensions

Photos extensions do not add extra sliders to the Photos editing interface. Instead, they replace the standard Photos tools entirely.

Photos extension replacing the default controls

As with normal editing in Photos, Photos extensions let you edit only one image at a time—you cannot use a Photos extension to correct white balance in a batch of ten images. When you select a Photos extension, Photos opens the extension and sends it the original file:

How Photos sends images to extensions

When you click the Save Changes button in the Photos extension interface, the Photos extension gives Photos back its editing instructions and a full-size JPEG, as shown here:

The extension passing the JPEG and instructions back to Photos

As a result, Photos extensions participate in the same non-destructive system that Photos uses for its own image editor. Photos stores the JPEG and extension’s editing instructions in the library. Then Photos creates thumbnails from the full-size JPEG and hides the original.

If you decide to edit the image again with the same Photos extension, Photos will pass the extension the original image, along with the extension’s editing instructions. Again, this is just like Photos does for its own editing system:

Editing an edited image with the same extension it was edited with

The Photos extension can show the original, as well as remove or adjust any previous edit made in that Photos extension. Even better, the Photos extension’s adjustment data is synced through iCloud Photos, allowing you to continue editing with that Photos extension on another device. That means you can edit with the RAW Power extension on a Mac and continue editing non-destructively on another Mac with the RAW Power extension, or in iOS with the RAW Power app.

This is also how the system works in iOS, with one important limitation. Photos extensions in iOS 12 are never passed a RAW image—they are always given a rendered JPEG. So don’t bother editing a RAW image with a Photos extension in iOS 12. I filed a bug about this some time ago; hopefully Apple will fix it in iOS 13.

A Fly in the Ointment

This approach works swimmingly as long as only one application or Photos extension edits an image. As soon as two editors enter into the mix, things go sideways. That’s because Photos can store the adjustment information for only one editor for any given image. As soon as the changes from a second editor are saved, the edit instructions from the first editor are discarded and cannot be recovered! The image data is preserved, but the option to make changes with the first editor is lost. This is another key point of confusion for customers.

To illustrate this problem, imagine that a user decides to use the Photos Auto Enhance feature on a RAW image and then use the RAW Power extension to apply a crop and vignette. Photos has a good auto enhance and, unlike the Photos Vignette tool, the RAW Power Vignette lets you pick the center of the vignette.

Step 1: The user clicks the Auto Enhance button and clicks Done. As described earlier, Photos stores a full-size JPEG and the adjustment parameters in the library (the original is present, but hidden):

How Photos modifies an image

Step 2: The user selects RAW Power from the Photos extensions menu. Here’s where we run into the first difference from the previous example: Photos sends the full-size JPEG (not the original RAW image) to the Photos extension. Photos sends a JPEG rather than the original because the user expects to see the auto-enhanced image in the Photos extension, not the unadjusted original. Since Apple does not publish its internal format for adjustments, nor does it provide access to the code for its adjustments, there is no way for a Photos extension to apply the auto enhance made in Photos. This is also true in reverse—if a user edits an image with a Photos extension, and then edits further in Photos, they are working in Photos with a rendered JPEG—Photos has no way to apply edits made by other companies’ extensions to the original file.

Step 3: The user applies Crop and Vignette in RAW Power and clicks Save Changes. This is what gets stored in the Photos library:

Losing edit data in Photos

Notice that Photos has just discarded its own adjustment data! While everything may look fine, the Photos editing information has been lost, and the user is never informed of this. While the user can go back into the Photos extension and adjust the crop or vignette (or any other adjustment in the extension), they cannot go back a step and fine-tune the sliders in Photos. Further, the user cannot undo the step before they invoked the Photos extension—the only option available is Revert to Original because there is no Apple adjustment data left to revert to.

Note that there are also now two JPEGs stored in the library, one from Photos and one from the Photos extension. This is necessary to maintain some semblance of non-destructive editing. The first JPEG is the one created by Photos as a result of the Auto Enhance step and handed off to the Photos extension. Remember that for the user to re-edit an image with the same editor, the editor must receive the “starting image” along with its editing instructions. In this case, the “starting image” is the JPEG made after Auto Enhance was applied. Though there are two JPEGs, only the final JPEG is visible in the user interface. The auto-enhanced JPEG is hidden; it’s only stored in the library to enable re-editing with the Photos extension.

“But,” you may say, “I can edit in Photos, and then use a Photos extension, and go back to Photos and continue editing.” True, but you are adjusting the JPEG sent back from the Photos extension, not the original adjustments you made in Photos to the RAW file. Imagine that you make the edited image black-and-white. This is what is stored:

Losing edit data with a Photos extension

In other words, you are stacking Photos adjustments on top of the RAW Power adjustments (and losing the RAW Power adjustment data in the process).

Also note that this loss of adjustment data will occur if you edit in the Photos extension first and then edit in Photos, or if you don’t edit in Photos at all, but instead do all your editing with two Photos extensions. The order doesn’t matter—just the use of two or more editors on the same image.

A RAW-specific note: If you first edit a RAW in Photos and then send the image to RAW Power or another RAW-friendly Photos extension like DxO OpticsPro, not only are you working in the Photos extension with an 8-bit JPEG, rather than a 12- or 14-bit RAW, you lose access to the RAW processing features that RAW Power or DxO provide. This severe loss of quality happens for any edit done in Photos, even a simple rotate or flip.

This gotcha also rears its ugly head if you want to use two RAW-capable Photos extensions. In this situation, which Photos extension should you use first? RAW Power has strong RAW editing controls, based on the same engine that Photos uses. DxO has high-quality lens correction. I believe it’s best to start with RAW Power to access its RAW-only editing sliders, and then use DxO OpticsPro, because DxO’s lens correction works on both RAWs and JPEGs.

I wish Photos would warn users whenever quality is being degraded or adjustment data is being lost. Photo extensions cannot warn users of this problem because of the way Photos extensions communicate with Photos—they do not even know there is a RAW original. This is a warning only Photos can provide. If this happens to you (and you notice it in time), you should:

  1. Cancel the edit in the Photos extension
  2. Revert to Original in Photos
  3. Re-enter the extension

I Exported a TIFF from a What?

When saving edited RAWs for posterity, photographers prefer 16-bit TIFFs because they can store all of the richness of an edited RAW in a universal image format. For best results, exports should start with an image format whose quality is equal to or better than the destination file format. RAW is superior to TIFF, and TIFF is superior to JPEG. Accordingly, if you edit a RAW in Photos and later export a TIFF, you get excellent results because Photos will reload the RAW, apply the edit instructions to the RAW, and then generate the TIFF.

That’s not true for Photos extensions. As I mentioned earlier, extensions are required to send JPEGs back to Photos, rather than TIFFs. If you use Photos to export an image edited by an extension, Photos uses that JPEG as the source. I’ll repeat that: your 16-bit TIFF will be created from an 8-bit rendered JPEG, not by applying the adjustments to the original, as Photos does for images edited by its own tools. Photos uses a lower-quality source than the destination—your exported image is basically a JPEG decompressed into a TIFF.

There is a good, if unsatisfying, reason for this: Photos cannot go back to the extension and ask it for a rendered, full-size TIFF because the extension might have been uninstalled, or the image may have been synced to a device that lacks the extension. This is another case where Photos should warn users. It’s unreasonable to assume users understand that their 16-bit TIFF is being made from an 8-bit JPEG.

A word to the wise: there is no point in exporting a TIFF from Photos if you edited the image with an extension. I had to add a button in the RAW Power extension to export TIFFs, which is kind of crazy but necessary.

Support for multiple editors as well as TIFF export would be greatly improved if extensions could return TIFFs to Photos—and if Photos could send TIFFs to extensions. This could be an opt-in feature for extensions and users wanting the best quality.

Don’t Edit with “Edit With”

Photos has yet another way to work with third-party editors and, unfortunately, it is both prominently placed in the Image menu and produces inferior results in most cases.

The Edit With menu.

This feature is intended to let users edit in applications that modify files directly, such as Photoshop and Preview, that do not have Photos extensions. However, any application that can edit images will appear, including some that don’t make much sense in this context. Confusingly, the list may also include apps that provide Photos extensions, if those apps can also edit images outside of Photos (as RAW Power does). The prominence of the command means it’s all too easy for customers to choose Edit With RAW Power instead of the RAW Power extension. Extensions, as you will see, integrate much better with Photos. (As a simple fix, Photos could filter out apps with Photos extensions from the Edit With list.)

Because Apple intended Edit With for use with destructive editors, Photos protects the original. Instead of sending the original file, Photos makes a copy in its library and sends the copy to the external editor. The editor modifies the copy, and non-destructive behavior is maintained. Sort of. Because the external editor is making destructive edits, all that Photos can do is maintain the ability to use Revert to Original. No editing information is stored in the Photos library (since there isn’t any), nor can the user revise edits afterward in Photos or the external editor. (It’s technically possible to have a more non-destructive workflow, but doing so requires storing the edits separately and managing them yourself. It demands great care and is highly error-prone.)

I mentioned that Photos starts by making a copy of the original. That’s not entirely accurate. If the original is a RAW, Photos sends a TIFF instead. Why? Because it must provide a file that the destructive editor can modify, and as I mentioned earlier, RAWs cannot be edited and rewritten. In addition, not all editors support RAWs.

Consequently, Edit With is a particularly bad choice for RAW editing. I field many questions from customers wondering why they are unable to edit their RAW with RAW Power. The reason is usually that they used Edit With instead of the RAW Power extension. It may be possible for external apps to hunt around the Photos library and locate the RAW, but that is sketchy and not recommended (sandboxed apps cannot do this anyway).

Conclusion: Non-destructive Editing Is Great, Most of the Time

Non-destructive editing provides users with valuable features like instant revert, A/B comparisons, and fine-grained control over the editing process. However, it establishes a significantly different interface paradigm that applications don’t communicate well to users. The non-destructive illusion leads many photographers to believe their originals are being modified when they are simply hidden. These photographers then protect their originals by duplicating images needlessly. While smart copying can minimize the disk cost, such duplication leads to complexity and visual noise due to extra images in the grid. Some education would reduce customer nervousness when editing prized photos.

In addition to its own editing tools, Photos provides a clean extensions interface that provides non-destructive editing and syncing through iCloud. It has some warts, including issues with multiple editor support, export, and the Edit With feature. However, with a few fixes, Apple could greatly improve the non-destructive workflow in Photos, both in quality and understandability. To that end, I have filed bugs with Apple with my suggestions. Hopefully, we will get some improvements in a future release of Photos for macOS and iOS.