Understand Photo Storage Model on Apple’s Photos app and PhotosKit
I remember the time that I was an Android Developer and worked on a photo editing app that can save edited photos into users’ device. By saying saving photos into users’ device, it literally did the following things:
Read the image’s memory from GPU back to the main memory as it uses GPU to render the effects.
Encode the image as JPEG format and save it to an internal file.
Copy or move the file to the DCIM folder and notify the system that the content of the folder has been updated and please construct the photos database accordingly.
Back then, we must carefully decided where the photo files should be put since it may be accidentally scanned by the system and users would report to you that why they saw the icon files in their albums.
The method we used was not the best method, however it’s still approved by the Android system, until the Scope Storage was introduced and the Android team decide to correct this media storage behavior. Working on different companies later on, one of my main job was revamping the media storage logic by using the MediaStore API.
That’s the story when I was an Android developer, and I am thankful that Apple has introduced the PhotosKit in the first place, which encapsulates all the methods we need to interact with the Photos feature in iOS and macOS.
Working with the PhotosKit is not just as simple as saving and reading photos from the system, because it provides more features to help users manage their photos and their versions in the non-file system form.
My apps like PhotonCam and MyerTidy integrate deeply with the PhotosKit and have leveraged some features that PhotosKit provides like:
PhotonCam allows you to save both DNG+HEIF as a single photo.
If you don’t want to save the DND files for storage size considerations, you can choose to save the photo applied with filters, frame or cropping effects as the “Edited” version, along with the original untouched version. Later you can revert it back to the original state.
You can choose which version to load in the photo editor and in the gallery view.
Below I will share some information about the storage model that associated with the PhotosKit and the Photos app in Apple platform. Note that on macOS, you can still (and most would) manage photos as files. However the same logic would apply when you use the Photos app and PhotosKit API on macOS.
Photo Asset Representation
Traditionally, a photo is represented as a file. It can contain encoded JPEG data or other new formats. While PhotosKit still uses file representation to storage the data in the disk, layers are built upon of the file system to enable more features like versioning and favoriting.
If you are dealing with the PhotosKit API, you may already know that the entry to access a photo in the Photos Library is PHAsset.
What you see in the Photos app library is the current representation of a PHAsset. By saying “current representation”, I mean:
If there are more than one versions in a
PHAsset, it chooses to display the current/edited version.
If HEIF/JPEG and DNG files are saved in a
PHAsset, it chooses to display the compressed format.
If it’s a live photo, it shows the current version of the photo and will play the live video when users choose to play.
The Information sheet shows the information of the current version. That’s, if it’s a HEIF+RAW asset, only the size of HEIF file will be displayed, which may be misleading when calculating the storage occupation manually by users.
PhotosKit manages the photo assets in its own database. When requesting a photo, you only get a PHAsset. To find more resources (or files) it associates, you use some of the methods provided PhotosKit to get the PHAssetResource. A PHAssetResource represents a file representation of a PHAsset, which can be not only the traditional image files like JEPG or DNG, but also some auxiliary files like AAE file, which stores editing metadata that can be used to render the edited version of a photo from its original state.
Here is the image to illustrate the storage model mentioned above:
A Photo Asset, or
PHAsset, represents a collective files and data that are needed to display an image in the corresponding apps.
The illustration uses a Portrait Image as an example. Apps that can render portrait effect need the auxiliary data that is embedded in the original file, not the current/edited file(though it still can contain the portrait auxiliary data, but most apps won’t do that).
The information sheet displays information of the current/edited version of the photo asset.
When loading and editing the same photo in PhotonCam:
When you select a photo to edit in PhotonCam, it choose the original version to load by default.
You can tap the “…” button on the right-top corner to choose which version to load.
If you were about to re-edit the depth effect of a portrait photo, make sure to choose the current/edited version to load.
When saving a photo, selecting “Save as New Photo” will create a new Photo Asset while “Save as Edited Version” will replace the current/edit version and its adjustment data.
Auxiliary data Representation
Aside from the image data that can be used to decoded and created a bitmap to display on screen, there are some auxiliary data that will be used when displaying images:
Apple / ISO Gain Map is used to render images in SDR/HDR compatible environment.
Segmentation matte is used to distinguish parts of the image like the hair and face of a person.
Disparity image is used to created a depth map for a portrait photo and can be later render a depth effect.
Those data is stored along the primary image within the same image file, and PhotosKit has nothing to do with those auxiliary data.
My thoughts on this design
Although adding an additional layer to provide the functionalities is not uncommon nowadays, but making the decision to “sandbox” the photo files and provide a Framework for developers to use back then was a great move, as users may tent to manage the photo files using the traditional file system, as what Windows or macOS would did.
Notes to PM/Developer
Given the information above, I believe there are a few things that should be noticed by Product Managers and Developers when building apps:
Make sure what versions of the photo users want based your apps functionality.
For social apps, be sure to default to use the current version of a photo asset, because users may have edited the photo using other apps and just want to post it on the social platform.
For photos/video editor, it would be better to having an option for users to choose which version to load, as users may combine and chain some editor apps to compose the final photo/video they want.
Product Managers who work on multiple platforms may lack a comprehensive understanding of the Photo Storage Model on the Apple platform. As software engineers, we can engage in discussions with the PMs to ensure that we deliver the most optimal outcome.
Notes to Users
For users that are not a tech “nerd”, if you understand the content above, you can discover more features when using your Apple devices:
What you see in the Photos app when you click and open view an image in the large view is the current version of this photo asset. So when you share this photo, it defaults to share the current version.
For photos that have JPEG+RAW tag on the top-left corner, if you would like to share both the DNG and JPEG/HEIF files to others, you should turn on the “All Photos Data” toggle in the share options page.
After editing a RAW photo in the Photos app, sharing it will defaults to the current JPEG version. So be sure to share all photos data to get the DNG files when you would like to view and edit it on your Mac.
Photos-backup apps may only support backing up the current versions of your photos, or the other way around. So be sure to do a quick test of your preferred Photos-backup apps to see their functionalities about photo versions, otherwise you may lost some photo data when restoring them on the other devices.
If you delete a photo that takes up about 10MB in the information sheet, the system may release more than 10MB, as the photo assets may have more than one versions.