Phone Camera Sensor Guide – There are more important things than megapixels

Unless you are a keen photographer, phones have become the main camera for most people nowadays. I haven’t used a dedicated camera for holidays in years, and most of my product photos are taken with my phone (currently, the Pixel 8 Pro).

Apart from the convenience of having an all-in-one device, phone cameras have become so good that there is little reason for most people to have anything other than a phone for photography, especially when most people only share photos online where the images inevitably end up being compressed.

However, the quality of photos produced by a phone will vary widely based on the physical camera sensor that’s used in the phone and many of the features built into the phone such as the ISP of the chipset used in the phone and the software processing down by the phone itself.

While most companies still use megapixels as the stand-out number when advertising the phone camera performance, this is only one element that affects performance, and it isn’t even the most important.

The article looks at the various technologies used in phone cameras and how they affect the quality of your photos.

What are CMOS sensors and why they are used vs charge-coupled devices (CCD)

CMOS (complementary metal-oxide semiconductor) sensors and CCD (charge-coupled device) sensors are the two main types of image sensors used in digital cameras.

CMOS sensors are the most common type of sensor found in smartphones today. Compared to CCD sensors, CMOS sensors:

• Are cheaper to manufacture
• Consume less power
• Offer faster data transfer speeds
• Provide better low-light performance
• Allow more on-chip processing

These advantages have led to CMOS becoming the dominant sensor technology over CCD in smartphones and most digital cameras.

Main companies producing camera sensors for phones

The main companies producing camera sensors for smartphones include:

• Samsung: The world’s largest image sensor manufacturer with sensors in many flagship phones from brands like Xiaomi, Oppo, Vivo etc.
• Sony: A leading sensor manufacturer providing sensors for Apple, Samsung, Xiaomi and other phones.
• OmniVision: Major supplier of sensors for phones from Chinese brands like Oppo, Vivo, and Xiaomi.

Flagship camera sensors

Samsung:

• ISOCELL GN1: 50MP, 1.2μm pixels, Smart ISO to minimize noise

Sony:

• IMX766: 50MP, 1.0μm pixels, omnidirectional PDAF, staggered HDR

OmniVision:

• OV50A: 50MP, 1.0μm pixels, support for seamless zoom, PDAF

Descriptions of various sensor types

Wide / Ultrawide sensors: Capture a wider field of view, useful for landscapes, architecture etc. E.g. 13MP ultrawide in iPhone 14 Pro.

Telephoto / Periscope sensors: Provide optical zoom for distant subjects without quality loss. E.g. 10MP 3x telephoto lens in S23 Ultra.

Macro sensors: Allow extreme close-up photography of tiny subjects. Dedicated 2MP macro lens in recent Xiaomi phones.

Time-of-Flight (ToF) sensors: Measure depth using infrared light. Used alongside RGB sensors for portrait mode, AR etc.

Depth sensors: Dedicated sensors for creating depth maps to simulate shallow depth of field in portrait shots. E.g. depth vision camera in iPhone 14 Pro.

Optical Zoom vs Digital Zoom

Optical zoom uses the optics of a telephoto/periscope lens to zoom in without reducing image quality. Provides up to 10x lossless zoom in phones like S23 Ultra.

Digital zoom simply crops and enlarges part of the image, reducing quality at high zoom levels. Best avoided for photography but can be useful for video.

Fixed-focus lens vs Autofocus

Fixed-focus lenses have a fixed focal distance and cannot adjust focus. Common in front-facing selfie cameras and basic rear cameras.

Autofocus lenses can automatically adjust focus by moving internal lens elements. Different AF technologies like PDAF, Laser AF, and Dual Pixel AF help achieve fast, accurate focus locking. Critical for photography.

Focal Length

The focal length of a lens (measured in mm) determines the angle of view and magnification. Shorter focal lengths provide a wider field of view. Longer focal lengths result in narrower field of view with higher magnification. Smartphone cameras typically have focal lengths from around 4mm (ultrawide) up to around 125mm (10x periscope telephoto) .

Phase detection auto-focus (PDAF)

PDAF uses sensor pixels to detect phase differences between light entering from opposite sides of the lens. This allows very fast and accurate autofocus tracking of moving subjects, important for photography/videography.

Megapixels

The number of megapixels (MP) determines the maximum image resolution. However, pixel size, sensor size and image processing also hugely impact image quality. E.g. main 108MP sensor in S23 Ultra uses pixel binning to take exceptional 12MP images.

Camera Sensor Size

Larger sensors can capture more light, reducing noise and increasing dynamic range. Most phones use tiny 1/1.x” to 1/2.x” sensors, while the Sony Xperia 1 IV features a large 1” sensor for significantly better quality.

Photosites

Photosites are the light-sensitive elements in each pixel that convert photons into electrons to form the image signal. Larger photosites perform better in low light. Advanced sensors use pixel binning, combining signals from multiple tiny photosites to increase low light sensitivity.

Shutter Speed

The shutter speed controls the length of the exposure to light. Slow shutter speeds are used in low light but can result in blurry images. Faster speeds help freeze motion but require good lighting. Smartphone cameras typically have electronic shutters with speeds from 1/8000s up to 30 seconds.

ISO

The ISO setting controls the sensitivity of the camera sensor to light. Higher ISO settings result in brighter images in low light, but also more visible noise or grain. Smartphones use advanced noise reduction processing to allow usable images even at high ISOs.

Aperture

The aperture adjusts how much light passes through the lens to the sensor by changing the size of the opening. A wider aperture (lower f-number like f/1.8) provides faster shutter speeds in low light but reduces depth of field. Most phones have a fixed aperture lens.

Optical vs Electronic Image Stabilisation

Optical image stabilisation (OIS) uses gyroscopes to move optical elements, countering device movement to reduce blur.

Electronic image stabilisation (EIS) uses digital processing like cropping and warping to counter device shake. Best paired with OIS for maximum stabilisation.

RGGB vs RYYB sensor filter arrays

Most camera sensors use a RGGB filter array with alternating red, green and blue color filters over each photosite to capture colour information.

Some sensors, like the main one in recent iPhones use RYYB, swapping green for yellow filters to improve low-light performance.

Quad Bayer Sensors vs Quad Pixel AF

Quad Bayer Sensors have four photosites behind each colour filter to improve light capture. Images are combined and interpolated back to standard resolution through pixel-binning algorithms.

Dual Pixel AF / Quad Pixel AF uses dedicated AF photodiodes within pixels for faster phase detection autofocus. Still provides a standard resolution RGB output image.

Software Aspects

Advanced software processing is crucial to maximize image quality from smartphone camera hardware. Key techniques include:

• Multi-frame noise reduction
• Semantic segmentation
• Super resolution / resolution synthesis
• Computational photography (Portrait mode, Night mode etc.)

Smartphone cameras also leverage AI and machine learning for scene detection, facial recognition, style transfers and other creative effects.

The software aspects of a phone camera performance should not be underestimated. On paper, Pixel and iPhones don’t have the best camera sensor hardware, yet they are consistently regarded as the best phones for photography.

HDR

High Dynamic Range (HDR) imaging combines multiple exposures to retain details in the highlights and shadows. All smartphone cameras use HDR, capturing bracketed frames which are merged automatically.

Image Formats

HEIF/HEIC uses advanced compression and supports HDR/computational features. Widely adopted but lacks third-party app support.

AVIF is an emerging format building on HEIC using the AV1 video codec for 30% better compression. Limited device support currently.

So, in summary, while megapixels make for nice marketing, factors like sensor size, lens quality, image processing and software ultimately determine real-world camera performance on modern smartphones. Understanding the hardware and technologies that go into smartphone cameras can help make better choices when selecting a device.