Galaxy S23 Ultra’s Secret Weapon Could Be This Samsung Camera Chip

Samsung’s Isocell HP2 image sensor takes photos at up to 200 megapixel resolution and adds new HDR tricks.
Samsung

With its new Isocell HP2 image sensor, Samsung will try giving smartphone photographers the best of both worlds: a high  200-megapixel resolution and good image quality in challenging conditions. 

The HP2 is in mass production, but Samsung hasn’t confirmed which phone it’ll arrive in. The sensor is expected to power the main camera on the company’s flagship Galaxy S23 Ultra phone, though, which likely will debut at a Feb. 1 Samsung event.

Image sensor designers face a tradeoff. Increasing resolution means each pixel on the sensor is smaller, and for a given technology, smaller pixels aren’t able to gather light as well. That means shots taken in low light are marred by noise speckles. They lose detail in shadowed parts of a scene. And they suffer blown-out highlights in bright areas like skies.

The HP2, though, brings new methods to counteract those problems and make the most of each photon of light, Samsung revealed exclusively to CNET. The HP2 is a bit smaller than its HP1 predecessor but each pixel is more efficient at gathering light, an engineering direction that enables higher resolution photos and a smaller camera bump.

The South Korean electronics giant’s sensor can gather light more effectively in the first place, boost high dynamic range (HDR) photos to cope better with scenes with dark and bright elements, the company said. And when shooting at the full 200-megapixel resolution, Samsung uses AI technology to help render the finest details.

It’s not yet clear how well the sensor will perform in real-world testing. But it’s no surprise Samsung is focusing on the technology. Camera improvements are a prime reason to upgrade phones, with better photos and videos more noticeable than marginally better processors, battery life and network technology.

“The full 200MP resolution especially shines when shooting at concerts or outdoors where there’s lots of detail to be captured,” said JoonSeo Yim, executive vice president of Samsung Electronics’ sensor business. “It may not be the predominant setting for most consumers, but we definitely see the need for highly detailed images.”

Apple, Samsung’s top smartphone rival, is likewise investing heavily in its cameras. Comparatively large lens elements protrude from the back of iPhone 14 Pro models to show off camera performance, and Apple has upgraded its sensors for better high-resolution and low-light shooting.

Better pixel binning options

One of the headline techniques for improving smartphone photos is called pixel binning. With it, groups of physical pixels can be combined into larger virtual pixels that gather more light when it’s dim, trading off resolution for lower noise and better color.

Samsung isn’t alone in using pixel binning. You’ll see it in the Apple iPhone 14 Pro, Google Pixel 7, Xiaomi 12T Pro and other phones, but the HP2 sensor is one of the most advanced. Apple and Google, for example, use 2×2 pixel binning that turn four physical pixels into one virtual pixel. Samsung’s flagship Galaxy S22 phones have offered 3×3 pixel binning since 2019, offering 108 megapixel photos in good light and 12-megapixel photos when it’s dim.

Samsung’s HP2 can take 200-megapixel photos under good conditions. When it’s dimmer, pixel binning groups pixels into 2×2 chunks for a 50-megapixel image. And when dimmer yet, Samsung’s 4×4 “Tetra2pixel” chunks take a 12.5-megapixel photo.

The two levels of pixel binning were available on the 200-megapixel HP3, announced in 2022. However, the HP3 uses smaller pixels that, while minimizing camera bulk, aren’t as good at capturing light in the first place. The HP1, announced in 2021, also had it. But the HP2 adds some other tricks the HP1 lacks.

Pixel binning ups and downs

Pixel binning has some other advantages. Cameras can crop in on the central portion of the image to zoom into more distant subjects. It’s a key foundation to the effort to give smartphones zoom abilities like traditional camera lenses. Pixel binning also opens up new options for high resolution 4K and 8K video.

Pixel binning has downsides, though. It takes a lot of battery power to process all those pixels, and storing high-resolution photos gobbles up a lot of storage space. And high-resolution sensors, while nice in principle, don’t achieve top image quality unless they’re paired with high-quality lenses.

“The full 200MP mode does require more RAM and power,” Yim said, which is why such high resolution sensors are only found on high-end smartphones.

One complication with the HP2 is figuring out color when shooting 200 megapixel photos. Digital cameras capture either red, green or blue light for each pixel, but the Tetra2pixel design means each 4×4 pixel group captures only one of those colors. To help fill in the color detail needed within those 16-pixel groups, Samsung uses an artificial intelligence algorithm, the company said.

Photography enthusiasts — a population likely to be very interested in the sensor’s high resolution — can shoot raw images at 200 megapixels, Samsung said. Raw images are larger but offer higher image quality and better editing flexibility than conventional JPEG or HEIC photos.

Samsung HP2’s image quality improvements

The sensor has other tricks to boost image quality, particularly with high dynamic range scenes with both bright and dark details. Here are a few:

• A technology called Dual Voltage Transfer Gate (D-VTG) gives each pixel a 33% better ability to gather light, which should improve image quality in dim scenes and cut back on washed-out white patches in bright skies.
• Samsung’s Dual Slope Gain (DSG) feature improves HDR photos by digitizing each pixel’s exposure data at two different scales to gather bright and dark data when shooting in 50-megapixel mode. The abundant pixels on the sensor mean some pixel quartets are tuned for bright light and others for dimmer light.
• A related feature called Smart-ISO Pro is a separate HDR technology that adapts to different scenes, employing different combinations of sensitivity settings appropriate for the different frames used to build the HDR photo.

Another new feature in the HP2 is an improved autofocus with a technology called Super QPD. It can spot either horizontal and vertical lines across 2×2 pixel groups, helping the camera lock onto details like horizons or tree trunks even when it’s dim, Samsung said.

Each HP2 pixel is 0.6 microns, or 6 millionths of a meter, wide. That’s a shade narrower than the 0.64 microns of the HP1, which means the HP2 is a slightly smaller image sensor at 11.3×8.4mm. For comparison, a human hair is something like 75 microns across. Combined into a 2×2 array for 50-megapixel photos, the pixel width increases to 1.2 microns, and in 4×4, to 2.4 microns.

“We expect that high-resolution image sensors will become a standard feature in future flagship smartphones,” Yim said. “Because of that, we think it’s important to continue our efforts, from advanced pixel processes below 0.5 microns to pixel performance and algorithms.”

Larger sizes are better at gathering light. The Samsung pixel sizes are pretty similar to the iPhone 14 Pro’s main camera sensor, which uses 2.44 micron pixels in 12-megapixel mode and 1.22 microns in 48-megapixel mode.

When it comes to video, the HP2 has many options. It can shoot 8K video at 30 frames per second by using the sensor in its 50-megapixel mode. It can shoot 4K video at 120fps, or, if Smart-ISO is engaged, at 60fps. For 1080p video, the sensor will shoot at 480fps without autofocus and 240fps with autofocus.

Correction, 2:48 p.m.: This story misstated the pixel size on the Samsung Isocell HP1 image sensor. Each pixel is 0.64 microns across.