February 27, 2018

Apple has applied for a foveated rendering patent

The eye of the beholder

Apple is probably going to bring out a headset with foveated rendering and 8K resolution.

Way back on the 4th of July 2017, we reported here that Apple had acquired a German company that made eye tracking technology. At the time, we speculated that Apple was planning on using the eye-tracking technology in conjunction with a process known as “foveated rendering.” We can now report that barely a month after that (in August), Apple applied for a Europe-wide patent for a foveated rendering chip.

This is a technique whereby the eye movement is tracked and only the area in the center of the vision is rendered in high resolution, whilst the peripheral area is rendered in lower resolution. This reduces processing time or power. It is based on the fact that the area at the center of a person’s vision is clearer, sharper and more focused than the area at the periphery of their vision.

Foveated image - a closer look

How foveated rendering looks

In the Apple patent, the area at the center of the user’s vision is shown at 8K resolution and this area is identified by eye-tracking. The eye-tracking updates rapidly, in real-time, and informs the graphics processor. The graphics processor in turn rapidly updates the relevant area of the display to make sure that the area the user is focusing on is displayed at the highest resolution.

The patent discloses a system that updates 120 times per second. The display latency for each eye is 240 Hz. But because both eye has to be updated, the effective system latency is 120 Hz.

Foveated imaging in action

Foveated imaging in action

The patent application is broadly worded to include different types of display, not just head mounted displays. These include computers, mobile phones, wall-mounted screens and even projectors. However, this is normal for patent applications to maximize any potential future benefit.

In practice, it would be of limited value in a wall-mounted or projector-based display. In a smartphone it would have value if the smartphone is used inside a headset display. The main application of the patent is obviously head-mounted displays. In other words, an Apple Virtual Reality headset.

How foveated rendering works

How foveated rendering works

Click Here to Leave a Comment Below

Leave a Reply:

CommentLuv badge