Sony Wants To Track The Eyes Of Disabled Players For In-Game Assistance

The proposed system is seeking to increase immersion.

Story Highlights

  • A new patent by Sony wants to track the gaze of players to personally help them in games.
  • It will track the gaze to provide virtual indicators with directional sounds to aid users during gameplay.
  • The company wants to assist disabled gamers by making it easier for them to spot details they miss.

Sony has long been helming the accessibility features in the gaming scene for disabled and vulnerable players. It has also been exploring many unique ways to help users during gameplay. A patent recently published by the company now wants to track the gaze of players to aid them during specific game moments. The legal doc talks about using a camera to track the players and keep track of ongoing gameplay for personal assistance.

The patent dubbed “GAZE-BASED COORDINATION OF VIRTUAL EFFECTS INDICATORS” will store the gaze data gathered from the player using a camera. The proposed system will be able to detect where the player is looking in the game. An indicator will pop up if the player hears a sound during gameplay but fails to look toward it or probe into the source of that sound. It will realize that the user may have missed the sound.

A method and system for providing gaze-based generation of virtual effects indicators correlated with directional sounds is disclosed. Gaze data is tracked via a camera associated with a client device to identify a point of focus within a three-dimensional virtual environment towards which one or both eyes of the player are focused,” mentions the Sony patent in detail.

The image shows an example of a virtual effect indicator as per the proposed system.
The image shows an example of a virtual effect indicator as per the proposed system.

Sony mentions that the user will be alerted about the missed directional sound by a virtual effect indicator that will be created using the gaze data. Now, most users would be able to identify the source of the sound, but those with a disability may struggle. The company has issued this patent to support select gamers who have a disability that hinders gameplay. This accessibility feature is meant to assist the gameplay where it’s needed.

It argues that there are already plenty of accessibility features that are used in games, but there is still room for improvement. There is a huge lack of accessibility features, especially for blind gamers or those with sensory disabilities. Tracking the gaze and assisting players would make it possible for more affectees with sensory issues to enjoy the game. Yet, those users could still play without extreme hand-holding in gameplay.

Therefore, there is a need to provide a service that provides virtual effect indicators based on certain cues that the player is not hearing various directional sounds using gaze-based detection.”

The image shows a method for detecting gaze to create virtual effects indicators correlated with directional sounds.
The image shows a method for detecting gaze to create virtual effects indicators correlated with directional sounds.

Related Content:

The gameplay design will become more inclusive using the proposed system by Sony. Multiple ways have been discussed in the patent that would be used to alert the player via indicators. A user may see written cues, an arrow towards the direction, a notification alerting the player toward the direction of the sound, and/or a combination of these.

Was this article helpful?

Thanks! Do share your feedback with us. ⚡

How could we improve this post? Please Help us. ✍

Get up-to-speed gaming updates delivered right to your inbox.

We don’t spam! Read more in our privacy policy.


Shameer Sarfaraz is a Senior News Writer on eXputer who loves to devoutly keep up with the gaming and entertainment industries. He has a Bachelor's Degree in Computer Science and several years of experience reporting on games. Besides his passion for breaking news stories, Shahmeer loves spending his leisure time farming away in Stardew Valley. His articles have been cited by VGC, IGN, GameSpot, Game Rant, TheGamer, GamingBolt, The Verge, NME, Metro, Dot Esports, GameByte, Kotaku Australia, PC Gamer, and more. Experience: 4+ Years

Related Articles