Sony Could Soon Punish You For Immoral In-Game Voice Chat Behavior

The patent seeks to help users with possible triggers for mental health issues.

Story Highlights

  • Sony has published a patent that will analyze the users’ emotional responses during in-game voice chat sessions. It will moderate toxicity if detected against the user’s moderation rules.
  • The offenders will be punished with different penalties depending on factors, including the number and type of offenses. The user could be banned or put into a training program before regaining access.
  • The hostile behavior could include profanity, harassment, harsh tone, swearing, etc. It will be automatically moderated to ensure a user does not feel distressed.
  • The voice chat will be uniquely altered for every player, and actions could include changing, censoring, or muting the hostile responses and the user.

Sony has published a new patent that will study the emotion of users during voice chats and give every in-game user a profile where they can specify moderation rules. The rules will let every player choose, and the system will automatically filter any specific immoral emotions or replies to ensure no trigger cues are activated for users to protect their mental health. Moreover, the offenders will be given different penalties depending on the severity.

The associated moderation action may include identifying a user who is the source of the triggering emotion (e.g., a specific player or spectator) and automatically adding the identified user to a block list or initiating another form of punishment or intervention (e.g., blocking access to gameplay session or platform, requiring remedial training as prerequisite to regaining gameplay privileges, in-game punishments affecting game status or gameplay).

The patent dubbed “EMOTION DETECTION & MODERATION BASED ON VOICE INPUTS” will ensure that offenders are automatically blocked or muted by the respective players as per set rules. Moreover, they will be given different penalties depending on the immoral conduct to ensure the game can become an enjoyable venture for the users. The penalties could be imposed for that gameplay session or may result in an outright ban.

The number of violations will only fuel the length and severity of penalties by Sony. As detailed, it may also use a three-strike plan before giving the final ban. All in all, it wants to ensure that the players have a pleasant time without any toxicity or hate abiding in the voice chats in a session, which can affect young gamers or users with mental health issues. It is a notable try to curb online bullying, and we may see it come into effect in the future.

A flowchart diagram showing emotion detection and emotion-based moderation based on voice inputs.

Moreover, players can dictate what they want to hear in voice chats. Any negative emotional replies, such as yelling, swearing, and the sort, can be avoided entirely or censored. It will significantly help users with mental health issues since voice chat can be automatically moderated. The patent puts great emphasis on the mental health and emotions of all players and how negative emotions can affect other users.

For example, a user may be stressed when other users are yelling or otherwise exhibiting anger. In such cases or in similar cases, the user’s mental health or sensory processing conditions may deteriorate or otherwise suffer when exposed to others’ negative emotions over extended periods of time.”

The system will moderate voice chats to confirm players trying to avoid “profanity, graphic content, bullying language, […] or other aggressive, hostile, or violent language (e.g., harassment, threats),” and the like, are safe from such behavior. It is worth noting toxic behavior could be textual, visual, or vocal in nature, as pointed out by Sony.

Thus, it will detect emotions and observe users based on the user’s emotion profile. As stated, any legal guardian can use the new system to stop or limit the child’s exposure to a toxic online game ecosystem. “Young players (e.g., children) may have parents, guardians, or other supervisors (e.g., relatives, child-care professionals) who wish to limit their charge’s exposure to such negative emotions and associated language.

Since every player can set individual rules for improved personal gameplay, the voice chat session could greatly differ for each person. However, an entirely unaltered session is also possible if the respective gamer does not specify any rules to curb any hostile retorts. 

Diagram showing an example implementation of emotion detection and emotion-based moderation based on voice inputs.

Different moderation actions can be chosen while moderating a specific emotional response, such as automatically muting or blocking the offender or changing the incoming words to censored voices. “The user emotion profile may include one or more moderation rules that specifies a moderation action responsive to one or more emotional states.

Moderation actions may include removing (e.g., deleting, muting, bleeping, auto-tuning, voice-generating) or modifying (e.g., blurring, blocking/blacking-out, auto-correcting or replacing with alternative language, emoji, graphics, UGC) the triggering language from the chat stream presentation provided to the user.”

In case the negative response is just harsh in tone, Sony’s proposed system will even be able to modulate the voice to take out undesired emotions and to make it sound more neutral, as mentioned by the patent. Thus, the discussed method will let the players stay in the chat session, allowing them to enjoy the experience in a moderated setting amid toxic players.

A user may therefore determine how to protect their own mental and emotional well-being and avoid stress related to such undesired emotions when interacting with other users in communication sessions. Such a user may also specify or customize moderation rules responsive to their own emotional states.”

If utilized, the patent may change multiplayer gaming as a whole, and we could see an automatic moderation system for voice chats in games. The system will allow every player to automatically censor or mute specific words or modulate the angry emotions of other players to create a controlled gaming session.

Similar Reads: Outer Wilds: Archaeologist Edition Rated For Nintendo Switch By ESRB.

Did you find this helpful? Leave feedback below.

Thanks! Do share your feedback with us. ⚡

How can we make this post better? Your help would be appreciated. ✍

Subscribe to our newsletter and get up-to-speed gaming updates delivered to your inbox.

We don’t spam! Read more in our privacy policy.


Shameer Sarfaraz is a Senior News Writer on eXputer who loves to keep up with the gaming and entertainment industries devoutly. He has a Bachelor's Degree in Computer Science and several years of experience reporting on games. Besides his passion for breaking news stories, Shahmeer loves spending his leisure time farming away in Stardew Valley. VGC, IGN, GameSpot, Game Rant, TheGamer, GamingBolt, The Verge, NME, Metro, Dot Esports, GameByte, Kotaku Australia, PC Gamer, and more have cited his articles.

Experience: 4+ Years || Education: Bachelor in Computer Science.

Related Articles

Join Our Community

Enjoyed this article? Discuss the latest gaming news, get expert help with guides and errors, and chat about all things gaming on eXputer Forums and Discord Server. Connect with fellow gamers who share your passion by becoming a part of eXputer's community.