Tencent Patents To Automate Live Game Stream Commentary Using AI

We could soon see live game streams with auto-generated commentary using AI technology.

Story Highlights

  • Tencent has published a new patent that aims to produce automated commentary during live game streams using AI. It will take out the tedious manual steps to ensure that players can focus on playing games while the AI handles the commentary, as the patent suggests.
  • The AI will utilize various innovative technologies to generate commentary in real-time using multiple attributes and objects. The commentary will be produced as the user plays the game on stream.
  • The patent will also solve issues like more computing power and electricity consumption and increase automation to aid streamers.

Live streaming has swiftly and thoroughly evolved over the last few decades, reaching new horizons thanks to the demand and availability of services like Twitch. Video games live streaming encapsulates most of the community, with users watching other streamers play their favorite games while delivering amusing commentary.

It seems live game streaming is about to get easier than ever before with a new patent published by Tencent. We came across the patent dubbed “COMMENTARY VIDEO GENERATION METHOD AND APPARATUS, SERVER, AND STORAGE MEDIUM,” which discusses automating commentary in live streams using the wonders of AI.

The patent notes that a game streamer needs to engage with the community by commenting on everything occurring while playing games, which can prove to be a monotonous task. It further states that manual commentary in live streaming requires a “long production process and high manual operation costs.

Tencent‘s proposed system will utilize AI to generate commentary for streamers as they play the game automatically. It will employ “a computer vision technology, a speech processing technology, and a natural language processing” AI technology to seamlessly generate the commentary during live game streams.

The patent contains highly technical terms and jargon, so it is difficult to decipher in a reader-friendly manner. In short, it works “through online analysis on the game instruction frame, the commentary audio is generated, the game video is rendered, the commentary audio and the game video are aligned in time to generate the commentary video.”

The image showcases a method for determining a target game event utilizing the proposed system.

While live streaming a game, any in-game activity, “game operation instruction,” performed in the game will be recorded and sent to the “commentary server,” which “can render the game in the commentary server 120 based on the received game operation instruction.” 

The patent elaborates on the commentary server module in detail; it is a back-end server behind any game being live-streamed online. The commentary server is designed to create online game commentary automatically using AI technology and push the live stream to the client devices viewing it online.

The further patent notes, “The commentary server automatically generates a corresponding commentary video stream during the game, and pushes the commentary video stream to a livestreaming terminal for playing, to improve the generation timeliness of the commentary video.”

The figure shows an architectural diagram of the working of a commentary system using AI.

It continues, “To generate the commentary video during the game in real time, in a possible implementation, online game video rendering and online analysis and commentary can be implemented through analysis on the game instruction frame.

As stated above, Tencent’s system can generate commentary in real-time during the live game stream, which comes with its slew of benefits. “There is no need to wait for the game to be over to obtain and process the game video to generate the commentary video, thereby further improving the real-time performance and timeliness of the commentary video generation.

It can consider various attributes of virtual objects in games before processing to create automated commentary. Some attributes include “location information, health point information, speed information, level information, skill information, feat information, equipment information, score information, etc.

Meanwhile, the objects can refer to any virtual object that the user can control, for instance, a building, playable and non-playable characters, and the sort for automatic commentary generation during live game streams.

The image reveals a lengthy flowchart of a commentary video generation method.

It is beneficial because it can automate many procedures behind the scenes during live streaming by utilizing artificial intelligence technologies. It will reduce the usage of computing resources and electricity, as per Tencent’s patent. The commentary will be created on the spot as the user plays the game on stream.

Additionally, the player could focus more on the game and leave the commentary to AI. The patent, if adopted, could transform the streaming service industry as we know it.

Video games of all genres, including “a sports game, a military simulation program, a multiplayer online battle arena (MOBA) game, a battle royale shooting game, a simulation game (SLG), etc.” can possibly use Tencent’s proposed method.

The patent sticks to using “a virtual online commentary scenario” as an example, meaning any multiplayer experience, like MOBA. Tencent explores a team fight situation in a MOBA game to elaborate on the working of the patent. The AI will use different variables to prioritize what to talk about during the game in live streaming. 

For instance, the number of virtual objects can determine what game event is selected to be commentated on by Artificial intelligence. “For example, if the game includes two team fights, where the team fight A has 3 virtual objects and the team fight B has 7 virtual objects, priority is given to the game event corresponding to the team fight B in selecting the game event.

The image shows a process diagram of the complete generation of a commentary video stream and a block diagram of a commentary video generation apparatus.

Tencent has been at the forefront of patenting innovative schemes alongside Sony and Microsoft to get ahead of its western competition. For instance, it only recently patented an ingenious Virtual Reality headset that seeks to solve some of the most prominent issues in the current VR devices and greatly increases immersion.

Tencent is highly popular for keeping several accredited games like PUBG, League of Legends, Ring of Elysium, Path of Exile, and more under its belt. The giant Chinese conglomerate is currently developing anticipated projects like the new AAA entry, Code: To Jin Yong.

What are your thoughts regarding Tencent seemingly tinkering with AI to automate gaming live streams in the future? Do you think automated live streams will aid streamers in focusing more on the gameplay only? Do let us know your views in the comments below.

Did you find this helpful? Leave feedback below.

Thanks! Do share your feedback with us. ⚡

How can we make this post better? Your help would be appreciated. ✍

Subscribe to our newsletter and get up-to-speed gaming updates delivered to your inbox.

We don’t spam! Read more in our privacy policy.

Subscribe to our newsletter and get up-to-speed gaming updates delivered to your inbox.

We don’t spam! Read more in our privacy policy.


Shameer Sarfaraz is a Senior News Writer on eXputer who loves to keep up with the gaming and entertainment industries devoutly. He has a Bachelor's Degree in Computer Science and several years of experience reporting on games. Besides his passion for breaking news stories, Shahmeer loves spending his leisure time farming away in Stardew Valley. VGC, IGN, GameSpot, Game Rant, TheGamer, GamingBolt, The Verge, NME, Metro, Dot Esports, GameByte, Kotaku Australia, PC Gamer, and more have cited his articles.

Experience: 4+ Years || Education: Bachelor in Computer Science.

Related Articles

Join Our Community

Enjoyed this article? Discuss the latest gaming news, get expert help with guides and errors, and chat about all things gaming on eXputer Forums and Discord Server. Connect with fellow gamers who share your passion by becoming a part of eXputer's community.