Nvidia's new RTX broadcast engine features green screen mode powered by AI

The latest GPUs from Nvidia's RTX series are the first to offer real-time ray tracing, making games more beautiful than ever before. However, they are also powerful computing platforms for AI development. Nvidia has introduced a new artificial intelligence application based on Tensor cores in RTX cards, called RTX Broadcast Engine. Among other things, the new broadcast engine can cut the background of a video stream to create an artificial "green screen".

The RTX broadcast engine is intended for game broadcasters and includes several functions. The Greenscreen RTX is undeniably the most impressive of them. Using the computing power of RTX cards, Broadcast Engine can process a video in real time and understand what parts of the streamer are and what parts of the background. So, you can replace the background with different scenes or make it totally transparent in the stream.

RTX Greenscreen is similar to the built-in features of some Logitech C922x and Razer Stargazer webcams, but Nvidia's solution should work with any video stream. Webcams with custom background suppression do not work very well anyway. If Nvidia's solution works well, it could make these devices much less attractive.

RTX Broadcast Engine also features a Face-Tracked Augmented Reality (AR) feature called RTX AR. It recognizes faces with enough detail to map your features and render them on a 3D model. Again, this feature is similar to what you can get with specific hardware items such as the iPhone. This camera uses IR points and special cameras to map your face and produce "Animoji" versions of your face. Nvidia does not need true depth data, the GPU simply calculates it with the AI.

Finally, there are RTX style filters, which use AI to apply filters to a full video. The idea here is that you can use the broadcast engine to match the style of a game or an illustration. In the demonstration video, Nvidia applies a "Starry Night" filter to a real-time video. We do not know how much this will match the look of a game, but it could produce interesting streaming effects.

Developers can request early access to RTX Broadcast Engine now on the Nvidia website. It will be released in the coming months. Nvidia is also working with the makers of the OBS streaming software to integrate the RTX Greenscreen feature.

Now read: