Video games have steadily risen in popularity for years. And with people looking for new ways to socialize and stay entertained during the pandemic, the trend has only accelerated. Gaming is now a bigger industry than movies and sports combined. Revenue for gaming grew 12 percent in 2020, up to $139.9 billion from $120.1 billion in 2019, according to a report by market research firm SuperData. And at one point last year, four out of every five people in the United States had played a video game within the previous six months.

“It used to be ‘what to watch’ and now it’s ‘whether to watch,’” venture capitalist Matthew Ball wrote. “And the answer is increasingly ‘no, I’m going to play a game.’”

So what’s next? Culturally, gaming will only continue to become more mainstream. But what tech innovations are shaping the future of video games, and how will they influence the gaming experience?

WHAT WILL THE FUTURE OF GAMING LOOK LIKE?

• Virtual reality

• Augmented reality

• Artificial intelligence

• Cloud gaming

• High-fidelity graphics

• The metaverse

Virtual Reality

For decades, virtual reality has tantalized gamers with the prospect of a fully immersive experience. But the technology has been slow to deliver on that promise.

Polygon’s Ben Kuchera put it bluntly last year: “VR has been five minutes away from some kind of breakthrough for about eight years.”

VR is still a niche category when compared to the rest of the gaming industry (it accounted for less than half a percent of all gaming sales in 2020). And despite its buzzy status, it continues to give many consumers pause.

“Right now we’re sort of in this trough of disillusionment about VR,” Kevin Mack, a VR game developer, told Built In in 2020. “There was a lot of hype around it in 2015 and 2016, and then the whole world sort of got butt-hurt that their first-generation VR headset didn’t instantly morph into the Holodeck.”

“The whole world sort of got butt-hurt that their first-generation VR headset didn’t instantly morph into the Holodeck.”

Although VR has yet to live up to the hype, tech companies like Facebook, Valve and Sony are busy trying to advance the industry, investing considerable resources to develop VR hardware and games.

Most VR headsets weigh over a pound and must be strapped tightly to a user’s face. It’s not terribly comfortable. You get sweaty and after a half hour of play your energy is sapped.

This experience chafes against that mode of playing which is typical of gaming enthusiasts — spending hours comfortably sunk into a couch. If VR hardware can’t align with the preferences of gamers, will it be able to survive? Until companies slim down their VR headsets, get rid of cumbersome connector cables and lower prices, most gamers — save the early adopters and tech enthusiasts — will continue to balk.

VR’s gaming future

Companies are busy making VR more appealing to a wider audience, and hardware prices are dropping. But even when those hurdles are cleared, the fact that the typical VR experience is so socially isolating might limit its upside.

“[VR] is a solitary experience. It’s a thing that you’re doing on your own and it’s a thing that you choose to do to the exclusion of anything else,” Mack said. He enjoys playing VR games, but if someone else is around, he thinks twice before strapping the headset on.

“I still generally wouldn’t really wear one that much at home if my girlfriend is there too,” he said. “Because I feel like I was completely cutting myself off from the social environment.”

Though he recognizes the limitations, Mack remains optimistic about VR’s future.

“VR, I think, will remain niche, but it could potentially turn into a big niche,” he said. “I think we’re going to see some very impressive stuff and very compelling stuff come down the pipe in the next couple of years.”

Mitu Khandaker, a professor at New York University’s Game Center, is hopeful about VR’s role in gaming, she said in a 2020 interview with Built In. Khandaker just doesn’t think it’s going to look like people alone in their homes playing through a headset, so much as a co-located experience that multiple people share in.

“I think that the future of VR is more through social VR,” she said.

Indeed, several VR games — such as Rec Room and VRChat — offer social experiences where users can interact and hang out with each other in real time.

If VR unlocks more, not less, connections with other people, it will be able to earn a prominent place in gaming’s future.

Augmented Reality

In the summer of 2016, parks and plazas were swarming with Pokémon collecting enthusiasts.

Pokémon Go is an augmented reality smartphone game in which colorful animals called Pokémon overlay a person's natural range of vision.

The game was most people's first experience with AR and remains the technology's largest success story.

But Pokémon Go's long-term success is due to its treasured intellectual property. There are many different games, novels, and movies where Ash Ketchum and Pikachu may be found. The game's virtual-real interaction between digital characters and realistic locales is the true secret sauce.

That's why AR is growing faster than VR: People want games that interact with reality rather than remove them from it.

“I don't believe AR entertainment will strive to be immersive,” Mack added. “In [Pokémon Travel], I would go to particular locations simply to catch a Pokémon. That's a big societal driver.”

The x-factor that catapulted Pokémon Go into a multi-billion-dollar craze was farther out into the neighborhood rather than deeper within goggles. Its success will motivate additional gaming firms to attempt to capitalize on customer desire for virtual-real hybrid games.

“I could picture a game like hide-and-seek or laser tag,” Mack added. “It's a natural fit.”

AR, says Rogelio Cardona-Rivera of the University of Utah's School of Computing, will be more fruitful ground for game makers than VR in the immediate future.

“Instead of completely simulating reality, I believe designers might better track complementing reality,” he told Built In 2020. “Then we could see some AR lessons reapplied to VR.”

Mobile AR gaming has gotten the greatest momentum thus far. But tech giants like Facebook, Apple, Snap, and Magic Leap believe the future of AR lies in custom-made glasses.

Artificial Intelligence

The idea of artificial intelligence has been expressed in gaming for decades — most prominently in non-player characters, like the colorful ghosts in Pac-Man or the innocent bystanders in Grand Theft Auto.

In recent years, gamemakers have taken a more sophisticated approach to NPCs. Many NPCs are now programmed with behavior trees, which allow them to perform more complex decision-making. The enemy aliens in Halo 2, for example, have the ability to work together and coordinate their attacks, rather than heedlessly beeline into gunfire one by one like they’re in a cheesy action movie.

Still, NPCs can only do what is written in their code. Their behavior, however intelligent it seems, is still determined in advance by the game’s designers.

In the future, could we expect to see more advanced AI appear in commercial games? It’s possible, according to experts, but not all are convinced it’s coming anytime soon.

“You can try and build a really cool, comprehensive AI system which is about letting a character behave in all kinds of ways the designer hasn’t anticipated,” Khandaker said. “But if there’s too much of that, there’s no guarantee about which way the story will go and whether it’s going to be any fun.”

In other words, even if we could give NPCs minds of their own and let them run free in games, odds are that their autonomy would result in a less fun experience for the player. A rogue NPC could decide to shirk its duty to help the player advance toward the next level or take the player on a nonsensical quest where nothing happens.

“Games are a pretty conservative industry, in terms of the willingness that publishers or studios have to take risks.”

Assassin’s Creed Valhalla

In addition to presenting game design challenges, free-range NPCs may be a non-starter when considered from a purely economic perspective as well.

“Games are a pretty conservative industry, in terms of the willingness that publishers or studios have to take risks,” Khandaker said. “Because there is such a great history in terms of design for what does work in games, there’s a real sense of wanting to keep doing that same thing.”

Putting more sophisticated NPCs in games may be possible. But if it costs a lot of money and fails to improve the player’s experience, studios lose an incentive to make it happen.

“The biggest challenge for AI is to mimic what is perhaps the most complex and mysterious capacity of the human brain: imagination,” Julien Desaulniers, the programming team lead of AI and gameplay on Assassin’s Creed Valhalla, told GamesRadar. “Having AI generate narrative content is taking this to a whole new level, one that not even all human beings can perform well.”

AI isn’t just part of the gameplay experience though. It’s part of the game-making experience. For several years now, designers have been using AI to help them generate game assets, which frees them up from painstakingly drawing each individual tree in a forest or rock formation in a canyon. Instead, designers can offload that work to computers by using a technique called procedural content generation, which has become fairly standard practice in the industry.

Procedural content generation is also used to create game levels — sometimes randomly — so the player can enjoy a fresh experience each time. (The 2016 game No Man’s Sky took this technique to the extreme, as the entire open-world environment of the game is procedurally generated and was not sketched out ahead of time by the game’s creators.)

Playing Smart

Some game designers use neural networks to create custom game levels for players, a method described by NYU professor Julian Togelius.

For example, in 2009, researchers quantified each player's preferences while playing Super Mario. Maybe a level had too many leaps and not few sewers, or too few money and too few dangerous characters. A computer was given player data. The computer then generated additional levels based on the player's selections.

While AI creates gaming components and even levels, human designers aren't in danger of losing their jobs just yet.

A human game development team can create a game from start with a level of excellence that an AI system cannot match, Togelius stated in his 2018 book Playing Smart.

Academics and game designers alike are still working on creating AI systems that are engaging for players.

Cardona-Rivera sees AI as a game master who directs a human player.

“Imagine having an AI ‘director' that watches what you do and guides your experience,” he continued. “That's sort of what my study — and other fascinating work in the area — tries to do.”

“Imagine having an AI ‘director' that watches what you do and directs the unfolding experience for you.”

Until they do, human designers and computer algorithms will continue to collaborate to produce the next generation of video games.

AI may not yet be able to create high-quality games from scratch, but it can give useful input to game creators. I see it often.

“It's uncommon to find a commercially launched game that doesn't ‘phone home' to the developer,” Togelius wrote.

Games often capture player data. This data is put into an algorithm that people use to adjust games depending on user preferences.

“Games are so much about creating an engaging experience,” Khandaker remarked. “That's what AI is for.”

Cloud Gaming

The future of gaming might take place on someone else’s computer. That is to say, in the cloud.

Cloud gaming offers users the ability to play video games streamed from tech companies’ faraway servers. In the same way they stream Netflix movies on their laptops without needing to pop in a DVD first.

In theory, this arrangement makes the gamer’s local hardware less relevant — they can stream the games regardless of their device.

And since cloud gaming is typically pitched as a subscription service. It’s moving gamers away from a mentality of owning physical media and toward one of renting digital content.

Even Big Tech is getting in on the action. Google released its cloud gaming service, Stadia, in 2019. And Amazon debuted its cloud gaming offering — called Luna — in 2020. Even Netflix — which, up to this point, has only made movies and TV shows — has shown signs of getting into cloud gaming. The company recently hired its first vice president of gaming.

Cloud gaming is expected to haul in $1.6 billion in revenue by the end of 2021. From more than 23 million paying customers, according to projections by gaming analytics firm NewZoo.

But the biggest hurdle cloud gaming needs to clear in order to become truly mainstream is to be able to offer a smooth, non-laggy gaming experience for users. And that sort of experience is hard to come by for anyone without a great WiFi connection.

Not only that, running a cloud gaming service is costly and computationally intensive. So getting the technology right will take time.

In 2019, Xbox head Phil Spencer told GameSpot as much. “I think this is years away from being a mainstream way people play. And I mean years, like years and years.”

High-Fidelity Graphics

In the pursuit of ultra-realistic graphics, video games have come a long way.

PC gaming companies like Nvidia and AMD have made great strides in creating graphics cards. That allow for high-fidelity gaming and techniques like ray tracing.

In the past, things like shadows and reflections and lens flares were essentially painted onto objects within the game. This gave the illusion that light was coming from the sun or moon and reacting as it would when it hit a surface. With ray tracing, an algorithm basically allows it to actually do just that.

The technology is expected to be a game changer — if only consumers are able to get their hands on it. A chip shortage has plagued the industry for much of 2020 and 2021. That’s mostly due to the sudden rise in popularity of cryptocurrency mining, which relies on the same hardware.

But not all games of the future will be designed for such realistic graphics. Especially not card games like Spider Solitaire or Klondike.

The way Mack sees it, there are two distinct routes game developers can take when it comes to graphics.

One approach is what you see happening in major triple-A games. Which is to hire tons of visual artists and technicians to supply vast amounts of art for high-fidelity graphics. That means big budgets, big teams and increasingly realistic graphics, down to every last speck of dirt.

The other approach is to produce a more stylized — in some cases cartoonish — aesthetic for your game. That way, the costs stay down but the game still looks cool and dodges the criticism: “It doesn’t look realistic!” Mack said this approach is becoming more and more common in the mobile VR space.

The Metaverse

No discussion of the future of gaming would be complete without mentioning the metaverse. A theoretical concept that has dazzled many of the world’s foremost tech companies.

A concept popularized by author Neal Stephenson in his 1992 science-fiction book Snow Crash. The metaverse is best understood as an online cyberspace. A parallel virtual realm where everyone can log in and live out their (second) lives. Ideally, the metaverse will combine both virtual and augmented reality, have its own functioning economy and allow complete interoperability.

While we may be a long way off from that, hints of the metaverse are increasingly evident. You see it in gaming platforms like Roblox, where luxury fashion brands like Gucci host events, and in games like Fornite. Where users can dress up as their favorite Star Wars or Marvel characters and watch virtual Ariana Grande concerts.

The metaverse, like the internet, will be used for more than just gaming. It may incorporate office work as well.

But gaming itself is expanding its definition. It’s no longer about competition — but connection. It’s what Keith Stuart describes as a “digital third place,” more closely resembling a skate park than an arena.

If the current trends and future forecasts of the gaming industry clue us into anything about ourselves. It’s that our desire to connect far outpaces our desire to escape.