Features

It was mid-summer, and Kedar Shashidhar ’15, ’16 (KEY) had hit a creative roadblock. He was leading a group, mostly of fellow students, in the development of a video game plotline, and the protagonist that he’d created seemed cookie-cutter. They were aiming for a psychological thriller, and got started, Shashidhar says, with “a typical kind of disgruntled, middle-aged, white male protagonist.” It felt flat. “None of us could buy it.”

When they scrapped the male protagonist for an older female lead, the story—and hence the game—started to take shape. It centers around Courtney Mayhall, a veteran police officer, and her adult daughter, who followed her mother’s career path. The two are thrust into a working relationship in pursuit of a serial killer haunting the backwoods of Georgia. They’re captured. And you, the player, as Courtney, have a set of decisions to make. There are tactical decisions about how to escape. But those decisions have consequences, compelling Courtney to make moral choices as well. And complicating matters further is the tension between Courtney, a sober and sassy veteran of the force, and her fresh-eyed and feisty daughter, Ava.

If you’re over 40, if the last video game you played was Pac-Man, or if video games conjure up images of greasy-scalped tween boys shooting cartoon rifles for hours upon hours in dark basement playrooms, then you might want to pause right here.

Here’s the brief tutorial: video games, still a young medium, are evolving in ways that were only dimly foreseen by their earliest creators and that are motivating critics and skeptics to take a second look. Video games, once considered a distraction at best and later blamed for everything from poor grades to violence and antisocial behavior, now elude generalization. They’re coming out of playrooms and into classrooms, as scholars in the arts, humanities, social sciences, education, and sciences are starting to find value in games for both teaching and research.

To get a sense of the landscape, it’s useful to draw an analogy to film. Historian Jeremy Saucier ’10 (PhD) is the assistant director of the International Center for the History of Electronic Games at the Strong National Museum of Play in Rochester. “Imagine that, when you were five or six years old, you watched The Fox and the Hound or Snow White and the Seven Dwarves,” he says. “And then imagine you never watched a movie again. And you thought that’s what movies were. You’re missing an enormous amount of incredible stuff.”

Like film, the world of video games has expanded to include multiple genres. There are role-playing games and simulation games. There are games that are played solo, and games played in groups or teams. There are massive multiplayer online games. There are blockbuster games that have helped catapult global video game sales ahead of film. There are noncommercial, experimental games that find their way into art galleries, and increasingly, games that incorporate complex scores composed by serious musicians.

Shashidhar’s game crosses a few of these genres. It’s both a role-playing game and an experimental game—though he does plan to sell it. An audio and music engineering major from Painted Post, New York, Shashidhar is a participant in the Kauffman Entrepreneurial Year, or KEY, program. Established with funding from the Ewing Marion Kauffman Foundation, the program allows successful applicants to spend a fifth, tuition-free year at the University to pursue coursework related to an entrepreneurial venture. He’s created a company, Blackout Games, and pulled together three programmers, two other sound designers, two writers, and a composer—all but one of whom are Rochester students or recent graduates—to create the game, called SightUnseen.

The game is entirely audio-driven. Courtney is in advanced middle age, and has lost much of her sight through macular degeneration. Dialogue, written by Katherine Varga ’15, ’16 (T5), an English major from New Britain, Connecticut, who cut her teeth in playwriting with campus theater groups such as TOOP—The Opposite of People—will be narrated by voice actors. But the player, as Courtney, will navigate forests, brush, and abandoned buildings through a dense soundscape.

“We can track the way audio interacts with the world and capture every reflection, diffraction, obstruction, and occlusion,” says Shashidhar, describing the three-dimensional audio model he’s developing. “We’re essentially creating an auditory snapshot of the world, and creating it with such accuracy that it’s like actually experiencing real life.” Audio verisimilitude requires not simply the replication of a single soundscape, but consideration of psychoacoustics, or the perception of sound. “Your ears localize in the real world, so you can tell the way the sound flows. We’re able to model that,” he says. The goal of the soundscape is to make you, the player—as Courtney— feel as if someone is behind you, above you, whispering in your ear.

When the first home video game entered the consumer marketplace 40 years ago—Pong, debuting just in time for Christmas 1975—it had already been about 15 years since the creation of the first games to be projected and played on a screen. Like many products in consumer electronics, video games emerged out of university laboratories in the years during and after World War II, rising in tandem with the development of computers.

But while commercial gaming may have arisen out of powerful computers in university laboratories, it’s the commercial gaming industry, which topped $100 billion in 2014, that’s driving a lot of computer technology today.

“The competition among game developers for the best graphics, the best sound, the best experience overall, is what pushes a lot of development,” says Ted Pawlicki, a professor of computer science at Rochester. In years past, graphic and sound elements in video games strained all but the most powerful home computers. Today, however, a typical computer used for writing papers, keeping track of personal finances, and even storing substantial photo, video, and music collections comes with greater memory and processing speed than most people need for such tasks.

The advancement of computer technology has benefited researchers. “People making the fast graphics cards weren’t thinking about physicists simulating planetary motion,” Pawlicki says. “They were thinking about selling games to kids at Christmas. It’s hard to say if those technologies would have been developed if it hadn’t been for the entertainment market.”

Meanwhile, the market has expanded far beyond kids. According to the Entertainment Software Association, an industry trade group, in the United States alone (relatively even with China as the world’s largest video game market, in monetary terms) nearly 60 percent of the population now plays video or computer games. Of those, more than two-thirds play on a video game console such as PlayStation or Xbox. Half of American households own a game console. Nearly 40 percent of American gamers are over the age of 35. And more than 40 percent are women.

As a professor and head of the undergraduate program in computer science in the Hajim School of Engineering & Applied Sciences, Pawlicki has encountered plenty of students interested in developing video games. As a means of training computer scientists, he believes video game creation is a powerful tool. “Video games really—and I believe uniquely—draw on every component of computing,” he says. “You need good algorithms, you need good coding skills. But you also need knowledge of good graphics, of user interaction. And you can’t ignore networking and security.” For computer science majors, producing a video game calls upon virtually every skill they’ve learned.

At the production end, video games can clearly be a teaching tool for programmers, graphic artists, scriptwriters, and composers. But what about at the consumption end? Can playing video games aid in learning? The answer among a small but growing group of educators ranges from a cautionary to a resounding yes.

Count Jayne Lammers, an assistant professor of teaching and curriculum at the Warner School of Education, in the “yes” column. In the world of skeptics and holdouts, Lammers has a lot of credibility. She’s a former high school English teacher and the head of Warner’s secondary English teacher preparatory program. As you’d expect, Lammers is an avid consumer of literature, storytelling, and the printed word. She began her career, she says, inspired “to help struggling adolescent readers.” Her students faced a number of barriers to reading. Among them, in Lammers’s view, was an uninspiring curriculum. “What I was required to teach was a programmatic, scripted kind of curriculum that seemed really beneath the 9th and 10th graders in my classroom,” she says.

She enrolled in the doctoral program in education at Arizona State in hopes that further study might help her to gather some new ideas. The summer after her first year, she received an email from her department asking her if she wanted to be a research assistant on a project led by two new faculty members. The professors were James Paul Gee and Elisabeth Hayes, and they’d come from the University of Wisconsin’s Games, Learning, and Society program. Their project concerned video games and literacy, and was premised on their belief that video games were self-teaching tools—or learning systems, as they called them—worthy of study and some type of replication as teaching tools that could enhance classroom learning in ways that traditional curricula weren’t.

Lammers wasn’t quite sure how to respond. Yearning for new ideas, the incorporation of video games into learning was far from anything she had in mind. “I thought video games were part of the problem,” she says. They were something “taking my kids away from books, and from reading and writing in authentic ways.” But she was a graduate student, it was a job, and she took it.

The project involved using video games to get girls interested in STEM—science, technology, engineering, and math—careers. Since men often credit video games with inspiring their interest in computers (something that Pawlicki confirms, in his own case and from many years teaching computer science), the thinking was that introducing more girls to video games might motivate them to follow a similar career path.

They focused on a blockbuster game, the Sims, first released in 2000, and now in its fourth version. Among the best-selling video games of all time, and notably popular among women and girls, the Sims is a life-simulation game, in which players create characters and shepherd them through daily life. In observing the girls at play, Lammers saw that the game was actually a form of storytelling. “They were manipulating their game to tell stories,” she says. “And they did that in either text or image-based stories.”

Lammers was captivated. As she learned more, she realized that outside of the game, young players were creating Sims fan fiction, and making Sims television shows and movies. “As I realized what they were doing, not only in the game, but also with the corresponding online space, how they were supporting one another, it was like a writer’s workshop happening right before my eyes with young women from all over the world,” Lammers says. “It made it hard to argue that there weren’t literacy practices happening.”

At Warner, Lammers has been studying the ways in which youths use online spaces to demonstrate fandom—literary and otherwise. She notes that readers of the popular Hunger Games series have created and shared role-playing games with Hunger Games characters, rewrites of the stories, and stories combining characters from multiple works of popular fiction, including the Harry Potter series.

Lammers has recruited several graduate students, among them Kristana Textor. Textor, a second-year doctoral student in teaching and curriculum at Warner, was a documentary camera operator and a part-time instructor in an after-school program in New York City when she first started thinking about games as a source of rich group discussion. Born in 1976, Textor calls herself “a former arcade rat,” with a lifelong love of games, her personal favorites ranging from the mesmerizing puzzle game Tetris to the first-person shooter blockbuster Call of Duty: Black Ops.

In the program, she and the high school students played plenty of games. “But then we would also break them down and discuss them,” she says. The games elicited discussions about culture, prejudice, ideology, the use of different musical genres to suggest setting and mood.

Textor says she had “no idea there could be a field of study tied to this.” But at a national gaming conference, she started talking to scholars. She applied for, and won, a fellowship from the Strong National Museum of Play, where she met Saucier, and later, Lammers.

Textor has fine-tuned her game-based learning program in Rochester, working with K-8 students in the summer enrichment Horizons program, a national initiative with which Warner has an affiliate, as well as students at East High School, which is being managed in partnership with the University.

She’s also trying to address a gap in knowledge and understanding between gamers and non-gamers. During three afternoons this past fall, Textor held an event called “Games for Curious Scholars.” In a laboratory in LeChase Hall that’s equipped with television screens, she brought PlayStation and Xbox game consoles as well as an array of games and books on gaming.

She invited members of the University community, particularly people with little knowledge of video games, to drop by and play, free from judgment or expectations, and with her support, if needed. Among her aims was to demonstrate just how broad an appeal games can have. “Video games are so interdisciplinary,” says Textor, who plans to continue holding the event in the spring. “You can really look at them from any angle. There’s the programming angle, you can look at them as cultural artifacts, you can look at them as works of art, and you can look at them as social environments. To me, games are at the heart of this wheel with infinite spokes off it.”

Among the people who came by Textor’s event last fall was Mike Jarvis. Jarvis is an associate professor of history and director of the digital media studies program, and admittedly, not as new to video games as the target participants.

He stopped by in between a presentation and a lecture he was to give to play Assassin’s Creed. As a historian, he has mixed feelings about the game. First released in 2007, Assassin’s Creed is described as “a historical fiction action-adventure.” The version Jarvis played involves a long backstory, but in brief, a modern character named Desmond Miles has been catapulted back into the past, to 18th- century North America, where he meets his ancestor, a half-English, half-Mohawk North American who goes by the name Connor.

“The landscapes are beautifully rendered,” Jarvis says. They’re also deeply researched. Although the game play consists of accomplishing various sorts of missions, the game allows you, through your character, to wander at will through its model of the natural and built environment. “It’s amazingly powerful to walk the streets of 1750s Boston and see the taverns and see the ships in the harbor,” Jarvis says of the game’s historically accurate rendering.

But the characters—“the actual historical characters you meet,” Jarvis says—“are not at all authentic.” They’re merely superimposed onto the 18th-century landscapes. But imagine if that weren’t the case. Imagine if the characters, like the physical setting, were deeply researched. What would Connor’s experience be, as half-Mohawk? What if the game permitted you to take on the role of a colonial goodwife? A free black? A slave? A revolutionary? A loyalist? How might the experience vary in each case? And most important, to Jarvis, as a historian, what kinds of historical questions might the experience of such game play prompt the player to ask?

To create such a game, he already had a lot of research groundwork covered. As a graduate student working toward his doctorate at the College of William and Mary, Jarvis, who may well be the world’s foremost expert on Bermuda, began to amass what became multiple filing cabinets full of information on the built environment of St. George’s, Bermuda—the oldest British colonial settlement still in existence—as it developed in the 17th and 18th centuries. After he completed his first book and earned tenure—“a wonderful thing, knowing you have security, and can then think of bigger projects,” he says—Jarvis revisited his mini-archive. He had recorded names and details of all the people who lived in every residence of St. George’s, through at least 10 generations. In 2013, he began using three-dimensional visualization software to create virtual versions of the settlement at multiple periods in its long history. With the help of undergraduate and graduate students, who have conducted more detailed research into the lives of individual residents, right down to the furnishings of their homes, he’s getting closer to the point where he will be able to use game-generating software to transform the static model into one that’s immersive and interactive.

He envisions the finished project, Virtual St. George’s, as a powerful teaching tool, a model that he hopes will inspire other historians. He anticipates that the project will prompt users to ask questions about aspects within certain time periods, as well as change over time. “You could literally pause a minute and consider from different angles the very same scene,” he says. “Mentally, it’s what we try to do as historians.” You could take on the role of a black slave, a white laborer, an artisan, and so forth. Then, jump ahead even 10 years, and everything might change. “Two snapshots across time invite you to think about the world as a very changeable place, rather than a static place, or to say, ‘it happened a long time ago,’ and to assume everything was the same across decades or centuries.”

Jarvis is focusing first on St. George’s in 1775. “The powerful story to tell there is that the American Revolution affected all the British colonies, and not just the 13,” he says. Bermudians were at the center of Atlantic world trade. Their food came from the North American colonies, which had just decreed that they would no longer trade with colonies loyal to the Crown.

“A lot of Bermudians were trying to figure out how to survive, and you had others who ideologically supported the Americans’ just grievances,” he says. “You would get a variety of different understandings, and hear very different versions of the Boston Tea Party,” depending on what role you played.

For his course, Digital History: Building a Virtual St. George’s, he has students design a game, which he assesses for its value as a historical teaching tool. In designing a historically grounded game, he says, “you don’t write a monograph. You write a multigraph.” In doing so, you script counterfactual history, stories of what might have been. And there’s no better way, Jarvis argues, to impress upon students a notion historians labor to instill: that the world we have is not inevitable. It’s the result of countless choices, each with its own set of consequences.

If games involve a series of choices with consequences, and if they can be made to simulate natural environments, urban environments, or familial environments—then perhaps, some game enthusiasts say, they might be used to make the world a better place.

Together, Pawlicki and Jarvis are overseeing the capstone project of four seniors majoring in digital media studies. The students—Gina Fabio of Auburndale, Massachusetts; Yukun Liu, of Beijing, China; John Lockard of Sayre, Pennsylvania; and Lean Mateos, of Elba, New York—are in the beginning stages of developing a game called Aster. According to Pawlicki, the game offers “a different take” on a traditional war game. “This one’s a little more socially conscious,” he says. “Rather than take on the role of a combatant, in this game, you’re going to play from the refugees’ standpoint.”

Fabio, who is focusing on graphics and storytelling, came up with the basic idea. The protagonist is seeking to rescue her friends and townspeople. There’s another problem, though, in addition to the risks of being captured or killed. Refugees are placed in a camp, which slowly evolves into a town, Aster. They need water, shelter, food, medical care, and clothing. As the player, you have to decide: where should limited resources go? To attempt more rescues? Or to take care of critical needs in Aster?

“There’s no combat,” Fabio says. “You spend your time avoiding enemies rather than fighting them. And if you do get caught, you can’t fight your way out of it. You have to talk your way out of it, or escape.”

While all four of the students had pitched ideas, Fabio’s intrigued the group because it appealed to them as gamers on multiple levels. Not least, it promised to be fun. But they were also compelled by the complexity of the problems it posed. As an exploration of actions and consequences, it seemed to have endless potential. “One concept we’ve talked about is the idea that, if there aren’t enough resources for the people you’ve saved, they would start to leave Aster, and go back to where they came from, because you don’t have enough to sustain them,” says Mateos, whose focus is programming.

Although none of the four group members put it quite in this way, Aster could easily be considered an outgrowth of the so-called serious games movement. The earliest incarnations of serious games were military simulations. But since the Cold War, policymakers in many realms have partnered with game developers to create simulations to help address public problems. Diane Tucker is the director of the Serious Games Initiative at the Woodrow Wilson International Center for Scholars in Washington, D.C. “Like games, policies also can be seen as systems—structures created by the interrelated rules, regulations, and other mechanisms by which policymakers try to close a gap between reality and a goal,” she wrote in a 2012 article “Gaming Our Way to a Better Future.” “Policies can have unintended consequences that a game version of a policy can reveal.”

Tucker is the creator of the Wilson Center game Budget Hero, launched in May 2008 and retired in August 2014. During that six-year period, the game was made available for free, used in classrooms across the country, compelling players to set priorities, examine trade-offs, and in the process, appreciate the complexity of the federal budget. The Wilson Center tracked data to reveal trends in policy choices and sought feedback from users. According to its own analysis, the majority of players reported that it changed the way they thought about the federal budget.

Another major figure in the serious games movement is Jane McGonigal, director of games research and development at the Palo Alto, California, nonprofit Institute for the Future. She’s a folk hero among gamers and, last summer, paid a visit to Rochester to address 200 high school students from around the globe housed at the River Campus for the International Baccalaureate World Student Conference. Among the reasons for her popularity among gamers is that she attacks some common, negative stereotypes about gamers head on. If gamers are lazy and lack motivation, she points out, how can it be that more than 10 million people around the world have spent more than 50 billion hours collectively, working, in teams, on a difficult, complex massive multiplayer online game called World of Warcraft? If gamers are governed by the need for instant gratification, how can it be that it takes, on average, 500 hours to reach the point of “leveling up,” where so many players report “the real fun” begins? Most important, she asks, what if all those hours of collective thought and effort were poured into the equally challenging task of trying to solve real-world problems?

McGonigal is part of the team that developed World Without Oil, a simulation of a world in which oil becomes scarce and that seeks to predict how societies around the globe would respond to an actual oil crisis. With funding from the Corporation for Public Broadcasting, a team of designers created the game, and then recruited nearly 2,000 participants, equally divided by gender, across age groups, and in every state of the United States and 12 other nations. The recruits played over a period of 32 days in real time, representing 32 weeks in the simulation.

The result was a fascinating case study demonstrating a mix of competition for survival, and later, widespread cooperation toward mutual survival. It’s arguable whether game behavior would approximate actual behavior. But what wasn’t contestable was that participants reported devoting an unprecedented amount of thought to the full range of consequences of oil scarcity. They’d witnessed, and participated in, a range of cooperative solutions that could provide a template for policymakers in actual environmental or economic crises.

McGonigal is an advocate of gamification, or the quest to better engage people in all kinds of tasks by turning those tasks into games. Even enthusiastic gamers, such as Textor, find some of her claims extravagant. When it comes to gamification, Textor says, it’s best to proceed with caution. She wants to make clear that advocates of using games as teaching tools are not automatically suggesting that games are a panacea for student disengagement, among other challenges many educators face.

“A lot of times, people will say, ‘Gamify! Let’s gamify the classroom because everybody loves games!’ It’s a little more complicated than that,” she says. “You have to consider the details of what you’re gamifying. It’s not just a matter of using carrots and sticks.”

But used thoughtfully, video games show potential in ways that Textor, Lammers, Jarvis—and perhaps other faculty at Rochester who have incorporated games into their teaching and research—hope will encourage more educators to explore them.

Regardless of what happens in the classroom, the gaming market is likely to continue to expand and to become even more diversified. The most dynamic sector (in terms of creative development, if not profit) is in indie games, made available for free or at low cost on websites such as Steampowered.com or Humblebundle.com. Fabio describes such sites as part of a new gaming culture. That culture values the social good, sometimes tying games, and their revenues, explicitly to social causes, or simply incorporating the moral element she finds compelling in many games. She’s accessed “a ton of games,” she says, on Humblebundle, which allows game buyers to pay what they want, and divides revenues between the developer and charity. She also describes the indie game culture as thoughtful. “There are a lot of games that have very emotional sorts of stories that really make you think.”

Mateos points out that the city of Aster is named after a flower. “The idea is that your city blossoms like a flower,” he says.

In fact, if you look at an aster, it looks much like a wheel, with a dense supply of spear-like pedals. It’s like Textor’s metaphor: a game that’s a wheel with infinite spokes.