already a subscriber? Register me
Subscribe now See my options

Stanley A. Miller II

MSOE students put reality before your eyes

Students from the Milwaukee School of Engineering are taking video games off the screen, out of the living room and into the real world with a system fusing augmented reality, GPS and off-the-self hardware that anyone can find.

The research project is called ARES - short for Augmented Reality Entertainment System - and it's the senior design effort of software engineering students Kevin Curtis, Michael McBride and John Schmitt.

"The potential of augmented reality is incredible," Schmitt said. "We're taking what you see and merging it with digital information. It's a natural way to make information easily accessible."

Augmented reality is a technology that overlays digital content such as graphics or 3-D models on top of real objects or environments as seen through video. Software developers are building it into travel and dining apps for smart phones, and it's being used in classrooms for education, labs for research and even professional football games on television to show the first-down line.

The game in ARES is simple: find players, zap players and score points - it's a bit like digital paintball or virtual reality laser tag. But the technology behind it is much more sophisticated.

ARES uses a laptop carried in a backpack, Web camera affixed atop the head and video goggles to create a digital heads-up display over the field of vision of the player wearing the gear.

Players can still see everything around them through the camera's video feed except now there is extra data layered over it making up parts of the game.

For example, this video overlay, which was projected on a screen for a demonstration, showed the time, the strength of the system's Wi-Fi signal and a long rectangular bar lining the bottom of the screen representing the player's "health."

With more development time, ARES could have other digital items added to its real-world view, such as power-ups granting players temporary boosts or virtual items restoring a player's health.

Players line up targets by angling the camera and then pull the trigger by clicking a mouse. Schmitt said that a Bluetooth remote would be more portable and satisfying, but hey, they had a limited budget and a lot of technology to build.

For example, the team designed video analysis software that tracks players seen by the cameras by the color of their shirts. During the demonstration, McBride, wearing a bright red shirt, moved around within the camera's view and a clear rectangle stayed centered on his mid-section.

"What we are doing is taking a view of reality through the video camera and adding useful information to it," McBride said. "It's looking for the biggest blob of color."

A GPS module attached to an ARES laptop tracks a player's movements and a wireless Internet connection keeps players linked to the server keeping score.

The GPS ensures players are near each other and stay within the field of play - an anti-cheating mechanism to maintain the integrity of the game. But in a more advanced version of ARES, the locational technology also would be key to placing digital items in the real-world arenas for players to snag.

Curtis said one of the inspirations behind the team's project was ARQuake, an augmented reality system created in the Wearable Computer Lab at the University of South Australia.

"We wanted to make a system that would support a lot of players," Curtis said.

In addition to designing the image analysis software, the MSOE team built the augmented reality overlay system, video input, the game client and the server software coordinating everything.

The current version of ARES, which will be on display along with other MSOE senior design projects Friday, "comes pretty close to our early mockups. The essence is there," Schmitt said.

Darrin Rothe, assistant professor of electrical engineering and computer sciences and the team's adviser, noted the ARES architecture could be used for other kinds of applications.

"With the framework in place, the modules can be used for a variety of purposes," Rothe said, adding in addition to the technical aspects of the project, the students are challenged with tasks such as documentation, the processes of teamwork and project management.

The team said one of the biggest lessons they learned while working on the project was the potential of augmented reality technology.

"We like to interact with the world and augmented reality is taking information off the computer screen and putting it into the real world," Schmitt said. "It's convenient, intuitive, immediate and seamless with the human experience."


What: Electrical Engineering & Computer Sciences design projects

When: 10:30 a.m. to 3:30 p.m. Friday

Where: Walter Schroeder Library and the Werwath Mall, 500 E. Kilbourn Ave., and the Allen-Bradley Hall of Science, 432 E. Kilbourn Ave.

Admission: Free

Stanley A. Miller II covers personal technology for the Journal Sentinel. Reach him at (414) 223-5162 or He tweets @stanmiller.

© 2015, Journal Sentinel Inc. All rights reserved.