A Ryerson architecture professor is heading a new and unique project to bring the digital world to reality using a camera and a QR code. By Sean Wetselaar
Gingerly, Kevin Pu places a tiny, fl at piece of black and white cardboard onto an empty lot in a carefully constructed, cardboard replica of a city block.
The fourth-year architecture student stands back for a moment to examine the empty lot. He shuffl es about, pointing a camera at the empty space from various angles. To the average observer, it’s a bizarre scene.
But then, Pu glances down at the monitor on his MacBook. Displayed against the backdrop of the cardboard city scape, is a tall, white, digital condominium, fl oating just above the black and white piece of cardboard he’d placed. Pu swivels the camera, and the shot of the building swivels as well. This is no computer generated environment – Pu can see the building in real context. And this Ryerson classroom is the only place in the world he can do it.
Ryerson is the frontrunner in a new research initiative in the architecture industry studying ‘augmented reality.’ The program links 3D renderings of structures to small, cardboard, high density QR codes (usually used to transmit Internet links via smartphone). Though the research is still in its infancy, the fi nal result could have a lasting impact on the industry.
“Architecture is all about visualization,” Pu says. “[You’re] trying to show your ideas through pictures. Historically, or conventionally, people’s architects show [their designs] through static images. ‘Here’s a pretty picture – it’s what the building looks like.’ But if you take it to the next level, ‘Here’s a pretty picture in context.'”
The research project was initially spawned when architecture professor Vincent Hui considered taking concepts from video games, like a heads up display, into reality and into his chosen profession.
“If we can see how many shots we have still in our gun or how many more points we have to get, why couldn’t we also see the virtual notion of our building in reality?” Hui says.
The technology works in conjunction with a 3D modelling program called Rhino. After creating a 3D model of their structure on Rhino, the architect can export it to a fi le recognized by Hui’s interface.
This file can be linked to physical high density codes, like the one placed on cardboard, which can then be moved and placed wherever the architect wants.
Then, using an HD camera, the user can pick up the digital signature of the fi le. On the screen linked to the camera, the architect sees not only the reality of the physical space, but a model of their structure in that space. Though initially these models are white ‘mass’ models, more textures are slowly being added. This visualization allows the architect to see everything that could potentially affect it in the field, from an unexpected shadow falling from an adjacent building to a blocked view that they had wished to preserve.
The initial stage of conceptual development came from a collaboration between Hui, and former student at the University of Waterloo, programmer Matthew Compeau.
The pair were reunited at a charity dinner last summer, and after agreeing to work together, Compeau spent several months planning and building the interface.
“There was a lot of preliminary work before we even decided to do this project just because I wanted to be really confident that it would actually work and that the research would actually be worthwhile,” Compeau says. “[This] kind of project pulls together a bunch of different technologies… So the essential challenge of the project is bringing those all together in a really efficient way so it can run on a fairly mobile device.”
Compeau coded a framework for the project using the programming language Java, which is able to run on multiple platforms, a key aspect of the project.
The project got off to a slow start late last fall while, after joining the team as research assistant, Pu tried to fi nd a balance between detail and processing speed with the ‘polygon count’ for the interface. Higher ‘polygon’ or ‘mesh’ counts allow for more detail, but also lags the program and makes practical use of the system difficult.
“So we’re playing around with that and then when we had it up to function, we were testing it on like a big QR code, and we brought it on site, we scanned it, and we had a couple shots of it actually working, but it was really slow,” Pu says.
After finding a balance for white ‘mass’ models without textures, the team moved on to adding detail to the images, like glass. But this posed another problem when additional details caused the system to lag once again.
Despite technical concerns, the group was able to move forward with the development, and capture an example of 1:1 scale when a student designed a wave-like addition to the entrance of the architecture building.
“That’s not Photoshop, that’s really what you would see through your iPad,” Hui says, gesturing at the image. “So that student’s design now, as they walk around, they can actually see their design in real time and real space virtually overlaid.
That’s pretty cool, right?”
Though the project is making strides in its 1:1 modelling, one of the areas Pu sees it having the greatest impact is in education, like the second year studio class where some of Hui’s students have been given a chance to test out the system.
“It was really fun where Vince is teaching a second-year studio and we were able to implement that into their studio time,” Pu says.
“And all of a sudden, Vince can also pick [the student’s project] up, turn it around, look at it in all four sides and say, ‘Wow, there’s problems here and here.'”
In its infancy, the augmented reality system achieved little that existing programs could not already do. Luke Kimmerer, a student in the studio class testing the system was unimpressed by early iterations of the system.
“It’s still in its early phase,” Kimmerer says. “It’s cool in theory, but I haven’t experienced it like they want to experience it yet… Until it gets to that point, I can understand why people argue it’s not different [than programs like Rhino] at all, but defi nitely when it gets to that stage, it’s going to be something pretty neat to play around with.” Some students in the class pointed out to Pu that the ability to model buildings based on high density codes was not that far removed from modelling systems like Rhino, which allow students to orbit around a stationary structure. But what Pu says these programs lack is the context that can be obtained by seeing models in reality, rather than through a modelling program.
“At the stage where it’s at that’s a valid question,” Pu says. “But what we’re trying to get at, is the idea to be able to have students visualize their projects in context and then scale in 1:1 perspective. So, obviously the computer screen is still your lens to this augmented reality, but it gives you a new perspective to a lot of things that a static image can’t give you.” Though the augmented reality project is making strides, it’s just one of many side projects Hui spends much of his time running.
His office in the architecture building is a confused jumble of pens hanging from magnets, Star Wars memorabilia, computer screens and desks which he wheels about the room depending on his needs.
As he talks, he perpetually pulls portfolios and fi les out of drawers for emphasis, or fl ips to fi les on his computers or iPad.
Hui says fi nding students work is very important to him, and when he can’t do that through industry connections (though he receives 30 40 requests for students from the industry per year), he often fi nds room for students to “guinea pig” some pet projects like the augmented reality research.
“I really like ensuring that my students get work experience because I believe that there’s only so much that you can be taught in school.
It’s really imperative that students go beyond simply the classroom,” he says.
“Far too often we, architecture as an industry, sells to the public the idea of the romanticized notion of the napkin sketch, the architect drawing on a napkin, going, ‘Here Mr. Client, that’s what we’re going to design.’ But that’s far from the reality.” Reaction to Hui’s system so far has been very positive, and the team has garnered interest from the industry with researchers in Spain publishing papers on the topic. Hui is also slated to travel to Rhode Island to speak to other building educators about the project, and he sees it spreading to other schools, and other research teams.
“Ultimately, we designed this to help students,” Hui says. “It’s not to make money off of, it’s not to do anything more than to just make sure there’s bridging of the gap between the digital to the real world.” Funding for the project has largely come from Ryerson, with Hui receiving the Learning and Teaching Enhancement Fund from the Learning and Teaching Offi ce at the university.
Hui credits their contributions to his research with the recent progress the project has made.
That progression is substantial.
From a simple tool with few features beyond existing technology in Rhino, Hui and Compeau are confi dent a relatively fi nished product, with options to edit and view projects on a 1:1 scale, in real time, are in the near future. Hui expects a fi nished tool by the fall, or summer 2014 at the latest, while Compeau suggests a relatively polished product will be available by the end of the semester.
The team has focused its development so far on the educational sphere, but given time their research could have an impact on the professional world of architecture, though Kimmerer and Compeau note that it would take a re-evaluation of workfl ow to emphasize more on design.
“Our development’s focused on the educational aspect, but I don’t see any reason why it would have to stay there,” Compeau says.
“I don’t see it as something that exclusive to academia,” Hui says. “But I think that as augmented reality in general as a digital platform continues to gain market share, I think that the casual user of electronics will fi nd uses of augmented reality.” Balancing his multitude of side projects, teaching career and personal life is not always easy for Hui. He worked for 10 years at Waterloo as a professor before deciding to move to Ryerson, despite the fact that his wife and newborn baby continue to live in Cambridge. But he says the progress his students are making, both on the augmented reality research and in a myriad of other fi elds, makes the commute worthwhile.
“I live in Cambridge… [But] I work in Toronto and sometimes I have to sleep at my sister’s place, sometimes my god kid’s place,” he says.
“I wake up alone in someone else’s bed, or on a couch. But I wake up with a smile on my face, because I’m coming back here.”
Leave a Reply