- Tech Portfolio
ADSC researchers are developing technology that can build virtual 3D replicas out of moving 2D images, even when the images contain similar textures and sizes.Daniel Lin is building upon work conducted at the University of Washington, which leveraged images from Flickr.com to construct a virtual 3D replica of Rome using Structure-from-Motion (SfM), which is the process of inferring a 3D scene structure from moving 2D images. They used common points between the thousands of images to reconstruct the city’s popular sites and landmarks.
Their parallel distributed matching system worked great when recreating cities such as Venice and Dubrovnik, Croatia, because those cities have many buildings that are distinctive and are of different textures and sizes. But bundler-style systems have difficulty distinguishing the makeup of modern cities, which contain repetitive strutures and similar textured regions.
Lin, who is collaborating with Singapore University of Technology and Design Assistant Professor Sai-Kit Yeung, is working to build a system that can accommodate all images and scenes, even where there is little texture and lots of repetition, such as a row of skyscrapers in Singapore or the flat walls of a home.
“We are trying to stabilize the system, so that it can works most of the time, avoiding potentially frustrating failures,” Lin said.
The basis of Lin’s research is feature matching, where a computer can look at an image or scene and recreate it using features, or distinctive image regions, of that image that can be described with a unique descriptor. By comparing one image to another, the algorithm developed by Lin, called RepMatch, can match identical points in each image.
“By definition, there is only one correct match, but many ways to match incorrectly,” he said. “Correct matches tend to be clustered together and incorrect matches tend to be scattered. This helps us get the most consistent set of matches from each scene. Once we do that, we can use a geometrical growing tool to get even more matches.”
Some of the benefits to Lin’s algorithm is it allows computers to estimate the camera positions in the images more accurately, in addition to holding images together and avoiding breakage when an image turns corners or go outdoors.
While Lin’s algorithm has made it possible to recreate these 3D images of a variety of spaces, the feature matching is extremely slow when using large data sets. Lin has begun developing a newer and faster algorithm that still guarantees the same amount of accuracy.
It’s Lin’s hope that this technology could be used in many different areas, from furniture sales and real estate to a possible alternative to laser scanners.
“The main problem is that our system is a bit too slow, but if our results improve, we could see this being used in many different fields,” he said.