3D models and AR in Basel

The purpose of this page is to expand our documentation of one of the key motivations for the AR-4-Basel project:

to produce the best possible experiences for users by having 3D models available in both the design and the presentation modes of the AR experience cycle.

There are three parts:

  • Rationale
    What are the relationships between 3D data with precision such as could be made available by the city and private businesses (e.g., Messe, Art galleries, etc) and the AR end user experience?
  • 3D models for Basel
    What is available for experimentation in Basel?
  • Potential Results
    How can using 3D models bring new value?

Rationale

One objective of the AR-4-Basel project is to help bring together in a new collaborative framework those who are working with 3D models, particularly in urban environments, and those who design and deliver AR experiences for the people in cities.

At the time of the establishment of the AR-4-Basel project, there is a general awareness that data can be attached with not only a geo-coordinate (e.g., GPS) but also an elevation. And that over time data will be provided at a very fine grain. Hence, a point of interest for an AR experience may be described as having a volume and, in some cases, that volume will be associated with a fixed position on the earth. Further, a space, a volume is described in 3 dimensions, both internal spaces and external spaces. Hence, there will be need for the use of 3D for high quality AR experiences.

In 2011 3D models of cities were known to be the subject of urban IT department development and many cities are investing or have invested in conversion of data sets from 2D to 3D, as well as from static data (that which is published and assumed to be unchanging over time) and dynamic data (that which is published in a fashion that reflects the continuously changing conditions at that space or environment).

One day people will expect their AR experiences to be so well developed that they will take the 3D model of the object or place of interest fully into account when designed. This will permit the augmentation to reflect a great deal more realistic information, giving the user shadows, realistic overlay/alignments and even having textures in the digital augmentation that reflect precisely the texture found in the real world. Unfortunately there are several components missing and that must be developed before users can experience the full benefits of 3D in their AR experiences.

In a general way, AR development and publishing tools and AR experience designers developers using the tools, have insufficient knowledge and experience with 3D.

Further, those who work with and create 3D models are not conversant in the language of AR or using the AR development tools to permit users to visualize their 3D creations.

Finally, there is a lack of standards for the use of 3D in AR.

When the bridges we seek to design and build between 3D and AR are in place, we can expect benefits such as those summarized in the table below.

Starting data Calculations
Manipulations		 What AR brings How users benefit
volume and precise data about buildings and structures (texture maps, coordinates of exterior corners, doors, windows, rooflines, interior spaces, etc) ingest and catalog (extract key features) with a feature recognition engine on the 3D model Use the "real" data from video and compare with 3D model as the basis for more accurate visual recognition of the user's position relative to the fixed building point of interest More accurate alignment of AR overlay with respect to real world stationary objects (statues, buildings, fountains, walls, roundabouts, bus or tram stops)
same as above calculate precisely the look/appearance of additional (augmentations) objects or labels (stationary or not) inserted where currently nothing appears With the 3D models, we can calculate shadows and occlusions More realistic integrations of the real and virtual objects or buildings
Data such as water quality, electricity use, natural objects (trees, rivers, parks) etc Attach the data to the place in space Real time volumes to show the quality, relative importance of consumption or production in situ Makes quantitative data more "real" for the inexperienced. Better method to visualize/communicate data

 

3D models for Basel

At the time of the writing there are three 3D models provided by the AR-4-Basel project for use and experimentation by developers and providers of AR tools. To learn more about the models prepared by virtual city SYSTEMS, go to this page. For more information about the GTA Geoinformatik model, go to this page.

Source data formats LOD
city of Basel GVA .shp 2
GTA Geoinformatik GmbH CityGML, KMZ, ESRI Shape file (.shp), DXF, OBJ, VRML 2
virtualcitySYSTEMS GmbH CityGML, KMZ, ESRI Shape file (.shp) 2

Potential Results

In the first table above we provide what we consider to be user benefits emerging as a result of 3D and AR in urban spaces. In addition to benefits during the experience phase (at the time of the presentation of the augmentation in alignment and integrated with the real world), we anticipate that there will also be benefits for developers during the design phase of the AR experience creation.

Developers who are using 3D models of the real world will be able to base their designs on these digital representations. By having a 3D model of a building, a monument or any object which is to be augmented, the design of the augmentation (its textures, sizes, colors, and other properties) can be better and performed more quickly. One could even have tools which take the 3D model and propose some augmentation properties based on best practices.

Further, we anticipate that knowledge of the 3D environment (even that which is not the exact POI for augmentation) when delivering the augmentation could have benefits. If the AR system can detect closeby objects as well as the GPS location and orientation of the user, then the shadows and occlusions to the digital objects would be calculated in advance of display and, in some circumstances pre-calculated. 

Conclusion

We look forward to there being more realistic and more innovative AR experiences as a result of the increasing collaborations between those experts in 3D and AR domains.