Programme Details [back to Top]

Part I – Market Forecasts

Although the market is young, the opportunities for Augmented Reality (‘AR’) are expected to be varied and large, including:

Applications and Advertising On Mobile Phones

Some mobile phone users will be willing to pay directly for real-time AR information and entertainment services. In addition, advertising will be a key revenue source in the mobile market. Location-aware mobile phones provide an excellent opportunity for advertisers to deliver AR overlay promotional messages to users wherever they are, whenever appropriate and in a highly personalized way. The mobile advertising business is expected to grow rapidly, reaching US$14 billion by 2011 (Strategy Analytics and eMarketer estimates).

Advertising, Promotion & Packaging

Companies in highly competitive industries, such as automotive and consumer packaged goods (e.g., soap), engage in a constant battle to attract consumer attention, provide more product information, and differentiate themselves in the consumer’s mind. Thus, they spend a large amount on innovation in advertising, promotion, and packaging. Procter & Gamble, the largest advertiser in the world, spent US$2.6 billion on advertising in just the U.S. in 2007 according to Nielsen. BMW’s Mini automotive division and Nissan both used AR technology in advertising campaigns during 2008 using technology from Metaio and Total Immersion. In each case, print advertisements were tied to online augmented brochures. While an initial benefit of AR in these situations is to a large extent novelty, AR can provide lasting benefits such as deeper information about products.

Games

The games market (not including gambling) is projected to reach US$68 billion by 2012 according to AdWeek. Companies in this market are always looking for technology to deliver new and more engaging experiences as they compete with large budget Hollywood films and other forms of entertainment. For example, 3D is being introduced in mainstream film and television. Modeling is used extensively in this market already. In addition, game console developers, such as Sony, Microsoft, and Nintendo, are looking to add new capabilities to existing consoles via add-on accessories, such as the Wii Fit (a sensor-enabled balance board accessory and game for the Wii). Adding functionality such as this helps drive incremental revenue from existing users and lengthens the product life of console platforms, which require huge up-front development investment.

Office, Industrial, Commercial, Medical and Scientific

AR systems can be introduced into office, industrial, commercial, medical, and scientific markets for a variety of tasks—from office automation to complex industrial tasks such as aircraft maintenance, where it would be a substantial advantage not to have to look back and forth between the subject and the reference manual. There is a significant need and substantial work being done in this area, suggesting an important long-term market opportunity. For example, BCC estimates the worldwide market for medical robotics (a closely related field) will be US$5.7 billion by 2011.

Part II – Technical Needs

The “big picture” vision for Augmented Reality is simple: the world as user interface – using real objects, places and people encountered in the world as reference points for additional computer-generated information. AR traditionally has focused on computer graphics objects being blended into real footage in real-time, but the field is expanding, and based on our market research we at IV’s Invention Development Fund believe that there is ample opportunity for innovation in many areas, including the following:

AR User Interfaces

The goal of AR is to provide a user with an extended fluid interaction with their environment. The UI is the most critical aspect in providing the interaction. Without the ability to “naturally” interact with the AR system, the augmentation is not

fluid or seamless. Consider the simple idea of a user physically moving an item in her physical environment and having it affect the AR environment; for example, the user pushes her cell phone away on her desk and it causes all calls to go to voicemail and a flag to be presented in her AR environment.

Why This Problem Is Valuable to Solve: Many previous solutions do not sufficiently address the basic components of intuitive and natural immersive user interaction.

AR Tracking

In order to properly match the graphics related to the real-world environment that the user is viewing at any given moment, it is necessary to know where the user is located in reference to his or her surroundings and where they are looking. This process is called “tracking” and has been one of the biggest challenges facing developers of AR experiences. Several aspects of the problem are challenging, such as the requirement for real-time performance and the fact that the view of the environment, whether presented via head mounted display (HMDs) or handheld device, may be constantly changing in unanticipated ways in many AR situations. In some cases, such as in medical and industrial applications, tracking must be very precise, but the environment can be tightly controlled. In other cases, such as games and applications used on mobile phones, tracking can be less precise but the use environment may be very challenging – unknown locations, lower performance capability of devices, intermittent connectivity, a variety of sensors, etc. Consider the tracking problems associated with the relatively simple task of deciding that the user has pushed her cell phone away on her desk. Why This Problem Is Valuable to Solve: Effective tracking must be accomplished to enable further steps in the AR process.

AR Object Recognition

Object recognition for a human user is formally defined as “the visual perception of familiar objects.” Historically, for an AR system (machine-vision applications), object recognition involves the accurate and consistent classification of an object into a group of predefined or prelearned objects that is defined by a set of associated data. Consider the problem of deciding that the object that the user pushed away from her was, in fact, a cell phone, and further that it was her cell phone.

Objects can include people, places and things in the user’s local environment with the potential addition of changing conditions such as weather, lighting (day/night) and extraneous/variable objects in a dynamic environment. Expertise from a variety of fields including sensors, Sonar and optical systems (physics), image processing, simulation and modeling (computer science), algorithm development (mathematics) as well as systems integration (mechanical engineering) may help provide solutions for object recognition. Why This Problem Is Valuable to Solve: Improved machine-oriented object recognition could help developers or service providers create interesting services for users of computers and other devices. In addition, it could help advertisers or other business partners provide more targeted, and therefore higher-value, information to end-users.

Audio & AR

Audio is one of the most important modes of communication for AR—especially since humans have a remarkable ability to locate a sound source with better than 5° accuracy in both azimuth and elevation. Although a typically immersive AR user experience is incomplete without an audio component, audio may be the only mode of communication available in certain situations (for example, in mobile applications, where safety is a concern, and situations where a video display is not available) thus requiring the AR application to integrate various augmented audio information streams with real audio inputs. For example, an AR application could allow a mobile phone user to spatially localize distinct sound sources when listening to multiple sound sources, such as during a game or a conference call.

There are many fundamental problems that need to be tackled before the aural user experience is more natural, immersive and fluid—for example, sound quality needs to be a realistic (it should be externalized and not sound like it is “inside the head,” for instance), natural extension of the acoustics of the user’s local environment. Why This Problem Is Valuable to Solve: New inventions are needed to enable users to intuitively manage various audio sources simultaneously, to author augmented audio environments, and to help developers solve fundamental augmented audio “display” challenges.

Model Creation in AR

Models are central to AR applications – they are used, for example, in tracking (to help determine where the user is and where she is looking), for determining what information is available to be presented over an image, and for the actual rendering of those overlays. However, the creation of models for use in AR is often an expensive and laborious process. In some cases, models can be created in advance for the specific application, and in others models must be created by the AR system “on the fly” based on environment sensing. For some uses, such as in medical and industrial applications, models must be very precise. In other cases, such as games and applications used on mobile phones, high precision often is not required, but the application environment may present unique challenges, such as unknown or rapidly changing locations, lower performance devices, and intermittent connectivity.

There is ample room for improvements in model creation that will increase the performance and enhance the capabilities of AR applications, and reduce the cost of their development. Consider the challenge of developing a model of a user’s desk environment in the context of the simple ‘pushing the cell phone away’ example mentioned earlier. Why This Problem Is Valuable to Solve: Models are at the core of several key aspects of the AR experience. The level of detail a model provides defines in large part the quality of the user experience. Innovation is needed to reduce the time and expense of model creation.

Data Fusion in AR

Data fusion is a technique that combines data from multiple sources (such as sensors) into a single, unified source. AR systems often operate in unknown and challenging environments where a changing array of data may be available. In addition, more and more varied data is becoming available to AR systems, driven by such factors as the proliferation of sensors and the increasing availability of open crowd-sourced databases. To provide the best possible experience for users, AR systems must be able to accurately and efficiently combine a broad range of data sources and types, including temporal and spatial data, sensor data, image data, audio data, video data, text, and web-content. Often, these data must be combined in real time. Why This Problem Is Valuable to Solve: Without the ability to perform data fusion, an AR system may offer incomplete or inaccurate information which limits the utility, and therefore value, of the system for users.

Data Privacy in AR

Providers of AR systems will need to address the fact that people have substantial concerns about sharing personal information — their location, identity, lists of things that they own or activities that they are doing — over the Internet. Unfortunately, it is fairly common for people to be caught on Facebook doing something that they would rather not be seen doing — either by being tagged by another person in a photograph or by haphazardly posting their own photograph. Even one incident of breached privacy could substantially reduce a user’s willingness to use AR applications. At the same time, sharing personal information can often allow applications to be more useful and compelling. For example, automatically adding location information to pictures greatly enhances the photo sharing experience. Trade-offs between privacy and increased application value created by the use of private information must be managed individually; users’ comfort levels will vary from situation to situation and from person to person. AR presents major new challenges in the areas of privacy and safety of data. This is due in part to the fact that location, audio, and other information will often be transferred in real time while a user is mobile and interacting with unknown and unfamiliar environments. Furthermore, it is unrealistic to have an AR system constantly seeking user verification of data policies. New, robust and scalable “real world” approaches to privacy and safety of data are needed. Why This Problem Is Valuable to Solve: Without solutions to address these concerns, the AR market cannot grow to its full potential.

Part III – Inventing in AR

We also will provide some overall considerations for inventing in the AR field.

Speaker’s Bio [back to Top]

Invention Capital – As Senior Strategist for Intellectual Ventures, Chris Somogyi is intimately involved with the entire innovation value chain. He focuses on the future of technology, products and markets and manages technology teams in Asia, invention teams in Australia and strategy teams in the USA.

Venture Capital - As a partner with Tokyo-based Pacific Rim Ventures, Chris Somogyi invested in numerous companies in the US and Europe, in areas including proteomics, drug discovery, biotech services, advanced diagnostic instrumentation, genomics, and diagnostic services. With CommTech Management, he invested in technologies developed at Stanford, UC Irvine, and SRI International through a grant-seeding technology elicitation program.

Entrepreneur- As a co-founder of Lucent Medical Systems, he brought a University of Washington technology from the research labs to a new commercial entity. As founder of Dentigenix, he assembled researchers from around the world to advance tissue engineering within dental applications. As co-founder and Chairman of Ratner Biomedical Group, he launched the first incubator for biomaterials and regenerative medicine new companies. As co-founder and Chairman of Healionics, he helped lead this new company to introduce novel biomaterials which heal. As co-founder and Chairman of Calcionics, he oversaw the team to develop a drug to control the calcification of blood vessels. As co-founder and Chairman of Inson, he helped the effort to introduce novel ultrasonically-controllable drug delivery polymer systems. As co-founder and CEO of Advanced Electroluminescent Sciences, he brought together researchers and essential intellectual property from around the world to develop low-cost solid state lighting devices based on organic LED technology.

Executive- As President and CEO of Lucent Medical Systems, he built the start-up from initiation through three rounds of venture capital and two strategic corporate partnerships with Abbott and Becton Dickinson. As CEO and Chairman of Dentigenix, he led the tissue engineering start-up through to acquisition by Europe’s leading dental company.

Board Leadership - Corporate board memberships have included: Ultreo, Inc., ScaleOut Software, Inc., Lucent Medical Systems, Healionics, Inc., Calcionics, Inc., Inson, Inc., and Ratner Biomedical Group. He also holds Advisory Board roles with several start up companies.

Engineer - Hands on experience in medical device new product development with American Hospital and Puritan Bennett. Inventor for 8 issued US patents.

Technology Licensing - With Stanford University’s Office of Technology Licensing, Managed Stanford’s portfolio of software and medical device inventions. Negotiated technology licenses and managed technology transfer relationships with universities, private research institutes, and government research laboratories in the U.S.A., Canada, Taiwan, Finland, Germany, Netherlands, France, Japan, Singapore, and New Zealand.

Angel Investor - A former member of the Executive Committee of Seattle’s Alliance of Angels, he has, through Somogyi Ventures, made dozens of investments in early stage technology companies across North America.

Community Contributor - He is a former member of the Board of the Technology Alliance, of the Executive Committee of Seattle’s Alliance of Angels, of the University of Washington’s Business School Center for Innovation and Entrepreneurship, of the University of Washington’s Technology Transfer Advisory Board, and of the Nanophotonics Initiative Committee. Mr. Somogyi is founder and primary supporter of the UW Bioengineering Summer Camp for High Schoolers.

Mr. Somogyi holds a BSE in bioengineering from Purdue University, an MS in bioengineering from Tulane University, and an MBA from the University of Washington.