Media Search

The Internet has provided people a vast platform for broadcasting, networking and information sharing around the world. This valuable tool of communication has become an integral part of our daily lives as we increasingly depend upon it to keep in touch with the rest of the world and its happenings. Information sourced from the Internet can come in a variety of media types: from text to images to audio and video clips.

Navigating this colossal amount of information necessitates the development of effective and accurate search engine technology. Current search technologies depend solely on text annotation databases to generate search results. This has spurred researchers to develop search services with enhanced functionalities for mining and retrieval of media data existing in other formats (such as pictures, video or audio) and integrating real-time contents such as those from sensors and social network sites. Future applications are potentially far-reaching in areas such as security, healthcare and entertainment.

NUS Research Capabilities

NUS offers supportive infrastructure and resources for researchers from the NUS School of Computing who continually blaze new trails in media search technology development in areas such as speech-based search, natural language processing, scanned text search, multimodal mobile music retrieval and interactive web-scale media search. Researchers like Prof. Chua Tat Seng and Prof. Mohan Kankanhalli are also active in international and local professional activities such grant programme management, event organization, and serving on technical committees and editorial boards.

The NUS-Tsinghua Extreme Search Centre (NExT) is a collaborative effort between the two universities established with an objective to carry out revolutionary research in media search technologies in areas of live media search, live forum search, distributed infrastructure as well as extreme database support. The NExT Search Centre is supported by a funding of S$10 million over five years by the Interactive Digital Media R&D Programme Office (IDMPO) hosted by Media Development Authority of Singapore, with combined contributions in kind from NUS and Tsinghua, bringing the total value of support for the Centre to approximately S$20 million.

Highlights

		Inventor Listing   Media Search Projects in the News

Areas of Research

	Web-Scale Media Search and Multimedia Question-Answering Question-Answering Multimedia Analysis and Synthesis Sound and Music Computing Geo-Referenced Video Search Natural Language Processing Information Retrieval

Virtual, Mixed and Augmented Reality

A boy’s daydream about his favourite science fiction story could now become a reality thanks to major breakthroughs in the world of virtual reality. Augmented reality (AR) has now charted new realms in virtual reality technology as conventional virtual characters break out of the digital world and burst into the real world environment to co-exist with us in never before imagined ways. One of the exciting possibilities on the horizon are holograms that respond to your touch and interact with their background in real-time.

AR technology has been available since the 20^th century, however its applications were mostly limited to military training and multi-sensory theatre productions whose high powered computers and technology were able to support the AR. As increased computing power becomes common in everyday devices such as personal computers, smartphones and PDAs, a broad range of applications in education, entertainment and architecture have emerged to make use of AR’s highly interactive platform to enhance learning and experience.

NUS Expertise

Although the market for AR technology is still maturing, Singapore has not ignored this small but emerging sector and has been able to groom research leaders in this new cutting edge technology. Within NUS, the Interactive and Digital Media Institute and the Department of Electrical and Computer Engineering are two prominent groups that spearhead research and development of AR technological applications that we can use in our daily lives.

MXR Corp, founded by NUS professors Adrian Cheok and Steven Zhou, is an example of a company that produces educational products based on AR technology. Some products of MXR Corp include magic cards and story books where characters are brought to life in 3D and interact with children in a fun and engaging way. Major corporations and organizational bodies like Walt Disney, Microsoft and the Singapore Tourism Board have subscribed to MXR’s AR technology, reflecting the exciting growth potential of AR in areas like gaming, entertainment and tourism. The University is now committed towards making AR and its applications more visible and accessible to the public through continued industry partnerships and participation in public technology forums and showcases.

Technology Offers

	Interactive Transcription System and Method Augmented Reality on Mobile Virtual Keyboard Markerless 6DOF Computer Vision Tracker for 3D Augmentation of Patterned Planar Surfaces

Inventor Listing

	Adrian Cheok Dr Cheok is the director of the Mixed Reality Lab (MXR) Henry Duh Read about Dr Duh's multidisciplinary work in Human Computer Interaction (HCI)  Ong Soh Khim Read more about A/Prof Ong's research group in the Department of Mechanical Engineering Steven Zhou Co founder of MXR Tan Ping Image processing and computer modeling

Areas of Research

	Researcher Area Of Research Tan Ping Sketch2photo: Internet Image Montage Symmetric Architecture Modeling with a Single Image Image Based 3D Architecture Modeling Single Image Tree Sketching Steven Zhou System and Method for 3D Stereoscopic Photo Creation (Funded by NRF PoC) Interactive Transcription System and Method Adrian Cheok Augmented Reality on Mobile Ong Soh Khim Virtual Keyboard Markerless 6DOF Computer Vision Tracker for 3D Augmentation of Patterned Planar Surfaces Henry Duh MobileAR Game Engine MobileAR Embedded System*

Education

IDM in education is one of the major focus areas of Singapore's efforts to grow its IDM sector. Key to these efforts is the development of human capital and R&D capabilities in IDM in education, and ultimately, Singapore's ability to translate these ingredients into successful products and services. The IDM in Education SIG seeks to address IDM-related gaps related to the area of education and match R&D results to the needs of the education sector. The SIG members is focusing on a few important areas such as the commercialisation of successful pedagogical models, tools and content as well as the development of innovative educational programmes and tools to enhance learning.

Education Using Musical Tools

	Dr. Wang Ye is an Assistant Professor with the Computer Science Department. Dr. Wang established the Sound and Music Computing (SMC) Lab within the NUS School of Computing. He is currently the head of the SMC Lab. His main research projects include the multimodal mobile music retrieval (M3R) and its applications to healthcare, and the NUS Mobile Music Group (NuMOG) for edutainment. Dr Wang Ye and his research team have been most recently working on two research projects: MOGCLASS and MusicalWalk.

Areas of Research

	MOGCLASS (Musical mObile Group for Classroom Learning And Study in Schools) MOGCLASS was conceived as a mobile network system for improving music education by collaborative participation, targeted for use by Primary 3-6 students. Its prototype was successfully deployed in three primary schools and the Muscular Dystrophy Association (Singapore). Read more... MusicalWalk In recent research efforts, Dr Wang’s group has developed new methods for domain-specific music retrieval. With these methods, songs can be delivered based upon particular perceived attributes, such as culture-genre or tempo. In further development of this work, his team has begun core research in music retrieval to facilitate rhythmic auditory stimulation (RAS)-based gait training for people with Parkinson’s disease.   Read More....