NRT Trainee Team Presentations

Fall 2022 Presentation Abstracts

FeelingPlus: A Design of Augmented Reality on Storybook as a Learning Tool for Young Learners’ Perspective-Taking and Implicit Story Comprehension

Meiying Chen, Jin Dou, Erqian Xu

Abstract

This project presents a novel system design of an augmented reality (AR) application called FeelingPlus which can be used as a learning tool for children to better understand the underlying meaning behind text in a storybook. It aims to create a learning space for young learners by providing expressive audio effects and an interactive first-person view of story characters to facilitate perspective-taking with multi-language options, thus helping children with implicit story comprehension. The target age group for this project is children between 5 and 8. To achieve our goal, three features have been built into this application, including emotional and customized audio synthesis, interactive role-play, and multi-language access. Challenges and future work will also be discussed in this report.

FeelingPlus Demo video script (pdf)

Briteller: Sensorimotor Learning for K-12 Machine Learning Education

Xiaofei Zhou, Pei Xiong, Qinqin Xiao

Introduction

With the increasingly important role that Machine Learning (ML) technologies are playing in our daily life, it’s more and more essential to start introducing related ML literacy to the young generation. As one of the most commonly used ML technologies in everyday life scenarios, recommendation systems are having a big impact on what information people and furthermore, influence people’s opinions and behaviors in many different aspects. Therefore, literacy related to recommendation systems is one of the ML literacy that K-12 students could benefit from, not only as potential future developers and designers of such technologies, but also as a more informed recommendation system users, and a citizen living in the era of Artificial Intelligence (AI).

However, with limited background in math, computer science and ML, K-12 students may find it abstract and inaccessible to learn ML literacy around recommendation systems. Therefore, we would like to propose a novel way of using light intensity as a design metaphor to make those knowledge components more tangible and accessible for young students. We designed an optical system named Briteller which serves as a recommendation system. By interacting with Briteller, kids can both understand the high-level function of a recommendation system and also develop a basic understanding of the inner working mechanisms of recommendation systems.

AR/VR visualization platform for the design of novel optical systems

Yiwen Fan and Matthew Ferguson

Introduction

Freeform optics expand what is possible compared with conventional optics by breaking rotational symmetry, allowing for advantages such as performance enhancement and volume reduction. There are also application-specific advantages such as increased spectral bandwidth and spectral resolution in spectrometers [1]. However, this not without drawbacks. Designing such systems can quickly become confusing and non-intuitive in the 2D environments of classical optical design tools such as CODEV and Zemax, which have limited 3D capabilities. The inherently 3D geometries of these systems can be difficult to visualize in 2D, and how altering said geometry affects the performance is not always straightforward.

In this project, we implement Hyperion, a 3D visualization platform, as a potential solution to these challenges. It utilizes a Unity-based user interface (UI) that is powered by an Eikonal+ Fortran backend, a lightweight raytracing software ideal for cross-platform development. There is an existing architecture utilizing this for the personal computer (PC) platform. By extending Hyperion to the mixed reality environment of the HoloLens 2, we enable the visualization of optical systems in a dynamic 3D environment, allowing the user to experiment with new geometries while receiving real-time performance feedback. This creates an immersive, intuitive, 3D design experience ideal for the challenges facing the design of freeform optical systems. For more information and demos go to: Hyperion Demo 2022