Automatic Synthesis of 4D Effects

Representative image: 
Abstract: 

Providing well-synchronized haptic effects (motion and vibrotactile effects) corresponding to audiovisual contents plays a large role in enhancing user's experience and immersiveness. A typical example is 4D system. 4D system refers to an immersive entertainment system that presents various physical effects, such as motion, vibration, wind, water, and scent with audiovisual contents. In particular, haptic effects takes almost 80% of the 4D effects . However, creating 4D effects is very labor-intensive, and it has been a major obstacle against the wider spread of 4D platforms. In this regard, we are working on developing algorithms that automatically synthesize haptic effects by extracting important features from audiovisual contents. For example, we did research on automatic synthesis of motion effects. We applied computer vision algorithms to estimate the motion of an object or the motion of a virtual camera in a scene, and automatically generated motion effects by mapping the estimated motion. Also, we studied of synthesizing vibrotactile effects automatically. In this research, we developed algorithms that substitute motion effects to vibrotactile effects.

Video: