FineSkills is a research project that tries to enable physically-based interaction (navigation and manipulation) in the commercial VR training simulator Magellan which is used for training of employees in the oil and gas industry who learn to perform safety critical tasks in a virtual refinery or offshore platform.

In the simulator several positions (head, hands) are recorded using motion tracking. A head-mounted display serves as output device. Traditional VR simulators use a 3d joystick as input device.

In FineSkills the 3d joystick is replaced with two data gloves. 3d hand models resemble the actual hand pose and position. The user can move his hands naturally. At collisions with virtual objects forces are applied to the touched objects which then move in a physically realistic way.

This attempt allows us to replace conventional “touch and click” interaction with the natural equivalents: pulling, pushing, grasping. Examples: Doors can be opened by grasping and pushing the door handle. Buttons can be activated by a touch of the fingertip. Such interaction is natural, intuitive and … more fun!