A VR controlled robot with head and hand tracking (PC software only, see VR-Robot-AVR). This project was done in collaboration with Tony Li who designed and built the mechanical robot hardware.
This software provides the interface between the user and the robotic arm. It uses the movement of the users head measured by the HMD to adjust the angle of a webcam on the robot (head mechanism not yet built). Additionally the relative position of the users hand, measured by one of the Controllers, is used to set the robot's arm desired end effector position. This, in turn, is used to derive the necessary angle of each of the arm's joints using a gradient descent inverse kinematics model.
Make sure to install the following plugins from Unity asset store and import to their default location
coming soon...
This software should support any OpenVR compatable VR headsets but has only been tested on the Valve Index
- In engine 3D representation of robot arm
- Blank callibration scene (i.e. prior to initializing webcam feed)
- Inverse kinematics model and solver
- World space UI
- HMD/user callibration
- Serial bus writer interface