Excalibur: Encouraging Embodied Exploration

  • Implemented human-level performance of the AI2THOR agent in new environments in Virtual Reality (VR) using an Oculus headset - Includes replicating continuous movement to discretized steps in VR to human-arm movement in real-world, mapping force to Oculus grip, Opening and Closing objects in VR, and rendering procedurally generated randomized scenes in VR.
  • Worked on building reinforcement learning environments for the agent and implemented the Proximity Policy Optimization Algorithm (PPO) for these environments and agents. Working on integrating semantic maps which will help in improving the performance of the agent.
  • Tech Stack: Python, Open AI Gym, Unity 3D, C#, Oculus VR programming, Reinforcement Learning