SurRoL: RL Centered and dVRK Compatible Platform for Surgical Robot Learning

The Chinese University of Hong Kong

Overview

Surgical robot automation has attracted increasing research interest over the past decade, expecting its huge potential to benefit surgeons, nurses, and patients. Recently, the learning paradigm of embodied AI has demonstrated a promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role in facilitating relevant research. Despite visible efforts that have been made on embodied intelligence in general, surgical embodied intelligence, which should be supported by well-developed and domain-specific simulation environments, has been largely unexplored so far. In this regard, we designed SurRoL, a simulation platform that is dedicated to surgical embodied intelligence with strong focus on learning algorithm support and extendable infrastructure design. With which we hope to pave the way for future research on surgical embodied intelligence.

Project Papers

Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learning
Yonghao Long, Wang Wei, Tao Huang, Yuehao Wang, Qi Dou [paper]
arXiv preprint: 2301.00452
Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot
Tao Huang, Kai Chen, Bin Li, Yunhui Liu, Qi Dou
IEEE International Conference on Robotics and Automation (ICRA), 2023.
Integrating Artificial Intelligence and Augmented Reality in Robotic Surgery: An Initial dVRK Study Using a Surgical Education Scenario
Yonghao Long, Jianfeng Cao, Anton Deguet, Russell H. Taylor, Qi Dou [paper]
International Symposium on Medical Robotics (ISMR), 2022.
SurRoL: An Open-source RL Centered and dVRK Compatible Platform for Surgical Robot Learning.
Jiaqi Xu*, Bin Li*, Bo Lu, Yun-Hui Liu, Qi Dou, Pheng-Ann Heng [paper] [code]
International Conference on Intelligent Robots and Systems (IROS), 2021.

(Included in dVRK Software Ecosystem as a community open-source ML tool)

Specifics

In SurRoL, we create assets of surgical tasks with the help of the powerful 3D modeling tool Blender, and then generate relevant collision models and urdf description models at the same time for physical modeling in the simulation environment. Once surgical tasks are imported to the simulator, the human can conduct surgical action using a human interaction device, and the interaction information is streamed to the virtual environment for physical simulation. In the meanwhile, the video frames are produced using the visualization engine of the simulator, which will be displayed on the monitor for human perception and interaction. The human action can be recorded for intelligent policy learning and the policy can also interact with the virtual environment, which forms the diagram of human-in-the-loop surgical robot learning with interactive simulation environment.

Task Demonstrations

We have established a spectrum of tasks given the dexterity and precision properties in the surgical context.

In detail, SurRoL has Fundamental Action Tasks: Needle Reach, Gauze Retrieve, Needle Pick and Needle Regrasp; ECM Fov Control Tasks: ECM Reach, MisOrient, Static Track and Active Track; and Basic Surgical Robot Siill Training Tasks: Peg Transfer, Bimanual Peg Transfer, Needle the Rings, Pick and Place, Peg Board and Match Board. These 14 tasks range from entry-level to sophisticated counterparts, which cover levels of surgical skills and involve manipulating PSM(s) and ECM.

  • Needle ReachTo move the PSM jaw tip to the location slightly above the needle within a tolerance, where the needle is randomly placed on a surgical tray, and the jaw is close and of fixed orientation.
  • Gauze RetrieveTo sequentially pick the surgical gauze and get it back (place it at the target position), with one DoF to indicate the jaw open/close.
  • Needle PickTo sequentially pick the surgical needle and get it back (place it at the target position), with an additional DoF indicating the yaw angle.
  • needle regraspTo hand over the held needle from one arm to the other arm with bimanual operations.
Needle Reach1 Gauze Retrieve2 Needle Pick3 needle regrasp4
css image slider by WOWSlider.com v9.0m
  • ECM ReachTo move the camera mounted on ECM to a randomly sampled position.
  • MisorientTo adjust the ECM's joint such that the misorientation with the desired Natural Line-of-Sight is minimized.
  • jquery carousel sliderTo let the ECM track a static target cube with red color, disturbed by other surrounding cubes, that mimics the scenario to focus on the primary instrument during surgery.
  • Active TrackTo keep the ECM tracking the moving cube, with a relaxed misorientation requirement but a chance to lose the target out of the view.
ECM Reach1 Misorient2 Static Track3 Active Track4
jquery image carousel by WOWSlider.com v9.0m
  • Peg TransferTo move the block from one peg to the other peg with single PSM, which requires collision avoidance and long-horizon reasoning.
  • bi pegtransferTo move the block from one peg to the other peg with bimanual operations.
  • needleringsTo pick up the needle and pass it through the highlighted ring which calls for high positioning and orienting skills.
  • Pick and PlaceTo place the colored jacks into the matching colored containers sequentially.
  • image sliderTo pick up and transfer the rings sequentially from the peg board to the peg on the floor.
  • Match BoardTo pick up the various objects and place them into their corresponding spaces on the board.
Peg Transfer1 bi pegtransfer2 needlerings3 Pick and Place4 Peg Board5 Match Board6
slider html by WOWSlider.com v9.0m

Control with Human Input Device

To incorporate human control of surgical robots in the simulator, we develop a manipulation system with human input devices. We use Touch (3D Systems Inc.) physical devices to simulate two master arms of the robot to tele-operate Patient Side Manipulators (PSMs) and the Endoscopic Camera Manipulator (ECM).

ECM Control

Bimanual PSMs Control

Deployment on the dVRK Real Robot

Below are the trajectories of four learned policies on the real dVRK which demonstrate that our platform enables the smooth transfer of learned skills from the simulation to the real world.

Contact

For any questions, please feel free to email qidou@cuhk.edu.hk


Html Counter hits since Jan 2023