Touch Surgery · Rigging tools, real-time rigs and AI tools
Touch Surgery
At Touch Surgery, which lets surgeons and medical teams learn and rehearse surgical procedures, I worked as a technical director on rigging tools, real-time rigs and AI tools. I built PyQt and Maya tools, automated pipeline tasks in Python and wrote an automated IK rigging system for the laparoscopic duodenal switch simulation. I also worked on a Python AI that turned MRI data into an augmented reality deformation system.
- Client
- Touch Surgery
- Role
- Rigging tools, real-time rigs and AI tools
- Year
- 2017 to 2019
- Tech
- Python, Maya, PyQt
The challenge
Touch Surgery lets surgeons and medical teams learn, prepare for and test themselves on surgical procedures any time, anywhere. It helps leading medical device and pharmaceutical companies speed up the surgical competence of their sales reps and medical education attendees, and with over 2 million users it also gives marketing teams a clinical audience for their product and procedure stories.
Every procedure needs realistic 3D anatomy that moves the way real tissue does, and the company's 3D work was a core part of what set it apart. That meant rigs that behave convincingly, and tools that let the team build and update them quickly.
What I did
I joined as Senior Technical Director, working with the team on rigging tools, real-time rigs and AI tools.
- I automated and scripted technical tasks for clients using the in-house Python library, and built PyQt and Maya tools to speed up rigging and animation.
- I created a class-based Python inheritance hierarchy that cut the code by over 80% while adding functionality.
- I created a Python AI that read 3D MRI point data and translated it into an augmented reality deformation system.
- I built TensorFlow and Keras models to analyse images, reaching 97.89% accuracy.
Laparoscopic duodenal switch. This bariatric procedure, also known as biliopancreatic diversion with duodenal switch, combines restriction (feeling full sooner, so eating less) with reduced absorption of fat. It was one of the more difficult surgeries Touch Surgery completed, and one of the more difficult rigs I have made. The internal organs had to flex as close to real life as possible, so I wrote a new automated 3D IK rigging system in Python. It focused on reducing the flipping that crossing vectors cause in a rig, and it could rebuild the rigs in a few clicks whenever the geometry was updated.
The outcome
The rigs and tools went into Touch Surgery's surgical simulations, including the laparoscopic duodenal switch procedure, used for medical education on phones, tablets and augmented reality headsets.
I was also Technical Director on Touch Surgery's midwifery best practice simulation, which was shown at the United Nations.
