We have successfully reached all of our goals for the SVD and SVD Encore. This page documents implementation performance and progress as of SVD Encore.
Performance
| Requirement | Target | Achieved |
| Registration Error | < 2 mm | < 2 mm |
| IR Tracking Rate | ≥ 10 Hz | 50 Hz |
| Tracking Drift over 5 minutes | ≤ 2 mm | 0.24 mm |
| AVP Motion-to-Motion Latency | ≤ 500 ms | 388 ms |
| Point Reachability Error | < 5mm – 2/3 sites | < 5 mm – 3/3 sites |
| IR Tracking Uptime | ≥ 95% | 99.3% |
Perception Subsystem
Our perception layer first performs a visual scan over the leg bones to gather a point cloud, and registers it to a CT scan reference point cloud to get an initial position. Then, our IR camera tracks the motion of markers attached to the bones and the arm end effector for fast, reliable transmission of their full 6 degree-of-freedom poses to the VR Digital Twin subsystem.
- Scanning trajectory gathers a dense point cloud of the bones on the operating table, then performs segmentation with a hand-tuned YOLOv11 model
- To ensure highest accuracy and success rate, the system can perform multiple scans, and combine those that the user identifies as good using a least-squares solver.
- Tracking drift is calculated and corrected using redundant information on the end effector pose– both forward kinematics and the custom IR tracker on the end effector should yield the same pose, and we can correct by the difference when they do not.
- IR trackers placed on the femur, tibia, and robot end effector are tracked in 6 degrees of freedom by our Polaris Vega VT camera at a rate of 20Hz.
- Tracking information is smoothed and made robust with an EKF.
VR Simulation & Digital Twin Subsystem
We are utilizing NVIDIA Isaac Sim to serve as the “imagination” for our Kuka Med 7. Registration and tracking data are sent over ROS to create digital twins of the Kuka arm, femur and tibia as they are tracked in real time on the Apple Vision Pro. The interface allows the user to select a target point in the simulation, and have the arm move to reach that point.
- Displays tracked 6 degree-of-freedom poses of bones and robot arm live with high precision
- Receives tracking information at 20Hz over ROS, integrates with tracking user hand motions
- Allows user to select 6 degree-of-freedom target point for the end effector to reach.
- Once a point is selected, the end effector will continually follow that point, relative to the tracked bone.
Motion Planning & Hardware Subsystem
Our system uses a Kuka Med 7 medical cobot for its motion planning stack. The arm is responsible for sweeping out our registration trajectories and will be responsible for cutting/drilling procedures in the Fall semester. Its medical-grade accuracy is important both for maintaining our desired goal of medical-grade accuracy for the entire end-to-end system and for providing a ground-truth point of comparison for our registration and tracking pipelines.
As of SVD, this subsystem also includes several safety features:
- Simple distance-based geofencing; the arm is not allowed to plan more than 0.3m (1ft) away from the tracked bones.
- Magnetically-attached custom end effector tip– preserves precision for validation, while coming off gently and easily to avoid damage to bones, arm, or users in the case of unexpected movements.
- A big-red-button local emergency stop which immediately cuts power to the arm.
- Two “validation” pedals– one locally wired to the system, the other transmitting its signal over the Internet. Teleoperated arm tracking stops while either of these pedals is pressed.