Stories from the bench.
One week. A new robot. A demo in Paris.
Porting the LLM cobot from a Franka to a UR5e in three days, an 11-hour solo drive, and Jensen Huang signing the robot.
Six years of shipping perception to cars
Segmentation, detection, tracking, depth and rain detection on embedded GPUs at ESR Labs, 2017–2023. The lessons that outlived the models.
Teaching a UR5e lab chores with AprilTags
Drawer, centrifuge knob, button, lid, rack. No world model, just a calibration chain and YAML pipelines. And the one inverse that cost 230 mm.
Humanoid teleoperation: SenseGlove + HTC Vive
Driving a humanoid's arms in Isaac Lab 2.2 with Nova2 gloves and Vive trackers. No headset, no camera hand-tracking.
LLM-powered cobot manipulation through speech
Whisper, GPT-4, a robot-agnostic control layer, CNOS + FoundationPose and annotated grips. You talk; the arm picks.
Hand-eye calibration, three ways
Every grasp depends on one transform. I compared a board, a fixed camera and a known-geometry "selfie" on a Franka FR3, and measured each.
First steps to teleoperation
Flex sensors, a RealSense T265 and a 3D-printed bracket on a glove. Cheap, smooth, and not scalable.
Quantifying NVIDIA FoundationPose in the real world
A Franka, a Zivid and a ChArUco board as an automated evaluation rig, and what masking RGB vs depth reveals about the model.
Training-free 6D pose with OnePose++
SfM on an ArUco board, LoFTR features per 3D point, PnP at runtime. Then rotation broke it, and se2-LoFTR fixed it.
6D pose estimation from synthetic-only data
Keypoint heatmaps, PnP and one normalisation swap. A Rubik's cube and a Robotiq gripper, trained only on Blender renders.
Metric depth without depth labels
Camera plus sparse LiDAR in, dense metric depth out, trained with view synthesis and the car's own speed. Pose quality turned out to limit everything.