This is an independent project to design and build a 6-DOF robotic manipulation platform end-to-end, from mechanical design through simulation-integrated motion planning, aimed at getting reliable manipulation out of low-cost hardware with no force/torque sensing or joint encoders. I'm using chess piece pick-and-place as a concrete, visually well-defined validation task for an otherwise domain-agnostic pipeline; the classifier and task logic are meant to be the only pieces that would need to change to move it into a different domain, like pharmacy or food-prep manipulation.

Mechanical & electronics: I designed the 6-DOF arm in CAD from scratch, including the end-effector/gripper geometry and separate collision vs. visual meshes optimized for real-time simulation, then 3D-printed and assembled the full arm. It's actuated with Feetech STS3215 serial-bus servos through a Waveshare driver board on a 12V/10A supply, with a wrist-mounted Intel RealSense D405 depth camera for direct gripper-to-object visual feedback. The servos have no output-side encoders, so I lean on the Feetech present-load register as a rough force-sensing proxy and built the rest of the control architecture around vision rather than proprioception.

Software (ROS2 / MoveIt2): I brought up a full ROS2 Humble workspace, including a custom hardware interface under ros2_control to bridge the Feetech servos into standard ROS2 control, implemented forward and inverse kinematics for the arm, and integrated it into MoveIt2 for motion planning using OMPL and CHOMP.

Simulation / digital twin (MuJoCo): I built a full digital twin of the robot in MuJoCo (the arm, gripper, and wrist-mounted depth camera with field-of-view matched to the real sensor's datasheet), along with a simulated camera ROS2 node that mirrors the real RealSense driver's topic structure exactly, so perception and planning code runs unmodified against either simulated or real hardware. Since clean rendered depth doesn't transfer realistically, I added a synthetic depth-noise model covering stereo triangulation error, range-edge dropout, and grazing-angle dropout, plus a MuJoCo ros2_control hardware interface so the same control stack can drive either the simulated or physical robot with no code changes.

Grasp planning & perception: A segmentation pipeline (plane removal + clustering into object instances, with self-filtering that excludes the arm's own body from the point cloud) feeds a scene-builder that turns each detected instance into a MoveIt2 collision object, both running live in the loop. Grasp candidates come from graspnet-baseline: a pretrained, class-agnostic, parallel-jaw grasp synthesis network (PyTorch, trained on GraspNet-1Billion), using its RealSense-trained checkpoint with no fine-tuning needed since it already matches the D405. It's exposed as a standard MoveIt2 GraspPlanning service and has been validated against a real RealSense scan: 340 raw candidates, 120 surviving collision filtering, with good scores.

What's next: Motion execution (transit plus grasp/retreat) is implemented and validated end-to-end against a live MoveIt2 move_group. Remaining work: a target-selection layer to decide which detected object to act on each cycle; a multi-view sweep-and-fuse step (FK-seeded, ICP-refined) for reacquisition and to resolve approach-angle ambiguity from self-occlusion; a confidence-gating layer that combines depth quality, ICP fitness, and grasp score before committing to an action; wrist-mounted visual servoing for the final approach, since there's no gravity feedforward or in-hand force sensing on this hardware and vision is the only mechanism left for correcting servo backlash/deadband at grasp time; and mapping the predicted gripper opening width to this gripper's actual joint value. Reinforcement learning is deliberately kept out of the pipeline entirely: segmentation, grasp synthesis, transit, and final approach are each either a geometry problem, a single-shot trained prediction, or classical feedback control, not a sequential decision problem with unknown dynamics that would need one.