Experiment overview
This experiment evaluated whether a low-cost, fully open-loop pipeline could perform reliable tabletop pick-and-place tasks. The system relies on a smartphone overhead webcam, ArUco fiducial markers, and a custom Python/Arduino host-firmware stack, explicitly avoiding encoders, force feedback, visual servoing, or standard robotics middleware like ROS.
System architecture
The architecture divides control into two distinct layers. A Python host handles computer vision, coordinate mapping, the inverse kinematics (IK) solver, and waypoint sequencing. The Arduino firmware manages I²C servo communication and smoothstep-eased motion profiles at 50 Hz.
Synchronization relies on a simple ASCII serial protocol. The host sends waypoints and waits for an "OK" completion token from the firmware. This avoids the need for a global clock or complex timing models for the mechanical arm.
Key engineering decisions
- Metric homography: Camera frames are warped to a 610x450 pixel destination, mapping 1 pixel directly to 1 millimeter. This identity transform eliminates scale-calibration errors.
- Analytic parallax correction: Uses known camera height and object height to calculate the displacement of an object's top surface relative to its footprint, replacing depth sensors.
- Closed-form IK: The solver maintains a level gripper as a constraint, collapsing the 5-DOF problem into a geometric triangle approximation tuned for this specific arm.
- Empirical constants: Mechanical flex, backlash, and mounting errors are compensated using named scalar constants rather than a feedback controller.
Outcomes and failures
The prototype successfully demonstrated an eight-step pick-and-place sequence within a constrained workspace. However, the project suffered from significant technical debt and abandoned refactors. An attempt to modularize the codebase was left incomplete, resulting in broken imports, naming mismatches, and configuration drift between the working and modular trees.
Data corruption also remained hidden due to improper error handling. A truncated calibration file was ignored by a bare 'except' clause, allowing the system to fall back to interactive calibration silently. Similarly, the IK logic diverged between the working code and the abandoned modular branch.
Engineering findings
- Intermediate representations should make difficult conversions an identity transform.
- Planar workspaces do not require camera intrinsics when fiducial-based homography is used.
- Completion tokens are more robust for sequencing than timers.
- Enforcing safety limits at the firmware layer is essential, as host-side constraints are insufficient for hardware protection.
Next steps
Future work requires resolving the modular refactor, implementing self-collision checking, and calibrating servo pulse widths. Connecting the existing 'S' speed and 'STOP' firmware commands, along with finalizing the multi-zone drop system, remains as the primary path for further development.