Grasp-point detection and robotic picking of mixed items
Items of different shapes and packaging lie mixed in the tote. An overhead depth camera captures the scene in 3D with the help of a projected dot pattern; a deep-learning model finds and scores the surfaces that suit the gripper. The robot picks the item at the highest-scoring point and places it in the order tote.
The selected grasp point, surface normal and approach direction are sent to the robot as a pose. An item below the threshold is skipped and re-evaluated in the next image; the number of picked items is reported to the order system.
- 01 Infeed
- 02 Pattern and image
- 03 Grasp analysis
- 04 Pick
- 05 Place
The mixed-item tote enters the cell and stops at the pick position.
- Grasp score
- —
- Candidate surfaces
- —
- Items picked
- —
Scenes are illustrative simulations; they are not customer references or performance guarantees.
Need
The items are not known in advance; pouches, boxes and bottles lie on top of each other at random angles. Plain or glossy packaging offers no texture for stereo matching, and wrinkled film or overlapping items may leave too little flat area for a suction cup.
Solution approach
The projector casts a random dot pattern on the scene; the pattern gives texture-less surfaces detail to match, and the stereo pair yields a dense depth map. A deep-learning model finds flat, unobstructed surfaces for the suction cup from the colour and depth images without recognising the item. Each candidate is scored by area, tilt and collision clearance; one below the threshold is not picked.
Process steps
- 01
Infeed
The mixed-item tote enters the cell and stops at the pick position.
- 02
Pattern and image
The dot pattern is projected and the depth image is captured.
- 03
Grasp analysis
The model finds and scores the candidate surfaces.
- 04
Pick
The robot picks the item by vacuum at the highest-scoring point.
- 05
Place
The item is placed in the order tote and the scene is imaged again.
Related applications
Let us assess your application together
We test your sample in our laboratory and report on resolution, optics and lighting selection.