An important application of autonomous robotic manipulation is in automation. Consider our robot on the right, its job is to find and pack items from a cluttered bin. This type of robot is particularly useful for warehouse automation and order fulfillment.
One key challenge is that the robot does not know anything about the clutter of objects in the bin in front of it. The robot has to understand which object to go for and how to manipulate it, either using its suction cup or gripper. Sometimes using just grasping or just suction is not enough. Perhaps the object is occluded by others or it’s in an awkward position. In this case, the robot first needs to plan to move some objects out of the way or to move the object away from others to be able to grasp it. Once picked, the robot should also verify the item and carefully place it in a box, ready to ship to the customer.
Building full robotic systems is key to developing state-of-the-art solutions and solving central challenges in robotics. Only in doing do we really understand what is possible. We have developed such systems to compete in the Amazon Robotics Challenges (2015 through 2017) that autonomously pick and stow objects from raw visual feedback. We’ve learned a lot from these experience and look forward to tackling many open challenges such as using tactile feedback to close the loop, learn manipulation primitives for different objects, and use other modalities such as language to target objects.
