Deep Learning of Force Manifolds from the Simulated Physics of Robotic Paper Folding

Who’s working on this: Tong, D., Choi, A., Demetri, T., Jungseock, J., Jawed, M.K.

Robotic manipulation of slender objects is challenging, especially when the induced deformations are large and nonlinear. Traditionally, learning-based control approaches, e.g., imitation learning, have been used to tackle deformable material manipulation. Such approaches lack generality and often suffer critical failure from a simple switch of material, geometric, and/or environmental (e.g., friction) properties. In this article, we address a fundamental but difficult step of robotic origami: forming a predefined fold in paper with only a single manipulator. A data-driven framework combining physically-accurate simulation and machine learning is used to train deep neural network models capable of predicting the external forces induced on the paper given a grasp position. We frame the problem using scaling analysis, resulting in a control framework robust against material and geometric changes. Path planning is carried out over the generated manifold to produce robot manipulation trajectories optimized to prevent sliding. Furthermore, the inference speed of the trained model enables the incorporation of real-time visual feedback to achieve closed-loop sensorimotor control. Real-world experiments demonstrate that our framework can greatly improve robotic manipulation performance compared against natural paper folding strategies, even when manipulating paper objects of various materials and shapes.

Publication: Tong, D., Choi, A., Demetri, T., Jungseock, J., Jawed, M.K., “A Method to Enable Robotic Paper Folding Based on Deep Learning and Physics Simulations” 2023

Funding: We acknowledge financial support from the National Science Foundation under Grant numbers IIS-1925360, CAREER2047663, and OAC-2209782.

YouTube: https://www.youtube.com/watch?v=sRpjZhPZSx0

GitHub: TBD