Browsing by Author "Ha, Sehoon"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Learning Human Search Behavior from Egocentric Visual Inputs(The Eurographics Association and John Wiley & Sons Ltd., 2021) Sorokin, Maks; Yu, Wenhao; Ha, Sehoon; Liu, C. Karen; Mitra, Niloy and Viola, Ivan''Looking for things'' is a mundane but critical task we repeatedly carry on in our daily life. We introduce a method to develop a human character capable of searching for a randomly located target object in a detailed 3D scene using its locomotion capability and egocentric vision perception represented as RGBD images. By depriving the privileged 3D information from the human character, it is forced to move and look around simultaneously to account for the restricted sensing capability, resulting in natural navigation and search behaviors. Our method consists of two components: 1) a search control policy based on an abstract character model, and 2) an online replanning control module for synthesizing detailed kinematic motion based on the trajectories planned by the search policy. We demonstrate that the combined techniques enable the character to effectively find often occluded household items in indoor environments. The same search policy can be applied to different full body characters without the need of retraining. We evaluate our method quantitatively by testing it on randomly generated scenarios. Our work is a first step toward creating intelligent virtual agents with humanlike behaviors driven by onboard sensors, paving the road toward future robotic applications.Item Learning to Transfer In-Hand Manipulations Using a Greedy Shape Curriculum(The Eurographics Association and John Wiley & Sons Ltd., 2023) Zhang, Yunbo; Clegg, Alexander; Ha, Sehoon; Turk, Greg; Ye, Yuting; Myszkowski, Karol; Niessner, MatthiasIn-hand object manipulation is challenging to simulate due to complex contact dynamics, non-repetitive finger gaits, and the need to indirectly control unactuated objects. Further adapting a successful manipulation skill to new objects with different shapes and physical properties is a similarly challenging problem. In this work, we show that natural and robust in-hand manipulation of simple objects in a dynamic simulation can be learned from a high quality motion capture example via deep reinforcement learning with careful designs of the imitation learning problem. We apply our approach on both single-handed and two-handed dexterous manipulations of diverse object shapes and motions. We then demonstrate further adaptation of the example motion to a more complex shape through curriculum learning on intermediate shapes morphed between the source and target object. While a naive curriculum of progressive morphs often falls short, we propose a simple greedy curriculum search algorithm that can successfully apply to a range of objects such as a teapot, bunny, bottle, train, and elephant.