Grasp Objects in Pybullet Environment, Apply inverse kinematics algorithm in simulated environment

Grasp Objects in Pybullet Environment 1. Project Objective • Get familiar with 3D robot simulation • Understand intelligent robot programming • Apply inverse kinematics algorithm in simulated environment 2. Project Steps 1. Install pybullet with Anaconda (recommended) ◦ Install Anaconda: [login to view URL] ◦ Install Pybullet: [login to view URL] 2. (If you have done the first step, please ignore) Install python3 (optional) Setup the python3 environment on your system. ◦ Windows 10: ▪ [login to view URL] ◦ Mac: ▪ [login to view URL] ◦ Ubuntu 16.04: ▪ [login to view URL] 3. (If you have done the first step, please ignore) Setup Pybullet Environment: ◦ Run following commands in the terminal: ▪ pip3 install pybullet --upgrade --user ▪ python3 -m pybullet_envs.examples.enjoy_TF_AntBulletEnv_v0_2017may ▪ python3 -m pybullet_envs.examples.enjoy_TF_HumanoidFlagrunHarderBulletEnv_v1_2017jul ▪ python3 -m [login to view URL] --arg_file [login to view URL] ◦ Pybullet Reference: [login to view URL] 4. Load assigned object in the environment ◦ In [login to view URL], change the variable object_path to your assigned object. ◦ In [login to view URL], update object_path to your assigned object. 5. Record the tray location in the environment ◦ run [login to view URL] ◦ Click the GUI and Press Q on your keyboard ◦ Record the tray_x and tray_y ◦ update tray_x and tray_y in both [login to view URL] and [login to view URL] ◦ Please make sure the tray and the object is not conflict. 6. Run [login to view URL] ◦ Adjust your hand position and grasp type by using the slider ◦ After you have adjusted your hand position. Press Q on your keyboard ◦ you can get 3 outputs ▪ handIReading: the pose of the hand ▪ palmPosition: arm postion ▪ Orientation: arm orientation 7. Find grasping trajectory ◦ Run [login to view URL] and record parameters for each step in lines 340 – 376 in [login to view URL] ▪ handIReading • handInitial • handClose • handOpen ▪ palmPosition and orientation • grasp_palmPosition, grasp_orientation • pu_palmPosition, pu_orientation • final_palmPosition, final_orientation 8. Update the outputs in step 7 to [login to view URL] 9. write code for the grasping task ◦ Use Inverse Kinematics Function in [login to view URL] to control the position and orientation of the arm ◦ Use Hand Direct Control Function in [login to view URL] to control the hand 10. Record a video for successfully trail 3. What you need to submit • Functional codes (30%) • Demo illustration (30%) ◦ A video that contains one trial of successfully grasping the target objects. • Project Report (40%) ◦ A project report that in academic paper format should be written for this project (IEEE conference templates) (1%). ◦ The report should include: ▪ Introduction (5%) • Background about robotic Object Recognition and Grasping (2%) • Procedure of the project (2%) ▪ Project Descriptions (10%) • Describe the procedure (2%) • Describe the applied algorithm/methods of the project (4%) • Save the grasping parameters of the hand and fingers into a “.csv” file. For example, if your object is “203”, save your parameters into a “[login to view URL]” including all parameters in step 7 (3%) • Include a picture of the hand gesture (1%) ▪ Results (10%) • Trajectory of the object in grasping (5%) • Snapshots of successful and failed grasps (if there are any) (5%) ▪ Discussion of results (10%) • Discuss the advantages and disadvantages of the designed hand gesture in grasping the object. Why is it successful? (5%) • Discuss the designed trajectory of hand movement. What are the key parameters and how are the parameters are selected? (5%) ▪ Conclusion (3%) • Conclude the report, what you have done (1%), what results you have got (1%), what worked and what did not (1%). ▪ Reference (1%)