Project overview

This is a course project of CSIE5074 Robotics, offered by Prof. Li-Chen Fu @ NTU EE. I led our team to win the first place of the course competition. My contribution of this work is the CGA theoretical development, robot motion planning, and the project management as a team leader.

We built a supervisory teleoperation kendo robot system. The robot is able to track the opponent’s pose and wait for the operator’s command to attack.

The kendo robot system is implemented in both ROS 1 Python and JavaScript and can be found in kendo_robot.

To detect and track the human body, we utilised a deep-learning-based human pose estimation algorithm (LOGO-CAP, CVPR 2022). By aligning the RGB and DEPTH images from the RealSense D435, we obtained the 3D coordinates of the human body.

An interactive GUI was created using JavaScript for visualisation and real-time implementation. With just one click on the GUI button, the operator can send an attack command to ROS (via a WebSocket), which controls the robot arm. The GUI is handcrafted based on ganja.js, a web-programmable platform developed by the geometric algebra community.

Drag the mouse cursor to change viewpoints and modify the codes to change parameters through the interactive GUI!

Visualisation, inverse kinematics, and trajectory planning are all handled using conformal geometric algebra (CGA), a powerful mathematical framework for representing geometry. CGA provides numerous geometric insights into the robot that are nearly impossible to achieve using traditional matrix methods.

• Team members
  • Wei-Hsuan Cheng (Team Leader): johnathancheng0125@gmail.com / r11631045@ntu.edu.tw
  • Wen Perng: b10901042@ntu.edu.tw
  • Che-Jung Chuang: r12021008@ntu.edu.tw
  • Cheng-Yen Yu: r12922135@ntu.edu.tw

Resources

• YouTube Demo Video
• Interactive GUI of Kendo Robot System
• LOGO-CAP Human Pose Estimation Algorithm
• The Geometric Algebra Community - biVector.net
• ganja.js

Project report in PDF

Download the project report.