Robot System
In this project, me and my colleague developed a visual navigation system on Jackal UGV robot. The objective is to return to the starting position, based on the image-based graph memory. For each episode, a demonstration trajectory is given. By leveraging such memory, the robot needs to find its location from the memory, and return to the specified place in the trajectory.
During a demonstration, the system constructs a graph-based memory. The topdown map in the video is just for visualization, and it is not used in the robot system. In each node, a panoramic image from ricoh theta is stored, and the edge represents the spatial relationship of each node. During the memory construction, the localization is done by VGM and SuperGlue.
At each episode, the target place is given as an image. The system finds the target place using the constructed graph memory, by comparing the target image and the stored image in each node. Once the target place is localized, the navigation path is generated using dijkstra algorithm.
After path planning, the robot navigate to the target place following the planned path. However, the graph memory itself only contains image information, and no other information. To inference the relative direction to the subgoal node, we used keypoing matching from SuperGlue. We can calculate relative orientation based on the matched keypoints between the robot observation and subgoal node.
* The topdown view is just for visualization, and it is not given to the graph-building system.
* The topdown view is just for visualization, and it is not given to the navigation system.
The target place is given by an image, and the system needs to find the target location in the graph memory