| Affiliations: | College of Engineering and Computer Science |
| Team Leader: |
|
| Faculty Mentor: |
Hwan Choi, PhD
|
|
Team Size:
|
5 |
| Open Spots: | 3 |
|
Team Member Qualifications:
|
PyTorch. CNN. Deep learning. Computer vision |
|
Description:
|
Deep Multimodal Terrain Recognition for Safe Robotic and Wearable Mobility refers to a perception framework that integrates multiple complementary sensors—such as cameras, IMUs, LiDAR, and proprioceptive signals—to accurately classify and predict terrain conditions during locomotion. By leveraging deep learning–based multimodal fusion, the system captures both visual appearance and motion/force cues, enabling robust recognition of stairs, slopes, uneven ground, grass, gravel, and other challenging environments. This multimodal approach improves reliability compared to single‑sensor systems, especially in real‑world conditions where lighting, occlusion, or sensor noise can degrade performance. Research in multimodal fusion for robot navigation shows that combining RGB images with LiDAR or IMU data significantly enhances perception robustness and navigation accuracy in complex environments . For robotic platforms, the system supports safer autonomous navigation by providing terrain‑aware decision‑making and traversability estimation. For wearable mobility devices—such as powered prosthetic legs, exoskeletons, and orthoses—it enables predictive control strategies that adjust joint impedance, torque, or gait mode before the user encounters a new terrain, reducing fall risk and improving mobility confidence. |