| Affiliations: | College of Engineering and Computer Science |
| Team Leader: |
|
| Faculty Mentor: |
Joseph J. LaViola Jr., PhD
|
|
Team Size:
|
7 |
| Open Spots: | 1 |
|
Team Member Qualifications:
|
Applicants should ideally have experience with Python, including scripting, working with APIs, and using libraries such as pandas, matplotlib, and seaborn. Familiarity with Large Language Models, particularly in areas such as prompt engineering, API usage, and context handling, is also preferred. Experience in Unity development, especially involving VR interaction, scripting, and API integration, is beneficial. Proficiency with GitHub, including command line operations like cloning repositories, pulling and pushing changes, merging branches, and submitting pull requests, is encouraged. Strong research writing skills, including the use of tools like Zotero and Overleaf for citation management, are advantageous. Finally, applicants should be able to take clear notes and report progress effectively. |
|
Description:
|
This project explores AI-driven dialogue systems in virtual environments using Large Language Models (LLMs), Visual-Language Models (VLMs), Gesture Recognition, and Virtual Reality (VR). The existing pipeline enables real-time interactions with virtual agents through ASR → LLM → TTS, built in Unity. The goal is to enhance these interactions by making them multimodal, incorporating gestures, visual context, and richer conversational dynamics. Students will contribute to research that expands the capabilities of interactive AI in immersive settings. |