Mahdi Davoodikakhki

Professor Steve DiPaola

Introduction

  This project is about making a 3D scene generation program. It aims to create 3D scenes and environments in Virtual Reality platforms. In this project, we tend to get the user voice input and use a speech recognition system to convert the user speech to text.

This project can be completed in four steps. First, we can detect some keywords linked to certain models and instantiate the requested models where the user asks for them. Second, We should try to get the whole sentences from users and understand and build the semantic graphs coming from the users. Third, adding a dialogue system between the users and the scene generator. With this dialogue, users would be able to ask for some changes, such as changing the scale, rotation, color, and position of objects. Fourth, with having a large 3D dataset model, users can ask for some different type of the instantiated models.

Team members and roles

  The team for doing this project has just one student, named Mahdi Davoodikakhki, and he gets guidelines from Professor DiPaola, who is the course instructor and his supervisor. Therefore, Mahdi is the responsible person for implementing the codes, design, writing reports, and other sections.

Implementing the Project

   We have two suitable software for implementing this project. The first one is the Unity game engine, which has a proper documentation and also a great asset store that can help us with finding the 3D models for implementing the project. The second solution is using Tivoli Cloud VR software. It seems to be an easier solution overall but for now, its documentation is not very clear as it is a rather new platform and it does not have a dataset of 3D models, but it can easily gather people in an interactive environment, which is a quite worthy advantage for using that.

We plan to try implementing things in Tivoli Cloud VR to test its abilities and take advantage of its interactive environment, but we will use Unity game engine as a backup software to finish the project in case things do not go as planned for this project on Tivoli Cloud VR.

Sketches and ideas

   Here, we show a few sketches from the possible outcome of each phase. Please note that the first phase is done partially and the next phases sketches are created manually.