Source_CodeYour state decides how the mission unfolds.
The assignment was to develop a Mixed Reality (MR), Augmented Reality (AR) or Virtual Reality (VR) training experience that places the player under cognitive, emotional or physical stress. An EmotiBit sensor is used to track the player’s vital body signals.
Our team chose to develop a Virtual Reality space station repair mission that combines all three types of stress into a single immersive training experience.
Gameplay
The player explores a damaged space station in VR and completes different repair tasks such as connecting cables, solving riddles and restoring navigation. While doing this, the player moves through a dangerous environment with limited oxygen and constant time pressure. Extra tasks can appear during the mission, so the player has to quickly decide what to focus on while everything feels unstable.
The experience changes while the player is playing. A sensor called EmotiBit is used to track the player’s vital body signals and provides information about the player’s current state. The recorded data is sent to the Serrala-KI-Server, where it is analyzed. The analyzed data is then sent to the Administration Panel. Based on this data, the Administration Panel determines the player’s physical, emotional and cognitive stress levels and sends them to Unity.
The data flow works as follows:
Based on the player’s stress level and game performance, the system adapts timers and environmental effects during the mission. The goal is to keep the game challenging but still playable and engaging.
One challenge was finding the right level of detail for the 3D models so they looked good and matched the overall style of the project. Several iterations were needed to arrange the UVs in a way that avoided blurry textures in areas where the player spends most of the time.
Another challenge was integrating the adaptive stress system into Unity. To implement the communication between the Administration Panel and Unity, scripts from the previous research project KeepCool first had to be understood and adapted for our project. Establishing reliable communication between all components required repeated testing and debugging.
For some team members, Unity was a completely new environment at the start of the project. By learning and working together, they were able to understand the engine and contribute to the development of the project. Another challenge was organizing the team’s workflow. At the beginning, development was sometimes more sequential, meaning that only one person worked on certain tasks at a time. Over time, the team improved communication and task distribution which resulted in a more efficient workflow.
