Planing Laboratory in Real Time and 3D using Mixed Reality Technology


Fraunhofer IPA in Stuttgart


Bachelor candidate, Mixed Reality Designer & Developer


HoloLens, Unity, 3DS Max

Developed in


I chose to apply for a bachelor candidate position in Fraunhofer IPA because I wanted to enrich my skill set in a new field: mixed reality. It was such an amazing experience to work with a whole new field, technology and people. I graduated with a pretty outstanding grade and I am very proud of it. 

Problems & Challenges

Visualizing complex room or laboratory plan

Robots and laboratory devices are massive

Mixed reality is still a new technology, its potential is endless- 

UX Process


A. Onboarding

– Research on mixed reality technology and hardware
– Introduction to Unity

B. UI/UX Design

– MR App design pattern
– LabPlaner Concept
– 3D Model & Animation

C. Programming

– C#
– Visual Studio
– Unity
– User Testing

D. Release

– Optimization
– Thesis

How It’s Done

1. Onboarding

I spent my first few days playing around with HoloLens. Then I researched more about the field, the technology behind it and what people have created for the device. This is very important since I had to have first hand experience first with the device to figure out what I can do to enrich people experience who are already using the device or who have not. 

Different Kind of Reality

In the virtuality continuum there is reality as it is, augmented reality, augmented virtuality, and virtual reality. And mixed reality is just a combination of them simply saying. 

Types of App

Just like mobile apps, there are at least 5 types of app that people can create for the mixed reality devices such as business, education, entertainment, research or construction app. 


Advantage of HoloLens for Lab Planing

Lab planing usually takes place in 2D medium such as paper, print out, etc. Even if it involves 3D objects, it still takes place in 2D medium such as computer screen. Using HoloLens lab planing will have several advantages.

Planing in real-time: Seeing is believing

Three dimensional objects along with its animation: see how it rotates and how it fits in the laboratory

1:1 size with real model. No scaling is needed.

2. Designing 3D Experience

Designing 3D experience is way harder than 2D experience such as mobile apps or website. Imagination played a big role in this process. Whenever I thought something at one point, I tried to draw it, in 2D but with perspective. It was not an easy process. 

I watched a lot of behind the scenes in interactive media such as movie to try to learn how they can bring imagination to life. I took their workflow and tried to apply it in my case. It works wonder. 


A. Flowchart

Visualizing one state to another

B. Table of Elements

Information ‚module‘ that can be displayed in one scene

C. Scene

Visualizing information that is displayed in one state or more

D. Storyboard

A compilation of all scenes


Flowchart is a helpful tool to visualize any state that can occur in the app and its relation between other states. 

Table of Elements

This table of elements is created by and helped me a lot to determine what kind of interface or elements that I wanted to put in the app. 


Just like in a movie, a scene describes one section of a story (or in this case a state). It consists description on the left side, in the middle is sketches of the visual, and on the right side is the elements that are going to be present.

Story Board

Story board is a collection of scenes that can be triggered in the app. It is always nice to have an overview of all things that need to be programmed in the app . 

3. Development

After finishing the concept, I started the development phase. Throughout the project I used kanban style project management using It is the easiest and fittest option for this kind of project since I was working by myself anyway. 

Since I didn’t know C# before, I spent the first few weeks figuring out how to write C# code. I also had no real experience with Unity

4. Result

I am extremely happy with the result that I have worked on for the 3-4 months. My supervisor and numerous exhibition visitors were also very excited to try Lab Planer. 

I worked closely with several students and people who work at the laboratory to ensure the app is easy for them to use in real world application

Play Video
Graphic: Personal | Wireframes: Personal | GUI: Personal | Illustration: Undraw | Photos: Unsplash | Mockups: Behance