LabPlaner

Planing Laboratory Plan In Real-Time Using Microsoft HoloLens

Client

Fraunhofer Institute for Manufacturing Engineering and Automation IPA.

Role

Programmer, Designer, Tester, Bachelor Candidate

Developed in

Stuttgart

Year

2017

Platform

Microsoft HoloLens

For the past couple of years Fraunhofer IPA has numerous ambitious projects in many sectors. One of those is the bio-engineering sector. In 2017 I was trusted to test the benefits of  using Microsoft HoloLens in laboratory instead of other hardware to simplify the process of designing a laboratory plan. I started the project with zero knowledge of C# and Unity3D: two of the most important foundations to create a mixed reality application for the platform. It’s not an easy journey, but after all the bumpy ride led me to a satisfying destination.   

Play Video
Video presentation of LabPlaner which was created at Fraunhofer IPA in Stuttgart

Mixed Reality is indeed a whole another world

From Virtual Reality to Mixed Reality

I began exploring another reality when I was an Intern at Daimler Financial Services. Two semesters after I decided to send my application as a Bachelor Candidate at Fraunhofer IPA. My application was accepted not long after. Three months after my first Mixed Reality app is successfully developed.

It was presented in front of people from the biochemistry industry, university and more during Fraunhofer IPA event nICLAS. 

 

Workflow of Mixed Reality Application Development

3 Months, 3D Modeling, 3D Animation

Like stated above I only had 3 months time to figure it all out. Or at least to try to make the app that I was going to make usable. And I did!

3 months is such a short amount of time to learn several things at one. Before the project began I already knew that I had to program the application using C#. And I had absolutely no experience of it since at the University I only learned C, C++, Java and JavaScript. 

But that’s real life of app developers. Sometimes you have to learn another language and there’s no downside of learning something new. Thus I learned C# and the impact of learning the language is real since it helped me to get through my job at Kommdirekt GmbH as UX Developer. 

Then I had to use Unity for developing the mixed reality app Interface. My knowledge was at the bare minimum at that time since I only used Unity to prototype a virtual reality application at Daimler Financial Services. And then again I tamed Unity. 

The later: 3D Modeling and 3D Animation were not a big challenge since I had learned those two at University of Applied Sciences Esslingen. 

There’s absolutely no downside of learning new things. In fact it helped me in several occasion that happened months after. 

8ninth's Design Pattern

Mixed Reality App and Its Design Pattern

Designing For Mixed Reality App

I couldn’t just put 3D models into Unity and called it a day. I had to carefully plan how the app would look like and how it should behave. From web and mobile app development I know I just had to create mockups and user-flow and then after I could leave the office and grab some protein shake. Web and mobile app share the same elements: they have button, text, input fields. But what does mixed reality have? That was the question which was haunting me during my first days (beside trying to fit HoloLens perfectly on my head)

Internet is sure full of talented and kind-hearted people. I found the amazing Design Pattern by 8ninth that helped me figuring out what the app should have and how it should behave.

 

Design Workflow

Using Existing Tools For Creating New Things

During the time I wrote my Bachelor thesis there’s no prototyping tool for Mixed Reality application and I am sure that there’s none nowadays. So I had to use any tool that I could use that time.

I began the design phase creating user stories. I then visualized the stories in a scene which includes various information like the story itself and the elements from the design pattern. Scenes that I created made a single story board. 

This approach helped me visualizing the app even before I began with programming and 3D animation. 

Designing 3D Application With 2D Tools

LabPlaner was tested by people from the industry and students from various universities.

More than half of the participants find it fun to use HoloLens

Usable App for Any User

Despite HoloLens weight and it’s inability to accurately capture user’s input testers found it fun to use HoloLens. 

There were not only one test happened during the development. At least every week the app was tested by my supervisor or other coworkers at Fraunhofer IPA. If no one tested the app during that week I would test it and switched my brain to user brain and not developer brain since developer often don’t see problems that users see. 

 

I’m Always Up For Something New, Fun and Exciting

To be honest this project was one of my highlight. I didn’t think I would be able to program such an app since I always put myself as a Designer and/or Conceptor. This shows myself as a man who always takes the risk to achieve something he’s never been achieved before. 

I’m looking forward to our collaboration if you’re interested. Drop me an E-Mail or just share this page and spread the word. 

It’s such an amazing project to close my final semester at the university