5 Taboo Things that HoloLens 2 Developers Don’t Want to Talk About
Not knowing these could cost you big time when turning your vision into a reality
Introduction: In this article, I will reveal to you 5 taboo things that HoloLens 2 developers don’t want to talk about. If you are looking to create a bright future with HoloLens 2, this is a must-read. If you just want to learn more about HoloLens 2, feel free as well. Ready or not, here they come:
1. HoloLens 2 graphics quality is visibly worse than HoloLens 1
I have been developing apps for the original HoloLens (aka HoloLens 1) for over 4 years. For all the criticism about its limited field of view, I understand the limitation and already got used to it. However, I never had a complaint about the graphics quality inside the letter box I see. It is simply stunning to see 3D content through HoloLens 1, unless your developer really didn’t know how to design for and optimize performance on HoloLens.
Unfortunately, I cannot say the same thing about HoloLens 2 visual display.
I have tried engineering versions of HoloLens 2 back in May 2019. I thought the visual quality of HoloLens 2 engineering versions was poor. The 3D objects and texts looked pixelated, and the color on white background appeared non-uniform and rainbow-ish. When I put back on my HoloLens 1, I fell in love with my HoloLens 1 again.
I remained hopeful and purchased a HoloLens 2 when it became generally available in the Microsoft store in July 2020. The shipment came and I was so thrilled to record an unboxing video, but then the truth hit me. I was so disappointed that I couldn’t even bring myself to make a video review after a full month of having my HoloLens 2. Here is what I saw and struggled with in the first 2 weeks after many eye calibrations:
a. The color on white objects do not look white on HoloLens 2. They would appear a dull pink or tinted green, especially in the lower corners of my field of view. There were also color distortions on HoloLens 1, but never did I imagine it could be this extreme on HoloLens 2, which is a newer, more expensive successor product.
b. Images and texts appear pixelated on HoloLens 2. Some people call these “screen door effects” as if you are looking out of your window with the mosquito net screen touching your nose. This effect was non-existent on HoloLens 1.
c. There were clear visible jagged lines near the bottom of my display. These lines make already pixelated images and texts even harder to see. I am not sure if these are manufacturing defects, but since Microsoft Store had a no-refund policy on HoloLens 2, I didn’t want to risk a physical return and lost development time.
Because of these deficiencies, I barely noticed the improvement in the size of field of view of HoloLens 2 when compared to HoloLens 1. After a month, I believe my brain and eyes have adapted to subconsciously ignore the non-appealing outer rim of the field of view. This effectively shrinks the usable field of view to almost the same as HoloLens 1.
What does this mean to you, Mr/s. Entrepreneur, if you are thinking of a HoloLens 2 project idea?
1. Do not overhype the experience. Instead, focus on the functionality and the problem it will solve
2. Use dark themes or less white colors in your designs
3. Use more animations / transition effects and less static content, so as to mask some of the color distortions
___
2. HoloLens 2 hand tracking is an improvement but still too slow
With HoloLens 1, the interaction model was rather simple: you move your neck to target the cursor in 3d space and you used 1 gesture “Air Tap” to click on objects. This was as close to using a mouse as it gets. Of course, it came with a lot of shortcomings too. Hand fatigue can happen when playing RoboRaid.
HoloLens 2 added hand tracking capabilities to allow your hands to do more. You are no longer limited to 1 “Air Tap” gesture, but instead have your 10 fingers at your disposal for many tasks. This all sounded fantastic in theory. However, in practice, I still have to rely on air taps more than 70% of my time. The reason is that unless a holographic object is very close to you, it takes too much effort to move into the right distance range for a near “touch” interaction. Moreover, I often find myself trying to press the same button twice, because it was out of reach or because the hand tracking was too slow to recognize it the first time.
Based on my experience with HoloLens 2 and Oculus Quest, which costs only roughly 1/10th of the price, I find the hand tracking accuracy, speed and smoothness to be on par. WHAT? Let me say that again, you could have had the same hand tracking feature for 1/10th of the price!
What does this mean to you, Mr/s. Entrepreneur, if you are thinking of a HoloLens 2 project idea?
1. Do not overhype the experience. Instead, focus on the functionality and the problem it will solve
2. Add visual / audio affordance to all UI elements to make it easier to use hand tracking properly. Yes, this will cost more to develop but it will help you win customers, generate more revenue, and save you tons of headache from customer support.
___
3. HoloLens 2 eye tracking is too jumpy for accurate input
What about eye tracking? Isn’t that the greatest technological advantage HoloLens 2 harbors?
Yes and NO. I must say that the Eye Tracking feature is a hidden gem of HoloLens 2. It is a truly innovative feature at play here. It tracks your eyes and allows the machine to guess user intent faster than before.
However, it is not like what you think yet.
HoloLens 2 uses 2 cameras to track your eyes, one for each eye. It measures your eye movements and calculates the direction your eyes are looking. However, because it does this measurement at millimeter scale very close to your eyes and also because your eyes move in jumpy motions thanks to the way eye muscles work, any slight measurement error can result in a large variance in the holograms you see when projected 2 to 3 meters away.
I am impressed by how accurate it is for a first-generation feature, but during my many tests, my eye-tracking based cursor rarely converges on the actual spot I was looking at on the first try. If a hologram is less than 1 feet in size and more than 3 meters away, it will probably be easier to use your hand tracking-based pointer instead.
Thus, before you think you could do something crazy like typing or drawing with your eyes only, please read the eye tracking specs and reference the official guide documentation on eye tracking.
What does this mean to you, Mr/s. Entrepreneur, if you are thinking of a HoloLens 2 project idea?
1. Do not overhype the experience. Instead, focus on the functionality and the problem it will solve
2. Use eye tracking for an unexpected pleasant surprise for the user, like an icing on the cake, but do not make it part of a mission critical feature.
3. Eye tracking is well suited for guessing user intent. For content where users would spend a bit of time with, we could implement a dwell function to confirm where their eyes were looking and provide that magical surprise effect.
___
4. HoloLens 2 development comes with its own quirks (aka bugs)
When the original HoloLens came out, it came with a lot of support. A brand-new Holographic Academy with lots of tutorials and example projects were launched. Each tutorial project worked almost perfectly. I still remember having fun with some of them. However, it seems that HoloLens 2 launch didn’t have so much attention as far as support goes. There were some new tutorials, but they weren’t at the same scope as the original ones, and thus weren’t nearly as fun.
On the other hand, the complexity of the code base has risen significantly since HoloLens 1 and HoloToolkit. The new MixedRealityToolkit is a great new tool and brings in a lot of great new features. It uses a service provider / locator pattern for core functionality and uses a profile / theme-based concepts for configuration and design. However, the most commonly used Unity3D editor software relies very much on the concept of prefabs and MonoBehaviour scripts, which are easy to learn and well documented. Unfortunately, the new MRTK patterns break that Unity design pattern. It’s like an unfamiliar onion with many layers. Moreover, with each new version of MRTK, things change, making it difficult to keep up. If one of those layers break, the whole thing breaks. That is the risk of having a complex system. For example, the ways buttons are set up in MRTK v2.3 and v2.4 are very different, making version upgrade a major task.
Other issues exist with the change in underlying computing processor platforms. HoloLens 2 uses ARM platform whereas HoloLens 1 uses x86 platform. In Unity 2018, you could set build target as ARM platform for HoloLens 2. In Unity 2019, there is a bug that prevents developers from building against ARM target. Instead, the official alternative requires setting ARM64 as build target, but that comes with its own limitations. For example, you could get this error: “BV4 MIDPHASE ONLY SUPPORTED ON INTEL PLATFORMS.” Another example is the holographic remoting connection issues on Unity 2019 making quick iterations very difficult. Long story short, I experienced enough pains to know these issues exist, and as an entrepreneur, you should be aware of them too.
What does this mean to you, Mr/s. Entrepreneur, if you are thinking of a HoloLens 2 project idea?
1. Have patience. HoloLens 2 technology is new and its software is still evolving, so expect hiccups and budget extra time for project completion.
2. Hire someone who is experienced with all these quirks and details and who isn’t afraid to tackle new challenges and obstacles.
___
5. There are very few apps specifically built for HoloLens 2 in Microsoft Store
Before you buy a HoloLens 2, please beware there is currently a very limited number of useful apps on the Microsoft Store that were specifically built for HoloLens 2. It won’t replace your smartphone just yet.
As a consumer, you could be quite disappointed with the selection. This should be no surprise, given that Microsoft’s re-focus on enterprise applications, most of which would remain private. This could also be due in part to the delayed delivery of HoloLens 2 devices to smaller / independent developers. Lastly, Microsoft also took a stab at the app market itself, publishing apps such as Guides and Layout, effectively taking some sections of app market away from alternative providers.
For me, the biggest loss was this amazing crime-investigation game called Fragments which was launched with the original HoloLens for free. You cannot even install Fragments onto HoloLens 2. I wish the original project repo could be made public for community developers to port this top-notch experience to HoloLens 2.
What does this mean to you, Mr/s. Entrepreneur, if you are thinking of a HoloLens 2 project idea?
1. The app market place for HoloLens 2 is still quite young and your killer app idea may have clear sailing.
2. The current overall market size may be too small for pure consumer uses. Your grand idea should focus on solving a real pain for a real customer.
___
Why do I develop for HoloLens 2?
So, I shared a lot of negative facts and opinions about HoloLens 2, you must be wondering, why? Why do I develop for HoloLens 2?
I see the future. I see HoloLens 2 as a great investment into that future.
HoloLens 2 may not be the greatest thing since the slice of bread, but it is in a comfortable form that allows for more extended use as compared to HoloLens 1. While its visual display and input mechanisms have shortcomings, these are the closest things we have to science fiction.
The spatialized computing experience is already a major breakthrough in our lifetime. I have no doubt in my mind that one day the physical form factor and price of an XR device will become so compelling that everyone would ditch their smartphones for it. There will be new tech companies large and small serving billions of customers using this new technology and benefiting the entire human civilization.
The question for you, Mr/s. Entrepreneur, is: Do you want a piece of that action?
