An Augmented Future

Augmented reality (AR) is a field of software engineering that continues to develop slowly but surely.

I wrote about nine critical companies in the AR space over three years ago, and while things have progressed since, there is still a lot more to come.

I previously named these companies as the AR9: AMD, Apple, Facebook, Google, Himax, Intel, Microsoft, Nvidia, and Snapchat. These nine continue to be well-positioned for AR, but instead of jumping into details about each, let's take a high-level picture of the AR landscape.

Augmented Reality: A Software Frontier

Augmented reality is simply the concept of adding software-based overlays and objects on top of the real world through camera-based hardware. AR is currently primarily available through smartphones and their ever-improving camera systems, but this is already moving towards eyeglass style headwear. Snapchat spectacles and Google glass have been early experiments in AR headwear, and rumors continue to swirl around Apple and an expected Apple Glass product. 

Strategically, Apple has made it very clear that AR is coming and has done so over the past couple of years with updates to iOS' ARKit. With continuous updates to ARKit, Apple is building its augmented reality expertise. The recent addition of LiDAR sensors to iPhones and iPads is another clear indicator that AR is top of Apple's mind. These sensors don't have many practical uses for these devices today, but their integration and data provide a basis from which to put similar sensors in other hardware solutions. 

While Apple is priming itself to be a significant player in the AR integrated device space, Snapchat is positioning itself to play a role in the AR world's advertisement infrastructure. Through their filters and other geo-based features, Snapchat's advertisement business will be well-positioned to sell virtual space in augmented reality once people are spending time there. 

At the chip level, both AMD and Nvidia are in the process of significant acquisitions to further improve their positioning in the ever-growing chip space. AMD's plans to acquire Xilinx and Nvidia's plans to do the same with ARM don't immediately look like plays towards augmented reality, though. That said, chips needed for augmented reality will not be so different than other chip requirements in other software fields, which means that consolidation in the chip space that helps improve the businesses will have positive effects on AR. 

Another component of augmented reality will be 5G, and its promise of high-bandwidth, low-latency cellular network availability. Augmented reality will require significant bandwidth and data transfer capabilities for hardware systems to show things in realtime on top of the real world. The current cellular networks don't have the reliability or throughput to accomplish seamless AR experiences, but 5G should. With the expected throughput of 5G when fully rolled out, it would be possible to perform many AR scenarios that would be painfully slow today. 

While augmented reality continues to make slow but steady progress, it seems like an inevitable next big focus area in the software landscape. Who knows, perhaps we won't be holding iPhones around much longer, and we'll instead have all that functionality in our headwear!  

Tricky Details

Company positioning aside, what makes AR so technically interesting? High fidelity graphics that properly overlay with the world introduce several puzzling software problems. One central concept in AR that has been just recently fully introduced to ARKit is occlusion: the ability for software objects to appear behind or covered by real-world objects. In use cases such as furniture positioning, it is critical that an AR object is properly occluded and appears behind real objects should they be positioned that way. 

A second major technical hurdle for AR is that the ideal user interaction pattern is some form of headwear to allow for augmentation in an individual's entire field of vision. Smartphone AR is a compromise in comparison, where augmentation is only visible through the phone's small screen. Building a headset that is both computationally powerful enough to perform intense graphical and network operations (should the overlays need information from the Internet, for example) and light and sleek enough for many people to wear, while also being affordable, is a tall order. 

Conclusion

With so many large companies investing in augmented reality technology, it seems inevitable that we will see more in this space over the next couple of years. While augmented reality and its new software interaction pattern may or may not go mainstream, several big corporations making big bets would indicate upcoming progress in this space.