AR Computer vision demo

In : 2018 ~ 2021

For : Verizon

I Was : 3D Generalist

I Did: Modeling, Texturing, Lighting, Rigging, Animation

I Used: Maya, Substance Painter, Zbrush, Photoshop, Mixamo

Rendering Engine: EvrEngine (Proprietary Verizon Game Engine), Unreal Engine

Additional Artist Credit: Peter Casey (3D Scan, Rigging, Animation), Jeff Smith (Rigging, Animation), Tim Samuels (Concept Art, Modeling, Texturing)

I worked on various types of AR PoCs and activation projects at Verizon. From working with a renowned publishing company to working closely with the Computer Vision engineering team. Below you can read about some of the AR work I did as a 3D artist.

Project Castle/Liana Demo (Computer Vision Demo)

We collaborated with the Design and Engineering team to develop creative concepts to showcase our latest Computer Vision technology. We chose a game-like experience to engage the audience better and highlight how much our CV technology can recognize the most complex objects. The 3D assets were also streamed to devices on runtime through proprietary 3D streaming technology called EvrVault, so users didn’t have to store any heavy 3D files locally.

We went through two phases for this demo. The first phase was our exploration and troubleshooting phase, where we designed smaller scope to make sure all the technologies and art assets would run properly. Once we had a successful result from the first phase, we went for a bigger scope with proper game elements; however, we couldn’t complete the second phase due to the COVID lockdown and the inability to access the physical AR targets we had in the office at the time.


Phase 1. Castle Demo

Highlight Reel

This was our first test of the experience. We had to ensure our CV technology could detect the miniature castle model as an AR target and consistently catch it as the end device moved around. 

For this project, I worked with a 3D scan artist and cleaned up and aligned the scanned data to the real-life castle model. We needed this scanned data as an occlusion mesh, so the 3D characters could be appropriately occluded behind the models as the camera moves around.

I also worked on fighting animation sequences for the two soldiers' characters along with another animation artist.   

Demo Closeup Video

Phase 2. Project Liana

Based on our research, we moved on to creating a bigger-scale AR experience. The scan/clean-up method is based on the test version we did above with Castle demo. Although we couldn’t finish the project, below you can check out some of the 3D scan cleanup/optimization/modeling work I did for the project.

  • 3D Scan Optimization/Clean up

We had a physical miniature map that we scanned for two purposes. First, we needed them to create occlusion mesh so that when we place AR 3D characters on top of miniature maps, the CV technology can correctly recognize the physical model and set the occlusion mesh accordingly, so the character doesn’t float on top of the models.

Secondly, we needed the texture data for the real-time rendering of AR 3D models.

We scanned a highly detailed model using two different types of Artec Scanners. One to scan the entire models and the other to scan the texture maps. Once the scan artist handed me the raw models, I worked on cleaning up the models, fixing any holes or artifacts, and projecting/cleaning the scanned textures. The model did not require high-frequency details, so I focused on ensuring the overall silhouette matched with real-life models.

The engineering team told me to keep the occlusion mesh under 100k verts per scenary, which was another factor I focused on achieving.

Sports Illustrated Swimsuit Edition AR

Sports Illustrated Swim 2018 is an AR experience that let you see the cover model in AR space. 

I was responsible for texturing two 3D scanned models. I was given model with raw projection textures on top to use as a guide. The challenge was to use photographs that were not optimized for 3D texture usage such as blurriness, unevenness, and shadows. So I mixed the method of doing photo projection as well as hand-painting on top of projected texture to achieve even skin throughout. 

In-App Demo Video

Rendering