This project proposes designs for an Augmented Reality (AR) application that allows designers to map out the space for their design with accurate dimensions by creating AR shapes for easier spatial visualization of their design as well as make non-destructive modifications to physical artifacts using AR sketching.
This was an individual Capstone project for my Masters Degree in UX Design where I was involve with the whole product development process starting with research and concept design and ending with delivering an interactive prototype running on an iPhone using AR.
15 weeks
Designing a physical product requires designers to use 3D modeling software that not only takes them away from the environment that they're designing for. This is where AR sketching comes in since it has the potential to allow designers to ideate and create 3D sketches of their ideas within the real world.
Although smartphones have the benefit of being accessible and easy to carry, users had trouble visualizing the sketch in 3D, trouble understanding the depth of the 3D sketch, and dizziness while using the app.
Through user research with interviews and usability testing, I discovered how designers work with designing 3D physical forms and understood the process and pain points that they had.
Initially, I intended to help the user ideate by sketching out their ideas using AR Sketching and to help users visualize their sketches in 3D through a phone screen. Therefore, I created a feature where users can place reference planes to use as virtual whiteboards to sketch on. The reference planes do help users visualize their sketches in 3D, however, there was still a problem because it was not suitable for ideation due to the increased cognitive load when ideating.
Therefore, I went back to the drawing board and came up with another idea, which was instead of sketching, users can quickly create 3D primitive shapes with accurate dimensions to "block out" spaces to help visualize their 3D designs in the real-world environment.
The outcome of this project was a fully functional AR application that allows the user to create 3D primitive shapes such as blocks and cylinders with accurate dimensions that can be placed in the real world.
Although the AR Sketching feature was not suitable for ideation, it was able to help users perform non-destructive modifications to physical artifacts by sketching on top of them.
From this project, I learned a lot about designing for AR on smartphones, designing for the physical context of the user, and how to effectively integrate interactive prototyping into my design process.
To understand more about the user interactions with AR on a smartphone, I conducted interviews with designers and usability tests on the an existing AR sketching application called Just-A-Line.
From the interviews with five 3D designers and artists, I found that sketching is always present in the initial stage of the design process, participants also used 3D modeling tools to help them get a better sense of spatial visualization and perception, and it’s important for designers to get an accurate physical representation of the space for their design within the 3D program.
I also conducted usability tests of an existing AR sketching application called Just-A-Line where I asked 2 participants to draw objects of different sizes in AR on their phones. From these tests, I found that users had trouble visualizing the sketch in 3D, trouble understanding depth of the 3D sketch, and dizziness while using the app.
To help illustrate the problem, here’s what the original process looks like for Bob, a sculpture artist that works with 3D forms.
The first problem for Bob is that CAD takes him out of the space that he’s designing for. AR allows Bob to come back and design in the space by putting sketches within the space that they’re designing for. However, the problem with that is that people don’t really know how to visualize 3D sketches through a phone.
To address the issue of visualizing 3D sketches through a phone, I designed a way to help users understand the 3D world and sketch in it better through a phone using... reference planes! These reference planes are placed in the real world with AR to help provide anchor points for drawing, similar to drawing on whiteboards floating in the air.
To make a 3D dimensional shape using sketches, the user can sketch different views of the product, then combine them together to create a wireframe of the 3D model.
From evaluating this design through concept and usability testing I found that, although the reference planes do helped users with understanding the 3D space for sketching, it is not suitable for early ideation due to the increased cognitive load and processing time for the users. This meant that users need to take a significant time and effort to think about how they want to draw things, which is not ideal for ideation since the friction in representing the idea prevents users from churning out ideas during the ideation phase of the design process.Perhaps Another Use for Sketching
Although AR sketching, even with the reference planes can't be used for ideation, users reported that they believe it can be useful in another part of their design process. With AR sketching, users can make non-destructive modifications to their physical designs. For example, drawing on top of an existing piece of furniture or sculpture to see how it might look if parts are added to it.
Since sketching wireframes on reference planes wasn't suitable for ideation due to the increased cognitive load, I still haven’t solve the problem of CAD/3D modeling taking Bob out of the space he’s designing for and can’t visualize the space easily.
When Bob 3D models something, what he’s trying to get is an idea of how the object fits in space. So I was wondering would a series of primitive shape be enough to give Bob the essence of how his design would fit in the space?
This is where I came up with the Shape Creator, which turned into my final design.
The core concept for the Shape Creator is to allow users to quickly create shapes with accurate dimensions and place them in AR. Users can place points and draw out dimensions which will be used to create the 3D primitive shapes. This can be used to help users visualize the space for their design with accurate dimensions.
From the usability evaluation, the Shape Creator was successful in helping users create shapes quickly and accurately. There was also positive feedback in helping participants get a feel of how things look in the space.
However, in each of the usability tests, participants needed a quick demonstration of how things worked before being able to get started, which signals a need for an onboarding experience. There was also a limitation for users when trying to get a precise dimension for their shape which could be fixed by grid snapping. There is still also a limitation to the available shapes that the users can placed.
So Bob can now create simple 3D models and block out his ideas right in the real world, without having to use a computer.
So with these designs, here’s how they fit into Bob’s design process.
When introduced to a new space, Bob can quickly use the Shape Creator to map out the dimensions and block out spaces for his ideas with dimensional accuracy. Then during the experimentation and iteration phase for his sculpture, he can use the AR Sketching to preview new features to add to the sculpture before physically modifying it.
