Initially, I wanted to explore gestures as a fun experiment. However, I realized that if one day AR/VR technology becomes ubiquitous, quick, intuitive gestures linked to everyday tools could dramatically increase user convenience. This idea inspired my project development. In this project, users can use three convenient tools by making specific gestures with their hands that represent the numbers 1, 2, and 3. By making these gestures, users can activate the corresponding tools to make daily life more efficient and enjoyable.

The flow of this project is presented below.

I brainstormed a range of everyday tools that work with different gestures. The key is to make sure that users can easily connect specific gestures to the tools they want. Initially, I considered gestures such as the binocular gesture to activate the zoom-in function and the frame gesture to summon the camera. However, these options didn’t align with my original intention of providing productive tools, and some gestures were difficult to detect reliably. Eventually, I opted for simplicity, creating quick links similar to those on the desktop and mobile interfaces, and using gestures to indicate one, two, and three. This design ensures that users can access the tools they need intuitively and efficiently.

The project's user interface is kept simple and intuitive, reflecting the fact that it is an everyday, convenient toolset. This simplicity ensures that each tool can be used quickly and easily without distraction.

The project was built using Unity’s UI Canvas feature to design the interface, along with C# scripts to enable core functionalities like the timer, to-do list, and scene transitions.

After exploring the Meta Interaction SDK’s sample scenes, I found out there are a lot of cool components and interesting interactions setup that I can take advantage of. I used its movable panel for the main UI panel and the album component from the sample scene. The SDK’s micro-interactions add a polished feel, making the experience more cohesive for users. I also integrated Meta’s building blocks to implement a pass-through feature. Overall, working with this SDK was exciting and underscored how rapidly the AR/VR field is advancing.

For hand tracking, the most important part of this project, I choose to use Unity XR Hands instead of Meta Interaction SDK’s hand poses plugin. Unity XR Hands offers more flexibility and a streamlined setup for gesture detection. I created three gestures—represented by one, two, and three poses—that are easily recognizable and reliable for detecting simple commands. Although more complex gestures can be challenging, Unity XR Hands is still highly effective and well-suited to many scenarios, and I’m eager to see its future improvements, such as supporting dual-hand gestures.

I plan to develop Handy Tools into a fully functional AR quick-link system similar to mobile shortcuts. Like the iPhone’s Shortcuts app, I want this project to make daily tasks more convenient by allowing users to assign specific hand gestures to their favorite tools or even applications on their devices.