Tools Used/ BOM
- Mixed Reality (MR) on the Quest 3 to scan surfaces where objects will be placed.
- Fast Whisper for accurate and fast speech-to-text transcription.
- Flux for generating images based on user descriptions.
- SPAR3D for converting these images into 3D models.
- Custom Runtime Importer to import and instantiate the AI-generated models in the MR environment.
- Interactive components were added to allow for user interaction within the space.
Overview
CasaVista uses generative AI to render and import customizable 3D assets in real-time, enabling a virtual staging process that is both cost-effective and highly adaptable. With CasaVista, real estate agents can walk potential buyers through an interactive experience, visualizing the property as their own. Buyers can request specific furniture or design elements using voice commands, and CasaVista generates and places the requested objects in the MR environment. The AI pipeline even allows for the creation of unique, custom-tailored designs that traditional staging could never provide.
Challenges
One major challenge was developing a seamless pipeline from voice input to a fully interactive MR environment. Ensuring the models were generated quickly and accurately while maintaining visual fidelity was no small feat. Additionally, integrating multiple AI models into a cohesive workflow required overcoming compatibility and latency issues. Finally, creating a polished mixed-reality experience that felt intuitive to users involved fine-tuning both the software and the user experience.
Accomplishments
We’re incredibly proud of building a fully functional end-to-end pipeline for generative AI in mixed reality. From voice transcription to AI model generation and interactive placement, we successfully brought together complex technologies to create a seamless user experience. Participating in the Founder Track helped us refine our vision for CasaVista as a viable startup, and we’re excited about the potential impact this could have on the real estate industry.
Reflection
This project taught us how to integrate multiple AI and MR technologies into a single, cohesive platform. We gained valuable insights into the startup process, including market analysis and customer engagement strategies. Additionally, we learned how to optimize performance in mixed-reality environments and address challenges like latency and model compatibility.
We envision expanding CasaVista beyond small-scale commercial real estate to residential real estate and even broader applications like event planning and interior design. Future iterations could include enhanced user interaction through hand tracking, support for collaborative multi-user sessions, and integrations with CAD software for more advanced customization. Our ultimate goal is to revolutionize how people imagine and interact with spaces, using AI to make staging accessible, flexible, and inspiring.