GardenMate Ecosystem
Custom Vision Transformer AI + Viro AR mobile platform with 12+ modules.
Personalized onboarding and secure multi-role authentication
System Overview & Engineering Longevity
A full-stack mobile ecosystem engineered for React Native and Expo SDK 54, integrating spatial augmented reality, custom deep learning computer vision, and a 4-tier commerce architecture. At its algorithmic core sits a custom fine-tuned Vision Transformer (ViT-B/16) model combined with the PlantNet botanic API in a weighted 60%+40% ensemble decision pipeline. Users can test-place photorealistic 3D botanical models in physical environments using Viro React AR plane detection before purchasing via Stripe, while vendors leverage camera AI to auto-populate marketplace listings with full botanical specifications in seconds.
Key Engineering Highlights
Architected with clean domain separation, strict TypeScript contracts, and automated telemetry logging.
AI Inference Latency
FastAPI microservice response time
ViT + PlantNet Split
Weighted ensemble decision engine
AR Plane Confidence
Tabletop surface detection stability
Integrated Modules
End-to-end full-stack ecosystem
End-to-End System Data Flow
How user inputs, computer vision pipelines, and transaction protocols flow across the stack.
Camera Frame Acquisition
User or vendor captures plant leaf or flower via camera; frame is downscaled and normalized into an RGB tensor.
60/40 Dual-Inference Ensemble
Image tensor is sent in parallel to PyTorch ViT-B/16 (60% weight) and PlantNet API (40% weight).
Confidence Fusion & Auto-Fill
Softmax probabilities are aggregated; if confidence exceeds 75%, botanical specifications are automatically populated.
Spatial Plane Mapping
Viro React initializes 3D coordinate planes on horizontal ground surfaces to anchor photorealistic 3D GLTF models.
Stripe Settlement & Commission
Order is finalized via Stripe; 10% platform commission is routed to admin ledger and 90% is settled to vendor wallet.
System Capabilities & 12+ Functional Modules
Hybrid ViT + PlantNet AI Ensemble (60% / 40%)
A proprietary dual-inference pipeline combining a fine-tuned PyTorch Vision Transformer (ViT-B/16) with the PlantNet botanical API. The model runs a 60%+40% weighted confidence fusion: 60% probability weight is allocated to the local ViT classifier and 40% to the global PlantNet API, maximizing accuracy across regional and exotic flora.
AR Spatial Visualizer ("Place Before You Buy")
Utilizes Viro React spatial computing to detect real-world horizontal surfaces (tabletops, floors, patio grounds). Renders photorealistic 3D plant models with dynamic shadows, 360-degree rotation, scale lock, and surface confidence scoring (e.g. Tabletop · 85%).
Vendor Snap-to-Autolist by Image
Eliminates manual catalog entry for plant nurseries. Merchants snap or upload a plant photo; the AI ensemble identifies the species (e.g. Tomato Plant · 75% confidence) and automatically populates botanical name, care requirements, sun exposure, watering cycle, and suggested price range.
Vendor Autolist by Name
Merchants typing a plant name (e.g. "Golden Barrel Cactus") trigger an automated knowledge lookup that populates scientific taxonomy, recommended soil type, growth rate, and watering intervals without requiring manual research.
Care Engine & Smart Reminders
A centralized plant care management system that generates individualized schedules for watering, fertilizing, repotting, and pruning for each plant in the user garden.
Multi-Vendor Plant Marketplace
A comprehensive marketplace connecting regional plant nurseries with plant enthusiasts. Features categorized discovery (Indoor, Outdoor, Flowers, Fruits, Deals), search filters, and AR preview badges.
Multi-Item Cart & Stripe Checkout
High-performance shopping cart supporting multi-vendor purchases, regional delivery fee calculations, free delivery threshold progress indicators, and secure Stripe payment processing.
Order Dispute & Photo Proof Portal
A dedicated ticket-based dispute resolution system for live plant deliveries. Customers receiving damaged pots or wilted plants can file a critical ticket, upload photo evidence, and chat directly with the vendor for resolution.
Executive Admin & Commission Engine
Central administrative portal for platform governance. Monitors total collected commissions (10% platform take-rate), pending vendor disbursements, active vendor counts, total users, and open complaint tickets.
Vendor Wallet & Settlement Ledger
Dedicated merchant portal displaying real-time revenue analytics, order status progression, weekly payout settlements, and balance withdrawal requests.
Gardening Community & Consultation
Interactive social hub where plant owners share progress pictures, ask botanical questions, exchange tips, and request plant health diagnoses from experienced botanists.
4-Tier Role-Based Access Control
Strict architectural segregation separating Customer, Vendor, Delivery Rider, and Super Administrator interfaces with Firebase Auth and scoped Firestore security rules.
Key Engineering Bottlenecks & Solutions
ViT Model Latency on Mobile Devices
Running a full Vision Transformer directly on low-end mobile hardware caused thermal throttling and 3s+ latency. Engineered a hybrid architecture where lightweight tensor normalization runs on-device, while inference is dispatched to a GPU-accelerated asynchronous FastAPI microservice, dropping response times to under 350ms.
AR Plane Jitter & Drift on Indoor Carpets
Standard plane detection struggled with low-contrast textured surfaces like carpets. Implemented a spatial confidence filter that locks coordinate axes only when surface feature tracking confidence exceeds 80%, providing rock-solid 3D plant stability.
Accuracy Variance Across Exotic Plant Classes
Standalone models suffered from class imbalance on rare plant varieties. Architected the 60%+40% ensemble fusion: fine-tuned PyTorch ViT handles common regional species with high precision, while PlantNet API serves as a global taxonomy fail-safe.
Vouch Keychain
Cryptographic QR item recovery & real-time telemetry.