A modern real-time computer vision experience that blends hand tracking, four-dimensional geometry, procedural rendering, and cinematic visual effects. Using only two hands and a webcam, users can summon, manipulate, and collapse a dynamic hyperspace object that appears to exist within the real world.
The project combines Apple's Vision framework, Metal, Core Image, and SwiftUI to create an interactive "dimensional drift" experience inspired by science fiction interfaces and cinematic visual effects.
/assets/screenshots/app.png
/assets/screenshots/bigbang.png
- Real-time hand tracking using Apple's Vision framework
- Dynamic palm detection and gesture analysis
- Four-dimensional rotating tesseract renderer
- Crystal energy core with procedural glow
- Animated galaxy halo surrounding the object
- Orbiting dimensional fragments
- Dynamic starfield that fades into the environment
- Space transformation effect during dimensional drift
- Camera distortion using Metal and Core Image
- Interactive scaling and rotation based on hand movement
- Big Bang event triggered through a clap gesture
- Real-time rendering at interactive frame rates
- Modular SwiftUI architecture
The application uses Apple's Vision framework to detect both hands in real time. Palm positions and landmarks are continuously extracted from the camera stream.
The tracked landmarks are converted into palm centers and gesture states. The midpoint between both hands determines the position of the dimensional object.
The distance between the palms controls the object's scale, while the relative orientation of the hands influences its rotation.
Instead of rendering a traditional cube, the engine projects a tesseract (4D hypercube) into three-dimensional space using six independent rotation planes.
The projected vertices are then rendered as layered wireframes to create the illusion of higher-dimensional movement.
Several procedural rendering systems enhance the illusion:
- Crystal energy core
- Galaxy halo
- Orbiting fragments
- Dynamic starfield
- Environmental darkening
- Camera lens distortion
- Big Bang collapse effect
GestureFX
│
├── CameraCoordinateMapper.swift
├── CosmicCameraController.swift
├── CosmicMetalView.swift
├── ContentView.swift
│
├── EffectsEngine.swift
├── DimensionMath.swift
├── HandGeometry.swift
├── HandSkeletonOverlay.swift
│
├── CosmicDimensionView.swift
├── TesseractRenderer.swift
├── CrystalCore.swift
├── GalaxyHalo.swift
├── OrbitingFragments.swift
├── StarfieldBackground.swift
│
├── BigBangEffect.swift
├── SoundEngine.swift
│
└── Assets
- Swift
- SwiftUI
- Vision Framework
- AVFoundation
- MetalKit
- Core Image
- Core Graphics
- Combine
- macOS 15+
- Xcode 16+
- Apple Silicon Mac (recommended)
- Webcam
Clone the repository.
git clone https://github.com/yourusername/Cosmic-Reality-Engine.gitOpen the project.
cd Cosmic-Reality-Engine
open ComicEngine.xcodeprojRun the application directly from Xcode.
| Action | Result |
|---|---|
| Show both palms | Summon dimensional object |
| Move hands | Move object |
| Move hands apart | Increase scale |
| Move hands together | Decrease scale |
| Rotate hands | Rotate tesseract |
| Clap | Trigger Big Bang |
- Hand occlusion using segmentation
- Volumetric lighting
- Reality warp shaders
- Portal opening animation
- GPU particle simulation
- Multi-object interactions
- ARKit support
- Spatial audio
- Full six-axis gesture recognition
- RealityKit integration
This project draws inspiration from cinematic depictions of higher-dimensional geometry, science fiction interfaces, procedural rendering, and interactive visual effects.
This project is licensed under the MIT License.
Anushka Prasad
README.md
assets/
screenshots/
app.png
bigbang.png