Skip to content

Add visionOS support #2

Description

@owenselles

Summary

Port CloudNow to visionOS so users can stream PC games via GeForce NOW in a visionOS window using a connected controller or keyboard/mouse.

What works without changes

  • All networking: SignalingClient, CloudMatchClient, GamesClient, ZoneClient, auth
  • SDPMunger (pure Foundation)
  • GCExtendedGamepad / GCMouse input paths in InputSender
  • All non-streaming SwiftUI views (Home, Library, Store, Settings, Login)
  • LiveKitWebRTC xcframework — already ships xros-arm64 slices

Required changes

VideoSurfaceView.swift — Full rewrite

layerClass override (AVSampleBufferDisplayLayer as backing layer) is tvOS/iOS-only. Replace with UIView + layer.addSublayer(displayLayer) + layoutSubviews frame management. WebRTCFrameRenderer core is unchanged.

InputSender.swift — Gate Siri Remote path

Wrap GCMicroGamepad, handleMicroGamepad, RemoteInputMode, and touchpad-as-mouse in #if os(tvOS). Extended gamepad + keyboard/mouse paths already work on visionOS.

StreamView.swift — Replace Siri Remote dismissal

.onExitCommand is silently ignored on visionOS. Replace with a SwiftUI .ornament toolbar or tap gesture. Evaluate fullScreenCover → dedicated WindowGroup for the stream.

GFNStreamController.swift — AVAudioSession guards

Gate .moviePlayback / .allowBluetooth options and deprecated requestRecordPermission callback behind #if os(tvOS).

QueueAdPlayerView.swift — Optional

Switch UIViewControllerRepresentable + AVPlayerViewController to VideoPlayer(player:) (5 lines, more idiomatic).

Xcode project + SPM

  • Add visionOS deployment target
  • Fork livekit/webrtc-xcframework, add .visionOS(.v1) to Package.swift platforms
  • Add NSMicrophoneUsageDescription to visionOS Info.plist

Verification

  • Build for xrOS Simulator — zero errors
  • End-to-end GFN session: video renders, controller input registers
  • Overlay show/dismiss works without Siri Remote
  • On-device AVSampleBufferDisplayLayer frame rendering confirmed

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestin progressWork item is in progress

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions