ThatXliner/NerdLens

Mirror your laptop in VR. Contribute to ThatXliner/NerdLens development by creating an account on GitHub.

1 stars · 0 forks · ShaderLab Topics: ios, unity, vr, cardboard, google-cardboard, google-cardboard-vr, cardboard-vr

README

NerdLens

Put your computer screen in a headset you already own. A wired, open-source VR desktop experiment for iPhone and Google Cardboard.

NerdLens captures your desktop, compresses it into JPEG frames, sends those frames over USB, and renders them on a virtual screen inside a stereoscopic Cardboard view. No expensive headset. No cloud streaming. Just a phone, a cardboard viewer, and an arguably unreasonable amount of curiosity. ImportantNerdLens is an early proof of concept, not a polished consumer app. The current mobile project targets iOS, requires a source build, and is best suited to hacking and experimentation.

What it can do

Stream a selected display or window into VR at a configurable target of 10–60 FPS Keep traffic on a wired USB tunnel through iproxy Show a live preview, connection state, frame count, resolution, and target FPS Move the virtual display closer or farther away without leaving the headset Recover from interrupted frame writes with automatic reconnect attempts Keep the iPhone awake during a viewing session

How it works

flowchart LR A["Desktop or window"] --> B["Svelte capture UI"] B -->|"JPEG frames"| C["Tauri / Rust host"] C -->|"TCP :12345"| D["iproxy USB tunnel"] D --> E["Unity iOS client"] E --> F["Texture on a VR display"] F --> G["Google Cardboard"] B -->|"distance setting"| C

Loading

The phone runs the TCP server; the desktop host connects to localhost:12345, which iproxy forwards to the attached iPhone. Each message starts with a one-byte type followed by a big-endian payload length and the payload itself.

Type Payload Purpose

0 none Keep-alive, reserved by the client

1 JPEG bytes Replace the texture shown in VR

2 UTF-8 JSON Update viewer settings such as display distance

Quick start You will need

macOS and Xcode An iPhone running iOS 13 or newer A Google Cardboard-compatible viewer A USB cable Unity Hub with Unity 6000.1.5f1 and iOS Build Support Bun, Rust, and the Tauri system prerequisites iproxy from libimobiledevice

Install iproxy with Homebrew: brew install libimobiledevice Then clone the project: git clone https://github.com/ThatXliner/NerdLens.git cd NerdLens 1. Build the iPhone client

Open the Test0 directory in Unity Hub using Unity 6000.1.5f1. Open Assets/Scenes/SampleScene.unity. Switch the build target to iOS and build the Xcode project. Open the generated project in Xcode, select your development team, connect your iPhone, and run the app on the device. Place the phone in your Cardboard viewer after the app launches.

The Google Cardboard XR package is already declared in the Unity project and will be fetched by Unity Package Manager. 2. Open the USB tunnel Connect and trust the iPhone, leave the NerdLens mobile app running, then start the tunnel in a separate terminal: iproxy 12345 12345 Keep this process running for the entire session. 3. Start the desktop host cd desktop bun install --frozen-lockfile bun run tauri dev In NerdLens:

Select Connect. Select Start Recording and choose a display or window. Put on the headset. Use the Distance slider to place the screen comfortably.

That is it. Your monitor is now wearing a monitor. Development Run these commands from desktop/:

Start the Svelte frontend only

Type-check the frontend

Build the frontend

Run the complete desktop app

Validate the Rust host

Troubleshooting The host says it cannot connect

Launch the iPhone app before selecting Connect. Confirm the phone is unlocked, trusted, and visible over USB. Make sure iproxy 12345 12345 is still running. Check that nothing else is using local port 12345.

The iOS build will not sign

Open the generated Xcode project and choose your Apple development team. Give the app a unique bundle identifier if Xcode reports a conflict. Confirm that the connected phone is enabled for development.

The stream freezes or stops

Stop and restart screen capture in the desktop host. Restart iproxy, then disconnect and reconnect from NerdLens. Try a lower target FPS; full-resolution JPEG encoding is intentionally simple in this prototype.

The view is uncomfortable

Adjust Distance before putting the phone back into the viewer. Take regular breaks. This prototype does not yet include a complete comfort-calibration flow.

Roadmap

Smoother, lower-latency video transport Better connection and error feedback on both ends Automatic device discovery and iproxy management Display placement controls and axis locking A more polished VR environment Packaged desktop and mobile releases

The wonderfully candid development story—including the failed first Unity project—is in the project journal. Contributing Bug reports, experiments, protocol improvements, and delightfully overengineered ideas are welcome. Open an issue or send a pull request with a clear explanation of what changed and how you tested it. License NerdLens is free software released under the GNU General Public License v3.0.

source