open-source RGB-D compression
For seamlessly streaming sensor data
digitizing reality.
We’re creating tools to construct the AR Cloud. A universal infrastructure upon which anyone can build anything.
release #1: the sensor stream pipeline
Compress and stream sensor data from any device to a remote server at super low latency.
release #2: coming soon
collaboration is key
We can only truly digitize reality by working collectively. So let’s do just that.
On-device Computation is Limited
There’s a problem with creating a digital twin at scale. On-device computation is limited. This means that mapping the entire world is not possible within the confined computational limits of local hardware devices.
Transmitted Off-Device
Unless someone invents an insane super chip that can handle 100x the computation with no trade-off on battery life, sensor data must be transmitted off-device and onto a remote server to properly support a true digital twin.
Moetsi Sensor Stream Pipe
We built the first component of the solution. The Moetsi Sensor Stream Pipe compresses and streams sensor data to a remote server at low-latency.
20x Smaller
The sensor stream data Moetsi currently supports is RGB/D/IR. RGB = Red Green Blue, D = Depth, IR = Infrared. We are able to stream color, depth and IR data at 20 Mbps (20x smaller than original data) with minimal overhead (~30 ms overhead/latency).
Compress Raw Frame Data
The Sensor Stream Server takes in raw frame data and compresses it to reduce bandwidth.
Decompress the Data
It’s then sent to the Sensor Stream Client which is able to then decompress the data at low latency for further processing.
Completely Open Source and Platform-Agnostic
And we’ve made the whole thing completely open source and platform-agnostic. Why? Well, we believe that the best way to create a truly universal, digitized reality is by working together. We’ve already pieced together the first part of the pipe by interfacing with the new Azure Kinect, but there are a lot of sensors out there and, as much as we’d like to, we simply can’t build interfaces with them all ourselves.
More Sensor Types, Devices and Compressions
So we’ve put it out there in the hope that the dev community will take what we’ve built so far and run with it. That inquisitive minds will evolve our pipeline to work with more sensor types, devices and compressions and create some killer applications. That way we can fast-track the creation of a digitized reality and all reap the benefits.
let's digitize reality together!
Sign up to get the latest on new releases and more …