Overview

Bronco Ember is a NASA TechLeap Autonomous Oberservation Challenge Winner Bronco Ember is a nascent wildfire detection system that uses a combination of artificial intelligence, edge computing, infrared imaging, and a 2 degree of freedom gimbal. The system is designed to be small and compact while still being able to provide large impact to the wild fire prevention and detection industry. The system has already undergone its first test flight that provided successful results and a proving ground to continue development on the emerging technology. The system is udergoing continued development to refine and improve the system that is capable of indentify a fire that is less than 20 feet in diameter in 5 minutes or less after first ignition. To learn more about Bronco Ember you can view some of the additional resources linked in this repository or feel free to contact Zachary Gaines (zegaines@cpp.edu) with any inquiries.

Flight Data

System Design

Typical positional encoding (as used in Transformer networks and Neural Radiance Fields) maps a single point in space to a feature vector, where each element is generated by a sinusoid with an exponentially increasing frequency:

Here, we show how these feature vectors change as a function of a point moving in 1D space.

Our integrated positional encoding considers Gaussian regions of space, rather than infinitesimal points. This provides a natural way to input a "region" of space as query to a coordinate-based neural network, allowing the network to reason about sampling and aliasing. The expected value of each positional encoding component has a simple closed form:

We can see that when considering a wider region, the higher frequency features automatically shrink toward zero, providing the network with lower-frequency inputs. As the region narrows, these features converge to the original positional encoding.

Software Architecture

We use integrated positional encoding to train NeRF to generate anti-aliased renderings. Rather than casting an infinitesimal ray through each pixel, we instead cast a full 3D cone. For each queried point along a ray, we consider its associated 3D conical frustum. Two different cameras viewing the same point in space may result in vastly different conical frustums, as illustrated here in 2D:

In order to pass this information through the NeRF network, we fit a multivariate Gaussian to the conical frustum and use the integrated positional encoding described above to create the input feature vector to the network.

Progress

We train NeRF and mip-NeRF on a dataset with images at four different resolutions. Normal NeRF (left) is not capable of learning to represent the same scene at multiple levels of detail, with blurring in close-up shots and aliasing in low resolution views, while mip-NeRF (right) both preserves sharp details in close-ups and correctly renders the zoomed-out images.

We can also manipulate the integrated positional encoding by using a larger or smaller radius than the true pixel footprint, exposing the continuous level of detail learned within a single network:

Related links

Wikipedia provides an excellent introduction to spatial anti-aliasing techniques.

Mipmaps were introduced by Lance Williams in his paper "Pyramidal Parametrics" (Williams (1983)).

Amanatides (1984) first proposed the idea of replacing rays with cones in computer graphics rendering.

The closely related concept of ray differentials (Igehy (1999)) is used in most modern renderers to antialias textures and other material buffers during ray tracing.

Cone tracing has been used along with prefiltered voxel-based representations of scene geometry for speeding up indirect illumination calculations in Crassin et al. (2011).

Mip-NeRF was implemented on top of the JAXNeRF codebase.

Citation

Acknowledgements

We thank the Bronco Ember team for the dedication and work on this project.
Program is funded by the NASA TechLeap Autonomous Oberservation Challenge
The website template was borrowed from Michaël Gharbi.