Jetson Nano with Vision Components Cameras

In partnership with the Seattle Aquarium, I’m mentoring a UW ECE Senior Capstone project this winter to develop a vision-based navigation sensor for the BlueROV. The requirements are still in development, but I wanted the students to be able to quickly get started with stereo cameras connected to a Jetson Nano.

Based on experience with the Trisect I knew this was non-trivial unless there was great support from the camera vendor, so I decided to do some pre-work to build a path to a minimum working example – hopefully saving the team from sinking a lot of time on a dead-end solution.

I started with the requirement to use Jetpack 6.0+ (a newer version than currently used on Trisect), and initially focused on the Raspberry Pi Global Shutter (GS) camera as a (very) low-cost, high-quality global shutter camera. After trying a few options I determined there wasn’t a stable, usable driver for the IMX296 sensor used in the GS (though we’re still considering a backport of the Pi driver).

By searching for modules with the same Sony IMX296 sensor, I discovered Vision Components camera modules. Though not a supplier I had considered (or heard of!), their parts are in-stock at Mouser and while more expensive than the RasPi part, they are still fairly inexpensive.

Even better, VC publishes a driver for Jetpack. The VC cameras contain a “lightweight” hardware abstraction which provides a single (relatively simple!) interface for a range of Sony sensors, but it still requires an offboard ISP (unlike the AVT modules which both provide a hardware abstraction and a powerful ISP), like the “libargus” system provided by Jetson.

Jetpack 6 transitioned to out-of-tree (OOT) module builds, which was a new development environment for me. Given my positive experiences with the Allied Vision Jetpack support on Trisect, I started with the AVT Jetpack 6 driver. It’s not quite as refined as their later Jetpack 5 driver, but gave me a pencil sketch for how to work with OOT modules.

I ended up forking the AVT driver into vc-jetson-driver-release, which removes the AVT driver and adds a new module built from the Vision Components patches and overlay files.

After a bit of pain, I got to a working configuration. Neither the build nor the installation process are automated at this point, but it is functional, which is good enough.

With the cameras working, I wrote a few helper scripts for streaming data with GStreamer and V4L, including a basic proof of concept for externally triggering the cameras with GPIO.

Categories: bluerov jetson nano nano_super rov_odometry