Nvidia Jetson TX1

Realtime object detection is one of areas in computer vision that is still quite challenging performance-wise. When it comes to mobile/embedded application, GPUs certainly make a whole lot of difference allowing to achieve practically useful speeds. For example, SSD model described below runs at ~8.5 FPS on GPU and 0.03 FPS in CPU-only mode on TX1 board.

Single Shot MultiBox Detector (SSD) is one of the fastest currently available approaches to object detection on images. It achieves accuracy comparable to Faster-RCNN while in most cases performing faster than YOLO model. SSD is created by Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg. and is published on arXiv in December 2015.

Installation

This assumes you are installing SSD on a clean JetPack 2.3.1 (containing Ubuntu 16.04 & CUDA 8.0).

SSD is implemented on top of modified version of Caffe framework. Instructions from BVLC does not fully work for TX1 so here is the process with few workarounds:

Getting SSD sources:

cd ~ubuntu

# install git
sudo apt-get update
sudo apt-get install git

# clone SSD git repo
git clone https://github.com/weiliu89/caffe.git caffe-ssd
cd caffe-ssd

# switch to ssd branch
git checkout ssd

Squeezing some extra speed out of TX1:

cd ~ubuntu

# save current settings
sudo ./jetson_clocks.sh --store default-clocks

# load performance-optimized profile
sudo ./jetson_clocks.sh

# to get back to default power settings:
# ./jetson_clocks.sh --restore default-clocks

Installing dependencies:

sudo apt-get install \
  libprotobuf-dev libleveldb-dev libsnappy-dev \
  libhdf5-serial-dev protobuf-compiler

sudo apt-get install --no-install-recommends libboost-all-dev

# install OpenCV libraries
~ubuntu/OpenCV4Tegra/ocv.sh

Editing Makefile.config:

cd ~ubuntu/caffe-ssd
cp Makefile.conig.example Makefile.config
  • Uncomment line (5) # USE_CUDNN := 1 to be USE_CUDNN := 1
  • Edit line (90) INCLUDE_DIRS := ... to be INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/

Editing Makefile:

Edit line (181) LIBRARIES += ... to be LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial.

Compiling Caffe:

cd ~ununtu/caffe-ssd
make -j4

Now get a cup of coffee, but small, compiling Caffe on TX1 doesn’t actually take that long.

Running

Wei Liu’s repo for SSD contains links to SSD models pre-trained on PASCAL VOC 2007+2012, MSCOCO and ILSVRC2015 datasets with VGG as base network.

In order to run model in test (inference) mode mode you also need a labels map and a model definition, which you can obtain by by running Wei Liu’s examples or by downloading them and a SSD300 trained on VOC0712 here.

In latter case, after downloading ssd-tx1.tar.gz:

# extract files
tar -zxf ssd-tx1.tar.gz
cd ssd-tx1

# run in test mode
~ubuntu/caffe-ssd/build/tools/caffe test \
  --model="test.prototxt" \
  --weights="VGG_VOC0712_SSD_300x300_iter_60000.caffemodel" \
  --iterations="536870911" \
  --gpu 0

You also may want to configure which video device to use by editing test.prototxt. By default device /dev/video1 is used which maps to external USB camera, but only if it was plugged after the boot-up:

video_data_param {
  video_type: WEBCAM
  device_id: 1 <--
}

Don’t forget to enable turbo mode (sudo ~ubuntu/jetson_clocks.sh).

Enjoy!