Code

The code was developed and tested in MATLAB R2017b with the following toolboxes on a PC with an NVIDIA Titan Xp GPU, 64GB of RAM and an i7 processor running Ubuntu 16.04.

Note that the GPU is needed to run FlowNet2 as the TensorFlow variant of the FlowNet2 as it is easier to use and needs cuda to run the pre-processing code.

The release version of the code differs from the development version of the code used for experiments in the paper in the following ways:

1. Uses FlowNet2 based on Tensorflow: This is easier to setup and run.
2. Uses MATLAB code: Easier to debug and not much slower than the python code.
3. Uses a simple proportional controller: A simple proportional controller provides equivalent performance to a well-tuned PID controller.
4. Uses only Foreground tracker: The difference in performance between using only a foreground and a foreground+background tracker was minimal.

You need to setup the following things to run the code:

1. PC with Ubuntu 16.04, ROS Kinetic, OpenCV 3.3.0, CUDA 8.0, Tensorflow GPU >= 1.4, Matlab >= R2017b with the following toolboxes and a WiFi module (We use this TP Link WiFi module).
2. Bebop2 with the firmware version 4.0.6.
3. Clone and setup the bebop_autonomy from here.
4. Our TensorFlow based modified FlowNet2 can be found in the Code/flownet2-tf-umd/ folder. Follow the instructions here to compile the FlowNet2.
5. In the code ComputeFlowWrapper.m change the path on the line 14 to point to the shell script. Note that due to a bug in the system command in MATLAB you'll need to use the full path. Change the following line: system('gnome-terminal -x /home/chahatdeep/WindowDetectionWithControlApril05/RunFlowNet2.sh & exit'); to system('gnome-terminal -x YOURFULLPATHTO RunFlowNet2.sh & exit');.

To run the code:

1. Connect the PC to Bebop 2 Wifi.
2. Run roscore on a terminal window.
3. Launch the bebop_autonomy ROS node by running roslaunch bebop_driver bebop_node.launch on a different terminal window.
4. Turn off camera stabilization by running the following command ./video_stabil.sh.
5. Run the MATLAB code Wrapper.m.