Controlling a Parrot MiniDrone with a Kinect sensor over WiFi

This article was written by Tanweer Ali, Senior Developer at AVG’s Innovation Labs in Amsterdam. Tanweer is leading research into Computer Vision with a focus on enhancing user privacy. He is an experienced developer with background in Scientific Computing, 3D Graphics and Numerical Methods.

IMG_3773
Drones, a word which a few years ago we heard only in a military context, now it has become a buzzword. Amazon is looking into delivery drones, DJI is selling lots of drones to hobbyists and film makers and Lily just announced the selfie drone. Like it or not, drones are here to stay.
For us, this new technology is interesting from a home security aspect. What if a drone could protect your home while you are away? How would that work?

As part of our research one of the things we looked into (and thought would be cool to share with everybody) was controlling a drone using a Microsoft Kinect.

For drones to be used indoors effectively, they must be smaller in size to better avoid obstacles. Also they should be easy to control for an average inexperienced user. This is why, for this test, we picked the Parrot MiniDrone, which seems to be one of the most popular options on the market. This is fairly cheap model, at under 100EUR, but comes with a self-stabilizing system which makes it easy to control for a complete novice and a small camera at the bottom that can take low quality pictures.

Controll-Drone-Kinect-2

This drone is designed to be controlled with a smartphone ONLY over a Bluetooth-LE connection using a Free-Flight 3 app from PlayStore or iTunes.

Luckily enough, the API to control the Parrot drone can be downloaded from GitHub. So we decided to play around with the APIs and see if we can come up with something interesting. Can we control this drone over WiFi or Internet? Can we give commands to this drone using a different controller like a Kinect Sensor? The answer is: “Yes We Can” 😀

Disclaimer: While this is not the first experiment of its kind, to our knowledge we are the first ones that managed to get this working for a Bluetooth drone.

Drones configuration kinect

So our entire network setup was as shown in the image above. The drone still gets its instructions from the Android device over Bluetooth, but the device now runs a socket server in the background and waits for clients to connect. Once a client is connected the device can forward the commands to drone. In our case the client is a windows machine with a Kinect sensor which connects over WiFi. This means that we can control the drone from another room. With a Peer-to-Peer server, we can control the drone from anywhere in the world over the Internet.

We defined some simple body gestures to control the drone such as take-off, landing, forward and backward.

If you have any feedback please get in touch with us over @avginnovation or drop me a line at tanweer [.] ali at avg [.] com

IMG_8210 IMG_37821

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.