Download - Android and OpenCV Tutorial
![Page 1: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/1.jpg)
Institute of Visual Computing
Android and OpenCV Tutorial
Computer Vision Lab Tutorial
26 September 2013
Lorenz Meier, Amaël Delaunoy, Kalin Kolev
![Page 2: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/2.jpg)
Institute of Visual Computing 2!
Tutorial Content
• Strengths / Weaknesses of Android
• Java vs Native
• Overview of available SDKs
• Complete example
• Assignment
• Project plan coaching
• Question session
![Page 3: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/3.jpg)
Institute of Visual Computing 3!
Android Software Development Kits
Software Development Kit (SDK)
Native Development Kit (NDK)
![Page 4: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/4.jpg)
Institute of Visual Computing 4!
Android
• Android 4.3 Jelly Bean
• Targeted compatibility version: 4.1
• Android SDK: http://developer.android.com/sdk/index.html
• Android NDK: http://developer.android.com/tools/sdk/ndk/index.html
![Page 5: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/5.jpg)
Institute of Visual Computing 5!
Java SDK
• Easy to setup and get started
• Typically only mean of accessing the camera
• Slow for core vision algorithms
• Slow for some sensor access functions
• http://developer.android.com/sdk/index.html
![Page 6: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/6.jpg)
Institute of Visual Computing 6!
Android NDK (C/C++, GNU/Linux API)
• Still easy enough to get started
• Offers full OpenCV functionality on the device
• Important: Have NDKROOT set as environment variable in Eclipse
• http://developer.android.com/tools/sdk/ndk/index.html
export PATH=/usr/local/bin:/usr/local/sbin:/Users/lomeier/android/android-ndk-r9:$PATH export NDKROOT=/Users/lomeier/android/android-ndk-r9"
chmod u+x /Users/lomeier/android/android-ndk-r9/ndk-build.cmd"
http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/android_dev_intro.html!
![Page 7: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/7.jpg)
Institute of Visual Computing 7!
Computer Vision Toolkits for Android
OpenCV
Qualcomm Vuforia
Qualcomm FastCV
OpenVX
![Page 8: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/8.jpg)
Institute of Visual Computing 8!
OpenCV on Android
• Main reference http://opencv.org/platforms/android.html
• Watch out! API has changed, use only documentation and tutorials from 2012/2013
• Start off with the OpenCV Tutorial #3: Native OpenCV
http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/android_dev_intro.html!
![Page 9: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/9.jpg)
Institute of Visual Computing 9!
Qualcomm Vuforia SDK
• Virtual Reality SDK for any Android / iOS device https://developer.qualcomm.com/download
http://www.youtube.com/watch?v=BlF60qYTrPI!!
![Page 10: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/10.jpg)
Institute of Visual Computing 10!
Qualcomm FastCV SDK
• Similar API to OpenCV
• Collection of low-level computer vision primitives
• On many functions faster than OpenCV
• https://developer.qualcomm.com/mobile-development/mobile-technologies/computer-vision-fastcv
http://www.youtube.com/watch?v=zTxpFCpG-GA!!
![Page 11: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/11.jpg)
Institute of Visual Computing 11!
Preview: OpenVX – the new standard SDK
• OpenGL equivalent for hardware-accelerated computer vision
• Co-developed with OpenCV
• All major industry players
• First draft end 2013
![Page 12: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/12.jpg)
Institute of Visual Computing 12!
Android SDK + Android NDK + OpenCV
![Page 13: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/13.jpg)
Institute of Visual Computing 13!
Android SDK Installation
• Download from: http://developer.android.com/sdk/index.html
• Unzip the directory to a local directory without spaces in the path
![Page 14: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/14.jpg)
Institute of Visual Computing 14!
Android NDK Installation
• Download from: http://developer.android.com/tools/sdk/ndk/index.html
• Unzip to same top-level folder as the SDK
![Page 15: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/15.jpg)
Institute of Visual Computing 15!
OpenCV for Android Installation
• Download from: http://opencv.org/downloads.html
• Unzip to same top-level folder as Android SDK and NDK
![Page 16: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/16.jpg)
Institute of Visual Computing 16!
Set up Eclipse / ADT
![Page 17: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/17.jpg)
Institute of Visual Computing 17!
Final directory layout
• After all SDKs are installed, the directory structure should look like similar to the screenshot below.
• Open the Eclipse folder and start the executable.
![Page 18: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/18.jpg)
Institute of Visual Computing 18!
Start Eclipse / ADT
![Page 19: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/19.jpg)
Institute of Visual Computing 19!
Start Android SDK Manager
![Page 20: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/20.jpg)
Institute of Visual Computing 20!
Install the Android Support Library
![Page 21: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/21.jpg)
Institute of Visual Computing 21!
Install the Android Support Library
![Page 22: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/22.jpg)
Institute of Visual Computing 22!
Android Project Setup
![Page 23: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/23.jpg)
Institute of Visual Computing 23!
Android Project Setup
• Right-click into the Eclipse project view
• Select Import.. – Existing projects into workspace
• Select OpenCV Folder
• All tutorials will show up in the project list
• Right click the Tutorial #2
![Page 24: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/24.jpg)
Institute of Visual Computing 24!
Import new project
![Page 25: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/25.jpg)
Institute of Visual Computing 25!
![Page 26: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/26.jpg)
Institute of Visual Computing 26!
![Page 27: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/27.jpg)
Institute of Visual Computing 27!
![Page 28: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/28.jpg)
Institute of Visual Computing 28!
![Page 29: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/29.jpg)
Institute of Visual Computing 29!
Project Properties Setup – ENV variables
![Page 30: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/30.jpg)
Institute of Visual Computing 30!
Commandline Setup
• Imported Eclipse projects can contain errors and fail to compile dependencies
• Set up your command line environment by adding this to your ~/.bashrc or ~/.bash_profile file:
export PATH=/usr/local/bin:/usr/local/sbin:/Users/lomeier/android/android-ndk-r8b:$PATH export NDKROOT=/Users/lomeier/android/android-ndk-r8b"
• Make the ndk-build script executable chmod u+x /Users/lomeier/android/android-ndk-r9/ndk-build.cmd"
source ~/.bash_profile"
![Page 31: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/31.jpg)
Institute of Visual Computing 31!
Commandline compilation
• Change to the folder to the project
• Force a ndk clean build
• Build / upload now in Eclipse
cd /Users/lomeier/src/android/OpenCV-2.4.6-android-sdk/samples/tutorial-3-native"
ndk-build -B"
![Page 32: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/32.jpg)
Institute of Visual Computing 32!
Launch Application
![Page 33: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/33.jpg)
Institute of Visual Computing 33!
![Page 34: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/34.jpg)
Institute of Visual Computing 34!
Put Phone into developer mode
Settings -> About phone
Tap on build number
Settings -> Developer options
Enable USB debugging
From: http://howto.cnet.com/8301-11310_39-57549207-285/restore-the-developer-options-menu-in-android-4.2/!
![Page 35: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/35.jpg)
Institute of Visual Computing 35!
Reset USB
UNPLUG USB. RE-PLUG USB. CONFIRM THE USB CONNECTION.
![Page 36: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/36.jpg)
Institute of Visual Computing 36!
Select Cell Phone
![Page 37: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/37.jpg)
Institute of Visual Computing 37!
Troubleshooting
Unsatisfied link error?
Force a native build on the command line.
cd /dir/to/OpenCV/samples"
cd tutorial-2-mixedprocessing"
ndk-build -B"
![Page 38: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/38.jpg)
Institute of Visual Computing 38!
Run the Sample (jni/jni_part.cpp)
• Standard example code extracts FAST corners
Mat& mGr = *(Mat*)addrGray;" Mat& mRgb = *(Mat*)addrRgba;" vector<KeyPoint> v;"" FastFeatureDetector detector(50);" detector.detect(mGr, v);! for( unsigned int i = 0; i < v.size(); i++ )! {" const KeyPoint& kp = v[i];! circle(mRgb, Point(kp.pt.x, kp.pt.y), 10, Scalar(255,0,0,255));! }"
![Page 39: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/39.jpg)
Institute of Visual Computing 39!
Class Assignment (jni/jni_part.cpp)
• Extend sample to extract and match ORB features between two frames
• Then color all pixels with ORB features translucent blue Mat& mGr = *(Mat*)addrGray; // This is the gray scale frame" Mat& mRgb = *(Mat*)addrRgba; // This is the frame in color" vector<KeyPoint> v; // You need to store the ones from the last frame
" " " "// and this frame"" OrbFeatureDetector detector(50);" detector.detect(mGr, v);! for( unsigned int i = 0; i < v.size(); i++ )! {" const KeyPoint& kp = v[i];! circle(mRgb, Point(kp.pt.x, kp.pt.y), 10, Scalar(0,0,255,100));!
! line(… from old point to new match)! }" OrbDescriptorExtractor …" BruteForceMatcher<cv::HammingLUT> …"
![Page 40: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/40.jpg)
Institute of Visual Computing 40!
Class Assignment
• Hand in until next Thursday (October 3, 2013)
• Screenshot (from device, not photos of display)
• Code from jni_part.cpp, inlined into email
• http://www.howtogeek.com/121133/how-to-take-screenshots-on-android-devices-since-4.0/
![Page 41: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/41.jpg)
Institute of Visual Computing 41!
Camera Calibration
![Page 42: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/42.jpg)
Institute of Visual Computing 42!
Camera calibration
• Additional tutorial if required
• OpenCV functions available
• Undistortion and camera matrix map from image pixel locations to normalized camera coordinates (“rays”)
Pinhole camera model
![Page 43: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/43.jpg)
Institute of Visual Computing 43!
Camera calibration
1. Find and match corners
2. Calibrate camera (undistortion and rectification)
3. Optimize matrix
4. Test result on image
bool findChessboardCorners(image, patternSize, corners, flags); "
double calibrateCamera(objectPoints, imgPoints, imgSize, camMatrix, distCoeffs, rvecs, tvecs, flags, criteria); "
Mat initCameraMatrix2D(objPoints, imgPoints, imgSize, aspectRatio); "
void getOptimalNewCameraMatrix(camMatrix, distCoeffs, imgSize, alpha, newImgSize, PixROI, cPP); "
void undistort(src, dst, cameraMatrix, distCoeffs, newCameraMatrix); "
"
![Page 44: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/44.jpg)
Institute of Visual Computing 44!
Project Plan and Management
![Page 45: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/45.jpg)
Institute of Visual Computing 45!
Project Proposal Q&A
• Papers / Scope
• Library choice
• Milestones
• Block Diagram
• Gantt Chart
![Page 46: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/46.jpg)
Institute of Visual Computing 46!
Diagrams in Report
• Data flow / block diagram
• Gantt Chart
![Page 47: Android and OpenCV Tutorial](https://reader031.vdocuments.pub/reader031/viewer/2022021923/58522d151a28abfa398d35a1/html5/thumbnails/47.jpg)
Institute of Visual Computing 47!
Project Management
• Use Github issue tracker and milestones http://github.com
• Use the Github for Windows / Mac OS app as GUI
• Simple GIT guide: http://rogerdudler.github.com/git-guide/
• GIT SCM book: http://git-scm.com/book/ch1-3.html