Announcement

Collapse
No announcement yet.

Using OpenCV in FTC

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by mikets View Post
    One more question: during initialization, the call to webcam.openCameraDeviceAsync() is basically delaying the startStreaming call until onOpened is called. When is onOpened is called? Why is onOpened called asynchronously? Does startStreaming take a long time? I am trying to create a setEnabled and setDisabled methods to call startStreaming and stopStreaming dynamically so that we can turn on and off detection as needed.
    The reason that onOpened is called asynchronously is because both opening and starting streaming can take a long time, e.g. more than a second.
    There are also synchronous methods available, e.g. here.

    Comment


    • #17
      If startStreaming takes more than a second, then I should not enable/disable streaming dynamically or I need to only do it once before competition starts. Is there a cheaper way to start/stop streaming? I assume startStreaming is doing more than just start streaming. I can't imagine it takes that long to start the camera streaming. Basically, I need a cheap way to pause/resume the callback to processFrame.

      Comment


      • #18
        Originally posted by mikets View Post
        If startStreaming takes more than a second, then I should not enable/disable streaming dynamically or I need to only do it once before competition starts. Is there a cheaper way to start/stop streaming? I assume startStreaming is doing more than just start streaming. I can't imagine it takes that long to start the camera streaming. Basically, I need a cheap way to pause/resume the callback to processFrame.
        How long it takes really depends on which phone and which camera. I honestly haven't really done in depth testing to calculate the average time for certain configurations. I'd encourage you to try it and see how it behaves.

        It's not really doing much more than just starting the stream. You can see the implementations here and here:

        Comment


        • #19
          I am browsing the source code and found in the method handleFrameUserCrashable, it's checking if the pipeline is not null before calling processFrameInternal. So, in theory, if I set the Pipeline to null, it will pause OpenCV processing of a frame. And I can set the Pipeline again if I want to resume. Do you see any problem in doing that?

          Comment


          • #20
            Originally posted by mikets View Post
            I am browsing the source code and found in the method handleFrameUserCrashable, it's checking if the pipeline is not null before calling processFrameInternal. So, in theory, if I set the Pipeline to null, it will pause OpenCV processing of a frame. And I can set the Pipeline again if I want to resume. Do you see any problem in doing that?
            Yeah, you can "disable" a pipeline by setting a null pipeline to the camera. It will have the effect of displaying the unprocessed image feed on the screen. You can also call pauseViewport() if you want.

            Comment


            • #21
              Cool. I think I got it down. Take a look at this implementation and let me know if you see any issues. Thanks.
              https://github.com/trc492/FtcTemplat...ocvVision.java

              Comment


              • #22
                While testing this, the robot control kept crashing complaining the Native OpenCV library is not found in the local storage of the Robot Controller. I must not have setup EasyOpenCV correctly. Basically, I have put a dependency in the TeamCode/build.gradle pointing to Maven. Should that be it for the trick or do I need to do something else too?

                Code:
                dependencies {
                    implementation project(':FtcRobotController')
                    implementation 'org.apache.commons:commons-math3:3.6.1'
                    implementation 'org.openftc:easyopencv:1.5.1'
                    annotationProcessor files('lib/OpModeAnnotationProcessor.jar')
                }
                2022-07-14 17:29:45.557 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: org.openftc.opencvrepackaged.OpenCvNativeLibNotFou ndException
                2022-07-14 17:29:45.559 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.openftc.opencvrepackaged.DynamicOpenCvNativeLi bLoader.setupOpenCvFiles(DynamicOpenCvNativeLibLoa der.java:248)
                2022-07-14 17:29:45.560 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.openftc.opencvrepackaged.DynamicOpenCvNativeLi bLoader.setupOpenCVNativeLib(DynamicOpenCvNativeLi bLoader.java:140)
                2022-07-14 17:29:45.561 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.openftc.opencvrepackaged.DynamicOpenCvNativeLi bLoader.loadNativeLibOnStartRobot(DynamicOpenCvNat iveLibLoader.java:107)
                2022-07-14 17:29:45.562 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at java.lang.reflect.Method.invoke(Native Method)
                2022-07-14 17:29:45.563 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.firstinspires.ftc.robotcore.internal.opmode.An notatedOpModeClassFilter.callOpModeRegistrarMethod s(AnnotatedOpModeClassFilter.java:516)
                2022-07-14 17:29:45.565 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.firstinspires.ftc.robotcore.internal.opmode.An notatedOpModeClassFilter.registerAllClasses(Annota tedOpModeClassFilter.java:120)
                2022-07-14 17:29:45.566 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.firstinspires.ftc.robotcore.internal.opmode.Re gisteredOpModes$3.run(RegisteredOpModes.java:198)
                2022-07-14 17:29:45.567 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.firstinspires.ftc.robotcore.internal.opmode.Re gisteredOpModes.lockOpModesWhile(RegisteredOpModes .java:319)
                2022-07-14 17:29:45.569 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at org.firstinspires.ftc.robotcore.internal.opmode.Re gisteredOpModes.registerAllOpModes(RegisteredOpMod es.java:182)
                2022-07-14 17:29:45.570 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.ftccommon.FtcEventLoop.init(FtcEventL oop.java:181)
                2022-07-14 17:29:45.571 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.robotcore.eventloop.EventLoopManager. startEventLoop(EventLoopManager.java:563)
                2022-07-14 17:29:45.573 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.robotcore.eventloop.EventLoopManager. setEventLoop(EventLoopManager.java:537)
                2022-07-14 17:29:45.574 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.robotcore.eventloop.EventLoopManager. start(EventLoopManager.java:479)
                2022-07-14 17:29:45.575 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.robotcore.robot.Robot.start(Robot.jav a:59)
                2022-07-14 17:29:45.576 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.ftccommon.FtcRobotControllerService$R obotSetupRunnable.startRobot(FtcRobotControllerSer vice.java:281)
                2022-07-14 17:29:45.577 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.ftccommon.FtcRobotControllerService$R obotSetupRunnable$1.run(FtcRobotControllerService. java:300)
                2022-07-14 17:29:45.578 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.robotcore.util.ThreadPool.logThreadLi feCycle(ThreadPool.java:737)
                2022-07-14 17:29:45.579 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.ftccommon.FtcRobotControllerService$R obotSetupRunnable.run(FtcRobotControllerService.ja va:291)
                2022-07-14 17:29:45.580 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at java.util.concurrent.Executors$RunnableAdapter.cal l(Executors.java:428)
                2022-07-14 17:29:45.581 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at java.util.concurrent.FutureTask.run(FutureTask.jav a:237)
                2022-07-14 17:29:45.582 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1133)
                2022-07-14 17:29:45.582 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:607)
                2022-07-14 17:29:45.583 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at com.qualcomm.robotcore.util.ThreadPool$ThreadFacto ryImpl$1.run(ThreadPool.java:793)
                2022-07-14 17:29:45.584 1501-1557/com.qualcomm.ftcrobotcontroller E/OpenFTC-OpenCV-Repackaged-Loader: at java.lang.Thread.run(Thread.java:761)

                Comment


                • #23
                  Originally posted by mikets View Post
                  Cool. I think I got it down. Take a look at this implementation and let me know if you see any issues. Thanks.
                  https://github.com/trc492/FtcTemplat...ocvVision.java
                  Nothing catches my eye immediately.

                  Comment


                  • #24
                    Originally posted by mikets View Post
                    While testing this, the robot control kept crashing complaining the Native OpenCV library is not found in the local storage of the Robot Controller. I must not have setup EasyOpenCV correctly. Basically, I have put a dependency in the TeamCode/build.gradle pointing to Maven. Should that be it for the trick or do I need to do something else too?

                    Code:
                    dependencies {
                    implementation project(':FtcRobotController')
                    implementation 'org.apache.commons:commons-math3:3.6.1'
                    implementation 'org.openftc:easyopencv:1.5.1'
                    annotationProcessor files('lib/OpModeAnnotationProcessor.jar')
                    }
                    Please read the installation section of the readme )

                    EOCV depends on OpenCV-Repackaged, which is set up to dynamically load the 10MB native library from internal storage rather than including it with the APK every time, to reduce APK size and keep wireless deploy cycles fast (especially when combined with OpenRC-Turbo). You mentioned you already had the OpenCV library imported into your project directly, so probably the best thing for your case is to exclude the transitive dependency on OpenCV-Repackaged.

                    Comment

                    Working...
                    X