Object Detection rather than Classification

[baqwas]

Hello,

This is a newbie question. Trying to assess my understanding of the game requirements.

In the first stage of the Autonomous mode, does it make sense to perform a simple object detection exercise (for a known game element - duck or team custom)? Is it safe to assume that only three objects will be detected by the robot's camera (team will do the tuning/tweaking for this) and the the largest detection box will the barcode level chosen randomly for the team?

Training the classification for the team custom element will take a few extra cycles (that are not under our belt presently) and trying to place the duck on the Hub during the EndGame will be self-defeating exercise. So essentially it is a tradeoff. By using a modestly sized team custom element we can easily determine the barcode level number and also be in a position to cap the Hub with it (hopefully!).

Does this approach make sense or am I, as a rookie team member, missing something that is very fundamental. Thanks.

Kind regards.

[11343_Mentor]

Like most things in FTC, there are multiple paths you can choose, and FTC always provides examples and sample programs to help teams achieve the basics. You didn't mention either what programming package you are using (Blocks/OnBot/Android) nor did you say what level of expertise you have in programming.

If you are a rookie programmer and using Bocks or Onbot, then a good place to start is for you to use the sample Concept TensorFlow program and just go for the duck. FTC will be releasing a program to create custom TensorFlow models, so using this would enable you to use your custom element, which then you could use for end game hub capping.

If you are a more seasoned programmer and using Android Studio, then using OpenCV with your custom element is actually quite simple. With detection, "bigger is better", and certain colors work better than others, but basically you can just look for blobs of your colored element within bounded areas of your screen.

[Lunerwalker2]

Adding on to the comment above, TensorFlow is certainly a viable option for detecting an object as with a good model it's actually pretty good at detection. You can then, for example, check if the object's location on the screen is within a certain range and from that determine what shipping hub level it corresponds to.

However if you are using Java (OnBot or Android Studio), and you want to learn some skills, then using an OpenCv pipeline via the EasyOpenCv library (https://github.com/OpenFTC/EasyOpenCV) is most assuredly the best (most consistent and fastest) way to complete the vision challenge, and as was also mentioned above, quite simple to do as the vision challenges for previous years are essentially identical to this one, meaning pipelines like the SkystoneDetermination (for the 2019-2020 game of detecting a yellow brick with a darkish blueish picture on it from a row of three yellow stones) example in the EOCV repo are more or less exactly what you'd use for Freight Frenzy with some changes depending on the color of your team marker and the positions of the sample regions based on your camera.

[Westside]

The sample OpMode called ConceptTensorFlowObjectDetectionWebcam simply displays telemetry of the Bounding Box sides for any recognition(s).

It can be adapted for Autonomous use, but it's helpful to first understand its internal logic. Here's a guide at the FTC Wiki:

https://github.com/FIRST-Tech-Challenge/FtcRobotController/wiki/Blocks-Sample-Op-Mode-for-TensorFlow-Object-Detection

The article has a section with hints on modifying the sample.

Many teams are working now to adapt that OpMode for Autonomous. Here's an example, with public comments added:

https://www.reddit.com/r/FTC/comments/qurrtj/where_to_put_code_after_detection_with_tfod/

[baqwas]

Sorry, Westside, for not getting back earlier. Our Blocks code requires Zoom to be pushed out to 1.25 and Confidence lowered to 0.4 for consistent/repeatable detection at 18" distance and 5" height with a Logitech C920 for all three ducks to be recognized (yes, there will be only one duck in the real world). Several suggestions have poured in from more experienced practitioners (just like yours); we're digesting these and hope to have some better results shortly.

The key observation is that the starter code works. I hope that other teams can wade in with this solution because it pushes the envelope for the Youth Members who are eager to get on with doing their own model building with the newly released ML kit. Thanks.

Kind regards.