Announcement

**rbFTC** · 01-04-2017, 12:49 AM

Our sdk is at 2.4 release, but we get these messages all the time. My assumption had been that these messages are the result of usb disconnect issues. They do not have a practical impact in the teleop mode as long as the frequency of the messages is low (we see ~ 7-10 per minute). For the autonomous mode, we try to come up with ways to mitigate the impact.

Interesting to know that you do not see these messages if the phone bypasses cpdm. I have not tried that.

**Philbot** · 01-04-2017, 10:18 AM

Originally posted by FTC4855 View Post

Hello All

We recently updated to SDK 2.61 and keep getting the following error message in our Robot Controller Logs

Code:

 ```12-30 15:57:07.759 28278-28386/com.qualcomm.ftcrobotcontroller D/D2XX::: **********************Device Opened**********************
12-30 15:57:07.773 28278-28565/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 09] -> [33 cc 80 40 09]
12-30 15:57:07.774 28278-28565/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB Servo Controller [A104ORNB]: comm timeout on payload
12-30 15:57:07.797 28278-28524/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [33 cc 80 40 1e]
12-30 15:57:07.797 28278-28524/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [A104OW52]: comm timeout on payload
12-30 15:57:07.922 28278-28512/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
12-30 15:57:07.942 28278-28541/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [33 cc 80 40 1e]
12-30 15:57:07.943 28278-28541/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VIFG]: comm timeout on payload
12-30 15:57:07.966 28278-28524/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [33 cc 80 40 1e]
12-30 15:57:07.967 28278-28524/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [A104OW52]: comm timeout on payload
12-30 15:57:07.983 28278-28541/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
12-30 15:57:08.022 28278-28512/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [00 00 00 00 00]
12-30 15:57:08.022 28278-28512/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AI049PGC]: comm error
12-30 15:57:08.026 28278-28552/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
12-30 15:57:08.084 28278-28541/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [00 00 00 00 00]
12-30 15:57:08.084 28278-28541/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VIFG]: comm error
12-30 15:57:08.126 28278-28552/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [00 9e 01 10 c0]
12-30 15:57:08.126 28278-28552/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [A7008OX6]: comm error
12-30 15:57:08.131 28278-28575/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
....

We have swapped our DC Motor Controllers / Core Power Distribution Modules and other modules with spares and still keep seeing the error.
This problem does not occur if the robot controller phone is connected directly to the DC Motor controller i.e. we bypass the CPDM.
We find it hard to believe that all 4 of our CPDM's are bad. Anybody else has similar issues or have ideas on what might be wrong?

FTC4855

The fact that the messages go away if you bypass the PDM is interesting...

I'm wondering if they go away because you are only connecting to a single motor controller, so the other controllers are not being accessed (compared to using the PDM with all the MC's).

Do you see the same errors when you only have a single MC connected to the PDM?

Also, I'd be interested to see if it matters WHICH channel on the PDM you connect to. Since there are two cascaded USB hubs in that device, not all the ports are identical (although I'm not certain which are which, it's likely that the low number ports are not on the same hub as the high number ports)

**FTC0417** · 01-04-2017, 10:24 AM

Thanks for this report; it's surprisingly helpful.

Though I can't fully explain the behavior, I have a hypothesis that these aren't disconnnect issues at all but simply the fact that that the timing and tolerances of MR motor controller USB responses can be longer when connected through the two USB hubs in the CPDM (the module has two four-port hubs that are daisy-chained; thus you see seven ports) than the 100ms request-to-response timeout allowed for in the SDK. Half of that 100ms can be taken by the motor controller itself, which leave 12.5ms for each of the two hubs (25ms for the USB request path and 25ms for the USB response path, times two hubs). That doesn't seem onerous on it's face, but the data smells otherwise to me.

Do you perchance see a difference if the motor controller is on port P0 of the CPDM vs port P6? Do you see any differences if the motor controller is the only device on the CPDM vs there being other controllers present?

**FTC4855** · 01-04-2017, 11:56 AM

We will experiment and report back to you later this afternoon after we get back from school whether using CPDM P0 vs P6 makes a difference.

@philbot and @ftc0417 we do see the errors with a single MC connected to CPDM.

The impact this has is when we are trying to do closed loop control of our flywheel motor it stutters and we never achieve our target control velocity.

**Alec** · 01-04-2017, 12:07 PM

Originally posted by Philbot View Post

... Since there are two cascaded USB hubs in that device, not all the ports are identical (although I'm not certain which are which, it's likely that the low number ports are not on the same hub as the high number ports)

Modern Robotics Support, can you please tell us which 3 PDM USB ports are on the primary USB hub, and which 4 PDM USB ports are on the secondary USB hub?

**Modern Robotics Support** · 01-04-2017, 01:38 PM

Originally posted by Alec View Post

Modern Robotics Support, can you please tell us which 3 PDM USB ports are on the primary USB hub, and which 4 PDM USB ports are on the secondary USB hub?

The PDM uses a TI TUB2077A USB chip. It is a single USB hub chip and there is no reference by TI to daisy chained hubs or primary or secondary hub ports so we are unable to answer to your question.

Hub latency is difficult to determine as the manufacture does not state it but in our testing it appeared to be in the low µs range and did not add any measurable time to message send and response times in our tests.

On average the Core Modules message response is 4-6 ms. This does not take into account the added overhead of the SDK and Android.

The datasheet for the TUSB2077A can be found at http://www.ti.com/lit/ds/slls414f/slls414f.pdf.

**rbFTC** · 01-04-2017, 02:20 PM

Originally posted by FTC0417 View Post

Thanks for this report; it's surprisingly helpful.

Though I can't fully explain the behavior, I have a hypothesis that these aren't disconnnect issues at all but simply the fact that that the timing and tolerances of MR motor controller USB responses can be longer when connected through the two USB hubs in the CPDM (the module has two four-port hubs that are daisy-chained; thus you see seven ports) than the 100ms request-to-response timeout allowed for in the SDK. Half of that 100ms can be taken by the motor controller itself, which leave 12.5ms for each of the two hubs (25ms for the USB request path and 25ms for the USB response path, times two hubs). That doesn't seem onerous on it's face, but the data smells otherwise to me.

Do you perchance see a difference if the motor controller is on port P0 of the CPDM vs port P6? Do you see any differences if the motor controller is the only device on the CPDM vs there being other controllers present?

Good to know that these are not USB disconnect related messages. On the other hand, we now have an unknown problem to diagnose

Just to add another data point to FTC4855's post, I am pasting the logcat from our robot controller. I do not think this adds anything extra, except showing that the error occurs for Servo and DIM controllers also. The message frequency from Yesterday's run seems to be much higher than what I noticed earlier. We have 4 motor controllers (7 Neverest40 motors), 1 Servo controller, and 1 DIM. We will be happy to do tests and provide more data if necessary.

01-03 23:44:52.167 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [33 cc 80 40 1e]
01-03 23:44:52.167 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VS98]: comm timeout on payload
01-03 23:44:52.892 12195-12407/com.qualcomm.ftcrobotcontroller I/RobotCore: robot battery read duration: n=4, mean=5.727ms sd=7.481ms
01-03 23:44:53.387 12195-12358/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
01-03 23:44:53.488 12195-12358/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [a9 03 19 a0 20]
01-03 23:44:53.488 12195-12358/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VBJK]: comm error
01-03 23:44:53.717 1668-2595/? V/AlarmManager: sending alarm {8e4d051 type 3 *alarm*:send_events}
01-03 23:44:53.726 1668-1668/? V/AlarmManager: done {8e4d051, *alarm*:send_events} [8ms]
01-03 23:44:54.391 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
01-03 23:44:54.491 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [00 00 00 00 ab]
01-03 23:44:54.491 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VS98]: comm error
01-03 23:44:55.868 325-679/? D/audio_hw_primary: out_standby: enter: stream (0xb8347c48) usecase(1: low-latency-playback)
01-03 23:44:55.973 325-679/? D/hardware_info: hw_info_append_hw_type : device_name = speaker
01-03 23:44:56.320 12195-12398/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
01-03 23:44:56.360 12195-12369/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 09] -> [33 cc 80 40 09]
01-03 23:44:56.360 12195-12369/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB Servo Controller [AL00XLTB]: comm timeout on payload
01-03 23:44:56.420 12195-12398/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [00 00 00 00 00]
01-03 23:44:56.420 12195-12398/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00YCFW]: comm error
01-03 23:44:57.852 12195-12380/com.qualcomm.ftcrobotcontroller W/RobotCore: Modern Robotics USB header sync bytes are incorrect
01-03 23:44:57.853 12195-12380/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [00 00 00 00 00]
01-03 23:44:57.853 12195-12380/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VSAM]: comm timeout
01-03 23:44:57.936 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [33 cc 80 40 1e]
01-03 23:44:57.936 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VS98]: comm timeout on payload
01-03 23:45:00.001 1668-2595/? V/AlarmManager: sending alarm {bf7acd4 type 3 *alarm*:android.intent.action.TIME_TICK}
01-03 23:45:00.006 2758-2758/? D/KeyguardUpdateMonitor: received broadcast android.intent.action.TIME_TICK
01-03 23:45:00.007 2758-2758/? D/KeyguardUpdateMonitor: handleTimeUpdate
01-03 23:45:00.069 1668-1668/? V/AlarmManager: done {bf7acd4, *alarm*:android.intent.action.TIME_TICK} [68ms]
01-03 23:45:00.128 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [33 cc 80 40 1e]
01-03 23:45:00.128 12195-12387/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VS98]: comm timeout on payload
01-03 23:45:00.283 12195-12358/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 40 1e] -> [33 cc 80 40 1e]
01-03 23:45:00.283 12195-12358/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB DC Motor Controller [AL00VBJK]: comm timeout on payload
01-03 23:45:00.393 12195-12351/com.qualcomm.ftcrobotcontroller W/RobotCore: [55 aa 80 70 20] -> [33 cc 80 70 20]
01-03 23:45:00.393 12195-12351/com.qualcomm.ftcrobotcontroller W/RobotCore: could not read Modern Robotics USB Core Device Interface Module [AL00XKYB]: comm timeout on payload

**5294-jjkd** · 01-04-2017, 03:45 PM

If you have any other powered USB hub available, it might be interesting to see if the same problem occurs with a non-CPDM and having the same number of core modules connected. Or even one, you said previously that you still saw equivalent errors with only a single core module connected through the CPDM.

**FTC0417** · 01-04-2017, 03:50 PM

Originally posted by Modern Robotics Support View Post

The PDM uses a TI TUB2077A USB chip. [...] The datasheet for the TUSB2077A can be found at http://www.ti.com/lit/ds/slls414f/slls414f.pdf.

Thanks for the clarification, which updates what I gather now is out-of-date information I was given previously. And thanks especially for the datasheet reference.

**FTC0417** · 01-04-2017, 08:42 PM

After some reflection, what I now think is happening here is the following:

The SDK successfully sends a read request to the controller.
The SDK (using a 100ms timeout) successfully reads the five-byte header of the controller response.
The SDK (again using a 100ms timeout) tries to read the variable sized payload data of the controller response. This sometimes fails, the SDK 'purge's the USB line (which may or may not be successful, but that's a story for another time), and the SDK issues the "comm timeout on payload" error. But unfortunately this purge doesn't always flush everything, as sometimes/occasionally not all the data has even arrived from the controller yet.
When next the SDK sends a new request and then tries to read the response from *that* one, it's seeing some of the data incoming from the *previous* response. It can't interpret this correctly and resychronize, which yields the "comm error".

I believe that the (a?) root cause of the problem is that the SDK's second 100ms timeout value is (as I have recently learned) incorrect: it should instead be a length-dependent duration. The SDK could also do a more robust job of re-establishing synchronization when that's been lost.

**FTC4855** · 01-04-2017, 11:35 PM

More interesting data after our testing earlier this evening:

It does not matter which port on the CPDM you are plugged into. When the issue occurs it will occur on any of the ports.
It is not dependent on the number of Core Modules that are connected to the CPDM
It is dependent on the type of phone you are using for the Robot Controller. It occurs continuously with Moto G2, occasionally with Moto G3. We will be testing with Nexus 5 and ZTE a little bit later this evening and post detailed results. We do not have access to a MotoG4.
We do not have an externally powered USB hub available at our practice location but will be getting one tomorrow and will test further.

FTC4855

**rbFTC** · 01-05-2017, 07:14 AM

We use moto g4 play from which I pasted the logcat I earlier. So this issue occurs on moto g4 play also.

**FTC4855** · 01-05-2017, 11:43 AM

Additional Information

Additional updates after further testing:

This issue occurs with all phone models that we have (ZTE, Moto G2, Moto G3, Nexus 5)
With some phones, the frequency of occurrence is less but if you leave the Robot Controller App running with the Driver Station hooked up the messages start showing up and as time passes the frequency of occurrence goes up.
At this point, we are inclined to agree with FTC0417's analysis which we quote below

The SDK successfully sends a read request to the controller.
The SDK (using a 100ms timeout) correctly reads the five-byte header of the controller response.
The SDK (again using a 100ms timeout) tries to read the variable sized payload data of the controller response. This sometimes fails, the SDK 'purge's the USB line (which may or may not be successful, but that's a story for another time), and the SDK issues the "comm timeout on payload" error. But unfortunately, this purge doesn't always flush everything, as sometimes/occasionally not all the data has even arrived from the controller yet.
When next the SDK sends a new request and then tries to read the response from *that* one, it's seeing some of the data incoming from the *previous* response. It can't interpret this correctly and resychronize, which yields the "comm error."

We think this is an FTC SDK Synchronization Issue.

FTC4855

**yjw558** · 01-05-2017, 02:02 PM

What's the real impact on Autonomous or TeleOps? We got these errors from time to time, but our app seems running fine. We thought these were just some random noises from SDK.

Announcement

Technology Forum Has Moved!

Weird Issues with Modern Robotics Core Motor Controller

Weird Issues with Modern Robotics Core Motor Controller

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment