Update on USB Disconnect Issues (unable to detect USB modules during scan)

[dlevy]

Thanks for the tip dlevy! I updated my RC to the latest SDK and the robot was running better than before. My DS was v 1.25 while my RC was 1.0.

And yes, I personally was caught off guard by our terrible run. We had always had problems but never to the extent we had them at the competition. I also realized that when I got an error in match 6 at our competition, I was connected to another team's robot controller.

Matching the versions should be on the checklist during robot inspection. I wasn't failing the inspection but just giving a warning that this could be one source of problems. I actually downgraded one team's DS to 1.2 since in the interest of time that seem like the easiest thing to do.

[Philbot]

H Phil,
It was nice to finally meet you as well. Your team's robot was really fun to watch toss blocks quickly onto that conveyor belt.

Thanks for the log file. Can I ask how this error manifested on the phone ( error message in red, screen freeze , no error reported)?

I don't have a lot of experience with the static discharge problem. Is there something about this new control system compared to prior years that makes the problem more acute? Or do you think this is just a matter of allowing for static sprays as done in the past.

Thanks,
David

I'm not sure how the error manifested on the phone this time, but in the past it usually does not indicate at all... simply stops sending data to the hardware, which runs on for 2.5 sec and then stops.
No exceptions indicated. When the app is restarted, it gives the usb failure condition.

In the past, the static lockups were attributed to the USB connection between the NXT and Sammantha.

Since my robot now has 5 USB lines, I suspect the problem may be 5x as bad

Several teams swear by the ferrite chokes on the USB lines, but since our MR cables went bad we have only been able to replace them with un-choked lines.
Could be an issue. An EE Judge at our event swears that the problem is resolved by grounding the -Ve battery line to the chassis, but I don't believe this has been allowed in the past.

Our best solution has always been anti-static spray on the field.

Phil.

[dlevy]

...in the past it usually does not indicate at all... simply stops sending data to the hardware, which runs on for 2.5 sec and then stops.
No exceptions indicated. When the app is restarted, it gives the usb failure condition.
..

Phil.

As a CSA i would have looked at your app versions, Wifi Direct groups, RC logs, and then the code. I did see a lot of mismatched versions on Sunday and attributed it to that. I could be have been wrong but once I saw that all bets are off. Knowing you, I'm sure that this wasn't your condition. So admittedly I would have made the mistake of attributing the problem to faulty wiring rather than static buildup. At a loss about what to do about that.

[Philbot]

As a CSA i would have looked at your app versions, Wifi Direct groups, RC logs, and then the code. I did see a lot of mismatched versions on Sunday and attributed it to that. I could be have been wrong but once I saw that all bets are off. Knowing you, I'm sure that this wasn't your condition. So admittedly I would have made the mistake of attributing the problem to faulty wiring rather than static buildup. At a loss about what to do about that.

We desperately need a way to detect this glitch and recover from it. (be it caused by a jiggled cable or a static discharge).

If one of our personal entertainment devices just rolled over and died when there was a USB cable jiggle, we'd never buy that company's products again.

Phil.

[Varun Singh]

I would like to add that the Ferrite Chokes do nothing to prevent ESD. We are still using the MR cables (we haven't had a problem with those at all surprisingly, but have had our fair share of problems in plenty of other areas). If USB is highly susceptible to ESD (which it shouldn't), that could be causing the static lockup issues. Also, I am not sure if this affects it, but our robot experiences a lot of lockups, even though all of our electronics are on a sheet of Lexan and are at least 1" away from any metal, but I just realized today that all the wires are in close contact, and so maybe that is affecting it. I have no way of testing however. Between the errors in the electronics, cabling, and SDK, this is becoming a very frustrating season for everyone, as testing any changes takes around 5-10 minutes to get rid of all the problems.

[Alec]

... If USB is highly susceptible to ESD (which it shouldn't), that could be causing the static lockup issues ...

USB is highly susceptible to ESD, and to EMI (i.e. the stuff that spews from those red and black cables going in and out of the PDM). When in doubt, just google USB ESD EMI.

We used to have a server PC at our office that in the winter time (low humidity) would freeze whenever you moved your hand 1 foot near a USB port. Certain regions of the country will fare better with a USB platform than others.

[FTC9958]

We desperately need a way to detect this glitch and recover from it. (be it caused by a jiggled cable or a static discharge).

If one of our personal entertainment devices just rolled over and died when there was a USB cable jiggle, we'd never buy that company's products again.

Phil.

While this post is a couple weeks old, I just wanted to applaud Phil for once again finding such an eloquent way of breaking this down. i.e. if something as simple as a cable vibrating caused any of our home appliances to fail, even temporarily, that company would likely be facing a system-wide recall.

And while we Coaches are sure that Tom Eng and the Entire FTC team are doing all they can, I'm still left with on unanswered question.... Who on the design side thought that multiple unsecured connectors (i.e. USB) were a good idea to put on a device that's hosted by a vehicle that's purposely designed to crash into things?

Not saying it's easy, but there does seem to have been some gaps in the process.

[FTC6981]

Is it just me, or is the silence from First and MR in THIS thread deafening?

[ViperMentor]

Who on the design side thought that multiple unsecured connectors (i.e. USB) were a good idea to put on a device that's hosted by a vehicle that's purposely designed to crash into things? Not saying it's easy, but there does seem to have been some gaps in the process.

I sort of have to agree that using USB cables was a really bad idea for this new platform. You could argue that we critics are saying this with the benefit of hindsight but unfortunately (for MR) that hindsight was already there. The only data connection that used to fail on the old system was the USB connection to the Samantha module. The connectors between the NXT brick and the NXT sensors and HiTechnic modules NEVER failed. Those telephone-jack-style connectors were virtually bullet-proof.

I must admit I was really disappointed when the new system was announced and it wasn't based on EV3. That would have been a far more robust system for this application. And it would have done away with the Samantha issues too (EV3 supports one of those tiny plug-in wifi adapters - no Samantha module required and no USB cables required). It would also not have suffered from the charge vs. download issue. Unlike smartphones, the EV3 brick can be charged and downloaded (and have the wifi plugged in) all at the same time.

[dlevy]

We desperately need a way to detect this glitch and recover from it. (be it caused by a jiggled cable or a static discharge).

If one of our personal entertainment devices just rolled over and died when there was a USB cable jiggle, we'd never buy that company's products again.

Phil.

No argument there. See item #5
http://ftcforum.usfirst.org/showthre...ll=1#post23938

[skatefriday]

No argument there. See item #5
http://ftcforum.usfirst.org/showthre...ll=1#post23938

Yeah, I think there are likely safety issues in resiliency scenarios
that need to be carefully thought through before any changes
to e-stop behavior are implemented.

That said, "full stop forever" is probably not the best default
behavior.

But the, "in match", USB problem has been shown through pretty extensive
testing to be largely solvable with proper strain relief. Unfortunately
that means custom 3d printed parts and some teams don't have
the ability to print parts. A nice low cost retail solution might be
something right up Andymark's alley.

FIRST's official position on the USB discovery problem on startup,
enumerated elsewhere in this thread, is to wait 30 seconds after powering
off the PDU for capacitors to discharge. Might be useful to take
a survey of teams when experiencing this problem as to whether or
not a timed 30 second power off is solving the problem. My team
has not seen the problem since we discovered this recommendation.

[dlevy]

I think the 30 second rule ( some say 35) applies to the time needed to wait after powering down the PDM for any reason. But I do recall that powering down the PDM had been an early recommendation to resolve the USB problem. I've never had to perform that for any of the robots I've triaged in training or competition. Usually a simple restart does the trick. If not I'll swipe close the app and restart. It's difficult to remember though because as you say "strain relief" is often required. I'll add to that - replacing modern robotics usb cables with the one's from Monoprice will have immediate results. http://www.monoprice.com/product?c_i...seq=1&format=2

[DanOelke]

But the, "in match", USB problem has been shown through pretty extensive
testing to be largely solvable with proper strain relief.

From what I've seen at matches - yes proper strain relief helps - but it does not solve the problem. I've seen robots with excellent strain relief stop mid match.

FIRST's official position on the USB discovery problem on startup,
enumerated elsewhere in this thread, is to wait 30 seconds after powering
off the PDU for capacitors to discharge.

Again - my experience working with many teams is that this helps but doesn't solve all the problems.

[ViperMentor]

From what I've seen at matches - yes proper strain relief helps - but it does not solve the problem. I've seen robots with excellent strain relief stop mid match.

Agreed, I have seen the same thing. The related issue is mechanical protection. I have seen robots with very poor "armor". The USB cable and sockets are not able to withstand any abuse, regardless of how good your strain relief is, or where you buy your cables from. One tumble off the mountain onto exposed cables, motor or servo controllers and either the cable or the socket or both can become permanently damaged. Then all it takes is some small mechanical vibration (again, doesn't matter how good your strain relief is) and you will lose connection. The team was able to diagnose this is by systematically tapping gently on every USB connector until finding the one that caused the robot controller to disconnect. In this case it was the cable that was bad but I have seen other cases where it was the socket on the MR module.

[dlevy]

"The USB cable and sockets are not able to withstand any abuse, regardless of how good your strain relief is, or where you buy your cables from."

I wouldn't discourage anyone from swapping out the MR cables with the one's from Monoprice. I've seen way too many of the former lose their springs and fail permanently.

They cost $1 a piece here: http://www.monoprice.com/product?c_i...seq=1&format=2.