Crashes after running scope+gen a few minutes

Just about everything about Red Pitaya
Post Reply
David Lee
Posts: 7
Joined: Sat Feb 21, 2015 12:35 am

Crashes after running scope+gen a few minutes

Post by David Lee » Sat Feb 21, 2015 12:53 am

Hi,
I just got my board, downloaded the SD image, booted it up on my network and installed scope+gen.
The USB console works fine. It indicates this version:
Red Pitaya GNU/Linux/Ecosystem version 0.92-65
The scope+gen version is:
Generator & Oscilloscope 0.92-233
I run it from chrome and loop back OUT1 to IN1. This is pretty awesome!

Pretty awesome until it crashes.

The console has this message:
------------[ cut here ]------------
Kernel BUG at c0435fec [verbose debug info unavailable]
Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 0 Not tainted (3.9.0-xilinx #1)
PC is at skb_panic+0x50/0x5c
LR is at skb_panic+0x50/0x5c
pc : [<c0435fec>] lr : [<c0435fec>] psr: 60070113
sp : c05ede50 ip : 00000001 fp : 00000808
r10: dc10a6c0 r9 : 00000000 r8 : 00000000
r7 : c0535058 r6 : dc344d00 r5 : dc344d40 r4 : dc345548
r3 : 60070113 r2 : 00000102 r1 : 60070113 r0 : 0000007b
Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
and a whole bunch more stuff.

I tried upgrading, but the system won't stay up long enough to upgrade.
Here's a photo of my USB power supply:
https://drive.google.com/a/coloradosens ... sp=sharing
The heatsink is warm to the touch, as is the whole PCB.

Any ideas to reduce the lamness of my out-of-the-box experience?

Cheers,
David

David Lee
Posts: 7
Joined: Sat Feb 21, 2015 12:35 am

Re: Crashes after running scope+gen a few minutes

Post by David Lee » Sat Feb 21, 2015 1:02 am

I was able to upgrade to:

Code: Select all

Red Pitaya GNU/Linux/Ecosystem version 0.92-378
And now after running scope+gen for a couple minutes it crashes with:

Code: Select all

redpitaya> skbuff: skb_over_panic: text:c02a7448 len:13962 put:13962 head:df45db00 data:df45db40 tail:0xdf4611ca end:0xdf45e140 dev:eth0
------------[ cut here ]------------
Kernel BUG at c0420d2c [verbose debug info unavailable]
Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 0    Not tainted  (3.9.0-xilinx #1)
PC is at skb_panic+0x50/0x5c
LR is at skb_panic+0x50/0x5c
pc : [<c0420d2c>]    lr : [<c0420d2c>]    psr: 60070113
sp : c05cde50  ip : 00000001  fp : 0000368a
r10: df717f00  r9 : 00000000  r8 : 00000000
r7 : c0519c78  r6 : df45db00  r5 : df45db40  r4 : df4611ca
r3 : 60070113  r2 : 00000102  r1 : 60070113  r0 : 0000007d
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
Control: 18c5387d  Table: 1f70404a  DAC: 00000015
Process swapper/0 (pid: 0, stack limit = 0xc05cc238)
Stack: (0xc05cde50 to 0xc05ce000)
de40:                                     0000368a df45db00 df45db40 df4611ca
etc...

I'd appreciate any suggestions.

Cheers,
David

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: Crashes after running scope+gen a few minutes

Post by Nils Roos » Sat Feb 21, 2015 1:14 am

I can't access the picture of your USB power supply - no permission. If the output current rating of the PSU is below 2A, that is known to result in volatile performance and crashes.

David Lee
Posts: 7
Joined: Sat Feb 21, 2015 12:35 am

Re: Crashes after running scope+gen a few minutes

Post by David Lee » Sat Feb 21, 2015 1:14 am

So, the crashing seems to be related to network traffic.

In order to increase the packet traffic seen by the system, I connected it to a hub, not a switch, so that all local
traffic would appear at the Ethernet PHY.

I increase the traffic on this little piece of sub-net by viewing a web cam feed on another
PC that is connected to this hub.

Whammo! Immediate crash.

So, at least it's not thermal!

I'm not going to run the system like this, but crashing due to incoming packets? That's anomalous.

David

David Lee
Posts: 7
Joined: Sat Feb 21, 2015 12:35 am

Re: Crashes after running scope+gen a few minutes

Post by David Lee » Sat Feb 21, 2015 1:20 am

Here's the link to the power supply pic.
https://drive.google.com/file/d/0B2zrnn ... sp=sharing

I'm an idiot and can't seem to get the sharing permissions to work.
Basically the sticker on the power supply says 3 Amps.

I'm pretty sure the net traffic is the issue, and would be happy to perform any tests suggested.

Cheers,
David

David Lee
Posts: 7
Joined: Sat Feb 21, 2015 12:35 am

Re: Crashes after running scope+gen a few minutes

Post by David Lee » Sat Feb 21, 2015 1:25 am

Wow, the system crashes immediately after startup when there's lots of packets...

Code: Select all

++ Running all startup scripts
Starting logging: OK
Starting mdev...
Initializing random number generator... done.
Starting system message bus: done
Starting network...
ifup: interface lo already configured
Successfully initialized wpa_supplicant
rfkill: Cannot open RFKILL control device
Could not read interface wlan0 flags: No such device
WEXT: Could not set interface 'wlan0' UP
wlan0: Failed to initialize driver interface
Starting sshd: OK
Starting nginx ... done.
++ Starting Red Pitaya IP discovery
/opt/sbin/discovery version 0.92-378-d3751f9
OK
Running user script from SD card ...
Feel free to add custom initialization commands here.

Red Pitaya GNU/Linux/Ecosystem version 0.92-378

redpitaya> skbuff: skb_over_panic: text:c02a7448 len:2050 put:2050 head:df5eb800 data:df5eb840 tail:0xdf5ec042 end:0xdf5ebe40 dev:eth0
------------[ cut here ]------------
Kernel BUG at c0420d2c [verbose debug info unavailable]
Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 0    Not tainted  (3.9.0-xilinx #1)
PC is at skb_panic+0x50/0x5c
LR is at skb_panic+0x50/0x5c
pc : [<c0420d2c>]    lr : [<c0420d2c>]    psr: 600d0113
sp : c05cde50  ip : 00000001  fp : 00000802
r10: dc7f7c00  r9 : 00000000  r8 : 00000000
r7 : c0519c78  r6 : df5eb800  r5 : df5eb840  r4 : df5ec042
r3 : 600d0113  r2 : 00000102  r1 : 600d0113  r0 : 0000007b
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
Control: 18c5387d  Table: 1f4ec04a  DAC: 00000015
Process swapper/0 (pid: 0, stack limit = 0xc05cc238)
Stack: (0xc05cde50 to 0xc05ce000)
Pretty fragile system.

Cheers,
David

David Lee
Posts: 7
Joined: Sat Feb 21, 2015 12:35 am

Re: Crashes after running scope+gen a few minutes

Post by David Lee » Sat Feb 28, 2015 4:58 pm

So, does anyone have any ideas on the root cause of the crash?

A little help?

Cheers,
David

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: Crashes after running scope+gen a few minutes

Post by Nils Roos » Mon Mar 02, 2015 11:29 am

Hi David,
The traces you provided indicate that the kernel is trying to expand a socket buffer beyond its capacity. This could be related to frames with a payload in excess of 1500 bytes (so-called jumbo frames).

Could you check if the network adapter that is generating the traffic is using jumbo frames ? And if so, disable jumbo frames and try again ?

The ZYNQ's on-chip ethernet controller does not support jumbo frames and the Red Pitaya's kernel is not configured for them, and there have been some reports of kernel panic in that scenario, so it might be worth looking into.

David Lee
Posts: 7
Joined: Sat Feb 21, 2015 12:35 am

Re: Crashes after running scope+gen a few minutes

Post by David Lee » Mon Mar 02, 2015 3:56 pm

Thanks for the reply Nils,

After working with the red pitaya for a while I believe we will chuck it into the box along with the other unused evaluation boards.

We had hoped to use the RP to learn Zynq, Vivado, FPGA, but most of all, analog I/O best practices. We are not so much interested in Linux, JavaScript, and web instruments.

The absence of a complete hardware schematic sealed its fate. I think we'll look at the offerings from Digilent.

Cheers,
David

Post Reply
jadalnie klasyczne ekskluzywne meble wypoczynkowe do salonu ekskluzywne meble tapicerowane ekskluzywne meble do sypialni ekskluzywne meble włoskie

Who is online

Users browsing this forum: No registered users and 20 guests