We have been working on a project to use the Red Pitaya as a fast, multi-channel lock-in amplifier. The user can choose which signals to route to the output DACs (e.g. ch1 X, ch2 Y etc); however we simultaneously would like to do away with our oscilloscope hardware and stream the lock-in data directly from the Red Pitaya.
The data we are collecting naturally falls into a "block"-type acquisition (i.e. a certain number of samples each time we receive a trigger), and currently we're storing each block in an area of memory over the High Performance AXI 32b Slave Port, at offset 0x30000000. A TCP socket server running on the host linux side then reads the data using mmap on /dev/mem, at the given offset.
Unfortunately, this causes the OS to be unstable (after a variable amount of time, we start to see spontaneous segfaults etc) -- we assume that this is because we're writing to a section of memory that the OS itself is using. We have modified u-boot.scr to include memmap=64M$384M as a boot argument, and changing the offset to match in Vivado (0x1800_0000); unfortunately this only causes the OS to freeze more quickly.
Any pointers on how to solve this problem would be greatly appreciated! On the plus side, the project works very well until it locks up the OS!
Data streaming via HP AXI to DDR3
-
- Posts: 790
- Joined: Sat May 23, 2015 5:22 pm
Re: Data streaming via HP AXI to DDR3
I'm not sure if I can help but I'm curious to know what happens when you write to an address that's outside of the DDR3 RAM range. If I'm not mistaken DDR3 RAM range is 0x0 - 0x20000000 and 0x30000000 is outside this range.jlarksman wrote:we're storing each block in an area of memory over the High Performance AXI 32b Slave Port, at offset 0x30000000.
Here is a short description of how I do something similar in my projects:
- I limit the amount of RAM accessible to OS by applying patches to devicetree and to u-boot. For example, if you need to limit OS to the first 256 MB of RAM then memory/reg should be set to <0x0 0x10000000> and fdt_high should be set to 0x10000000.
- then I use the value from the previous step as the starting address for my direct memory access from FPGA.
-
- Posts: 4
- Joined: Fri Nov 11, 2016 10:08 am
Re: Data streaming via HP AXI to DDR3
Dear pavel,
Thank you for your quick reply! In our experience, writing outside the 0x0 -> 0x2 range of the DDR3 results in an unstable OS, but somehow the FPGA and host OS are able to communicate. We did wonder if it wrapped around to somewhere else in the memory space? Nevertheless, data we wrote to that address with the FPGA would reliably read on the host using mmap of /dev/mem at the same offset!
I have made the changes that you have suggested:
Thank you for your quick reply! In our experience, writing outside the 0x0 -> 0x2 range of the DDR3 results in an unstable OS, but somehow the FPGA and host OS are able to communicate. We did wonder if it wrapped around to somewhere else in the memory space? Nevertheless, data we wrote to that address with the FPGA would reliably read on the host using mmap of /dev/mem at the same offset!
I have made the changes that you have suggested:
- modified u-boot.script, run
Code: Select all
mkimage -A arm -T script -C none -d ./u-boot.script ./u-boot.scr
- used dtc to get the devicetree file, and modified reg from to
Code: Select all
<0x0 0x20000000>
Code: Select all
<0x0 0x18000000>
-
- Posts: 790
- Joined: Sat May 23, 2015 5:22 pm
Re: Data streaming via HP AXI to DDR3
I'd suggest to try just three modifications:
- set high and/or fdt_high to 0x08000000 in u-boot.script:
https://github.com/RedPitaya/RedPitaya/ ... script#L15 - set reg to <0x0 0x08000000> in devicetree
- use 0x08000000 as DMA offset in your FPGA configuration
-
- Posts: 4
- Joined: Fri Nov 11, 2016 10:08 am
Re: Data streaming via HP AXI to DDR3
Dear pavel,
Thanks for your continued help. I have tried as you suggested, but to no avail! I've tried it on clean versions of both the stable and beta OS, with the same outcome each time -- kernel panic.
I'd like to isolate whether the problem is on the linux end, or the FPGA end. When I make the modifications to devicetree.dtb and u-boot.scr, linux doesn't report any changes in memory tables. If I run I get the following near the top:
and if I run I get
Have you ever noticed whether these two outputs change when the memory restriction is working correctly? I'm surprised that lowmem is still reporting 480 MB after the change, but possibly I'm looking in the wrong place?
Thanks again!
Thanks for your continued help. I have tried as you suggested, but to no avail! I've tried it on clean versions of both the stable and beta OS, with the same outcome each time -- kernel panic.
I'd like to isolate whether the problem is on the linux end, or the FPGA end. When I make the modifications to devicetree.dtb and u-boot.scr, linux doesn't report any changes in memory tables. If I run
Code: Select all
dmesg | less
Code: Select all
[ 0.000000] Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
vmalloc : 0xde800000 - 0xff800000 ( 528 MB)
lowmem : 0xc0000000 - 0xde000000 ( 480 MB)
pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
.text : 0xc0008000 - 0xc0841544 (8422 kB)
.init : 0xc0842000 - 0xc0886000 ( 272 kB)
.data : 0xc0886000 - 0xc08e99c0 ( 399 kB)
.bss : 0xc08e99c0 - 0xc092a014 ( 258 kB)
Code: Select all
head /proc/meminfo
Code: Select all
MemTotal: 445168 kB
MemFree: 378064 kB
MemAvailable: 412872 kB
Thanks again!
-
- Posts: 4
- Joined: Fri Nov 11, 2016 10:08 am
Re: Data streaming via HP AXI to DDR3
Just to follow up, I have solved this by appending to my bootargs in u-boot.scr. I am unsure why the other approaches didn't work. However, now /proc/iomem, /proc/meminfo and dmesg all report the change in memory size, and my segfaults / kernel panics have disappeared.
Thanks for all the help!
Code: Select all
mem=384M
Thanks for all the help!
jadalnie klasyczne ekskluzywne meble wypoczynkowe do salonu ekskluzywne meble tapicerowane ekskluzywne meble do sypialni ekskluzywne meble włoskie
Who is online
Users browsing this forum: No registered users and 17 guests