New feature: high speed continuous recording

Discussions about active development projects
Post Reply
Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: New feature: high speed continuous recording

Post by Nils Roos » Fri Apr 17, 2015 7:21 pm

Hi,
Thanks for trying it out !
What you observed is consistent with the performance I had in my test-setup. Transmission cannot keep up with acquisition beyond 1/4th of 125MSps - ie. decimation factor -d 4 . I would attribute the corrupted packets to buffers being written to in mid-transmission. This should not occur as long as you keep the decimation factor at or above 4.
I plan to rewrite the FPGA code this weekend to fill the external RAM via AXI_ACP, thus maintaining cache coherence. I had observed much better performance from cached memory, but had to disable cacheing on the buffers because of the missing coherency over AXI_HP. It will share a bus with the ARM cores then, so I am not entirely confident that this experiment will end up successful.

@fisafisa:
I propose to tag each 2044 samples with the sequence 0x7fff, 0x8000, TSh, TSl for channel A and 0x8000, 0x7fff, TSh, TSl for channel B where TSh and TSl denote high and low 16bit of the timestamp; I expect the tagging combinations of full scale samples not to occur 'naturally' in sampled signals.
I think tagging is necessary because the transfer operates too far down (up?) the network stack to accurately predict packet boundaries.

Laurent
Posts: 18
Joined: Mon Jun 23, 2014 6:38 pm
Location: Paris

Re: New feature: high speed continuous recording

Post by Laurent » Mon Apr 20, 2015 6:48 pm

I got an error when I try to run in server mode:

Code: Select all

'/tmp/rp_remote_acquire -m 2 -p 9930 -u -c 0 -r -d 0'
it says:

Code: Select all

listen failed, Operation not supported
any clue?

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: New feature: high speed continuous recording

Post by Nils Roos » Mon Apr 20, 2015 7:14 pm

Laurent wrote:any clue?
:idea: Yes

Now that you bring it up, I guess server mode does not make a lot of sense for UDP.

I think we should disallow the tool to be configured that way.

Laurent
Posts: 18
Joined: Mon Jun 23, 2014 6:38 pm
Location: Paris

Re: New feature: high speed continuous recording

Post by Laurent » Tue Apr 21, 2015 7:11 pm

I'm not so sure it is useless:

udp transfer can be used both ways, whatever the server or the client is.

When RP is a client, the target computer is a server and must listen for the RP to connect and send data. RP must know the target IP

If RP could be a server, it would wait for a client to connect and send the data to the client. No need for RP to know the target computer IP.

this makes more sense, doesn't it?

fisafisa
Posts: 29
Joined: Fri Jun 06, 2014 6:02 pm

Re: New feature: high speed continuous recording

Post by fisafisa » Thu Apr 23, 2015 8:15 am

@fisafisa:
I propose to tag each 2044 samples with the sequence 0x7fff, 0x8000, TSh, TSl for channel A and 0x8000, 0x7fff, TSh, TSl for channel B where TSh and TSl denote high and low 16bit of the timestamp; I expect the tagging combinations of full scale samples not to occur 'naturally' in sampled signals.
I think tagging is necessary because the transfer operates too far down (up?) the network stack to accurately predict packet boundaries.

Hi!
I am trying to understand the problem from your statement?
The risk is that the timestamp gets mixed in with the data?
With udp it should not be a problem.
With udp you can send individual packets without fragmentation and be sure of the packet content.
You need to make sure that the packet-size + header is less than MTU (typically 1500)
Also, if switches are involved, one needs to make sure fragmentation does not happen later. DF bit set in header. (but this I have never tried) If receiver or switch cannot handle packets that size, data will not be accepted.
"IPv4 and IPv6 define a minimum reassembly buffer size, the minimum datagram size that we are guaranteed any implementation must support. For IPv4, this is 576 bytes." (http://www.masterraghu.com/subjects/np/ ... sec11.html)
Large datagrams are also possible up 65507bytes if MTU is set to 65535 or larger.
Having said that, the fact that the detination commits an error in the reassembling order would be strange...
In your case you are sending 2048 * 4 bytes, I guess, so unless MTU is set larger, about 11 datagrams would be sent.
Maybe it would be better to either reduce packet size to 150 samples or to increase MTU?
In any case, all of this is speculation as I do not understand the problem :D
For TCP is all much worse, as there are no boundaries in the communication...

Ciao
PS consider also UDP multicasting as an option. This would allow sharing data among several users on the network.

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: New feature: high speed continuous recording

Post by Nils Roos » Thu Apr 23, 2015 10:57 pm

fisafisa wrote:The risk is that the timestamp gets mixed in with the data?
It seems you understood my concerns quite well.

I wasn't aware that UDP packets below MTU size are guaranteed not to suffer fragmentation at source - and packets marked with DF wont be fragmented during transit.
OK, let's say I put a timestamp on every packet of 250 samples, to stay below the minimum guaranteed packet buffer size - I don't like to use the DF option.
I choose 250 because it allows better translation to 125MHz / 2^x than eg 256 samples, and also because it is convenient from the FPGA implementation POV.

Any objections ?

User avatar
redpitaya
Site Admin
Posts: 880
Joined: Wed Mar 26, 2014 7:04 pm

Re: New feature: high speed continuous recording

Post by redpitaya » Fri Apr 24, 2015 4:17 pm

Hi, guys

Sorry that we are not more active on the forum and hacking but
we have a lot of work with upcoming stuff.

Great work.
And stay RED


Regards

fisafisa
Posts: 29
Joined: Fri Jun 06, 2014 6:02 pm

Re: New feature: high speed continuous recording

Post by fisafisa » Fri Apr 24, 2015 11:22 pm

no objections.
Are you going to release the final version as an instrument in the bazaar?
In fact, is the bazaar used at all anymore?

Filippo

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: New feature: high speed continuous recording

Post by Nils Roos » Mon Apr 27, 2015 8:20 pm

The final application will also be submitted to the bazaar and will contain the web-app frontend as well as the command line utility.
As to whether the bazaar is still used, all I can say is that I use it from time to time.

On to other things:
Timestamping

I have next to no practical experience with the topic, so I'll lay my plans out here and hope somebody more knowledgeable in that matter will put forth an opinion.

I am running a 64bit counter with the ADC clock. The value of that counter is stored at the instant the first sample in a block of 250 is acquired, and will later be transmitted together with those samples.
The counter can be reset to a preloadable value with an external reset signal. Together with the option of using an external ADC clock, these measures should enable syncing the acquisition of a group of Red Pitayas: 1. preload the reset value to a common value, 2. give the external reset to the group.

fisafisa
Posts: 29
Joined: Fri Jun 06, 2014 6:02 pm

Re: New feature: high speed continuous recording

Post by fisafisa » Tue Apr 28, 2015 9:21 pm

125Mhz 1.25E8
64 bit 1.6E19
running time is E11 seconds - very good!

How do you do the external trigger? one of the RPI I/O pins?
My bet is that the electrival characteristics of those pins will not allow for a very sharp edge to the FPGA???
I guess reaching a 1us precision in the 0 time stamping should be feasible
That would be good enough for many applications.

As for syncing multiple RPIs, there will be anumbetr of samples of skew due to different propagation times.
Hopefully that would be constant so it would be possible to calibrate the system.

It all sound good to me!

Ciao
F

Post Reply
jadalnie klasyczne ekskluzywne meble wypoczynkowe do salonu ekskluzywne meble tapicerowane ekskluzywne meble do sypialni ekskluzywne meble włoskie

Who is online

Users browsing this forum: No registered users and 29 guests