New feature: high speed continuous recording

Discussions about active development projects
Post Reply
bexizuo
Posts: 9
Joined: Mon Oct 20, 2014 9:21 pm

Re: New feature: high speed continuous recording

Post by bexizuo » Tue Mar 31, 2015 9:25 am

Hi Nils,

please could you update latest binaries ? i don't have prepared environment to compile for redpitaya :-(

Thank You

Kind regards

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: New feature: high speed continuous recording

Post by Nils Roos » Thu Apr 02, 2015 12:15 am

I've attached a zip with updated components. They have been tested with the latest ecosystem available from redpitaya.com (Downloads -> SD Image).
  • copy to /tmp and unzip
  • "chmod 755 /tmp/preview_rp_remote_acquire"
  • "cat /tmp/ddrdump.bin >/dev/xdevcfg"
  • "insmod /tmp/rpad.ko"
  • "/tmp/preview_rp_remote_acquire -h" (lists usage instructions)
The remote acquire tool will stream data from either ADC channel to a socket or file in binary format.
With UDP, the highest rate is 30MSps (decimation factor 4) with the full 14bit resolution (expanded to short).

Have fun ;O)
Nils
You do not have the required permissions to view the files attached to this post.
All in a day's work for Bicycle Repair Man

Aragorn2101
Posts: 7
Joined: Sun Mar 22, 2015 10:54 am

Re: New feature: high speed continuous recording

Post by Aragorn2101 » Thu Apr 02, 2015 7:33 am

Hi Nils,

Thanks a lot for the update. I'm going to test it as soon as I get to the lab.

Just one question: what do you mean by "(expanded to short)"?

noah
Posts: 33
Joined: Fri Jan 23, 2015 9:53 am
Location: Zurich

Re: New feature: high speed continuous recording

Post by noah » Thu Apr 02, 2015 7:51 am

@Aragorn2101
Aragorn2101 wrote:what do you mean by "(expanded to short)"?
I think he means that the 14 bit are extended to a 16 bit short integer.

Will test it today, thanks for the update!

noah
Posts: 33
Joined: Fri Jan 23, 2015 9:53 am
Location: Zurich

Re: New feature: high speed continuous recording

Post by noah » Thu Apr 02, 2015 3:57 pm

So I had my attempt with your new 'preview' today and am quite astonished!! I'll briefly share my results.

I wrote a little C program, that acts as a TCP client and listens to the incomming connections from the Redpitaya. All you have to do is to start the Server on the Redpitaya with the command like this (assuming you already loaded the new FPGA image and kernel module):

Code: Select all

./preview_rp_remote_acquire -p 1234 -m 2  -r -c 0 -d 8 -k 4096
This will start the Server and each time you connect a client, it will sample 4096K Samples from channel 0 (A) with 15.6 MS/s and than transfer them to the client. So to get those samples run on the client computer:

Code: Select all

./ddrdump_rx 4096
To create a socket, listen to the redpitaya and receive 4096KB of data. It will than write the values in mV to a file in /tmp/out.txt which you can plot using the supplied Matlab script.
Important when using this template is, that you always 'listen' (the parameter after ./ddrdump_rx) to as many bytes as you send on the Redpitaya (the -k parameter).

My test setup: Redpitaya (192.168.0.100) and Macbook Pro (192.168.0.101) running Ubuntu connected via a Gigabit switch.

Don't know if it helps anybody, but i thought i'd share my work 8-) source is attached.

TODO:
  • Transmit over UDP to get less overhead
  • Cyclic transmission without data loss
  • Data loss detection if Sample rate is too high for the transmission
Have a great easter!
Cheers, Noah
You do not have the required permissions to view the files attached to this post.

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: New feature: high speed continuous recording

Post by Nils Roos » Fri Apr 03, 2015 5:19 pm

Hey Noah, thanks for giving it a go and putting some code up yourself !
noah wrote:I think he means that the 14 bit are extended to a 16 bit short integer.
Spot on.
A further note: The samples are captured and averaged MSB-aligned, so the averaged values at decimation >= 4 are actually 16bit values. Thus you can benefit from the additional resolution that averaging yields.

Some further comments on the transfer size parameter: If you omit the -k parameter or supply a value of 0 (the default), the data transfer will continue indefinitely - until the connection breaks or remote-acquire is interrupted with ctrl-c. The -k parameter was primarily intended for use with mode 3 (write to file).
All in a day's work for Bicycle Repair Man

noah
Posts: 33
Joined: Fri Jan 23, 2015 9:53 am
Location: Zurich

Re: New feature: high speed continuous recording

Post by noah » Fri Apr 03, 2015 6:03 pm

Nils Roos wrote:A further note: The samples are captured and averaged MSB-aligned, so the averaged values at decimation >= 4 are actually 16bit values. Thus you can benefit from the additional resolution that averaging yields.
Yes i figuered that out, thanks!

I have another question: The transfer is too slow for the highest few sampling rates. Is there a functionality built in your code that detects a buffer overflow? It would be really good to know if all sample are valid after receiving.

Do you have the code shared btw?

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: New feature: high speed continuous recording

Post by Nils Roos » Fri Apr 03, 2015 7:07 pm

The code for rp_remote_acquire can be found in this repository.
The modified ecosystem including the code for the kernel module is in this branch of my RP fork.

There is currently no detection of a buffer overflow condition. If the data cannot be sent before new samples are written to unsent buffer-space, the old samples will simply be lost. It is possible to detect this in the transmitter, but we had no good idea for a suitable reaction to that condition. Any suggestions ?
All in a day's work for Bicycle Repair Man

fisafisa
Posts: 29
Joined: Fri Jun 06, 2014 6:02 pm

Re: New feature: high speed continuous recording

Post by fisafisa » Fri Apr 03, 2015 10:38 pm

I would keep the source as simple and self consistent as possible.
Try to shift as much processing as possible downstream on the receiving server.
Said that I would:
1) time stamp (relative) each packet sent so that any hole can be detected at the receiver.
2) in case of overrun, drop data until the system is resynced to the next packet boundary.
If there is loss of data let's at least mantain a regular and predictable behaviour.
This will be expecially useful once you will be looking into sending both ADCs as it will avoid swapping channels.

I am looking forward to see the final product.

Ciao

Laurent
Posts: 18
Joined: Mon Jun 23, 2014 6:38 pm
Location: Paris

Re: New feature: high speed continuous recording

Post by Laurent » Fri Apr 17, 2015 6:01 pm

small test:

with this command:

Code: Select all

/tmp/rp_remote_acquire -m 1 -a 125.40.1.188 -p 9930 -u -c 0 -r -d 0
and a labview udp listener on the network (Int16 data), we obtain the following performances:

- transfer speed reported by rp_remote_acquire: 59.7MB/s (1-59.7/256 = 76% data lost)
- labview received 25Msamples/s (50MB/s) in packets of 16384 points digitized at 125Msamples/s (9.7/50=19% packet losses)
- 0.5% of the packets were corrupted (non-continuous data->recorded at different times).

just wanted to let you know. This still is very interesting for a number of applications. Thanks for your work.

Laurent

Post Reply
jadalnie klasyczne ekskluzywne meble wypoczynkowe do salonu ekskluzywne meble tapicerowane ekskluzywne meble do sypialni ekskluzywne meble włoskie

Who is online

Users browsing this forum: No registered users and 1 guest