Page 1 of 1
Continous acquisition on OS 0.94
Posted: Wed Dec 02, 2015 9:03 pm
by jskl
Hi guys!
I am developing simple app to continous acquisition using RP API, but I am wondering how to improve its speed.
https://github.com/mateuszjaskula/RedPi ... pp-solaris
Currently I am able only to save without losing any samples with decimation set to 1024 (copying + saving takes about 25ms a bit too long for me).
What I want to achieve is saving with decimation 64.
Do you have any ideas?
Thanks!
Re: Continous acquisition on OS 0.94
Posted: Wed Dec 02, 2015 10:00 pm
by Nils Roos
There are several factors that limit the speed in your application:
- The librp functions use the 16k sample buffers. Access to these buffers is rather slow, at about 4MB/s (estimated)
- Even though the function is called rp_AcqGetOldestDataRaw() it does some processing on the data, limiting the throughput even further
- Converting the samples to text before saving takes time and increases the amount of bytes to write to file
I suggest you first measure the time it takes to just use the rp_AcqGetOldestDataRaw() function, without doing anything with the data. If that is not fast enough for you, you will need to do without the RP-API.
If the speed is sufficient, you could try to save the binary data and do the conversion later.
Re: Continous acquisition on OS 0.94
Posted: Wed Dec 02, 2015 10:26 pm
by jskl
Thanks Nils for very fast response!
Actually calling rp_GetOldestDataRaw() takes ~3,6ms, buffer length on decimation 64 should be ~8,3ms.
I know that I'm making big overhead using text data, but I want to collect timestamped data.
Do you know how /tmp/ram is mounted? Because I don't see any difference between them in saving whole buffer (~22ms without timestamps and ~72ms with).
Re: Continous acquisition on OS 0.94
Posted: Wed Dec 02, 2015 11:03 pm
by Nils Roos
Do you know how /tmp/ram is mounted? Because I don't see any difference between them in saving whole buffer (~22ms without timestamps and ~72ms with).
I would guess that you are seeing no difference because for sd-card the data is first deposited in a ram-based cache and only later committed to the medium, when the fprintf() calls have long finished.
Re: Continous acquisition on OS 0.94
Posted: Thu Dec 03, 2015 12:58 pm
by jskl
Yeah, any ideas how improve this I/O performance?
I have tried to force fsync() after each full buffer write to file, but it made it even longer, only improvement is that with fsync there is no peak jumps to ~1sec in saving data, but it tooks regularly over 134ms, so some data is lost.
Re: Continous acquisition on OS 0.94
Posted: Thu Dec 03, 2015 4:12 pm
by Nils Roos
Well, for starters, let's do a bit of research.
Wikipedia tells us that the guaranteed sequential write speed for a sdhc class 10 card is 10MB/s, but that real-world performance - eg for anything that is not bulk data to a single file - can also be significantly lower. I'd say that your situation is quite close to the ideal case as long as you don't do the fsync(), so let's go with 10MB/s.
2MS/s on one channel converted to text with timestamps equates to approx. 30MB/s . It seems that saving text to a file on the sd-card is not going to work at your target rate, no matter what you try.
As an alternative I would propose a usb-drive or a network share.
Here is a thread that discusses this in some detail.
But that still does not help with the overhead of the text conversion. As a next step, you could try what the speed is if you do sprintf() instead of fprintf(), to isolate the conversion effect and get a feeling of its impact.