Hi guys!
I am developing simple app to continous acquisition using RP API, but I am wondering how to improve its speed.
https://github.com/mateuszjaskula/RedPi ... pp-solaris
Currently I am able only to save without losing any samples with decimation set to 1024 (copying + saving takes about 25ms a bit too long for me).
What I want to achieve is saving with decimation 64.
Do you have any ideas?
Thanks!
Continous acquisition on OS 0.94
-
- Posts: 1441
- Joined: Sat Jun 07, 2014 12:49 pm
- Location: Königswinter
Re: Continous acquisition on OS 0.94
There are several factors that limit the speed in your application:
If the speed is sufficient, you could try to save the binary data and do the conversion later.
- The librp functions use the 16k sample buffers. Access to these buffers is rather slow, at about 4MB/s (estimated)
- Even though the function is called rp_AcqGetOldestDataRaw() it does some processing on the data, limiting the throughput even further
- Converting the samples to text before saving takes time and increases the amount of bytes to write to file
If the speed is sufficient, you could try to save the binary data and do the conversion later.
-
- Posts: 5
- Joined: Thu Sep 25, 2014 10:00 am
Re: Continous acquisition on OS 0.94
Thanks Nils for very fast response!
Actually calling rp_GetOldestDataRaw() takes ~3,6ms, buffer length on decimation 64 should be ~8,3ms.
I know that I'm making big overhead using text data, but I want to collect timestamped data.
Do you know how /tmp/ram is mounted? Because I don't see any difference between them in saving whole buffer (~22ms without timestamps and ~72ms with).
Actually calling rp_GetOldestDataRaw() takes ~3,6ms, buffer length on decimation 64 should be ~8,3ms.
I know that I'm making big overhead using text data, but I want to collect timestamped data.
Do you know how /tmp/ram is mounted? Because I don't see any difference between them in saving whole buffer (~22ms without timestamps and ~72ms with).
-
- Posts: 1441
- Joined: Sat Jun 07, 2014 12:49 pm
- Location: Königswinter
Re: Continous acquisition on OS 0.94
I would guess that you are seeing no difference because for sd-card the data is first deposited in a ram-based cache and only later committed to the medium, when the fprintf() calls have long finished.Do you know how /tmp/ram is mounted? Because I don't see any difference between them in saving whole buffer (~22ms without timestamps and ~72ms with).
-
- Posts: 5
- Joined: Thu Sep 25, 2014 10:00 am
Re: Continous acquisition on OS 0.94
Yeah, any ideas how improve this I/O performance?
I have tried to force fsync() after each full buffer write to file, but it made it even longer, only improvement is that with fsync there is no peak jumps to ~1sec in saving data, but it tooks regularly over 134ms, so some data is lost.
I have tried to force fsync() after each full buffer write to file, but it made it even longer, only improvement is that with fsync there is no peak jumps to ~1sec in saving data, but it tooks regularly over 134ms, so some data is lost.
-
- Posts: 1441
- Joined: Sat Jun 07, 2014 12:49 pm
- Location: Königswinter
Re: Continous acquisition on OS 0.94
Well, for starters, let's do a bit of research. Wikipedia tells us that the guaranteed sequential write speed for a sdhc class 10 card is 10MB/s, but that real-world performance - eg for anything that is not bulk data to a single file - can also be significantly lower. I'd say that your situation is quite close to the ideal case as long as you don't do the fsync(), so let's go with 10MB/s.
2MS/s on one channel converted to text with timestamps equates to approx. 30MB/s . It seems that saving text to a file on the sd-card is not going to work at your target rate, no matter what you try.
As an alternative I would propose a usb-drive or a network share. Here is a thread that discusses this in some detail.
But that still does not help with the overhead of the text conversion. As a next step, you could try what the speed is if you do sprintf() instead of fprintf(), to isolate the conversion effect and get a feeling of its impact.
2MS/s on one channel converted to text with timestamps equates to approx. 30MB/s . It seems that saving text to a file on the sd-card is not going to work at your target rate, no matter what you try.
As an alternative I would propose a usb-drive or a network share. Here is a thread that discusses this in some detail.
But that still does not help with the overhead of the text conversion. As a next step, you could try what the speed is if you do sprintf() instead of fprintf(), to isolate the conversion effect and get a feeling of its impact.
jadalnie klasyczne ekskluzywne meble wypoczynkowe do salonu ekskluzywne meble tapicerowane ekskluzywne meble do sypialni ekskluzywne meble włoskie
Who is online
Users browsing this forum: No registered users and 3 guests