Continous acquisition on OS 0.94

Discussions about active development projects
Post Reply
jskl
Posts: 5
Joined: Thu Sep 25, 2014 10:00 am

Continous acquisition on OS 0.94

Post by jskl » Wed Dec 02, 2015 9:03 pm

Hi guys!

I am developing simple app to continous acquisition using RP API, but I am wondering how to improve its speed.
https://github.com/mateuszjaskula/RedPi ... pp-solaris
Currently I am able only to save without losing any samples with decimation set to 1024 (copying + saving takes about 25ms a bit too long for me).
What I want to achieve is saving with decimation 64.

Do you have any ideas?

Thanks!

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: Continous acquisition on OS 0.94

Post by Nils Roos » Wed Dec 02, 2015 10:00 pm

There are several factors that limit the speed in your application:
  1. The librp functions use the 16k sample buffers. Access to these buffers is rather slow, at about 4MB/s (estimated)
  2. Even though the function is called rp_AcqGetOldestDataRaw() it does some processing on the data, limiting the throughput even further
  3. Converting the samples to text before saving takes time and increases the amount of bytes to write to file
I suggest you first measure the time it takes to just use the rp_AcqGetOldestDataRaw() function, without doing anything with the data. If that is not fast enough for you, you will need to do without the RP-API.
If the speed is sufficient, you could try to save the binary data and do the conversion later.
All in a day's work for Bicycle Repair Man

jskl
Posts: 5
Joined: Thu Sep 25, 2014 10:00 am

Re: Continous acquisition on OS 0.94

Post by jskl » Wed Dec 02, 2015 10:26 pm

Thanks Nils for very fast response!

Actually calling rp_GetOldestDataRaw() takes ~3,6ms, buffer length on decimation 64 should be ~8,3ms.
I know that I'm making big overhead using text data, but I want to collect timestamped data.
Do you know how /tmp/ram is mounted? Because I don't see any difference between them in saving whole buffer (~22ms without timestamps and ~72ms with).

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: Continous acquisition on OS 0.94

Post by Nils Roos » Wed Dec 02, 2015 11:03 pm

Do you know how /tmp/ram is mounted? Because I don't see any difference between them in saving whole buffer (~22ms without timestamps and ~72ms with).
I would guess that you are seeing no difference because for sd-card the data is first deposited in a ram-based cache and only later committed to the medium, when the fprintf() calls have long finished.
All in a day's work for Bicycle Repair Man

jskl
Posts: 5
Joined: Thu Sep 25, 2014 10:00 am

Re: Continous acquisition on OS 0.94

Post by jskl » Thu Dec 03, 2015 12:58 pm

Yeah, any ideas how improve this I/O performance?
I have tried to force fsync() after each full buffer write to file, but it made it even longer, only improvement is that with fsync there is no peak jumps to ~1sec in saving data, but it tooks regularly over 134ms, so some data is lost.

Nils Roos
Posts: 1441
Joined: Sat Jun 07, 2014 12:49 pm
Location: Königswinter

Re: Continous acquisition on OS 0.94

Post by Nils Roos » Thu Dec 03, 2015 4:12 pm

Well, for starters, let's do a bit of research. Wikipedia tells us that the guaranteed sequential write speed for a sdhc class 10 card is 10MB/s, but that real-world performance - eg for anything that is not bulk data to a single file - can also be significantly lower. I'd say that your situation is quite close to the ideal case as long as you don't do the fsync(), so let's go with 10MB/s.
2MS/s on one channel converted to text with timestamps equates to approx. 30MB/s . It seems that saving text to a file on the sd-card is not going to work at your target rate, no matter what you try.

As an alternative I would propose a usb-drive or a network share. Here is a thread that discusses this in some detail.

But that still does not help with the overhead of the text conversion. As a next step, you could try what the speed is if you do sprintf() instead of fprintf(), to isolate the conversion effect and get a feeling of its impact.
All in a day's work for Bicycle Repair Man

Post Reply
jadalnie klasyczne ekskluzywne meble wypoczynkowe do salonu ekskluzywne meble tapicerowane ekskluzywne meble do sypialni ekskluzywne meble włoskie

Who is online

Users browsing this forum: No registered users and 2 guests