High Data rate File Sink dropped data

I am trying to record data directly from a UDP stream to file. I have
four
channels at 1.2MHz sample rate the code is below. Basically it opens a
UDP
source and connects to a file sink. However, It is losing data when
writing
to the file. If I stream a signal with a one second burst the spacing is
not
correct. Also, I have another c++ program that reads the UDP packets and
records directly to file, and there are no lost UDP packets nor lost
data
with this program. Also I have checked that the UDP packet size is less
than
1472, it is currently at 1024 bytes.

Is there a way to see if file_sink is not able to handle that data rate,
or
to see if the UDP source cannot keep up?

#!/usr/bin/env python

from gnuradio import gr
from gnuradio.wxgui import waterfallsink2, stdgui2, fftsink2,scopesink2
import wx
from optparse import OptionParser
from gnuradio.qtgui import qtgui
import os
import time
import thread
import threading

class gnuradioGUI(stdgui2.std_top_block):
def init(self,frame,panel,vbox,argv):
stdgui2.std_top_block.init(self,frame,panel,vbox,argv)

            self.sr = 1200000
            self.channels = 4
            self.FileLength = 5 #seconds of data in each file
            self.filenum = -1
            self.baseName = "udp_1k."
            self.file_increment()
            self.ipAdd = "192.168.100.178" #the ip of this computer
            self.port = 3456
            self.bufSize = self.channels*4096


            self.signal =

gr.udp_source(gr.sizeof_short,self.ipAdd,self.port)

            self.Update()

            self.thread = threading.Thread(target=self.FileThread)
            self.thread.start()

    def Update(self):
            self.lock()
            try:
                    self.disconnect(self.signal,self.dest)
            except:
                    print "could not disconnect"
            self.file_increment()
            self.dest = gr.file_sink(gr.sizeof_short,self.saveFile)
            self.connect(self.signal,self.dest)
            self.unlock()



    def FileThread(self):
            while 1:
                    time.sleep(self.FileLength)
                    self.Update()

    def file_increment(self):
            self.filenum = self.filenum + 1
            self.saveFile = self.baseName + "%d"%self.filenum
            print "writing to file:  " + self.saveFile

if name == ‘main’:
app = stdgui2.stdapp(gnuradioGUI,“CAIS signal using GNU Radio
GUI”)

    parser = OptionParser()
    parser.add_option("-w", "--file", dest="filename",
                    help="write data to FILE", metavar= "FILE")
    (options, args) = parser.parse_args()

    app.MainLoop()

    os._exit(1)

CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,
is for the sole use of the intended recipient(s) and may contain
confidential and privileged information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply e-mail and destroy all
copies of the original message.

On Mon, Aug 1, 2011 at 10:24 AM, Jason T. [email protected]
wrote:

to see if the UDP source cannot keep up?
It’s most likely that your disk is not able to keep up. I’ve used GNU
Radio’s file sink to save gigabytes of data at 25 MHz float I&Q, but I
used a RAID5 device to do it. You probably need more spindles.

Tom

On 08/02/2011 08:11 PM, Tom R. wrote:

Is there a way to see if file_sink is not able to handle that data rate, or
to see if the UDP source cannot keep up?

It’s most likely that your disk is not able to keep up. I’ve used GNU
Radio’s file sink to save gigabytes of data at 25 MHz float I&Q, but I
used a RAID5 device to do it. You probably need more spindles.

Tom

Tom:

It’s not strictly disk-limited, since Jason observes that with a non-Gnu
Radio program, he can record direct to disk from
the raw UDP stream, and not lose anything.


Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium

On Tue, Aug 2, 2011 at 8:19 PM, Marcus D. Leech [email protected]
wrote:

to the file. If I stream a signal with a one second burst the spacing is

Radio program, he can record direct to disk from
the raw UDP stream, and not lose anything.

Ah, I misread that. In my mind, he said something else… Still, I’ve
never had an issue with the file sinks. More data is required.

Tom