Ensuring all the data gets received through UDP socket in Python -

possibly related questions seem close don't describe issue understand it:

• reading data udp socket
• python udp socket semi-randomly failing receive
• python raw socket listening udp packets; half of packets received

problem

long file sent line line doesn't go way through udp on loopback.

long story

i have long file consisting of lines , breaks identical program on udp locally. let me emphasize the program sending packets on udp (there no choice here), , cannot feasibly modified process ack requests etc. while sending.

it looks (this tail):

stimulustime 56398  signal(0,2) -79.5457  signal(0,4) -81.7426  signal(0,6) -83.9978  signal(0,9) -63.3755  signal(0,11) -15.6045  signal(0,13) 31.1299  signal(0,16) 75.7539  signal(0,18) 98.301  signal(0,20) 98.301  signal(0,22) 48.4546  signal(0,25) 3.73159  signal(0,27) -49.9798  signal(0,29) -77.8449  signal(1,0) -22.0332  signal(1,2) -60.6384  signal(1,4) -98.0858  signal(1,6) -86.4579  signal(1,9) -68.9173  signal(1,11) -31.5552  signal(1,13) 35.2906  signal(1,16) 77.0686  signal(1,18) 96.3836  signal(1,20) 95.7246  signal(1,23) 25.6074  signal(1,25) -20.2101  signal(1,27) -60.2933  signal(1,29) -83.8169  signal(2,0) -31.8826  signal(2,2) -53.5045  signal(2,4) -89.9895  signal(2,7) -84.4503  signal(2,9) -59.7016  signal(2,11) -12.8569  signal(2,13) 28.857  signal(2,15) 58.0577  signal(2,18) 96.4222  signal(2,20) 79.783  signal(2,22) 58.6463  signal(2,25) -3.24883  signal(2,27) -45.5  signal(2,29) -88.8937  signal(3,0) -18.6625  signal(3,2) -53.3978  signal(1,16) 58.784  signal(1,17) 44.7782  signal(1,18) 6.247  signal(1,19) -12.0855  signal(1,20) -33.7644  signal(1,21) -49.4406  signal(1,22) -67.5791  signal(1,23) -92.0336  signal(1,24) -93.9841  end 

i wrote code takes file , sends line @ time on udp locally, , code receives , parses based on data type.

sender:

import socket import sys  # sends udp test data piped in stdin listener. # ex: cat sampleoutput.txt | python testsender.py   udp_ip = "127.0.0.1" udp_port = 5000  print "udp target ip:", udp_ip print "udp target port:", udp_port  sock = socket.socket(socket.af_inet, # internet                      socket.sock_dgram) # udp  # send stdin if len(sys.argv) < 2:   while true:     line = sys.stdin.readline()     if line:       sock.sendto(line, (udp_ip, udp_port))     else:       break    # file arg else:   myfile = open(str(sys.argv[1]), "r")   while true:     line = myfile.readline()     if line:       sock.sendto(line, (udp_ip, udp_port))     else:       break   sock.close() 

listener:

import socket array import array  udp_ip = "127.0.0.1" udp_port = 5000  sock = socket.socket(socket.af_inet,  # internet                      socket.sock_dgram)  # udp sock.bind((udp_ip, udp_port))  while true:     data, addr = sock.recvfrom(1024)  # buffer size arg      print data     # write file later testing     # file = open("testdummy.txt", "a")     # file.write(data)     if data == "end\n":         break 

i able use above listener produce test file original program, should work. unfortunately, fails around 500 lines of payload, tested tail -n 500 testdummy.txt | python testsender.py, although it's random. specifically, listener not receive of sent lines before sender exits, leaving hanging, waiting "end\n" string.

as understand it, socket in blocking mode--how can prevent occurring?

my first advice you, don't use udp if want transfer files sequence of lines preserved, use tcp if don't want code alot. reasons are;

1. udp unreliable protocol, in sense packet sent not guaranteed received recipient.
2. udp doesn't guarantee sequence of packets being received, because udp packets may go recipient via several routes (hops between computers). latter sent packets can take short route , reach recipient before former sent packets. ("end\n" packet can come before other packets)

tcp on other hand reliable , sequence of packets received guaranteed. ideal file transfer.

but don't worry file sharing applications bear share, bit torrents make use of udp there additional coding have do.

1. you need implement acknowledgement protocol, in need have unique id each packet send recipient , when packet received recipient should send acknowledgement packet sender id saying packet received.
2. if in case packet got lost , didn't reach recipient (no acknowledgement recipient) must resend packet again (and again) until acknowledgement.
3. you can control order not sending next packet until acknowledgement previous one.