Sockets - send()

[Brian K. White]

----- Original Message ----- 
From: "Nancy Palmquist" <nlp at vss3.com>
To: "Kenneth Brody" <kenbrody at bestweb.net>; "filePro List" <filepro-list at lists.celestial.com>
Sent: Thursday, October 30, 2008 7:04 PM
Subject: Re: Sockets - send()


> Kenneth Brody wrote:
>> Quoting Nancy Palmquist (Thu, 30 Oct 2008 18:02:31 -0400):
>> 
>>> Anyone using the sockets logic in filePro, I was wondering if anyone 
>>>  had an idea about this.
>> [... sends 4000 bytes through socket, but recv's 1490 on the other end...]
>>> rx(4000)=""
>>> length=recv(accept_handle,rx)
>>>
>>> If I display the value of LENGTH, it will show "1490" and I am  
>>> missing the end of my file.
>> [...]
>>> Any suggestions would be helpful.  All the records that are smaller 
>>> than 1490
>>> seem to work flawlessly.
>> 
>> It's a part of the TCP/IP design that messages can be broken up into
>> smaller chunks, which can be reassembled at the other end.  In you call
>> recv() for a length of 4000 before they have all arrived on the other
>> end, recv() can return just the part that has been received.  If you
>> call recv() again, you'll get the next chunk that comes, and so on.
>> 
>> In this case, it appears that the connection has been set to break things
>> up into 1490-byte chunks.  (If you check your network settings, you will
>> probably see an "MTU" value of 1500.)
>> 
>> As I recall, filePro's RECV() function has a flag to tell it to wait until
>> the entire length has been received (or an error occurs), but I'm not in
>> front of a system where I can check how to pass it.
>> 
> I looked in my docs for sockets and do not see anything more than ,flags in the 
> descriptions.  (From the on-line manual)
> 
> status = RECV(handle,dest [ ,len [ ,flags [ ,noblock ]]] )
> 
> status = RECVLINE(handle,dest [ ,len [ ,flags [ ,noblock ]]] )
> 
> 
> 
> Receive a message from a connected socket. Fills in the field in "dest" with the 
> result. Returns the number of characters received, or a negative number 
> containing the error number. A maximum of "len" characters will be read. 
> RECVLINE() will read up to len characters, or until a new line character is 
> received. The new line character, if any, will be discarded for RECVLINE(). 
> "Len" defaults to the length of the destination field. "Flags" defaults to zero 
> and should currently not be given any other value. "Noblock" defaults to zero. 
> It a non-zero value is passed, the function will return immediately, even if 
> less than "len" bytes are received.
> 
> 
> So I guess ,flags is not functioning yet.
> 
> How would you make it cycle around?
> 
> Would it look like this
> 
> again:
> then: length=recv(accept_handle,rx)
> 
> if: total_length lt "4000"
> then: line=line{rx;tot_length=total_length+length;goto again
> 
> Or should I force it to get it in pieces by making rx(1000) and repeating recv() 
> 4 times.
> 
> NOW line would have all the stuff that was sent, am I right???


Probably only most of the time, not by any garantee.
If Ken is right that fragmented tcp/ip packets are why you are
receiving less than the total 4000, well packets may be broken
any time to any size for any number of reasons by any device
along the way between sender/receiver.
So simply requesting a packet that is smaller than your own box's
mtu in no way ensures that you will receive 1000 bytes every time.

However _most_ ethernet devices have an mtu of 1500, and your
traffic is only likely to pass through a very few connections that have
lower mtus (Because of say, PPPoE eating some bytes as overhead
for the PPPoE protocol itself, this is generally just residential
DSL, but possibly some types of vpn  might incur some overhead too) 
TCP/IP itself uses up 44 bytes of each packet too, so the payload
that holds your data is actually smaller than that.

So, _usually_ 1000 bytes of payload data are going to be small
enough that nothing ever ends up needing to break the packet
into smaller packets. 1000 bytes from you at the application
level will produce a packet of at least 1044, possibly a little
larger like 1052 if you are passing through a
something-tunneled-within-something protocol like pppoe.
1350 or 1400 or even 1450 are probably still small enough.
The overhead eaten by a wrapper protocol is usually only a
few bytes per layer. Ex: 1492 vs 1500 , 8 bytes for PPPoE.

I'm surprised you have to worry about this at all.
Usually this is all transparently handled by the OS
or by _something_ such that an application never has
to be aware of it. The OS or the nic driver or something has
reassembled fragmented packets before you see them at
the application level. I don't really know where it happens,
so even if applications do have to reassemble their own packets,
still the user never has to. IE, you don't have to worry about
this when using telnet/ftp/rsync/http, not even netcat.
So, if even netcat is doing something to handle this,
then I'm surprised filepro isn't doing the same.

Usually the only reason to think about mtu is for performance
reasons. If you have a dsl connection that you know can't
pass full 1500 byte packets without breaking and
reassembling them, then you set your pc or servers mtu to 1492
so that your pc doesn't send packets that will need to be split.
This makes your connection run at it's best possible speed
most efficiently. But without doing that, everything still
works because the breaking/re-assembling happens transparently
at some lower layer.

Actually, maybe that's the problem. Maybe you are passing
through such a connection and have NOT set your pc's mtu to be
small enough to produce packets that fit without splitting.
Sending or requesting only 1000 bytes at a time at the application
layer may not actually help after all, because maybe the OS
will coalesce the data into the largest chunks it thinks it can
use, which might be 1456 bytes of payload per packet,
to produce packets of 1500 bytes each.

Or perhaps it's nothing on your end but the person at the other
end needs to set their mtu lower if they have dsl.

Sorry for all the maybes.

-- 
Brian K. White    brian at aljex.com    http://www.myspace.com/KEYofR
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!