last day (14 days later) » 

10:32 AM
1
A: Swift TCP: safe buffer size for `Darwin.send` function

JeremyPI didn't know there was a limit to the size of the buffer you can pass to send(). However, it is not safe to return the pointer you get as a parameter to your block in data.withUnsafeBytes. I don't think you can guarantee it points to a valid memory region after the block returns. You should t...

 
As far as I now can say the limitation is physical and not logical. As I mentioned in my post my application crashes when I pass a very large buffer to the function. I modified my code to send inside the closure (good catch) but the result is the same. Right now I'm rebuilding the my custom send algorithm to split the data into smaller packages and see if the mentioned issue will go away (I think it should), but my question remains the same: what buffer size is safe for Darwin.send!?
Okay I can now confirm that this approach by slicing data into smaller packages works great and as expected (at least on my MacBook).
 
@DevAndArtist There is no limit to the buffer you pass to the send system call. What happens is it sends what it can and returns the number of bytes sent (or -1 if there is an error). The size of the buffer is not the problem.
 
So that might mean that I have to check if the result of send is not equal to the my data I was sending and resend the rest (on each iteration I'll check for -1). Is that correct? I'd now assume, that this way it wouldn't matter if my data is bigger than the socket buffer size.
 
@DevAndArtist That is correct. I think I would do it by putting my fragment in a loop and then replacing the data by a subdata (using the subData() method) until my data was of zero size(or an error occurred).
I'm assuming this is TCP. If it is UDP there absolutely is a limit to the size you can pass in one send call.
 
Yes this is TCP. I've rewritten my code to use repeat while for both send and recv and the result is way more dynamic and different than I would have expected.
send takes only one iteration to send all the data (for my current data) but recv takes way longer to receive everything and it needs way more itereations.
I've got one question left, in my TCP framework I'm sending the data size first (a single Swift.Int) so the receiver can preallocate enough storage. How can I make my application safe and ensure that all 8 bytes for the data size are transmitted?
I'll accept your answer, because you gave me enough feedback to find the right solution.
 
 
4 hours later…
2:15 PM
First of all thanks for the acceptance
What happens when you do a send is (if the buffer is small enough) the operating system will grab as much as it can of the buffer and send it. For 8 bytes, that will certainly be the whole thing. It might not actually all get sent in one packet. TCP has a thing called a "window" which is how much of the stream it can buffer at once. This is typically 65k. The window moves through the stream as it gets asks for each packet.
So, if you end 65k of data and the packet size is 1k, TCP will send 65 packets to the receiver. The receiver sends back ACKs that acknowledge how much of the window it has got. When the first packet arrives, it sends an ACK for 1k. The sender then knows it can discard the first 1k and make 1k available at the end of the buffer.
If the receiver gets packets 1, 3, 4 it will ack the 1 but not the 3 and 4 until the 2 arrives. After a certain amount of time, it will assume the 2 went missing and ask for a retransmission. Once it is received it will ack the 4 it already has and the sender can move the window on so that packet 5 is the first packet.
This is all transparent to you except that if the available space in the window is smaller than your buffer, the send call may block (normal if there is no space) or may return a value smaller equal to the amount of window available.
Anyhow, if you send the 8 bytes and send() returns 8, you can be 100% certain that all 8 bytes have been transmitted. The underlying TCP software guarantees it will get to the receiver unless something catastrophic happens e.g. somebody pulls the wire out of the ethernet adapter.
 

last day (14 days later) »