« first day    last day (18 days later) » 

9:47 AM
I added net.ipv4.tcp_tw_recycle=0 to /etc/sysctl.conf and also tried the opposite but didn't change anything
Also, shouldn't I have the same issues with my minimal client/server example if that was the case?
 
We're in "this is down to the configuration of the host you're running things on" territory
which is extremely fun, BTW, but not so much when you're X km away from it and probably without a way to sniff out network traffic to figure out what is happening
 
10:05 AM
I find very strange that I have an opposite type of communication, where the client sends a stream of data to the server, and that one works without any issue
that is, I can send files of same size opening multiple concurrent connection and the data is received without any problem by the server
 
Is is roughly the same volume of data?
 
but when is the server sending data, this issue happens
yes, always try to test with files of 20MB, and server -> client fails sometimes even if I run a single request manually
why did you say yesterday that the issue is in the server?
I am mostly try to change the client, but without success, but I don't really see what I could change in the server, since it is simply using 2 tokio APIs
 
You said you had issues getting the full data with raw curl, yeah?
as in, client-side
 
10:31 AM
let me explain
I have an HTTP server that uses a Tcp client to connect and receive data from a Tcp server
I use curl to send request to the HTTP server that will forward the request to the TCP server using its TCP client
 
11:15 AM
do you have any idea of where to look for possible errors?
 
11:36 AM
libpcap and a suitable client (wireshark) to sniff everything going past
you should see every leg of the way
including where the bytes are dropped off
 
12:21 PM
ok, but what should I look for exactly in the sniffer?
 
12:44 PM
You check whether all your data is going through at every step of the way
and if it isn't, then you know where the buck stopped
 
I was looking at the code and I can reproduce the issue with a client that runs on its own (and not in the http server)
I use stream.take(file_size) and then use that stream to read the file data
could it be that is the use of take the issue?
since from the official doc I read "This function returns a new instance of Read which will read at most limit bytes, after which it will always return EOF (Ok(0)). Any read errors will not count towards the number of bytes read and future calls to read() may succeed."
what I need to do in the client is: 1) read start request 2) read FILE_SIZE bytes 3) read end request
I wonder if using that take is causing the error that sometimes make the 3 steps fail (even though even step 2 fails because it reads less bytes than it should)
 
1:41 PM
I'm trying with Wireshark, a single request has > 10k TCP packets, could you be more specific on what to filter?
 
1:57 PM
I can see in wireshark that even when the transfer fails, the data of the end request (point 3 explained above) is sent (I can see the TCP packet in wireshark), what does it mean?
 
 
7 hours later…
9:17 PM
nearly home, just landed in the netherlands
something something fun weekends something
I find it doubtful but it doesn't hurt to try
@Nick there is a possibility but I need to look into it, that the take is working on raw tcp bytes including the header
 

« first day    last day (18 days later) »