« first day (3463 days earlier)      last day (1471 days later) » 

12:00 PM
and the TCP implementation works beautifully. it can handle 100 clients or more
 
mr5
thread { while (true); } [16]
^ imagine your thread pool is limited to 16 only
 
then that is Microsofts problem
bro
 
mr5
and each thread have an infinite loop
 
bc the ThreadPool is in the standard library. furhtermore, im pretty sure that a large number of tasks can be run on the 16 threads seeing as how the TCP version can sustain 100 Tasks for its client handlers
Task execution is delegated to the ThreadPool
i dont have any control over that
 
mr5
I mean, I thought you're the one spawning those Task?
 
12:02 PM
nope. i explicitly avoided Threads in favour of Tasks
 
you can't service more requests than you have threads in your threadpool
of course you can queue them up, but that's not the same
 
you can, as Task execution is interleaved. my TCP server says otherwise
ill get a picture
 
mr5
while (!cancellationToken.IsCancellationRequested) remove the piece of code
 
then it isn't being run concurrently
 
i have done. it doesnt help
 
mr5
12:03 PM
as far I as understand things, the HandleClient should only be called once for each request
 
mr5
why are you looping over it?
 
thats what happens when i try to run 20 TCP clients at the same time. it works. and Tasks are concurrent.
just click on the 'Image not found'
 
@MikołajLenczewski in fact it is written that there are 20 threads :P
 
mr5
are you getting the total active threads using System.Diagnostics.Process.GetCurrentProcess().Threads.Count?
 
12:05 PM
thats my current test program
maybe i havent pushed it to Github yet
it uses Task[] to store the client tasks
 
Why do you use #if?
 
mr5
await Task.Factory.StartNew(TestSocketClient).Result; uhm wut
 
Task.Factory.StartNew(Func<Task>) returns Task<Task>
the innermost task is the actual TestSocketClient method
i await that instead of the outermost Task, which represents the spawning of TestSocketClient on the ThreadPool
see above that, where i await Task.Factory.StartNew(TestSocketServer)? that call passes right down to the next line, because i dont explicitly await the innermost Task
 
mr5
I get easily confused by a nested Task
 
it took a while to get my head around it
yeah
"why the fricc are you not stopping, reeeee"
@Hozuki those are btec cpp preprocessor directives. it allows me to change what the code does by commenting 1 line out
 
mr5
12:09 PM
            int totalActiveThreads = 0;
            for (int i = 0; i < clientCount; i++)
            {
                Task clientThread = clientTasks[i];

                bool threadWasActive = activeTasks.Contains(i);
                if (threadWasActive)
                {
                    totalActiveThreads++;
                }

                Console.WriteLine($"[Client task {i}] Approx bandwidth: {clientBandwidths[i]}, was active? {threadWasActive}");
            }
 
look at the top 2 lines
 
mr5
I don't think this is true
 
Yeah, usually you'd just compile it into the program and decide on behavior using a runtime flag, not a compile-time flag
 
mr5
because Task can be reused while it still in active state
 
12:10 PM
its never resused, and the activeTasks hash set is set once the client receives at least one response packet
so it must be valid
the tasks are awaited once, at line 201
im pretty sure
and never again
 
mr5
Use the combination of this and this to get the total active threads.
 
why do i care about the threads again?
that would show me the ThreadPool threads, which are not my concern
just because i am using Tasks
 
mr5
just so you have the correct info. you say it stops after 16 threads. that might not be true
 
on UDP yes. on TCP i can do more than 16 clients (up to 100). thus, i think that its a packet-not-arriving issue
i have 20 clients going at once
that is TCP
 
mr5
If I would be making something like that, I think I would do everything synchronously and let the upper abstraction handle the concurrency.
 
12:14 PM
16 threads means 16 requests being handled concurrently
 
anything beyond that isn't being handled concurrently though
 
the first thread to finish starts the next request
 
even pseudo-concurrency is ok, as long as each Task gets its alloted time slot (which is the case, look at the image above)
but it doesnt matter
 
12:15 PM
even though most likely, 16 threads does not equate to 16 concurrent threads
CPU might only be able to handle 8
 
i can run 20 clients at once. the thread is not the problem
 
yeah, not really important that it's concurrent
it just has to get through a lot of requests per second
 
mr5
why are you looping over a client request again?
 
which line? which file?
 
mr5
line 11 from there
 
12:18 PM
that cancellationToken represent when the client handler must exit (i.e. when the server must be shut down). so i check whether the cancellation has been requested by the user, and if so i stop responding to requests
otherwise, i continually process client requests
inside of the while loop
 
mr5
why can't you loop on the client side?
 
the client side just continually sends requests and listens for a response
 
mr5
yeah basically
there's also a loop there
 
i have to be able to shutdown both the server and the client
yes
 
mr5
but I'm not so sure why you need to loop over the same request?
 
12:19 PM
thats the client side
im not looping over the same request
 
mr5
Is HandleClient called everytime there's a request?
 
mr5
oh
 
its called once, when the server receives the first packet from a client. look at RunAsync
thats the full code
 
Why is there so much code ~_~
 
12:21 PM
bc Sockets are verbose, and i like spaces
 
mr5
is there any built-in event from the UDP library you're using whenever there's a request received?
 
this is the UDP library
 
mr5
it would be helpful if you can provide a very simple pseudocode of all the flow
 
sure, let me do that
 
mr5
I mean from System.Net.Sockets
HandleClient is called only once in the entire app lifetime right?
 
12:25 PM
very abstract pseudocode
oops, there should also be a registeredClients.Add(request.RemoteEndPoint) in there, after line 21
 
You know you can still use a Stream on top of a Socket, right?
It makes shit easier.
 
over UDP?
 
Sure.
 
hi
 
new NetworkStream(socket)
 
12:27 PM
hi shad
"The following code example demonstrates how to create a NetworkStream from a connected StreamSocket and perform basic synchronous blocking I/O."
 
List<string> accountList = new List<string> {"123872", "987653" , "7625019", "762401", "7625011"};

int i = accountList.FindIndex(3, x => x.StartsWith("762"));
why this not work?
 
since UDP is connection-less, i highly doubt it
 
I am using a predicate
 
"The NetworkStream class provides methods for sending and receiving data over Stream sockets in blocking mode." udp is not a stream socket
for TCP, it works fine. NetworkStream cannot work with UDP
 
Alright, shit for you I guess.
 
12:28 PM
yep
very shit for me
 
mr5
@Shad because you started at index 3 and 762 is at index 2
 
nvm
xD
 
@mr5 is the pseudocode enough?
 
mr5
yeah I think so
so each time a client requests, the server assigns a Task with an infinite loop in it right?
 
yes
once for each client
 
mr5
12:31 PM
yeah. that's pretty bad
 
how would you do it?
it allows more throughput
since multiple requests can be handled in parallel
the bottleneck is in receiving the requests
 
It's okay if the handler is properly async.
 
mr5
I'm thinking of just one loop in the server and just distribute the response accordingly.
 
that means that i cannot handle requests in parallel :(
also, i gotta go
 
This is the basics of a high-performance concurrent server gist.github.com/Deathspike/bb1260215e6b165b463e872d89417210
 
12:33 PM
thanks for the help
i sincerely appreciate it. ill be back later
imagine using TcpListener and UdpListener, not masochistic at all XD
 
mr5
yeah sure the project you're doing also interest me
might write one also for the sake of learning
I've never written one so shrugs
 
@mr5 It's not so hard :)
 
socket async event args are cool, but hard to get right. ive also never really made anything with 350 MB/s bandwidth before so its exciting to do that
but i actually gotta go. thanks again for the help and i might be on later to ask for mor
have a good one
 
mr5
@Hozuki yeah your code is more easily understandable =P
dang with this your you're
"more easily" is this grammatically correct?
 
@mr5 That's why I wrote it :-P
More easily is indeed grammatically correct.
I actually wrote a (bad) HTTP server thingy back in the day. It's probably still on GH. But the snippet I just posted is essentially the basics of it.
 
12:53 PM
im back
lets go
its simple
i grant you, but have you benchmarked it?
@Hozuki
 
@mr5 good rule of thumb is if it has two or more syllables, you add "less" or "more" before it
harder vs more difficult
though my guess is there are exceptions. there are always exceptions
 
1:10 PM
@mr5 I strongly recommend SignalR seriously don't start this conversion from the beginning in a few weeks.
@Neil fastly is one such exception
 
@Squirrelkiller you don't say fastlier
 
You don't say fastly
Adverb of fast is fast
 
eh? no, the rule of thumb was whether or not to say more easily or easlier
what makes that distinction is the number of syllables in easily
fastly is two syllables
fast is one syllable
 
fastly is not a word
 
and in fact you can say faster
 
1:13 PM
ah right
Wrong direction of exception lol
 
but again, I don't deny that there are exceptions :P
the rule of thumb with languages is that there are always exceptions to said rules
 
@MikołajLenczewski just like a static variable
 
1:29 PM
Morning all, I could use some advice because I don't want to waste anymore time running in circles. I have an entity called "Task" in a productivity application. Users may take a lunch while they have an active "Task", but an active task is not required to start their lunch. Obviously this means that a separate entity is required, but I have to ensure a relationship exists between the task and the lunch in the event that the user starts their lunch with an active task.
I figured a 0..1 to 0..1 relationship would be ideal, but entityframework doesn't make that easy.
 
hey
FindLastIndex(startindex);
is not giving my that startindex? (that startindex satisfies the predicate)
 
so startindex is a delegate?
that name is pretty bad
or you are doing something that you didn't expect
 
FindLastIndex seems like it should search in reverse order for the first instance of a string and return its index
 
do you want something like .Skip(startindex).ToList().FindLastIndex(somePred);?
 
@ntohl what do you mean?
 
1:41 PM
@MikołajLenczewski I asked Shad
 
Time for some easter holiday, cya o/
 
@MikołajLenczewski btw I check if you can make a NetworkStream no matter if it's UDP
 
oh. I mean you might have a static variable Configuration.TcpOrUdpEnum, and condition the #ifs with that
 
"The NetworkStream class provides methods for sending and receiving data over Stream sockets in blocking mode."
#if is a preprocessor directive, so you can only use it with #define XXX
unless im missing something
so static variables shouldnt be able to be used with an #if
 
1:44 PM
you should not use preprocessors, just like Hozuki said
 
why not?
its a simple test app
why bother writing 7 more lines of initialization and checking code
 
mr5
@MikołajLenczewski I mean, a single loop that breaks down all the client details and assign them each Task to handle that request. The Task does not need to have an infinite loop
 
when i can use 1 #define and #if ... #endif
 
cos it's C#. And also you have to be careful with your rebuilds. If VS is too smart about what dll should be rebuilt
 
theres a "clean solution" and "rebuild all" for that
 
1:45 PM
you already have those ifs. So it's a search and replace
 
thats more effort than i am willing to invest for something that has absolutely no effect on my code
sorry
 
than I ask back. Why do #ifs if there is a better way with same amount of code?
 
its not better in any way
it does the same thing
with the same amount of code
why bother
@mr5 that might work, ill have to check
 
same question asking back. Why bother with #ifs
when the other solution is better for other cases. Like not in a simple app
 
ive written the #ifs already, theres no point changing it now
and this is a simple test app
if i was doing somethign more complex i would not use them
for obvious reasons
 
mr5
1:50 PM
while (!token.IsCancellationRequested) {
	// perhaps, add a wait & notify logic here as to avoid overheating the CPU
	var client = ...
	var request = ...
	processClientRequest(request)
}

processClientRequest(request) {
	var response = ...
	socket.send(response)
}
@MikołajLenczewski something like that ^
make the processClientRequest asyncronous to have a concurrency
 
yeah, i was gonna implement that rn actually
cheers for the heads up, ill see if it makes a difference
 
mr5
how do you win against PA using BB? DOTA2 question.
okay. there's not a single article in the internet to counter PA as BB
where's squirrelintraining when you need him?
 
hes in training right now
 
mr5
training how to win against me. good idea.
 
2:01 PM
@mr5 there is some rock paper scissors in that game you know...
don't pick BB against PA
 
mr5
yeah but I want to
I just lost against PA last game I want to know if there's any item to counter PA
 
mr5
I've tried halberds, octarine core, lotus orb yet I still got bamboozled with its RNG crit
 
@mr5 blade m is pretty handy
 
Jason Punyon on April 09, 2020
Unfriendly comments are an issue in our system because of the effect that their tone has on their recipient’s and future readers’ willingness to contribute to Stack Overflow. The solution to these issues isn’t to argue about circumstance or intent. The only remaining option is to work on the comments themselves.
 
2:04 PM
let her crit herself
I had some success with halberd
@MikołajLenczewski I checked what is the error, when you try to make a NetworkStream from an UDP client ^
 
maybe somebody read amazon.com/C-7-0-Nutshell-Definitive-Reference/dp/1491987650 - C-7-0-Nutshell-Definitive-Reference By O'REILLY and could share some thoughs, is it worth the time, pros and cons?
 
good morning
 
2:21 PM
@ntohl the problem is that a UDP client has a Dgram socket, instead of the Stream socket the NetworkStream requires
sind Dgram doesnt guarantee packet transmission or reliable ordering, i would assume you cannot create a network stream from it
 
That's not problem. I was curious which part will throw up and how
 
oh really? what was the problem
 
technically you could have an unstable Stream tho. Also if both party use .Connect, it might work
there were no problem
 
I was curious which part will throw up and how
 
2:23 PM
fair enough
 
since educational code...
 
@mr5 so, using your approach with a handler task, i still couldnt handle more than 16 clients and my bandwidth went from 350 MB/s to 90MB/s
large oof
my latency went right down though
which is pretty cool
* a request handler task instead of a client handler task
 
@MikołajLenczewski Last time I bench-marked that style of code it reached millions of requests/second. And I don't have a good PC.
The whole trick is that it uses I/O completion ports and the scheduler to maximize concurrency and threading. It pushes both to the max.
 
mr5
@MikołajLenczewski eh can you show how you done it?
 
what probably hurts me most is that when using DotTrace and profiling, i spend over 90% of my codes time in System Code
but im dropping packets
le large oof
and what little time is spent in user code, is in methods such as TaskCompletionSource.SetResult
-_-
 
2:36 PM
Refactor to have one task accepting clients, one task per client to read data, and queue up the packets for processing for another task.
You won't drop packets that way.
 
@Hozuki whilst i doubt millions of requests per second, that just affirms that i shouldnt use my code in production (which i never intended to)
yeah i was thinking of that
which is a good shout
 
The queue should follow a typical subscriber/publisher pattern with n threads (e.g. NOT TASKS you want the thread pool for the readers) that consume
 
but ideally i would use a single task
you gotta love compromises
and to be fair i had that in the og code, but i had to scrap that and start again bc it got too cluttered
 
Well Windows' Winsock has an internal queue for UDP packets that your program needs to read out. It has a max size, so if you're not fast enough reading them out, Windows drops em.
You'll probably see different behavior on Linux tho.
So you have to minimize the amount of time spent of spooning them out.
 
why are we here, just to suffer?
how do i optimise the standard library?
 
2:41 PM
You don't. You run on Linux instead.
 
too true
i actually want to see how it would perform
would you be running linux by chance?
 
Nope.
 
Just for my information, why are you so obsessed with the performance?
Because #nerd and it's awesome?
 
bc whats the point of writing slow code? when does it ever make sense to do (apart from when prototyping)
if im not proud of my code, why bother?
im not going to put out shit, am i
dont get me wrong
if you are writing a one off, then sure. writing something obscene
that takes a night to complete, i dont care
 
2:45 PM
I don't look at the performance of my code all that much. I just make sure it uses appropriate theory and is very readable, even at the cost of some performance.
 
more power to you
 
Well, if you're just toying around, it's all fine. But if you've got a product to build, obsessing over performance is not a very good thing to do. I just wanted to point that out.
 
If money and time was no object I'd make everythin fast as it could be
 
i prefer to write fast code. i consider it to be an appropriate sacrifice to make something slightly harder to read if i get some more performance
well, i am in the coronavirus holidays, and this is a toy project
i have like 1 or 2 stars on github
 
Toy project I would absolutley be obsessing ^^
 
2:47 PM
Yep, for a toy, everything goes.
 
its not a major library that people rely on or anything, so i think i can afford to care about performance
how the hell do i change the github account used to push changes in visual studio
 
mr5
@MikołajLenczewski _ = HandleClientRequest(...) // can you just make it fire n forget?
 
mr5
does await SocketAsyncOperations.ReceiveFromAsync(...) idles and awaits from incoming request?
 
it shouldnt idle, but should be asynchronous
it uses events at its core, as with SocketAsyncEventArgs, and theres simply a TaskCompletionSource that wraps the event api
btw
163 MB/s bandwidth with the fire and forget version
cant handle more than 16 clients tho
or rather, cant handle 20
 
mr5
3:02 PM
the server should be idle just act on every request.
 
it now does its work on a single thread, but that thread is pinned at 100%
looking at the CPU usage graph, i can see that
thats exactly what the server does, it only acts on each request
but i think theres too many requests per second
and that packets from some clients are lost
in fact, that is the only problem that can be occuring
 
mr5
can I clone your repo?
 
dude go for it
wait
let me push the latest changes
 
mr5
welp, I'm playing dotes rn. I'm not sure if my 8GB RAM can handle Chrome (~20tabs), VS and DOTA2 and the same time.
 
if you want to help out, i would be very grateful
i dont really mind when
i have no clue why it shows up as EnderEnterprises on github
im pushing as myself
whack
 
mr5
3:09 PM
silly business name ^^
 
yeah
too true
made it when i was younger
but we move
ok
turns out i had 2 github account
thats fixed now
 
mr5
what does in means in a function parameter?
oh so to make it a const
 
3:30 PM
yh
basically eliminates an unnecessary compiler value copy
tbf, 195 MB/s is not too bad
thats better than 1Gbps
so would saturate Cat5e
the problem is on the number of clients
from looking at the profiling
over 95% of my codes time is spent in System Code from the standard library
i dont know if i can make it any faster
other than by using more than 1 thread
which is painful, as a library shouldnt necessarily do that
it should be as lightweight as possible
what do
?
 
mr5
3:45 PM
cloning
 
please elaborate
 
mr5
I'm cloning your repo lol
 
mr5
I want to see myself why it is limited to 16 clients only
 
i thought you meant that as an answer to my moaning and was confused
 
3:58 PM
is it limited to 16 on your system as well or is my system just massively screwed up?
 
Hey All, my company is starting to develop some applications using asp.net core. Most of our web servers are IIS, and more of our servers are Microsoft Windows Servers.
However, may I please know if we should start investigating whether or Not we should look into deploying to Kestrel webservers(regardless of whether they are hosted on Microsoft Windows Servers or Linux Servers)? Basically, would there be application speed improvements if we used Kestrel webservers for our ASP.NET core applications (as opposed to deploying them on IIS )?
 
if you use kestrel, it is mostly just cross platform. its a basic web server. put it behind a suitably configured nginx reverse-proxy and you should be good to go. im not sure how cross platform IIS is (i assume its windows only)
in fact theres docs on how to do this on microsofts asp.net docs
 
4:20 PM
If there is No huge application speed performance improvement, and if my company Only really deploys ASP.NET applications in Microsoft Windows Servers with IIS then maybe I should just stick with IIS on Windows Servers?
Yes IIS is only for Micrsoft Windows SErvers
 
by all means keep what you have already, but i would reccomend testing and benchmarking a linux box with kestrel and nginx to be sure
just to be sure that it cant beat your IIS servers
 
I have code I just wanted to check for the swagger /docs path and now it is magically returning a favicon path first. How do I approach so my middleware does not auth against the /docs path?

            if (context.Request.Path == "/docs")
            {
                context.Response.StatusCode = 200;
                return;
            }
in .net core how would you allow AllowAnonymous to the swagger /docs route?
 
 
1 hour later…
5:37 PM
@mr5 did you take a look? is there anything i can improve on?
 
mr5
5:48 PM
just finished gaming
 
mr5
I'm going to look now
 
mr5
err
what are the prerequisites?
@MikołajLenczewski The requested address is not valid in its context.
I'm just seeing hello world in the console
 

« first day (3463 days earlier)      last day (1471 days later) »