« first day (2 days earlier)   

1:45 AM
It might look slow from normal UX perspective but the query will return 100 elements to be shown in his feed, so he will take some time to read them and move to next page so the 2-4 seconds is an acceptable time

also the query will return 100 elements that will be sub-paginated into 10 pages of 10 for another system making possible for user to flip pages without latency

I'm very happy and satisfied with this result so far, even though is not perfect, is a giant improvement considering that i was given this task from an horrible legacy system in which the engineer himself who created the s
 
 
5 hours later…
6:41 AM
@RafaelLima I am glad you are happy with it. It is certainly much better than it was.
 
7:03 AM
Did you try the lazy load query I suggested above? Have you been able to figure out how big the temp table is for that "New York" query? Do you have high levels of concurrency? What language is the client application written in?
 
 
6 hours later…
1:08 PM
@user1191247 I cant use the lazy load because this involves directly altering the query which i cant because.
 
@RafaelLima I know you said you cannot change the query but I am still interested to know how much better it performs? You will have to fix the query at some point, as relying on a huge temporary table will only work as long as you don't see increased concurrency or the dataset growing so that the temp table needs more than the max 4,294,966,272 bytes. I am intrigued by the "engineer himself" not wanting to fix the query.
 
I cant use the lazy load because this requires directly altering the original query which i cant.

I think the biggest issue related to this problem is that it comes from a legacy node.js backend which was written by someone with very little knowledge who decided to make his own home made "framework" for dynamic query generation

it doesn't follow any widespread pattern as rest, graphql, hql ... anything... it is just a bunch of ifs elses and string concatenations that generate a sql from a json request...
@user1191247 the lazy loaded query [with the memory settings already adjusted] runs in about 1 second...
 
@RafaelLima Wow! That's a lot slower than I was expecting. I thought it would be less than 100ms. Oh well.
If you don't have to worry about concurrency, it does not matter so much.
Good luck with keeping it alive!
 
1:25 PM
Thanks everybody who tried to help me =D
 
2:10 PM
 

« first day (2 days earlier)