« first day (3858 days earlier)      last day (1080 days later) » 
00:00 - 14:0014:00 - 19:00

2:00 PM
@python_user I don't think even SSDs can perform truly simultaneous reads of non-contiguous memory
Confidence: 75%, will immediately capitulate if any contradicting evidence comes to light
 
thanks guys, I will see if I there are some benchmarks done on SSD vs HDD directory traversal
 
yeah, I'm pretty sure the firmware can only handle one thing at a time
But there's no "wait for the disk to spin around" downtime. There might be other, electronic kinds of downtimes.
so yeah, go investigate and let us know :P
 
I don't think it's impossible to design a drive that can do parallel reads, I just don't think consumer hardware does it
 
A fair comparison is not trivial to do, because IO on an SSD is much faster to begin with.
@Kevin there's probably a lot of shortcuts being taken in addressing data
nobody wants a lead to every single bit
 
Ugh all my google searches for "ssd simultaneous reads" just turn up a bunch of junk about doing one read at the same time as one write
 
2:03 PM
I suspect "SSD concurrent reads" might help
hehe, so much for that
 
maybe "concurrent" has better SEO magic than "simultaneous". Stranger things have happened.
The fancier the words you use, the less likely you are to get pages written by John Q Public for their very common question
 
if you get too much fancy, that wont get anything either :p
 
concurrent is not the best word, but Kevin's probably right about that being least noisy
 
Hmm, quora.com/… makes it sound like multi-reads are possible... Let's google some of these TLAs.
Ok I ended up at csc.lsu.edu/~fchen/publications/papers/TOS16.pdf, which I can barely understand 5% of, but the mere existence of the paper implies that SSDs can have levels of parallelism more interesting than one read plus one write
 
:52162432 Do you think I can adapt this to search for the substring? Because of dirpath.

for dirpath, dirnames, filenames in os.walk("."):
    for filename in [f for f in filenames if f == 'NSUNS4.exe']:
        print(os.path.join(dirpath, filename))
 
2:15 PM
@Kevin neat
 
I believe it's the OS's job to recognize when it needs multiple reads in no particular order, and ask the drive "hey, get these values and ping me when they're all ready". I have a feeling that this doesn't happen in Python very often.
FWIW, here's my not-particularly-good BFS walker prototype dpaste.org/gZTS
 
somehow I suck at copy pasting the code, SyntaxError: invalid non-printable character U+200B
some invisible character snuck in
 
I wonder if Windows provides an interface for its search indexes...
@python_user Ah, this is why I usually use pastebin over dpaste -- sometimes the latter puts weird stuff into my clipboard.
 
I was searching for the "Raw" option, but Copy Snippet did the job
 
Please transport me back in time to 1998 when copy-pasting plain text would almost always give you plain text
 
2:46 PM
Unrelated, somewhat abstract, question. I have a matrix multiplication equation containing a number of variables, and I want to solve the equation in terms of the variables on the LHS of the equation. My code produces the correct answer, and I see that the solutions are quite simple, requiring only grade school arithmetic...
So, my question: Am I just lucky? Or is there some underlying property of "fundamental" linear transformation matrices that makes them easier to solve than your average system of nonlinear equations?
I've 80% proven to myself that if you have a collection matrices, and any variable in those matrices appears exactly once, then the symbolic representation of their product will contain no exponents. So that at least establishes something of a ceiling for the complexity of the system of nonlinear equations that get fed into nonlinsolve. But I don't think that's sufficient to prove that the solution will also be non-complex.
 
@Kevin I have a vague hunch that it's related to matrix inverses, which are fairly simple for 2d see e.g. Cramer's rule. Not sure, though.
 
Hmm, interesting. I also happen to know that none of my basic matrices have a determinant of zero, unless the solution given by sympy would throw a ZeroDivisionError.
Determinants and inverses are related, so that's an avenue worth researching I think
 
yes, exactly nonzero-determinant matrices are invertible
 
Ok guys I think I solved it
for dirpath, dirnames, filenames in os.walk("C:\\"):
    if r'steamapps\common\NARUTO SHIPPUDEN Ultimate Ninja STORM 4' in dirpath:
        print(dirpath)
    else:
        print('no')
 
There are a number of fun facts intertwined here. Since the determinant equals the product of eigenvalues, zero determinant means that at least one eigenvalue is zero. This means that there's an eigendirection that gets collapsed (this is not jargon but you probably understand what I mean). Consequently, you can't "undo" the operation involved because you don't know where vectors came from after a projection. Hence zero determinant implying non-invertability.
 
2:55 PM
With this, if that path is not in the directory it just print no
 
Related, actually concrete and practical question: is it possible to tell sympy that it should look out for w*z - x*y in an expression, and replace it with the symbol "det" if that produces a simpler result? I don't think I can do a simple subs call, since my equations have -w*z + x*y. The sign is wrong (yet still expressible in terms of "det")
 
Howdy guys...

Any interested in puzzle solving heuristic algorithms using python?
 
I hope this is not a job offer
 
@Dave Nice work.
 
@Kevin All thanks to you. You gave me the idea
 
2:59 PM
Oops, never mind my concrete and practical question, I just tried expr.subs(w*z-x*y, det) and it is smart enough to notice the sign difference
I could have sworn it didn't work the last time... Maybe I forgot to save.
 
@Kevin Anyway is there a way to limit os.walk to only 5? Like
for dirpath, dirnames, filenames in os.walk("C:\\"):
    if r'steamapps\common\NARUTO SHIPPUDEN Ultimate Ninja STORM 4' in dirpath[5]:
 
@AndrasDeak Cool, this formalizes a suspicion that I had earlier this week. Glad to know I wasn't on a wild goose chase.
@Dave Limit it to five what? Characters? s in dirpath[:5] would do that, but I don't think that "steamapps/whatever/naruto 4" in "C:/pr" would ever evaluate to True
 
@Kevin I actually tried that, and it didn't work. Glad if it does!
Ah, I know. I defined det to be equal that expression first :)
 
@Kevin No if I do this.
for dirpath, dirnames, filenames in os.walk("C:\\"):
    if r'steamapps\common\NARUTO SHIPPUDEN Ultimate Ninja STORM 4' in dirpath[5]:
 
perhaps that was your mistake too
 
3:04 PM
print('yes')
It would print yes for all the directories. But if I see yes more than 5 times then I surely know that the directory is there
 
If you want os.walk to skip over directories that are more than five directories away from the root, for example c:/program files/foo/bar/baz/qux... I'm not sure how to do that. It's easy enough to count the slashes and continue if there are too many, but it won't be any faster because os.walk will happily continue to dive deeper and deeper anyway
 
@Kevin that still not what I'm asking for haha. Let me write the question fully and then paste it
 
If you simply want the loop to stop after it prints yes the first time, then add a break statement on the line after print("yes")
 
@Kevin ok yes this is what I was saying
 
Or put the loop in a function and return the path
 
3:08 PM
I think this is better
 
Quite possibly :-) all other things being equal, the simpler approach is usually better.
 
3:25 PM
Hi guys, wanted to ask have an exercise to do with query string filter, for example, my query string looks like this /product?query=EQUAL(id, "abc"), there might be different functions than EQUAL, is there a way that comes to your mind how to do this? I thought maybe using a lexer to solve it.
 
is this flask? you can use something like requests.args['query'] I guess
I have not used flask much, but passing op=EQUAL in the query string, would be easier
 
Yeah, but I will have to translate that query to sql, forgot to mention that.
Also there might be other functions like AND(a, b) like (Filters only values for which both a and b are true.) for example /?query=AND(EQUAL(id,"first-post"),EQUAL( views,100))
So my main maybe question would be is there an elegant solution how to translate it into sql? :)
 
I dont want to give you any wrong suggestions, someone more experienced than me will pitch in if they know
 
I'd probably whip up a custom language parser that can turn your query into an abstract syntax tree, then a custom evaluator to turn it into sql (or your ORM's querying interface)
Putting a lot of emphasis on verifying that the system will never allow the user to access tables or rows or columns that they don't have permission to see.
 
Thank, will look into it!
 
3:36 PM
(I will not dare to mention the forbidden approach, and I will refuse to acknowledge that a forbidden approach even exists after the end of this parenthetical)
 
What do you mean by the forbidden approach? :)
 
Sir, I do not know what you mean
Although... I am getting an idea. Where's my mad science blackboard...
 
Cool, I am just interested in ideas, there might exist easy solutions, don't want to over engineer.
 
If you're the one generating that query string to begin with, perhaps you could express it in a different form that doesn't require a full blown lexer/parser
 
3:51 PM
It is just a homework for a company
 
For example organizing it into Reverse Polish Notation: views 100 EQUAL "first-post" id EQUAL AND
 
oh nice, maybe i could even try that :)
 
RPN is very very easy to parse and evaluate, the only remotely difficult bit is lexing the tokens. If only you didn't have string literals, it would be as simple as tokens = the_query.split()
 
Thank You Kevin for this idea, will definitely look into this!
 
@Kevin Huh, that's surprisingly slow
 
3:55 PM
Re: overengineering. It's possible that the answer the company is looking for is "I refuse to implement this algorithm, because the easy approaches are very vulnerable to sql injection attacks and/or privilege escalation". So I might be overengineering by suggesting any solution at all :-P
"User-controllable input should be vetted very thoroughly before being used in the logic of an algorithm" often gets rounded down to "never use user-controllable input in the logic of an algorithm"
 
I could reproduce the factor 100 once, where os.walk took 2 seconds to find the target and my own code took 250. But now all of a sudden my code takes 2 seconds and os.walk takes 4. There must be nasal demons at work
 
Caching, I forever blame caching
@Aran-Fey Maybe I'm just very unlucky and the target directory I used can be found in fewer DFS rounds than BFS rounds.
DFS will locate C:/a/a/a/a/a/a/a/a/a/a/a/a/a/a/a/a/a/naruto4 very quickly, and BFS will probably locate it only after looking everywhere else
 
Caching seems plausible. I changed a dir_entry.is_dir() to dir_entry.is_dir() and not dir_entry.is_symlink(), and now it's taking forever again
 
4:13 PM
To answer my own question, there is a way to programmatically interface with the Windows search index. However, my eyes fell upon the letter sequence "OLE", which means that using it is a pain in the butt
 
Looks like a dupe
 
If I understand it correctly, it's asking a specific objective question, so that's fine. My dupe sense is pinging softly though.
 
the answer just points out what the OP did wrong but doenst address the question, I have no idea how people upvote here
they just nuked the answer
 
Yeah, that answer doesn't answer the question as I interpreted it.
As for whether the OP's goal is a good idea... There aren't many plausible scenarios where you would need to control what class gets invoked by list literals. But I usually don't penalize OPs for wanting to do something silly
 
4:19 PM
I have asked questions out of curiosity, and I didnt want to close vote without asking experts, this was the dupe I was thinking of stackoverflow.com/questions/19083160/…
 
start_time = time.time()

for dirpath, dirnames, filenames in os.walk("C:\\"):
    if r'steamapps\common\NARUTO SHIPPUDEN Ultimate Ninja STORM 4\data' not in dirpath:
        continue
        print('yes')
    else:
        print('no')

print(f'\nFile has finished downloading in {(time.time() - start_time) / 60} minutes')
My code takes 1.1674083153406778 minutes
Anyway I can optimize?
 
That's what we've been trying to figure out while you were gone :-) Short answer: we don't know.
 
Uh, yeah. Stop iterating once you find the folder you're looking for.
 
Yeah, didn't I suggest break a while back? Continue is basically the opposite of break.
 
@Aran-Fey I choose this drive because the folder is not in it.
@Kevin No break won't work for me
I'm going through 7 drives
 
4:22 PM
@Dave can you elaborate why break wouldn't do it
 
@Aran-Fey I've been Kevin'd here... but yes, don't forget your OS will cache a lot of things... so if it's something you've recently accessed - it's still likely to be in memory and the OS can just serve that back (if it knows no modifications have been made) rather than walking the disk again...
 
@ParitoshSingh If I use break it would stop on the first drive. or second, or third. The folder might even be in the last drive
 
@Dave if you write the condition such that it only breaks if you find it..then..well..it will only break if you find it.
remember, you're the programmer here. program according to what you need
 
It's a good thing to stop searching in the first drive if you find the file in the first drive. Why keep looking for something you already found?
 
also why bruteforce this search? Surely steam must have some pre-set places to install at. and it should be perfectly acceptable to stop searching eventually
 
4:24 PM
If your program is breaking before you find the file, then there's a logic error somewhere in your code, and it's fixable
 
Despite the nasal demons affecting my test results, I maintain my opinion that a BFS across all drives is the fastest option
 
If i try to use a tool and it decides it needs to go through every single folder on my system without warning me..i'd consider myself fairly pissed off by the end of it
 
@ParitoshSingh Yes, I discovered earlier today that there are five locations where the game is usually installed. It would be a very very good idea to check those locations first before searching anywhere else.
 
Especially considering that if i dont have it installed in a usual spot, i probably can manage telling the program where to look.
 
My program is not using break Kevin and all others. Break does not solve my problem because I need to search all drives of the user. And as I said, the folder can be in the last drive. There's no use if you break at the last drive after searching 7 or 8 drives. The program would still take 5 min
 
4:27 PM
the break isnt to fix the case where the file is present in the last drive. it's to help with all other cases
to ask a flip question, if you do find the file in the first drive, why the heck is the loop still running
 
What I'm hearing is "it is an intentional part of my requirements that the program searches through all seven drives even if it finds the directory in the first one". In that case, there's no problem; your program does what you want it to do, and you can ignore all our break-related advice.
 
That's why you use a BFS, so you don't waste time searching the entire drive A when the folder you're looking for is actually on drive B
 
@Aran-Fey Yes ok this is more what I want. But I have no idea of BFS
 
One possible explanation for this requirement: suppose the computer has multiple users, each one having their own personal drive. Dave's mod installer program shouldn't stop when it finds Alice's steam library; it should continue to find Bob's and Carol's And Ethan's, and then ask the user which one they want to install it under
This kind of scenario could even happen if Alice and Bob and Carol and Ethan all coexist in the C:/ drive
 
Kevin, you're awesome, just fyi.
 
4:32 PM
That's a good edge case haha.
@ParitoshSingh This is my original code. With break. But then as I said it took over 5 min.
buff_size = ctypes.windll.kernel32.GetLogicalDriveStringsW(0,None)
buff = ctypes.create_string_buffer(buff_size*2)
ctypes.windll.kernel32.GetLogicalDriveStringsW(buff_size,buff)
letters = list(filter(None, buff.raw.decode('utf-16-le').split(u'\0')))


for letter in letters:
    for dirpath, dirnames, filenames in os.walk(letter):
        if r'steamapps\common\NARUTO SHIPPUDEN Ultimate Ninja STORM 4' in dirpath:
            print(f'{letter} {dirpath}')
            break
 
well yeah, if you do a naive search, you can expect it to take this long, or even longer depending on the files present
 
Use it like find_bfs(['C:\\', 'D:\\'], 'directory/you/want/to/find')
 
@Aran-Fey And I came here to ask how to optimize my code and got responses about break lol
 
(Like I said, takes like 7 lines of code. *cough*)
 
breaking is an optimization... If the program requirements don't forbid it
 
4:43 PM
Considering that they clearly had a version with break already apparently, (and managed to get rid of it on their own) I'm sure they're not worthy of the escape hatch you're providing here Kevin.
Anyways, it's good to see that, in all the discussion we did so far, apparently we only "got responses about break lol". So i suppose that's that.
 
hello, I want use python and arduino But for connection suggestion python 2.7 and import serial .What is the alternative in Python 3?
Does anyone have experience in this field?
 
Personally, whenever I ask a question about optimization, I always secretly hope I'll get a response like "oh, just set experimental_speeder_upper to True in the config", and I'm always disappointed when I don't get that response. I'm not joking.
 
@Aran-Fey is there a reason why you used a while loop instead of for? line 21
 
@ParitoshSingh haha. be chill like Kevin.
@Kevin Thank you Kevin. I'll use break for now
 
I set an impossible standard of chillness B-)
@Salio I believe there is also a serial module for Python 3. Should work basically the same.
 
4:48 PM
@Aran-Fey Yep. Once again I don't know how to do that. I asked you for an example when you brought this up the first time cough cough
 
is serial pyserial? link
 
@python_user I'm pretty sure that calling next(scanner) can throw an OSError, so if I hadn't been too lazy to add another except, the while loop would've made sense :D
All the is_dir() and is_symlink() calls should actually be wrapped in a try as well
 
is the scanner offering something that perhaps could be replaced by pathlib's .iterdir?
 
@ParitoshSingh thanks Have you worked with this module?
 
@Salio no, just googled it
 
4:51 PM
ahh, I dont know what objects they are and what attributes they have, it seemed redundant to emulate a for loop, until you explained your reason
 
@python_user I have found this before, but in practice it has problems
 
@ParitoshSingh I don't think so. scandir yields DirEntry objects, which may (or may not) already have metadata about the file cached, speeding up calls like is_dir(). I don't think Path objects can do that.
Functionally they're equivalent though. It's just a small optimization.
 
ah i see
 
I own an Arduino and I wrote code for it only a year or two ago, almost certainly in Python 3.X... I wish I knew where I saved my source files.
 
@Kevin
@Kevin This module does not configure Python 2 as well. I do not know why?
 
4:55 PM
I don't know what you mean by "configure" there.
 
@Kevin The connection between Python 3 and Arduino
 
Hmm, I don't remember if I had trouble connecting
 
thanks
 
o/
(Random gormlessness. Disregard.)
 
Anyone have experience with optimization algorithms in Python? I mean algorithms heuristic exe Bat or Ant
 
5:11 PM
Bat optimisation?
I found out a few weeks ago that Barnacles Mating Optimizer (BMO) exists... so I'm curious to know what these bats are doing
 
Ahh, microbats. I should have known!
 
@roganjosh NO boy this means en.wikipedia.org/wiki/Metaheuristic
 
I'm well-aware of what metaheuristics are
 
@roganjosh :)
CloudSim is used in Java for computer computing. Is there an example in Python?
 
5:23 PM
If I get some time in a bit, I'll try dig out one of our internal presentations that went over some benchmarks and some libraries that have broad coverage of these evolutionary algorithms. Suffice to say, though, that things appear to be getting a bit desperate in this field and people are just throwing any old animal behaviour, apparently, into code and making something "new"
If my memory serves, it was something like 196 biologically-inspired metaheuristics and almost none of them were supported by anything more than a couple of papers (Ant Colony Optimisation is obviously an exception to this)
 
Exactly
my question CloudSim is used in Java for computer computing. Is there an example in Python?
CloudSim written in Python or something like that ..
 
5:59 PM
cbg
How would I find the min number here?
sum_num = 0
max_num = 0
min_num = max_num

while True:
    num = int(input('Enter number: '))

    if num == 0:
        break
    else:
        print(num)
        sum_num += num
        if num < min_num:
            min_num = num
        if num > max_num:
            max_num = num
print(sum_num,max_num,min_num)
 
Set min_num = math.inf at the start
 
set it to np.inf instead because you're a rebel
 
@Salio Ah, it wasn't an internal presentation, it was part of our Ensemble events. You can find the presentation from my colleague Darian and the University of Manchester here. It covers the types of algos you're talking about, benchmarks them, some of the issues with the proliferation of metaheuristics and also the relevant python libraries
 
Lol, It is not really a code for me. It is a code to be graded and they have not taught about anything but just loops and conditions. So have to just use those two concepts here
Else I would just use a list here, sigh
 
Well, you can always just set it to None and modify the condition accordingly
the core theme is simple, an accumulator for min can't start with 0. (just like an accumulator for max shouldn't either, if negative numbers are involved)
 
6:03 PM
But no -ve numbers are involved
 
Consider that there were. Trying to convey the theme here, not trying to just solve for this specific problem.
 
So I would check if the value is None right, else assign?
 
you'd assign if it's None unconditionally.
...funny sentence, because i suppose that is a prerequisite condition
i amend my statement to say, you'd assign if None as a sufficient condition on it's own
 
@python_user Best avoided then, remembering Kernighan's Law.
 
@ParitoshSingh Kay let me see
 
00:00 - 14:0014:00 - 19:00

« first day (3858 days earlier)      last day (1080 days later) »