« first day (3792 days earlier)      last day (1139 days later) » 

12:53 AM
Looking for any advice/knowledge re: interaction of threading and async. Googling for the two turns up many sites about using one versus the other. We have a situation where we use a library module that uses threading to do every-so-often push of buffered data to the mother ship, but wondering how this might conflict with other concurrency libraries (async, gevent, etc.)
I've looked at the gevent bits, and see that it has a monkey patcher to modify threading, sleep, et al. But I've been asked to consider the case where this every-so-often thread is running while async routines are being run also.
My initial sense is that it shouldn't matter, but would really like some hopefully better-enlightened thoughts.
I may just have to cook up a demo to see, but you can never be sure your demo is comprehensive of odd cases.
 
 
5 hours later…
user13727121
6:22 AM
when it comes to rounding off number, is using int more accurate than using floor division?
 
7:19 AM
hi it was so long since i quit this chat
 
7:32 AM
@CoreVisional what do you mean by rounding off number ?
 
@XavierCombelle been a while since I've seen you around - that's for sure :)
 
8:16 AM
@CoreVisional only "round" rounds. int truncates decimals. floor "rounds down" aka finds an int smaller or equal to the value being floored. So, the term "accurate" depends entirely on what behaviour you wanted.
import math

math.floor(2.6)
Out[2]: 2

int(2.6)
Out[3]: 2

round(2.6)
Out[4]: 3

math.floor(-10.8)
Out[5]: -11

int(-10.8)
Out[6]: -10

round(-10.8)
Out[7]: -11
(and in a similar vein as floor, ceil "rounds up")
 
Cbg
 
cbg @AndyK
 
@XavierCombelle hey back!
 
thanks
 
@XavierCombelle that's quite fun, I'm working with your surname's homonym, who is the boss of my current company. Christophe.
 
8:33 AM
how is the work at anybox ? @AndyK
 
@XavierCombelle many things to do. Deploying many stuff on K8s, but working on some python on a recent user story. So far, so good.
 
@PaulMcG There's no problem in principle with async+threads. Every async framework should come with a thread backend, and you can spin up your own as needed. Just mind the usual thread sanity, i.e. avoid too much shared state – if you deadlock the event loop by accident, that's as evil as it gets.
We have a service app that mashes together several async frameworks, threads and processes, and code contributors are mostly of the scientific kind ahem. Even with that we haven't encountered a nuclear meltdown yet.
 
8:49 AM
memo to self after finding two long unreadable functions with name foo and foo2. Never use numbers to differentiate function names
Man it really shows that the software was developed in a few years by a single guy without any code review
 
user13727121
@ParitoshSingh Like this example: dpaste.com/FN8NP8RJB, both methods output the same results, and from your example, I guess using int may be the right approach since I only want the result to be an integer
 
9:08 AM
@CoreVisional you can also always just use a floor division. //
that's what i'd probably do here
 
9:27 AM
stupid question, I know nothing about async, django channels and websockets. But when you have websockets isn't the idea that your session with the server persists? And not that each request has to make a new connection? Because we use websockets, but the first line of our receive_json function does: close_old_connections(), which to me seems like defeating the whole purpose of websockets. Or am I misunderstanding something?
 
9:51 AM
@Hakaishin Are you sure that also closes the current connection?
 
No for sure not, if the first line of the receive_event would close the connection the code would break I think. I just read this: channels.readthedocs.io/en/stable/introduction.html and to me it seems like you don't want to close and reopen connections when you receive an event, but only when you start and stop your application
 
Right. But I'm thinking maybe that code only closes redundant duplicate connections - for example, if a user had your website open and you had a websocket connection open, and the user refreshes the page, then their browser will open a new connection even though the old one is still alive
 
ah I see, that makes sense
 
10:22 AM
@CoreVisional "I only want the result to be an integer". OK, but both approaches Paritosh showed you give you an integer :(
Did you understand the whole point of what Paritosh told you?
 
using argparse, i do have an optional argument for file which am trying to use it with/without typing the flag. am using the following for it script.py PositionalFile -i OptionalFile so it's possible to call it without the flag? -i
like that way --> script.py PositionalFile OptionalFile
 
code markdown is backticks...
 
@αԋɱҽԃαмєяιcαη Use nargs='?'
 
parser.add_argument('-i', nargs='?', type=argparse.FileType())
i tired that actually, but if i called it without the -i flag, then am getting --> unrecognized arguments:
 
Since you don't want it to be a flag, you can't name it '-i'. Name it "optionalFile" or something
 
10:32 AM
711
Q: Argparse optional positional arguments?

Waldo BronchartI have a script which is meant to be used like this: usage: installer.py dir [-h] [-v] dir is a positional argument which is defined like this: parser.add_argument('dir', default=os.getcwd()) I want the dir to be optional: when it's not specified it should just be cwd. Unfortunately when I d...

 
@Aran-Fey Thank you. got it now. so nargs here treating it as optional so no needs to specific -
@JonClements Thanks for the ref.
 
10:44 AM
Is there such a thing as an async yield from or do I have to write a loop?
 
@CoreVisional for 2 non-negative numbers x and y: x // y = int(x/y). if either x or y are floats, the x//y would still be "floor division" but would give the answer as a float ending in .0, so .9//.04 = 22.0
the length of a list is always a non-negative integer and 2 is 2 (an integer) so the 2nd case can never occur for your case.
@CoreVisional as an aside, when you want the median of a list of numbers, you have to sort them before trying to select the middle elements
 
11:00 AM
Why the heck is it math.inf and not float.inf, ugh
 
Do you mean float('inf') ?
>>> math.inf == float('inf')
True
 
IEEE inf, presumably
 
@JonClements I mean, why does math.inf exist but float.inf doesn't
 
class attributes don't inflate the size of instances, right?
 
be a bit weird doing something like: (3.0).inf... ?
 
11:13 AM
hey quick question , how to use if on 4 objects at the same time

I want to make sure that at least 1 of the 4 inputfields is not empty

i'm trying out flask_wtf, It's a bit trickier compared to js.
 
First steps: learn about or and and.
 
@JonClements The same thing can be said about every class attribute, like (3.14).fromhex('a')
 
ah, "at least 1 of 4", that's slightly trickier
 
if username or surname or childname or wifename:
// do stuff ?
 
@AndrasDeak right
 
11:15 AM
yes I will google that

@Aran-Fey thanks !
 
@LoopingDev that would actually work if "empty" also means "falsey"
 
@AndrasDeak thanks a lot !!
 
if it were something more complicated, checking actual conditions, you'd probably want to use a loop over conditions and use any
 
@AndrasDeak any?
 
any() and all(), you should look those up too
 
11:17 AM
@Aran-Fey you need a loop
 
Why am I not surprised
 
@MisterMiyagi like an animal
 
@AndrasDeak got it ! , i will look them up ,
@MisterMiyagi thank you :)
 
@LoopingDev just so you know, these replies you're also posting point to specific messages. I'm the only one talking to you right now.
 
@AndrasDeak calling it "vintage" makes it easier to excuse all the suffering as a form of art.
 
11:22 AM
@AndrasDeak got it, I will separate messages in the future , sorry about that.
 
@LoopingDev it's no issue to me :P
 
lol ok
 
11:50 AM
hey, when we use Flask_Wtf we usually pass the form to the render_template like
`return render_template('test.html',form=testform)`
is it possible to pass it to the layout html file ?
 
12:02 PM
@LoopingDev What is the layout.html file?
 
12:22 PM
@roganjosh the extended Html file that we use on other html files using jinja2
 
12:39 PM
How do you guys save configs for production systems? Currently we have a custom format and save 1 string in the db and then parse that string. Ros for example uses a file in yaml format to store configs, but since the app is all browser based interacting with files is cumbersome. What is a good way to store and version configs?
 
@Hakaishin I assume you are not looking for something like Puppet/Ansible?
 
hmm, too many graphics, too little code
:D
 
1:00 PM
Morning. I'm trying to understand why this code pastebin.com/GQ4v95vU returns the desired output when the dtype is object but returns an argmax() error when it's string.
 
@n8-da-gr8 there's too much going on in that one line, try picking it apart step by step
 
Will do. I was beating my head against a wall so just started trying random things. I thought idxmax might be trying to parse the string some how.
 
@MisterMiyagi I guess already a standard format which allows nesting like yaml would be better than our custom 1 level deep format. And also a bit less effort then the full blown solutions
 
1:20 PM
@Hakaishin As for format, we are pretty happy with ini/toml for easy things and JSON/YAML for complex services. A simple script that fetches configs from git can get you a long way for deployment.
 
Sounds about right, I just don't see why use ini style like we do right now, since it's a subset of yaml. I mean not literally but functionally
 
1:38 PM
ini is much easier to work with for simple cases. YAML has a few nasty cases where it's trying to be clever.
 
@JonClements you are back as a mod! (did you leave ...?)
 
Yeah... came back last April-ish or something?
 
morning cabbages, folks!
very curious about what prompted your re(mod)turn @JonClements
 
after a break and watching things... I kinda felt things were looking a little more upwards... (plus I'm a masochist)
 
1:53 PM
but mostly that last part
@JeB welcome. Please don't ask for help here with fresh questions on the main site as per our rules.
 
JeB
ok thank you
 
Your question is tagged properly so you'll probably have answers soon, pandas is a pretty high-traffic tag.
 
 
1 hour later…
3:09 PM
Hi
can anyone convert c# to python ?
 
@karan hello. I don't think so.
 
Oh okay
 
 
1 hour later…
4:23 PM
o/
 
I'm trying to help out somebody who has a FASTA format data file with DNA sequences that's over 1GB in size. They're trying to get rid of duplicate sequences, so naturally a set came to mind. What would happen (hypothetically) if the size of the set exceeds available memory?
 
Memory Error
 
Does Python use swap?
 
4:25 PM
yes
 
I can't remember
 
python doesn't use swap, but the operating system might
 
I thought swapping memory was the OS's jurisdiction?
 
okay, so given that it's only 1GB or so, any reasonably recent computer should be able to handle it
 
yup I should think so
 
4:28 PM
are the sequences "sorted"
 
no
Well, I don't think so, the OP never mentioned that they were.
 
@MattDMo reading might have considerable overhead depending on reader
 
generally speaking, when you're doing that kind of bioinformatics work (assembling short reads into a longer sequence), the FASTA file wouldn't need to be sorted.
@AndrasDeak The individual sequences are being read by an iterator, so that shouldn't be a problem
 
but you can just set a ulimit (linux, right? :P) and try/except Memory Error
@MattDMo OK
 
I smell an XY problem. Are you able to comment on the current symptoms at all?
 
4:42 PM
I have zero understanding of the problem space but I'm going to suggest a bloom filter anyway. It's a set-like object that gains memory efficiency at the cost of the occasional false positive when doing "is object in the set?" tests
Unless I'm thinking backwards, if you use it to perform duplicate removal, the result will sometimes still have dupes, but it will never incorrectly exclude an actually unique element.
 
Since these sequences are all streams of "ATCGN" characters (and maybe one more, $ or *, I don't recall), some other optimization might also be possible. I was thinking a trie might help.
 
Darn, I was thinking backwards. It always detects true duplicates, but it sometimes excludes actually unique values.
If the string can only be a combination of 7 characters, then it isn't too painful to pack two characters into one byte
You can pack eight characters into three bytes if you love bit arithmetic
 
5:09 PM
You can also just dump it in the set, since 1GB isn't all that much.
 
For a 1GB dataset it's unlikely that a set representation would use more than 4GB of memory in Python. Nowadays that's surely a task that can (and hence should) be done in memory?
 
Maybe 1GB isn't much for you. [posted from my Amiga]
I have experimentally determined that I love bit arithmetic enough to write a ATCGN-to-bytes encoder, but not enough to write a bytes-to-ATCGN decoder
 
Air
What's the spec for all things crazy-go-nuts university anyway
 
@Kevin wouldn't it be enough just to define a custom codec for Python to provide both encoding and decoding?
 
I could do that but I think I would still have to draw the rest of the owl
I could write a decoder if I was confident that the operators >> and << and & and | were all at most O(log(a)* log(B)) for integer operands A and B, but I don't know if that's the case for very long ints
I guess B is always a one byte int for my purposes, if that makes things easier
 
5:26 PM
"always" as in "on any architecture that will ever run Python"? D:
 
Let's make our lives difficult and say yes
Unless that leads to the result of "it's totally impossible, let's give up and have lunch", in which case no
 
wait, I want my free lunch!
 
 
1 hour later…
6:30 PM
cbg. I've just starting trying to pick up sphinx again and my eyes are glazing over. The quickstart doesn't really help too much, and I don't think I trust the random tutorials I have dredged up. I don't suppose anyone knows a decent resource that covers a sane subset of features?
 
Air
okay... before I potentially brick something... does anyone here know ahead of time what happens if you put two different packages into Pipfile that import under the same module name?
in my case specifically, psycopg2 and psycopg2-binary
I'm expecting one will just be inaccessible but trying it feels spooky
 
Ah, nm, this guide looks sufficient!
 
 
3 hours later…
10:03 PM
when I create a new virtual env, is wheel supposed to already be included? for some reason it isn't for me.
I have to manually install it with pip or poetry
 
@Code-Apprentice me too
it's not part of the standard library
 
anyone know why my shell would autocomplete git checkout th to git checkout this_branch_has_been_merged_and_deleted_in_remote_and_in_local?
I seem to remember that something needs to be pruned after git branch -d, but I can't remember what, for the life of me
 
I can't repro with a branch I just created and deleted, but that might be too simple a test. Are you sure it's missing from git branch -a?
And what shell is that? Bash, zsh, git bash, ...?
 
I'm using zsh, but I'm pretty sure that's agnostic
 
10:12 PM
git branch -a shows me two sections - one in white and one in red. It looks like the white stuff is local, and the red stuff is remote (but I think the red is indicative of some form of "defunctness"). The deleted branch is shown in red as remotes/origin/deleted_branch
 
all my remotes are red in bash
 
github does not list this branch, so that's confirmation that it's been deleted in remote
 
I have this piece of code:
import cv2

w, h = 200, 200

vid = cv2.VideoCapture(0)

while True:
    ret, frame = vid.read()
    frame[frame[..., 0] > 150] = [255, 255, 255]
    frame[frame[..., 0] <= 150] = [0, 0, 0]
    cv2.imshow('frame.jpg', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

vid.release()
cv2.destroyAllWindows()
 
@inspectorG4dget your local probably doesn't know that
 
for sanity: I did do a git fetch
 
10:14 PM
did you try git fetch --prune?
 
It video captures in 2 colors; black and white. The formula for whether each pixel should be displayed as black or white
is if the blue value of each pixel is less than 150, or more than 150.
 
-p
--prune

    Before fetching, remove any remote-tracking references that no longer exist on the remote.
 
@inspectorG4dget you probably have a stale remote tracking branch. You need git fet... yah, what andras said
 
How will I go about to replace that random formula with this function that returns whether a color is closer to black or closer to white?
def black(color):
    r, g, b = color
    return not 0.2126 * r + 0.7152 * g + 0.0722 * b < 128
 
I was worried that git fetch -p would remove local branches that I haven't yet pushed to remote. Please confirm?
 
10:15 PM
@inspectorG4dget read the docs?
 
@AndrasDeak ok, so it isn't included automatically...and for some reason one of my dependencies doesn't declare wheel as a dependency and just craps out on an exception when it isn't installed.
@inspectorG4dget from my experience it doesn't work that way
 
I wouldn't expect "prune" to delete anything that has a branch somewhere, but I don't normally use it
 
I did read the docs. I guess my question is "does remote tracking branch that no longer [exists]" refer to a branch that was once online but is no longer online and has no local changes? Because if yes, then I need a way to prune specific branches, which I haven't been able to find
(with many thanks to @AndrasDeak and @Code-Apprentice)
 
def not_black(color):
    return (color * [0.2126, 0.7152, 0.0722]).sum(-1) < 128

...

whites = not_black(frame)
frame[whites] = 255
frame[~whites] = 0
@AnnZen something like that ^ give or take a swap of the colours
 
@AnnZen very difficult for me wrap my head around the question. Which of the following two RGBs would you say is "closer" to black: (0,10,100) or (0,11,99)
 
10:20 PM
@inspectorG4dget where did you find "remote tracking branch that no longer [exists]"?
 
@AndrasDeak oh, let me try...
 
oh sorry. s/branches/references. I typo'd while copying
 
The part I quoted from git-scm.com/docs/git-fetch is very exact
 
@AndrasDeak If I understand that code correctly, then color is a very poor parameter name...
 
@Aran-Fey well, it's a lot of colors :P
 
10:21 PM
I guess another problem here is that I don't understand what a reference is
 
@inspectorG4dget what do you mean by "online"?
 
that's nowhere near idiomatic code but I can't do more for Ann
 
by "online", I mean "on the remote", which in this case is GitHub
 
as the admin of 4 python facebook groups she can fill in the cracks and wrinkles
 
@inspectorG4dget Or to answer your question more directly. A "remote tracking branch" is a local copy of a branch in a remote. These branches start with the remotes name, such as origin/master. You may have a local master, too. These are really two different branches.
whether or not the branch has local changes does not matter. The remote tracking branch is only a copy of the remote branch and is independent of any local branches. Local branches can be configured to "follow" a remote tracking branch. This allows us to do git pull and git fetch and git push without explicitly specifying the branch. The tracked branch is assumed.
 
10:24 PM
@inspectorG4dget did you also read git-scm.com/docs/git-fetch#_pruning?
 
I'm curious, if you had a function that takes an rgb tuple as input, how would you map that over the 3rd dimension of the image?
 
So when the documentation says "remove any remote-tracking references that no longer exist on the remote" it means exactly that. It will remove a branch origin/foo if foo has been deleted from the origin remote.
 
@Aran-Fey is that a non-rhetorical question and is it aimed at me?
 
It's a serious question, but not aimed at anyone in particular
 
@inspectorG4dget And to clarify git fetch --prune will not touch any local branches.
 
10:26 PM
@AndrasDeak Thanks!
 
@AndrasDeak negative. Thanks for the link
 
@Aran-Fey The function I showed takes any array where the last axis is colour, and works with that. So it does what you seem to be asking about.
 
@Code-Apprentice this explains quite a bit. Thank you
 
@inspectorG4dget it's linked from the --prune docs
 
@inspectorG4dget That's not the question though; I already have a function for that.
@AndrasDeak I'm learning everyday :)
 
10:28 PM
@inspectorG4dget glad to help. I found Pro Git very helpful in learning and understanding these concepts. I highly recommend the first 3 chapters for better understanding of git commands and how they work at a high level.
 
@AnnZen without knowing what "closer to <color>" means, I don't really know where to begin, to "replace that random formula"
 
@inspectorG4dget The random formula is color[2] <= 150 or color[2] > 150.
 
"Pro Git"? link please? or is that just git-scm?
 
@inspectorG4dget translation: "I've found this code that works with numpy arrays, I want to replace the logic with this native python function which throws an error if I pass it an array"
 
@AndrasDeak What I want to know is how it would be done if you had to use Ann's original black function, which takes a 3-tuple as input. Or is that not possible?
 
10:30 PM
@Aran-Fey not with native numpy, no
 
@AnnZen which random formula are you trying to replace?
 
the only reasonable solution is to vectorize your function to make use of broadcasting
 
@inspectorG4dget yah, it's the book on git-scm. Google "pro git" to get the exact path.
 
That's a relief, that means my google-fu isn't as terrible as I feared
 
@Code-Apprentice The first one.
 
10:31 PM
@AnnZen First of what?
 
specifically not in the function will break on numpy arrays which refuse to support bool for a very good reason
 
@Code-Apprentice The first one I posted.
 
@Code-Apprentice ahh, much thanks. I keep learning just how much of that book I haven't fully grok'd
 
here:
frame[frame[..., 0] > 150] = [255, 255, 255]
frame[frame[..., 0] <= 150] = [0, 0, 0]
 
@AnnZen that isn't a function
 
10:32 PM
@Code-Apprentice Oh, I meant condition.
my bad
 
@Aran-Fey I mean there's an abomination called numpy.vectorize but that's terrible smoke and mirrors
 
Others here know more numpy than I do. I would try something like this:
frame[not black(frame[..., 0])] = [255, 255, 255]
frame[black(frame[..., 0])] = [0, 0, 0]
 
@Code-Apprentice there are multiple issues with that, alas
 
I was afraid so. It's my naive attempt at numpy.
 
the smaller issue is that black needs all three colour channels to be passed, the larger is that as I've mentioned not will break numpy arrays
or I guess the other way around: numpy arrays will break not
 
10:36 PM
not doesn't broadcast across each element of the array?
 
no, because not is not bool(...) which is an error
~ (binary not) is overloaded to do element-wise logical not for bool arrays, but there's also a not inside black
and black unpacks the first axis which won't work with arrays containing the RGB channel in the last axis
 
thanks for the explanation. I'll leave the numpy to the experts. I no longer have any professional reasons to learn it, at least not at my current job.
 
it's probably more trouble than it's worth getting into without a good use case
 
yah, numpy is very powerful, but difficult to get my head around without a specific application to use it for.
 

« first day (3792 days earlier)      last day (1139 days later) »