« first day (3511 days earlier)      last day (1423 days later) » 

wim
12:03 AM
I wonder if Spyder has already registered a handler for the same signal
does this return something other than the default int handler?
>>> import os, signal
>>> sig = signal.CTRL_C_EVENT if os.name == "nt" else signal.SIGINT
>>> signal.signal(sig, lambda: None)
<built-in function default_int_handler>
the default behavior for the interrupt is to raise a KeyboardInterrupt exception, but maybe spyder overrides that with a "restart the kernel" handler of some sort ?
 
@BlackThunder People who are looking at those questions will be looking at those questions (usually either following a tag with the new feature or just filtering by tags) so still no
That said, I remember that being a version issue so did you try the fix on their github issues page?
 
 
7 hours later…
7:33 AM
@smci Just rolling with wim's version of import module @ constraint here. It would not be a problem reading that information from elsewhere when not given.
I see merit in supporting both ambient and explicit constraints.
 
7:46 AM
@DeveshKumarSingh Your code is indeed not well-typed. mypy (and for that matter, your class) has no guarantee that self.check is not replaced in a subclass. As far as type checking is concerned, only the static input and output type are relevant. This information does not include side-effects on type-constraining other variables.
If you want this to be well-typed, check should return the value: def check(self) -> int and def def f(self): self.a = self.check() + 1.
 
or do a = typing.cast(int, self.a)
mypy still complains if check isn't defined in a class, btw. So "it could be overridden in a subclass" can't be the reason why it doesn't accept it
 
8:06 AM
True. After careful pondering: Outside of the function itself, mypy sees only the signature, not the body.
 
soon they'll add @typing.inline for cases like this, and not long thereafter we'll have to rename typing to java
(seems my optimism tank for today is already exhausted despite the fact that it's only 10AM)
 
Well, PEP 591 explicitly notes that it's what Java does as well.
 
user11702787
8:45 AM
I get this byte output from a device b' 4.6 g \r\n'
 
user11702787
what is ` \r\n `?
 
@MisterMiyagi Cool, that works
class F:

    def __init__(self, a=None):
        # type: (Optional[int]) -> None
        self.a = a # type: Optional[int]

    def check(self):
        # type: () -> int
        if self.a is None:
            raise
        return self.a

    def f(self):
        # type: () -> None
        self.a = self.check() + 1
 
user11702787
@Arne okay thought first that this was from the device
 
But this seems a bit off to me when I read it.
 
8:52 AM
@We..are..one it is. The device follows the windows standard for indicating a newline.
@DeveshKumarSingh I recommend against doing such setups in the first place. It means your class is can be inconsistent.
 
@MisterMiyagi You mean not using something like check to assert self.a value in the first place
 
A class should never fail based on internal state.
 
Which avoid typing issues as well
What would be a better way to do it? For more context, this is a class in pip repo for which I was trying to complete type annotations, and it has a similar structure to this
 
user11702787
so I only want the float value from the device output ` b' 4.6 g \r\n'` my aproach was to convert the bite to a string
`data = data.decode("utf-8", "strict")` use regex to filter the float from that string and set the type to float is there a faster way to do that operation ?
 
It's hard to answer "what's the best way to extract the data I need" questions without knowing exactly what your data looks like
it could be b' 4.6 g \r\n'[7:10] for all we know
 
user11702787
9:08 AM
@Aran-Fey the "structure" is ` b' 4.6 g \r\n'` or ` b' 4.0 g \r\n'`
 
so then float(b'...'[7:10].decode()) should do it
 
are there any rules of thumbs for using @property and @{method}.getter/setter in python oop?
because one article says use property when you have derived attributes and another uses getters and setters for private variables
 
if you make a getter and a setter for a private variable, it's no longer private, is it?
 
the first one's correct, the 2nd one's a java programmer
 
9:13 AM
@Permian use them if you can't achieve what you want without them
 
@Permian close the tab and pretend you never saw it
 
it's not "wrong", but it's teaching you bad habits
 
two articles say different things
 
Why does that article invoke name mangling rather than just a single underscore to indicate that it's private
 
9:17 AM
Because there's no better way to learn a language than writing a tutorial
 
I've read the whole thing now and I don't feel like it explained anything to me. If anything, I just have more questions. Good going, Data Camp
 
@Permian That one seems better
 
@roganjosh omg
i had learnt that material from a similar blog
 
@DeveshKumarSingh Ugh. pip has some dark corners as far as typing is concerned. I'd prefer to take a look at the actual code before making a recommendation.
 
@Permian The overall premise is fine but it just breezes through everything
 
9:22 AM
@Permian Use property if a) you don't want a setter, or b) the attribute is computed.
 
a) means read-only
 
@MisterMiyagi It's pip._internal.configuration , the function _ensure_load_only asserts if self.load_only is None.
 
@MisterMiyagi a) i thought property and setter went together?
b) what do you mean by computed?
 
and then used at places like set_value where we use self.load_only later
 
@Permian properties consist of getter, setter and/or deleter. Each is optional.
 
9:24 AM
@Permian no property --> no setter, but property -/-> setter
(Property is a necessary condition for setters. But leetcode doesn't teach you this jargon.)
 
@Permian the attribute value is computed via some function based on internal state of the object. For example, say you have a time duration internally storing seconds. You could define a @property def hours(self): return self._seconds // 3600.
 
@AndrasDeak why does property help here?
 
currently --strict-optional is set to False, but I was trying to remove it from the file as part of adding type annotations
 
@Permian try an example, see how it behaves
 
@MisterMiyagi ok
 
9:27 AM
Hello, I just joined
 
@Permian because property allows to only an attribute with only a getter, without a setter. By default, classes provide internal getter/setter/deleter for all attributes.
 
@AravTaneja Hello
 
@MisterMiyagi ugh
 
What?
 
thanks guys, i think i get this now
tough stuff all this closures, decorators and property
 
9:38 AM
You won't learn it and understand it if you don't try and see in practice
 
cbg-ning everyone
 
cbg
 
hope everyone is doing well.
I've got a quick question
@AndrasDeak o\ Sir
when I'm doing this on the console from random import randrange followed by a randrange(0,7), I have random integer from 0 to 7
however, if I'm doing this
def new(rand_val = randrange(0,7)):
    print(rand_val)
 
2592
Q: "Least Astonishment" and the Mutable Default Argument

Stefano BoriniAnyone tinkering with Python long enough has been bitten (or torn to pieces) by the following issue: def foo(a=[]): a.append(5) return a Python novices would expect this function to always return a list with only one element: [5]. The result is instead very different, and very astonish...

default arguments are evaluated at function definition time, not at function call
 
@AndrasDeak and this mean ...?
 
9:47 AM
this is only a "problem" when the default is mutable, or when it should be dynamically regenerated like in your case
@AndyK this means you have to use some kind of sentinel value as the default and generate the default value in the function body
def new(rand_val=None):
    if rand_val is None:
        rand_val = randrange(0, 7)
    print(rand_val)
 
10:15 AM
@AndrasDeak Only realised it > 2 min after typing. :/
@AndyK Did you encounter this in some actual code, or is this just toying around?
 
 
1 hour later…
11:24 AM
stackoverflow.com/q/62041655/4799172 typo. I'm kinda curious how that jumble of city names hones in on a bar in London with no apparent link to those other city names. Some super-level marketing just waiting for someone to screw up on Nominatim? :P
 
if a:
	b = "foo"
while True:
	if a:
		print(b)

b = "foo"
while True:
	if a:
		print(b)
what do you think of changing the first variant into the second one, just to not have the pycharm warning, this var might be unused?
in unnecessarily define b even though I don't need it sometimes. But I satisfy my ocd, by not having this warning :D
I don't get why pycharm can't understand this simple example. Like obv b will always be defined if the code reached print(b) sounds pretty easy
 
Why would you enter the while True loop if a is falsy?
 
oh it does other stuff ofc
 
@Hakaishin PyCharm has no guarantee that a has the same truth value on both tests.
@Hakaishin and initialising b is costly, I assume?
 
why not? It goes from line a used the first time to a used the second time and sees that a was never written to
no it is literally a string in this stupid trivial but still annoying me example
 
11:37 AM
We have no idea if a changes value inside the loop
 
@Hakaishin so what? We have threads, and we have arbitrary __bool__.
 
no it doesn't change
 
@Hakaishin is there a reason why you don't use it as a literal print("foo"), then?
 
I use it multiple times, so to not copy paste
 
Okay. Without knowing anything else about the problem, the second variant seems fine to me. Negligible overhead, and actually less code complexity.
 
11:41 AM
Cool, thanks. That's what I thought too. But it still annoys me every time, I have to change working code, to get rid of warnings
 
Dunno. These warnings are often not precise in their message, but show that your code has notable complexity.
Depending on how the rest of your code looks, it might be worth factoring out the entire b part - since, apparently, it is separate from the rest of the loop body.
 
I mean to be concrete, it's about showing an image with cv2. So I think it does belong in the body. I do some image processing. Either saving the results only or also displaying it. So it does belong in the loop
    window_name = "CSI Camera"

    if camera.isOpened():
        if args.show_img:
            window_handle = cv2.namedWindow(window_name, cv2.WINDOW_AUTOSIZE)
Also is there a good way to prevent pythons gc to not collect my handles and other objects which need to stick around?
 
FWIW, I stopped paying attention to Pycharm's warnings roughly 10 minutes after I first started using it. The vast majority of them isn't helpful IMO
 
No I really like them. And they helped me catch a few bugs tbh
 
11:59 AM
@Aran-Fey I find them rather helpful. Turned off only a few.
 
Let me rephrase that: Many of them are equally useless as they're useful
def foo():
    foo = 5
"shadows name 'foo' from surrounding scope", boohoo
 
how is that not usefull? You just destroyed your function, which is a terrible idea
 
I didn't "destroy" my function, I just can't access it inside foo. Which is irrelevant unless foo is recursive
 
@Aran-Fey I've seen people bitten by this. But it's more helpful for people with less experience.
 
Sure, it can be helpful, but overall I find there are too many "false positives" so to speak
 
12:11 PM
That's fair, I'm pretty sure linting is a holy war because of the "false positives" :)
 
I mean even if false positiv. It helps you think more about your code structure, which I think is always good
 
12:32 PM
Cbg
 
cbg
@Aran-Fey I seem to recall in VB that use to be a way to set the return value of the function :)
 
Yeah. Much better than a return statement, isn't it? :p
 
it's amazing it didn't catch on more :)
 
@JonClements I somehow remember MATLAB having caught that.
Or at least something very close.
function ave = average(x)
    ave = sum(x(:))/numel(x);
end
 
1:12 PM
@Aran-Fey I agree that the inspections don't quite report what's wrong. They're often enough a hint to me that something is fishy, though - my function and its internal variable don't represent the same idea, why should they have the same name?
 
well, sometimes words are vague enough to be a suitable name for both a function and a variable
 
better strive to be precise, not vague
 
I don't think that's true as such. Language is rarely precise; usually you infer some information from the context. For example, "hammer" can be noun or a verb. It would be perfectly fine to name a function "hammer" and then also create a variable "hammer". I don't think it's necessary to name them "hammer_the_verb" and "hammer_the_noun"
 
hammering and hammer :)
having a var and a func have the same name just sounds like trouble. But you do you
 
1:35 PM
umm... I'd have thought this'd come up before in some way... anyone know if there's a reasonable dupe for this one
(the confusion here appears to be thinking that builtin factory types are keywords when they're obviously not...)
 
user11702787
2:06 PM
is there a way to "translate" C++ Code into Python if you dont know C++ ? or just learn the basic structure ?
 
2:17 PM
Umm... what do you mean by "or just learn the basic structure"? I'd have thought the first part of your message up until the "?" was pretty much obvious though?
 
2:39 PM
cbg
 
2:58 PM
cbg
 
@We..are..one If you have some C++ code, and you can wrapper it into a C-style lib (by making a non-object API wrapper for classes), then you can call the C lib from Python.
 
wowsers... one of my old clients is reaching out to me again... they had someone else doing a few design bits, but it appears they kinda ventured into trying to do a DB update - which they've done by joining tables by the primary key rather than the customer_reference field... needless to say - that's completely yammed up so very many things
 
...and they don't have a backup?
 
shrugs - waiting for 'em to give my credentials access back - I know I put plenty of redundancy and a mix of cold and semi-warm storage for backup
replication is across 4 data centres anyway so
 
3:28 PM
In [23]: @ass_ret
    ...: def foo(x):
    ...:     foo = x + 1
    ...:

In [24]: foo(1)
Out[24]: 2
 
Oho, how does that work? I would've assumed ast-rewriting, but AFAIK that doesn't work in a REPL
 
Bytecode rewriting.
 
@IljaEverilä probably from fortran
 
@AndrasDeak Makes sense.
 
that's how fortran functions (not subroutines) work, or at least used to work (I have no idea about modern fortran)
 
3:32 PM
@IljaEverilä Is there a module for that?
 
types and dis came in handy for this little bastardization.
 
@MisterMiyagi when I was creating a piece of code actually.
 
So basically it's magic, ok
 
Mar 18 at 14:30, by PM 2Ring
@JonClements Pascal functions also use that syntax for returning a value. I found it a bit weird at first, coming from C.
@Aran-Fey That's reasonable, but I'd still prefer to not shadow a function's name with a local name.
 
okay... took me a few minutes to remember, but everything on the system has "triggers" that cause any updates/insertions/deletions to be logged into a separate system... so can use that to "revert" the DB as it were... which of course will cause another few hundred million rows of triggers sighs
@PM2Ring was fairly sure I was having a deja vu moment when I said that :)
 
3:50 PM
@JonClements that sounds... horrific
Why wouldn't you (they) back things up before doing such a major update? :'(
 
I've not been involved in the project for well over a year now
see ^^^ - it seems the web design guy was trying to be helpful
 
Yeah, I meant "they" but was invoking some sense of introspection
 
(why oh why you'd give a design guy access to the core infrastructure is beyond me)
oh, there's hot, warm and cold storage for the DB's, there's also replications and backups
 
You gonna have to roll back every one of their changes in the end?
 
the thing is, I wasn't told about this until now, so everything's already replicated and the hot storage has already been affected, the warm and cold are now too old
@roganjosh ultimately yes
 
3:56 PM
@JonClements A sound investment they made by going forward with that project, then :P
 
hey... it's a great DB distributed and auditing infrastructure setup - just never thought they'd let some guy that was doing a few graphic changes enough access to actually do that yam
 
Again, I'm not not directing it at you. I mean them. They let some web guy loose on a project and now have to pay someone external to come reverse that project
 
Sure - didn't think you meant me anyway. Gonna grab a cuppa and finish off a SQL query... but unlike some it seems, I'll make sure it actually does what it's meant to do
 
@AndyK Might want to reconsider your design. There are only few cases where such a default argument is legitimately desirable. Stumbling over baked-in defaults often indicates a design smell – for example, a function new should generally not return an existing value, i.e. not its argument.
 
4:14 PM
omfg... I'm yamming sure there was an index on a table that doesn't appear to be there now
this is frustrating
 
someone fixed it :P
someone knew a guy and the guy fixed it
 
obviously "fixed it" :p
/me growls
okay, so re-creating that index inside a transaction block of updates from the audit tables back to the main tables... going to take ages to run, but not my problem... see what happens later
 
hehehe
except it probably wasn't garbage to begin with
 
yeah, but now it's his (in a way)
 
4:22 PM
guess at the end of the day - don't let your designer have access to your DB trying their best - instead of someone that spent huge amounts of time and deliberation designing and implementing the back end (with 20+ years of experience) ? p
 
pizza from the pizza shop
but at least one man's incompetence is another man's job security
 
@JonClements but did the new schema look fabulous? Trumps practicality every time
 
Fabulosity trumps practicality? Isn't that in the ZofP?
 
@roganjosh I used to connect to datacamp for supplement material when teaching until students kept turning in weirdly structured code or getting confused between my style and theirs (people learning tend to be very literal) so yeah, its very Java-ish Python overall
 
On my sorta-retired project, I asked them to pull a backup and send me a copy so I could investigate something locally. Apparently the IT dept's "system" for doing this takes over a week so I occasionally get calls saying "this doesn't look like it's working" and I give them a reassuring "I know", and the call ends.
 
4:31 PM
sounds worryingly familiar
 
lol @JonClements that's what half my contracts feel like :)
 
don't even get any satisfaction out of "I told you so..." any more :p
 
that stops being fun at 10 years
before that your still trying to fight it, at 10 years you realize you can't win and your best fun is going "okay, your bill will be X amount because of that" and just getting to work
 
@PaulMcG I think so. It definitely mentions something about practicality, so it's a safe bet to assume it is indeed part of it :P
 
bbias... dinner and then need to look at some C# stuff
 
4:42 PM
(you need to see sharp when you look at it)
 
5:08 PM
@LinkBerest I'm reminded of a line from the Monty Python Fish Licence sketch. That's not a Python tutorial! It's a Java tutorial, with the word "Java" crossed out and "Python" written in in crayon.
 
Hello
 
5:29 PM
Hello
I want to increase sound volume by multiping an AudioSegment with a number.

Error: TypeError: can't multiply sequence by non-int of type 'float'
Maybe i must convert self.sound_volume to dB and then use + operator.
 
Rather than using [1] * 0.1 you need to use a loop in some form, such as a for loop
 
Does system sound have any concept of dB?
It seems unlikely to me that there is some calculation involving dB when I move the volume slider on my laptop... even less likely when I'm sliding the volume slider of an app which is tempered by my own system settings
 
# boost volume by 6dB
louder_song = song + 6

# reduce volume by 3dB
quieter_song = song - 3
But self.sound_volume is beetween 0 and 3
0 is mute
1 is normal volume
3 = 3*slice
How can i use self.sound_volume to change Volume of AudioSegment?
 
Ok, let's stop this here. We have no idea what your code is and, on every occasion that you post blocks of code, you mess up the formatting... for months, now
 
Uh, no @roganjosh. I'm clairvoyant.
 
5:41 PM
@JossieCalderon Hopefully you're telepathic too, so you can beam the solution over :)
 
When you read some of the comments on some of the questions, you can wonder , on some specific occasion, what is going in people's mind. The best answer is often Wu Wei or let it go.
 
sound_volume_in_dB = 10*math.log((self.sound_volume),10)
That's all i wanted!
 
I got to 3:36 on that video and I have no idea what it has to do with people commenting on questions
 
@roganjosh doing can impact your ability to complete the project, because it's better to have something out of your control to help you.
 
5:56 PM
We're on different pages here because I don't understand that either
 
Like waiting for one of your genius answers instead of rushing to submit my code.
 
@ChrisP Using logarithms to adjust sound volume is going to be rather slow. Better to use multiplication. And for speed, use Numpy, unless the sound library you're using already has a function to handle that stuff.
 
6:10 PM
Hi
Can anybody tell me how to use asyncio.lock??
I want to run my non-coroutine function like in a queue, so that only one function gets processed at a time
 
How have you tried to use it, @White Lightning?
 
6:25 PM
@JossieCalderon the illusion of action , yes. But this video , is it the best one to illustrate that? Thank you, though.
 
@BlackThunder Can you clarify your problem? The linked docs have an example on how to use asyncio.lock.
If your reply is "but my function is not async", then directly go to my counter question "then why do you use asyncio?".
 
class Third(First, Second):
one sources online says it looks right to left (ie second, then first)
another says its left to right
which is it?
 
Is one of these sources the official documentation?
 
See for yourself - print(Third.__mro__)
 
(It is left to right, by the way.)
 
6:37 PM
@MisterMiyagi thought so
no it was from blogs
 
Someone wrote a blog saying right-to-left? Where is it?
 
i cant find it now, ive read so many
 
Not depth first, tho. If B and C inherit from A, class D(B, C) mro is D, B, C, A, not D, B, A, C (which is what I would interpret "depth first" to mean)
 
6:57 PM
@Permian If you haven't already seen it, you should probably take a look at super() considered super by Python core developer Raymond Hettinger.
 
wim
7:18 PM
@Permian Maybe you were reading the blog upside-down or through a mirror
 
it was written in Hebrew
 
wim
@Aran-Fey it's usually not the name shadowing itself that's a problem, it's the mental stumbling block of seeing the same name used for two different things. readability counts.
 
right, but I think it's usually pretty easy to tell that the variable isn't being used as a function (because it isn't one), so it's hard to get them mixed up despite them having the same name
 
@PM2Ring oh! I'm gonna steal that - this is a huge problem with tutorials as I've seen them
Esp. when dealing with how Python uses "abstract classes" versus Java (these are horrible most of the time - so shoehorn Java idea as Python)
 
wim
it is, but it takes the wetware an extra second or two to parse. you've got to pause and figure out if it's some kind of clever monkeypatch or what
 
7:29 PM
plus the IDE might colour it differently
 
especially the tutorials which use a bunch of if/else if statements in __init__ to have polymorphism instead of using annotations
@AndrasDeak do you need it translated?
 
I agree that it's not ideal, but sometimes it's the least bad option IMO. The 3 other options essentially being 1) use a more detailed name (hammer_the_noun), 2) add an underscore (hammer_) and 3) use a different name altogether (the_hammer? percussive_maintenance_tool?)
 
@LinkBerest only if that swaps the mro!
 
My problems with the 3 options being 1) the super long name hurting readability even more, 2) having to remember to append that pesky underscore, and 3) not being able to think of a suitable name
 
Choose pmt. Satisfies the requirement without being an improvement!
 
7:36 PM
great :P
 
heh, my last analyst job was all about the MRO but I don't think that's what you mean (determining the maintenance repair operations cycle of aircraft)
also swapping those would be bad ;)
 
@Aran-Fey Pro challenge: Satisfy the line length requirement by using a linter with integer overflow. ;)
 
@LinkBerest did they use percussive maintenance tools as part of the operations?
 
hahaha, that's... ambitious
 
@LinkBerest I was talking about Permian's yet another random-blog-induced confusion
 
7:38 PM
At a utility engineering company I worked for, it was called the "BFH Alignment Tool"
 
only when the engine wouldn't fit
heh, if I see "wordpress" or "medium" in the uri of a tutorial - I just take it with a grain of salt
 
any special feelings about seeing "github"? Asking for a friend
 
Any platform that lets anybody write their thing is dubious. The actual source is the important bit in this case, not the platform.
 
Speaking of words that can be verbs or nouns, here's a lovely version of Massive Attack's Teardrop, performed by the Live From Here house band and featuring Sarah Jarosz on vocals. The band are nominally folk / country / bluegrass players, but they're very versatile, and I reckon they do a pretty good job of playing trip hop.
 
My last three recommendations (looking at Google) which have those in their name are: "Why you shouldn't learn to be a data scientist", "You should be a data scientist and here's why!", and "The statistical Greek Alphabet in Python"
The last one is actually really nice (low-level but nice)
 
7:48 PM
Link please?
 
@Aran-Fey same
 
:(
 
honestly if its an academic journal or industry leader I'm more critical (errors in testing, bias, all that) so its kinda backwards for me :)
I will note, I'm only talking about links I give/get for education or just to read. If I'm talking hiring working collaborate code trumps all (so own website, github, bitbucket, etc are all good) - but I don't know what context you were asking about github in
Also, considering my last answer on SO is to point someone at a github issue - if its a package I use it also gets more (and more critical) attention
 
same context as the other 2 websites you mentioned - tutorials
(yes, I was indeed crazy enough to host mine on github)
 
oh, then yeah same (I use all of them mind you and found all of them to be the same mix of good, bad, and ugly....err, yours looks good of course :) ;) )
 
7:59 PM
good, good, that's a relief
 
@Aran-Fey any ideas for your next article?
 
well, of course I have 2 in the pipeline that I'm struggling with, but I'm also thinking of tackling a super-beginner-tier topic like "what's the difference between print and return"
 
^ I get tons of question on that from students
 
but realistically speaking there probably won't be any updates for a while
 
(mostly all fall along the lines of Java has type in the function signature so Python's way confuses them)
 
8:07 PM
@Aran-Fey Good choice. Maybe include a sidebar about REPLs printing the result of unassigned expressions (that aren't None).
 
@Aran-Fey Just get famous enough so that aran-fey.github.io becomes its own brand.
@PM2Ring REPLs versus Notebooks versus Terminal versus IDE seem to cause a lot of confusion indeed.
 
@PM2Ring good idea, that probably plays a big role in the confusion
 
online REPLs versus REPLs versus Console versus IDE is the tutorial I'm writing right now
 
For the sake of not getting anyone's hopes up too much, I feel like I should explain that I'm the kind of person who invests their spare time into one particular thing for a few months before moving on to another particular thing. Right now, that thing is watching youtube videos. When it'll return to "write stuff for the website", I don't know
 
Watched any good Hitch-slaps recently? :P
 
8:12 PM
the whole "move online RIGHT NOW!" orders states came out with combined with teachers (all grade-levels) trying to adapt caused a lot of confusion in this area....and I want to avoid the headache next year (I'm writing two - one is "how do I use online repl.it as an instructor" the other is "what's the difference between ...." for students)
 
@roganjosh No, actually :( I'm starting to get into longer debate videos, but I've unfortunately discovered that I can't listen to Hitch for longer than a few minutes. He tells nice stories, but stories isn't what I want to hear
I've been watching Rationality Rules and The Atheist Experience, mostly
 
He didn't used to be all about religion. The monologues aren't always great, but on older videos he takes calls about politics etc. which is a bit more dynamic
 
hey guys i made a hat matrix code it works but its kind of slow is there anyway i can make it more efficient

    hat_matrix_array = []
    for i in range(processedTrainingImages.shape[0]):
        print(i)
        xtx = np.dot(processedTrainingImages[i].T, processedTrainingImages[i])
        inv_xtx = np.linalg.pinv(xtx)
        x_inv_xtx = np.dot(processedTrainingImages[i], inv_xtx)
        hat_matrix = np.dot(x_inv_xtx, processedTrainingImages[i].T)
        hat_matrix_array.append(hat_matrix)
 
Hmm, not sure if I'm interested in politics. But for the sake of Hitch I'll give it a try
 
8:24 PM
Hammered
 
@KhwajaHussamQuasmi I suspect I'm not the best person to answer that, but the print call will be really expensive if you have a lot of iterations
 
@KhwajaHussamQuasmi beyond removing print maybe you could use QR decomposition over SVD but the asymptotic complexity maybe the same - I'm working that out in my head so might be off. What's the goal of this?
 
can a print call be expensive? doesn't it just print the value below is the main computation
 
inexpensive things can be very expensive depending on the size of the data/complexity of the operation going on
 
you can always test that yourself with line_profiler
 
8:28 PM
@LinkBerest its for a face recognition system have to make it using linear regression
@roganjosh 40 iterations
 
QR is linear algebra and if you have the QR decomposition then setting up the hat matrix should be fairly simple (I just would have to work out if its more efficient for this)
 
i dont mean to bother you guys but i still dont get the concept of it being expensive. like isn't "i" incremented like its just a count
 
That's not the expensive part. The expensive part is passing it to stdout
import time

start_1 = time.time()
for x in range(10000):
    y = x ** 2 # some pointless calc
    print(y)
print(time.time() - start_1)

start_2 = time.time()
for x in range(10000):
    y = x ** 2
print(time.time() - start_2)
 
however if its based on facial recognition SVD would likely be faster (I would be surprised if it wasn't)
 
@LinkBerest ill go through this
 
8:36 PM
It's not scientific, but that simple test gives:
0.5890011787414551
0.003988504409790039
 
@roganjosh oi kind of get it now
 
If you want to time things normally, make sure you use something like timeit. That was just a throw-away example to highlight the issue. Still, it doesn't answer whether you can do better with your existing code, but I don't think I'll be able to help with that part
 
i removed the print(i) but tbh i didn't notice any difference ill look up qr and svd decompositions
 
@KhwajaHussamQuasmi Don't try QR with this, its stable but not efficient. The main methods for facial recognition are eigenfaces (which is your SVD & Principle Component Analysis or PCA), linear discriminant analysis (LDA), independent component analysis (ICA) and locality preserving projections (LPP)
In your case you may be able to find a more efficient equation (and I would definitely read up on PCA analysis for Facial Recognition) but I would bet its in the normalization that you have the most area of "increasing efficiency"
for instance, reading up on why (due to lighting) the first three results are thrown out in certain applications
 
8:51 PM
:49486769 I have to it using LRC  can't use PCA or the other eigenfaces
i fixed the code turns out the hat matrix wasn't slow it was the down sampling rather using

cv2.resize(img, None, fx=downSampleConstant, fy=downSampleConstant, interpolation=cv2.INTER_CUBIC)

i downsampled my matrix by slicing
return img[::12,::10]
 
sorry I meant look up SVD with eigenfaces (they both use this)
 
@LinkBerest o okay ill go through this also
 
either way, 90% of the time if my method isn't efficient and my algorithm doesn't have any obvious flaws: I check my data is good (clean, normalized properly, etc) before looking at the algorithm
 
@LinkBerest but that also needs expertise, considering "algorithm doesn't have any obvious flaws"
 
catch-22 isn't it ;)
 
8:56 PM
Not really. One has to learn profiling :P
 
I think my original profiling when I was younger was: "when I get to that point where I'm about to pull out my hair, I would look at the cleaning part and go.... oh!"
 
is there anyway to slice my matrix down to 10x10 automatically without having to tell how much to slice away ?
 
@KhwajaHussamQuasmi depends on what you really mean. arr[:10, :10] will give you at most 10x10
 
9:12 PM
stackoverflow.com/q/62038456/4799172 needs more focus. There is a lot of infrastructure needed to get away from their broken use of global
 
9:22 PM
Closed, thanks guys
 
10:12 PM
cabbage!
 
cbg :)
 
Hope all you are fine! I'm struggling with an interesting problem that I've never met before. When I import any python module, it says [00:00:00] Reading files |||||||||||| and it takes forever.
What can cause it? Any advice?
 
Importing where?
There normally isn't some kind of timer
 
python shell or running a python file, doesn't matter
It happens in the AWS instance, I tried different conda environments, then removed conda and installed modules with plain pip, but it seems it doesnt have to do anything with those
 
I've never seen this. What is the OS? You're saying that import pandas (or some heavy library) in a python shell triggers some counter?
 
10:32 PM
Ubuntu. It doesnt have to be heavy library, it happens while even importing urllib3
Not only in python shell but also while running code as a seperate python file
I'm using python3.6.9
 
I've never seen this phenomenon and I can't even begin to think of suggestions, sorry. It'll have to be picked up by someone else
 
Thanks for your interest!
 
@roganjosh are you familiar with AWS stuff? Sounds like an AWS-specific problem. Like, AWS lambdas execute a single function and I think they pull in everything from scratch each time. Such a progress report would not be out of the realm of possibility in this situation. But as I've never used anything like this myself I'm only guessing.
 
@AndrasDeak Not really. I suspect it's some subset of that massive umbrella but I can import big libraries with no effort, and the whole point of AWS is scalability... so it seems odd there would be some progress bar on an import
 
@KeremZaman just to be clear, if you only have import urllib3 in a python script and try to execute that, you see this message and it gets stuck?
if there's anything non-trivial about the situation you should prepare an MCVE to help whoever knows AWS help you
 
10:49 PM
@AndrasDeak yes, I just tried it and same thing happened, it gets stuck.
I think it's an AWS specific problem, too but I'm not sure I can make a reproducible demo for this problem
 
Tried it in what, exactly?
Andras mentioned Lambda but there are plenty of other services
 
Oh sorry, it's EC2. To be more specific, it's p3.16xlarge spot instance with deep learning ami but I also tried without the environments that deep learning ami provides
 
There are several ways to set the AWS environment to print timestamps at the start of jobs or file loads (including imports) so probably that
 
But why does it get stuck?
 
Because its not configured correctly: are you using AWS Glue (which would have major problems with pandas - doesn't work for C only pure python), EMR, Lambdas?
I would research "how do I import packages in Amazon AWS _____?" then try what you find -> then ask if problems
where ____ = whatever implementation your using
 
10:59 PM
No, not any additional AWS services. I use ec2 instance like a local computer.
 
Ok, which is what I thought when you said EC2 instance
So are we sure we're talking about imports and not just pip install?
 
Yes definitely we are talking about imports
 
<-- it can't be pip install because they're importing from the standard library
I'm back to my original position; I'm out. I have no thoughts on how this can be approached or debugged. You're credible enough for me to take time to try, but I have absolutely no idea where I'd put my efforts
<-- watched too much Dragon's Den recently. Also, I need sleep. Rbrb
 
Thanks for your effort! @roganjosh rbrb!
 
11:22 PM
@KeremZaman I just spun up a new instance of ec2, connected, made a quick script and ran it (with python3 run_test_urllib.py) to check if they've changed something (I rarely use ec2 as just a virtual machine anymore) and saw no issue. So I would definitely guess at a configuration issue -> I would check that you are logged in with full permission and that you haven't changed any default settings....I might also check if there is any issues in the area as it could be a connectivity issue
but again it would be guessing beyond that
 
@LinkBerest I guess there may be some IO/CPU usage limit issues
 
That's still a configuration issue with virtual machines (granted also a pricing issue with AWS but then they are just setting an upper limit within the configuration)
anyway, I'm out. adios all
 
Thanks for your help @LinkBerest , goodbye!
 

« first day (3511 days earlier)      last day (1423 days later) »