« first day (3070 days earlier)      last day (2103 days later) » 
00:00 - 22:0023:00 - 00:00

00:35
nice!
can someone help me with multi line regex with pipe characters
 
1 hour later…
01:42
Any good ways for hover-over or variable inspection with a jupyter notebook?
01:54
whats up everyone
jupyter notebook is making me angry, because I can't figure out how to debug lines so that I can try code snippets on the active variables
I don't really use jupyter notebooks, so I can't help you. However maybe you could try pasting the code in a real python text editor to see if it's simply a syntax error?
I don't normally use that garbage either. Yeah, I could write everything in a real python IDE and then move it into the notebook.
Its so painful to use I'm almost certain I'm doing something wrong.
Show me a picture of your notebook.
I just learned it in a couple of seconds.
@Mikhail
02:14
@Mikhail yeah, you probably are, sorry :c
So, how do I put a break point to inspect a variable, and bring up a python console in Jupiter?
IPython has its uses, but that isn't one of them
use an IDE or something.
So, is it not possible?
I don't know that; it might be but you'd probably be able to do it 10x more effectively on an IDE
Basically, there was a decision made to use this crap for course material (because its easier) and I need to test it out.
02:19
protip: if you want to inspect something at some point, it's best to break the cell and continue your code in another one
Ah, you're a C++ user. Okay, I forgive you
I've written a few internal python tools that are used by over 5 people :-)
02:45
Cbg
Straw poll: is having a seperate function that is wrapped by a method of a class and a method of child class bad form, or sensible? I'd like to take a setting (it's a default image) at the app level, at the Class(room) level and individually at the Student level (Student classes belonging to a Class class.
Said method would essentially do similar things, check image exists, process it if needed, copy to application folder, with names and locations particular to each use case - so using the same function wrapped by methods on each of the classes seems DRY to me - how about the room?
 
3 hours later…
06:09
cbg
I got a question, I want to implement search and filter function. and for now I know I can do it by following sql.
select * from t where country='UK' and info like 'asdnasda'.
if the user don't input the country, I have to build another query sql, like:
select * from t where info like 'asdnasda'.

But what I want to know, it's whether I can do it like if this filter condition doesn't input just match all. for example something like:

select * from t where country='%s' or "*" and info like '%s'.
06:30
Anyone help?
@Mikhail my impression is that jupyter was meant to be pretty. Period. Debug with a debugger. I can recommend pudb
@FrankAK perhaps just skip over "where country = 'UK' and" portion if no input is given while constructing the query?
06:53
yes
How can I do that? I tried to ask at (stackoverflow.com/questions/55135980/…), but no one answer it right now.
@FrankAK you expect an answer within four minutes?
Haha
and why are you asking in the Python room?
SO, Can I just say because I love python
I love you guys :)
is there a more appropriate active sql room actually? I've been wondering too
06:58
you know there is an SQL room?
Okay, I see.
Well I mean he's using python, so it's reasonable to ask here
Maybe SQL doesn't have what he's looking for, but python's mysql module (or whatever module he's using) does
@Aran-Fey Yep, I do. but I just make liiiiitle joy, to say I love py.
07:31
serveo.net down for anyone else?
@towc yup
fantastic
Pro-tip: isitdownrightnow
"Down for more than a week" isitdownrightnow.com/serveo.net.html
it doesn't say anything about a week for me
it was working well yesterday evening
huh
oh, I see the week now
well, this is confusing
Perhaps it was also down the last time they checked, dunno
07:35
could be
 
1 hour later…
08:55
cbg
cbg
Is there a way in python to know what function or object you're currently in?
you can inspect the stack
nice, looks good
is it reliable?
I remember some vague warnings about looking at the stack in python
I have a guess but only a guess so I'll refrain from answering
I'd expect caveats to be listed here
>>> def foo():
...     def bar():
...         return inspect.stack()
...     return bar()
...

>>> [stack.function for stack in foo()]
['bar', 'foo', '<module>', 'run_code', 'run_ast_nodes', 'run_cell_async', '_pseudo_sync_runner', '_run_cell', 'run_cell', 'interact', 'mainloop', 'start', 'launch_instance', 'start_ipython', '<module>']
without ipython fluff:
>>> def foo():
...     def bar():
...         return inspect.stack()
...     return bar()
...
>>> [stack.function for stack in foo()]
['bar', 'foo', '<module>']
that's quite the overhead in ipython, actually
09:14
Hello everyone,

I am having some issues trying to use 'multiprocessing' module... May I ask you how to solve them?
I am starting working with neural networks using Keras, and I am interested in doing several hyperparameter sweeps to see which combination works better.

Anyway, to do so I pretend to use 'starmap' function and it seems not to run even while using the most basic example I could come up with.
Which starmap?
multiprocessing.Pool.starmap()
I see
Can you remove keras and put together a very small example to see if that works with starmap?
Sure. What I am actually trying is:
from multiprocessing import Pool
import itertools

if __name__ == '__main__':
    a, b, c, d, e, f, g, h, i = [1, 2], [3, 4], [5, 6], [7], [8], [9], [10], [11], [12]
    comb2 = list(itertools.product(*[a,b,c,d,e,f,g,h,i]))
OK, so far so good
09:21
It does not output any error message. It just gets stuck running forever
there's no multiprocessing being used in that...
mp = Pool(1)
results = mp.starmap(addition, comb2)
mp.close()
mp.join()
Forgot the last part, sorry!
that's also in the if __name__ block, right?
NameError: name 'addition' is not defined
Yeah. Now I see that this apparently works. But when I import my self-made modules to the file, it starts running infinitely
So that's what you need to debug. Odds are it's running a lot because your comb2 is huge, or because the function you're applying takes too long to run. Monitor your CPU and memory use.
09:25
Ouch. Let me try copying all my code once again...
also, check if you need to convert comb2 into a list; odds are starmap will eat any iterable, so you don't have to have all of it in memory
from multiprocessing import Pool
import itertools

def addition(a,b,c,d,e,f,g,h,i):
	return a+b+c+d+e+f+g+h+i

if __name__ == '__main__':
    a, b, c, d, e, f, g, h, i = [1, 2], [3, 4], [5, 6], [7], [8], [9], [10], [11], [12]
    comb2 = list(itertools.product(*[a,b,c,d,e,f,g,h,i]))
    mp = Pool(1)
    results = mp.starmap(addition, comb2)
    mp.close()
    mp.join()
yup, can confirm it works fine
However I have been recently told that Keras does already paralellization by default, is that true?
I don't know but I wouldn't be surprised if it did
09:28
So that I would not need to use starmap at all
you can monitor your CPU use to find out
parallelization only helps if you have available unused capacity
I will do so... thank you so much for your help Andras, I really appreciate it :)
So 1. check CPU and memory to see if it's doing actual work rather than idling, and whether you're running out of memory which causes it to "hang", 2. if it's doing actual work, you're probably making it do a lot of work and it's merely taking its time to finish. In this case I'd try changing starmap to imap (and your function to addition(*args) to go with it, and see if you get results one by one
no problem
09:30
I will also check what is wrong with my modules, as it gets stuck when I import them
Actually module* in singular!
ah, that's probably a very different issue then
keras+multiprocessing might not be trivial stackoverflow.com/questions/42504669/…
I see..
I will have a look! However, odds are that I will end up coming back with more questions... Let's hope for the best! Thank you back again Andras! :)
Good luck, I won't be able to help with the keras part though (others might)
no worries
09:46
@Arne If you are looking for something reliable I suggest you use multithreading modules, You actually need to find out which process/thread you are in.
what do you mean?
Lazy solution is to run multiple python scripts from bash :-)
Why are the both of you talking about multiple threads and python scripts when Arne was just asking about the function you're in during execution?
cbg. looks like an interesting convo
re keras, the parallel processing there would be considered when you need to train 1 neural network and utilize all your resources.
if the goal is to run through various hyperparameters to find some good ones, you can consider grid searching. Alternatively, keras multiprocessing wont really help you, you're trying to run entire model code multiple times. multiprocessing "should" be the way to go.
but i am not sure how well that would all behave when it comes libraries like keras, that do a lot of work in C.
10:02
:45620257 Hmmm, Because what I'm saying is running multiple threads to track executions or event you can do something like this:

class myThread (threading.Thread):
   def __init__(self):
      threading.Thread.__init__(self)
      self.FncName = 'INIT!'
   def run(self):
	self.FncName = 'Run!'
that's...a horrible XY problem
^
if i have a simple program, i now have to worry about creating threads, and making sure everything works fine
"In case you don't have enough problems, add threading"
¯\_(⊙︿⊙)_/¯
Also the library has built-in threading support
which library?
10:07
Keras?
hes probably referring to keras
So...I suggest clarifying which one of two conversations you're referring to.
yes, keras supports multithreading when you want to train 1 model quicker by using all resources.
especially if the last message wasn't related to what you're talking about
context saves lives
And the kernel saves the context
10:09
:|
@ParitoshSingh Hmm, I almost use multithreading all the time in my scripts and everything works fine
@ParitoshSingh Yeah, my intention is to use starmap to do grid searching in parallel
Yeah, i have never attempted to pair that with a model script, but you "should" be on the right track, assuming keras, or to be more specific, tensorflow (assuming tf backend) behaves well inside multiple processes. @MisterTellini
(sorry for double ping)
Don't worry!
I am hoping that's feasible... but it's 'working' all the time, so I will see how to solve it
(And by working, I mean stuck...)
10:20
start with a mock model. Something trivial that takes perhaps 10-15 seconds to train.
try to use your starmap around that model, and see if it gets stuck
if it does, you know theres some kind of issue you need to resolve first. searching online once you know what youre looking for makes things easier to debug
also, as I said, switch to imap which doesn't block
ooh. ive never used imap before. i guess it doesnt try to get results in the order the items/jobs were arranged yeah?
that might make it troublesome trying to figure out which hyperparameters gave which results.
It does, but it's lazy. There's imap_unordered if you don't care about order
oh. ooh. til
so if the issue is a lot of combinations + lot of work for each combination, then switching to "generator of combinations" + "generator of results" then they'd see results trickling out one at a time
but I already said as much half an hour ago
10:26
concurrent futures map behaves like imap ?
no idea about async
but there's also async map in multiprocessing :P
imap acts like python 3 map acts like python 2 imap
10:44
But, if you are eager to train a set of hyperparameters, I guess starmap will be necessary as it is the only one that deals with N>1 parameters, am I right?
starmap is a convenience function, you can change your function definition or wrap it to unpack the arguments and use regular map
>>> from itertools import starmap
...
... def inner(a, b, c):
...     return a + b + c
...
... def wrapped(args):
...     return inner(*args)
...
... iterable = [([1], [2], [3]), ([4], [5], [6])]
... print(list(starmap(inner, iterable)))
... print(list(map(wrapped, iterable)))
[[1, 2, 3], [4, 5, 6]]
[[1, 2, 3], [4, 5, 6]]
that's why there's no need for istarmap and other cruft
if you'd want to Do It Right you could make wrapper a proper decorator
11:25
cbg-you-more
 
1 hour later…
12:41
@X4748 Oh, I don't care about the thread. It was only curious about module/class/function.
I also have a tangible argument now to discourage people from trying to debug "weird python console behaviour"
# when just running python from a terminal:
>>> (lambda: [print(s) for s in inspect.stack()])()
FrameInfo(frame=<frame at 0x7f43a75a4370, file '<stdin>', ...)
FrameInfo(frame=<frame at 0x7f43a98809a0, file '<stdin>', ...)
# when running python from PyCharm's python console:
(lambda: [print(s) for s in inspect.stack()])()
FrameInfo(frame=<frame at 0x7fb69405eff0, file '<input>', ...)
FrameInfo(frame=<frame at 0x7fb694093570, file '<input>', ...)
FrameInfo(frame=<frame at 0x7fb6a26c4b80, file '/home/.../pythons/python_3.8.pre_alpha/...)
FrameInfo(frame=<frame at 0x555bb6bced70, file '/home/.../pythons/python_3.8.pre_alpha/...)
FrameInfo(frame=<frame at 0x7fb6940946b0, file '/home/.../.local/share/JetBrains/...)
FrameInfo(frame=<frame at 0x7fb69401d850, file '/home/.../.local/share/JetBrains/...)
8 additional levels! Good luck finding out where exactly things go wrong.
13:06
Guys what would You prefer for microservice, Falcon or Flask ?
since I don't know falcon, I'm gonna say flask
*Same here
13:25
Since I disagree with the fundamental design decisions of Flask, I compete instead (WebCore: 1/3rd the size of Flask ;) #ShamelessSelfPromotion
that was not an option :P
XP
How many other frameworks you know of feature built-in support for annotation-based typecasting? :DDD
<3 Python 3
anyone use combinations with backtracking. I saw a beautiful solution using them as boundary bound
14:15
Uhhhh… huh. Well then, that's gosh-darned weird. Travis, apparently, pre-installs Numpy… but only on certain build matrix combinations. And Numpy registers a console_scripts entry point that explodes on .load(). That's what I get for tying my tests to non-controlled namespaces, I guess. :|
Pre-installing anything sounds a bit weird. What if you're using travis to test numpy?
Damn you, f2py2e raising SystemExit!
2py4me
cbg
@Marius I've used falcon in the past, but it has been a while. Seemed nice and lightweight, didn't try to do too much for me. But I would probably use it just for internal microservicing, and use a more full-featured framework for external-facing server.
@amcgregor Hah, you are like me tilting at the pandas windmills with my little littletable project. ("I need to read a 5-line CSV file." "No problem, just install pandas and numpy and use df.csv_read()")
14:34
@PaulMcG There have been several examples of "serious WTFs" as generated by other frameworks as I dev'd WebCore. E.g. Django's multiple database support is a 21K line diff that would DOS Trac on attempts to view. Multiple DB support in WebCore… 52 additions, 30 deletions, basically just a for loop w/ exc. handling. (Ref: application.py#L72 there)
I strongly favour a drive in the opposite direction of the kitchen sink. ;P
14:47
@PaulMcG FWIW only noobs suggest that
Pandas is not a yamming csv reader
@AndrasDeak All software can be boiled down to a CSV reader. Any sufficiently advanced technology is indistinguishable from a CSV reader. … I've written CSV (technically pipe-separated… HL7) stream-processing pipelines in Brainfuck, on a related note. If even a language with only 8 operations can CSV… ;P
… it's pretty "lowest common denominator", then, eh?
15:27
@davidism: will wtforms 3.0 ever be released?
Not soon given all the other stuff I have to do.
If you or someone else wants to help, I'm happy to add maintainers.
Honestly I'm not even sure what 3.0 is supposed to be. The original maintainers dropped off after starting it, so master is in the weird intermediate state where I can't use it to make new 2 releases either.
@davidism I was asking because I really miss the MarkupSafe handling.
Sorry, I dont' think I could commit to another project either though.
15:56
@MartijnPieters Wait… what? The nebulous nature of your last statements here has given me a shock of adrenaline that I need to be worried for MarkupSafe… what's going on?
@amcgregor wut?
@MartijnPieters Exactly my question. ;^P
@amcgregor: I just wish that WTForms widgets would use the markupsafe.escape() function to handle label values, so that I can hook into that with __html__ methods.
Catching up on backlog, though, it's WTForms, yeah. TIL WTForms doesn't use MarkupSafe.
it's a feature that's missing right now, someone coded it up and landed the changes on master last summer.
16:00
I wish there was a central PEP-like registry of protocol methods. :|
property(as_html) vs. to_html() vs. __html__ vs. __html_stream__ (generator) vs. …
Nah, Python projects don't need that much overhead.
MarkupSafe has won that race.
Except cinje has extended it to support streaming (__html_stream__) generation. ;^P
@MartijnPieters I'm referring to central documentation of these methods… what… overhead? Wat?
hello
I need some help with django can someone help me?
quick question
I have this
@amcgregor right, but who is going to collate that documentation, how are new methods discovered, how are conflicts handled?
== overhead.
16:08
Only in the scenario where you "invent" a protocol you wish to see adopted. No overhead for people not publishing protocols, and arguably negative overhead for those trying to implement/use the protocols, as it'd be searchable documentation, instead of "shared secret knowledge" or library-specific non-standards.
the obvious solution to that is coming up with a proper, new standard...
… only leaves everybody with n+1 "standards".
Actually, the searchable documentation aspect would be invaluable if you are trying to come up with your own protocol. Examine the competition, and possibly not re-invent the wheel if there's an existing protocol that's suitable. Encouraging compatibility, DRY, and re-use is a good thing, no? ;D
I have a queryset in django and return in template with {{data}} result is <QuerySet [{'avatar': u'media/Capture.PNG', 'email': u'[email protected]', ...}]>.Is there is a possibility to extract only one column? like {{ data.email }}
If I use {{ data.0 }}, returns {'avatar': u'media/Capture.PNG', 'email': u'[email protected]', ...}
def toto(a=[]):
    a.append('toto')
    return a
if I do toto() twice, I will have ['toto','toto']
Heck, if it had a PEP-like status, I could have potentially been forewarned that __set_name__ would be coming down the pipeline, since my own declarative implementation had to roll its own naming hook, or the similar protocol there to notify a parent class that it's been subclassed: __attributed____init_subclass__
16:17
what is the best way to have the list immutable?
@AndyK Not necessarily "best", but one effective approach would be to tuple-ize it. However, your function definition there is a "trap for new players". The list is allocated at function definition time, not on each call to the function!
Basically: don't put mutable types as default values, or exactly this will happen at the least convenient time.
(Making the list "immutable" is XY to the actual problem: using mutable types as default values in the first place.)
@AndyK initialize inside the function
Indeed, a=None with an inner if a is None: a=[] can be a viable workaround.
default = object()
def toto(a=default):
    if a is default:
        a = []
    a.append('toto')
    return a
cheers @AndrasDeak & @amcgregor
16:21
None for default can also work if a will only ever be something that is not None
and of course then you do if a is None: a = []
no problem
Heh, @AndrasDeak, I hesitate to construct more canary singletons when, in this situation, None would never be an expected value. (None.append = explody.)
Yup, but it's better [read: I prefer] to give a proper answer then loosen according to constraints then to give a specific half-general answer that breaks a month from now when a might be None for some reason.
Thanks
Indeed. Counter-example: reading a bare object construction a month later would be similarly 'wat' unless already well familiar with the pattern. A suitable case for an inline code comment. ;)
I'm actually surprised there isn't a Sentry singleton constant to go with None to handle cases where None is actually an acceptable value. Huh.
class Sentry(None): ... ;^P
Alas, NoneType is not an acceptable base type. XP
16:37
What if you wanted to pass Sentry?
wim
wim
the problem is libraries that try to use None as a magical value. don't do that - bad API.
it's a plague that then has to infect the users of your library too.
@AndrasDeak Key point to interpretation of Sentry: it's never a valid or ordinary value, and should never be utilized as one. Whereas None has direct equivalence to nil or null.
wim
wim
your interpretation of Sentry is how None should be used in the first place
@wim My ACL implementations use predicates that either return True (authorize), False (reject), or None (no opinion; pass the buck to the next ACL rule). Three-state boolean operations can be useful. ;P
wim
wim
I'm thankful that TOML at least took an opinionated stance on this
(try to TOML dump a dict with some None in it and see what happens)
16:51
That's like saying "we can't use datetime, JSON can't serialize it". :/
Get a better serializer. ;^P
wim
wim
meh, sometimes simplicity/lack of features is a good thing
I will pick json over yaml anyday
Irony being, in picking JSON, you are picking YAML. Just a subset of it. >:D
An overly-restrictive subset for many datatypes. E.g. enjoy your floats. (We all float down here.)
But it's a nice subset that's easy to work with.
And produces erroneous results for financial information. >:P
(Or requires hoop jumping to serialize non-standard types. For example. — identifying the "type" of an object by the presence of a specific key is a tendency from JS that has always weirded me the f- out.)
I never tried floats with JSON, but that's a good thing to know if I ever try it in the future.
16:58
Isn't/wasn't yaml unsafe?
wim
wim
yes like pickle
pretty much every ruby gem was compromised due to yaml
It can deserialize arbitrary object references. That makes it "somewhat dangerous".
(As well as construct arbitrary instances.)
Only as dangerous as eval in a box
Funnily enough, you can deserialize a reference to eval and call it by "instantiating it" within YAML. >:D
wim
wim
17:02
pyyaml 5.1 should release in the next few days, they are deprecating use of yaml.load without explicitly specifying loader class
is there a difference between yaml vs yml?
wim
wim
no
(This type of shenanigan is why my "Python reference" DB field type prefers to only serialize the short name of an entry_point registered plugin; if you enable full dot-colon path loading, it's on you.)
wim
wim
some people like 3 letter file extensions for....nostalgia? i don't know why.
probably traumatized windows users
Every time I see .htm I want to slap a b*.
17:05
.htmb?
lol. uh, no, replace b* with "yam"
Swear word partial redaction for public safety. ;^P
That would be confusing, I normally navigate projects by looking at the extension of the file and directory first.
:adds to read-later, makes a reminder to make her own variant of that:
wim
wim
17:20
this is very concise, you can read in 2 mins.
Not if I'm writing prose in my head at the same time. :P
wim
wim
Wish it had been around earlier, maybe I would not spend years trying things only to reach all the same opinions.
rb folks
@wim I lean heavily towards the idea that the best code, the least buggy code, the most featureful code, the fastest code… is the code you do not write or include. Megaframeworks like Django, Plone, or even Pyramid rub me the wrong way out of the gate on fundamental design. Like petting a shark backwards. They additionally persist in vigorous backwards-rubbing with the proliferation of framework-specific solutions and adapters.
(And encouragement of framework-only thinking. I don't need a distributed task queue, I need Celery! Because Django!)
I prefer flask because it's simple.
I am using it currently with a side project that I testing new technologies I never tried before with. It's essentailly a backend for a Twitter clone IOS app I am making.
17:29
It does take many of the decisions out of your hands in a way similar to a megaframework. E.g. use Jinja. ;^P
(From my side, why on Earth does a framework care about a templating engine?)
I thought flask uses Jinja already?
It does, I'm agreeing that having those decisions made for you does simplify things… initially.
OTOH, its concept of "blueprints".
17:41
On the gripping hand: Flask may be ~3x larger than it needs to be, for what it does. ;^D (Sloccount of ~3300 vs. 952.)
And yes, that's very tongue-in-cheek. Still, a third the lines to execute… :makes note to profile benchmark to see exactly how many calls are issued to give a "hello world" response:
Having a third the lines won't matter for tomato if they're executed ten times as much.
18:14
Meh, I am way too far into a project to switch frameworks now, so I'll just stick with Flask.
For sure! Practicality beats purity, and for any real (not imagined/practice) problem, go with what you know best. ;)
I just keep ending up noticing chat and piling things onto my own to-do list. XD
18:33
Any of you agree with flagging this as a dupe or do you think the difference (series vs dataframe) is worth keeping as a separate question? stackoverflow.com/questions/55126339/…
The answer is ultimately very similar.
A series is practically a single column dataframe so I would flag it.
Right, that was my thought as well.
19:13
stackoverflow.com/questions/55149263/… off-topic, only one more vote needed
I have this code
        for productobj in products:
            """
            -get the productquantity object and instance related to each product
            -get the quantity related to each product in the cart
            - get the total which is product price * quantity

            """
            objqt=ProductQuantity.objects.filter(product=productobj).first()
            qt=objqt.quantity
            subtotal=Decimal(productobj.price * qt)
            total += Decimal(subtotal)
giving
unsupported operand type(s) for +=: 'float' and 'decimal.Decimal'
what is the total initialized as? 0?
@AlexanderReynolds total=Decimal(0.00)
Hm, is it possibly getting overwritten as a float at some point?
Can you post all of the code between the initialization of total and that end line? Also, which line is the traceback pointing to for the error?
@AlexanderReynolds alright let me do
19:21
If the error is pointing to the line total += Decimal(subtotal) then you could do total = Decimal(total) + subtotal just to be sure all the types are good
But you shouldn't need to do that if total is indeed a Decimal type the whole time.
@AlexanderReynolds error coming from the total
So possibly you can't add the value from a DecimalField to a Decimal?
total =math.fsum(pro.price for pro in products)
@ebong that's a lot of code and the indentation (or lack thereof) is a mess. Please use a code paste service.
you might have various combinations of buggy class attribute references
@AlexanderReynolds possible since i am still getting errors
19:32
@AlexanderReynolds thanks i tried it , no more getting the error.
@AlexanderReynolds OP says it's not a dupe, so I'm hesitant to hammer it.
@ebong You should generally avoid initializing Decimal objects with floats, unless they're integers, or integers plus exact binary fractions like .5, .25, .125, etc. Use integers or strings. If you convert an arbitrary float to a Decimal the Decimal will simply maintain the error hiding in the float.
@PM2Ring so please give me an example on how to properly initialize it. I am a novice
@PM2Ring Note that OP stated that in their original question, before I pushed it as a dupe. But fair enough.
@ebong When you read the product price, it's a string, right? So convert that string to a Decimal directly, don't convert it to float and then convert that float to Decimal.
@AlexanderReynolds I figured that. But if the OP doesn't agree with the dupe target, I don't think there's much point antagonizing them. Maybe find a better target? I can't suggest one because I don't know Pandas. In fact, it's on my ignored tags list. So I'm a bit nervous about improperly hammering Pandas stuff. ;)
19:50
@PM2Ring +1 — float is a lossy representation; you can accidentally and subtly alter your data by passing it through that number encoding. Decimal works best with strings.
@PM2Ring Yeah I was iffy on it. I think I'll retract it and give a good answer.
@ebong If for some reason you cannot get the prices as strings, some other (stupid) software has already turned them into floats, then you should process them to make sure the digits are correct. One way to do that is to multiply the value by 100 and round it to the nearest whole number of cents. Then you have an integer you can safely convert to Decimal, and then divide by 100 (as a Decimal) to convert back to dollars.
Mar 4 at 19:17, by wim
is there any valid use case for doing Decimal(myfloat)? Questions like that make me think perhaps it should just be a TypeError in the first place.
Or at least emit a RuntimeWarning. :/
That's probably a good idea.
what other type conversions give a runtime warning?
20:06
Seems to be a convention with numeric problems, ref: Numpy utilizing it for overflows/underflows in binary operations.
RuntimeWarning: overflow encountered in power
So while technically this is a type conversion concern, it's actually a loss of precision concern under the hood.
wim
wim
anyone good with .travis.yml? I don't understand why the py 3.7 runtime only tested against 3.13 and not against 3.13, 4.2, 5.1 here
travis-ci.org/wimglenn/oyaml/builds/… click "View config" tab to see the markup
@amcgregor alright, fair example. but anything in the standard lib?
From sciencealert.com/… A commemorative 50p coin honouring Hawking, with an engraving of a black hole (sort of), and the black hole entropy formula.
that is cool
I think I want one...
20:15
@amcgregor I'm just iffy on runtime warnings being shown for a mistake that the programmer makes. The std lib should be relatively agnostic to why you need to do something, and converting a floating point number to a Decimal doesn't seem in principle like a bad thing. The point of using a Decimal is to avoid loss of precision from that point forward, not before then. IMO it's too similar to int(floating_point_num) which would just be unnecessary to issue runtime warnings.
Slight difference, though. int(float) seems (from above) explicitly intentional: throw away the decimals. Decimal(float) seems like an unintentional mistake. (Thought to use Decimal, so bonus points! … aaaaand supplied float source data. Good jerb.)
wim
wim
seems cute, until it swallows all your other coins.
(Once it's float, there's no reason to not keep using float, really, except to avoid compounding any error present, though still, it's too late.)
@amcgregor I couldn't disagree more. A floating point value doesn't have a loss of precision from itself. There's no reason why stopping compounding errors from that point on is in any way indicative of an error on the programmers part.
20:32
Pathological case: an invoice object containing ten line items for ten cents per. Total: 0.99(9*)¢. Decimal that, you don't have a dollar. (Though the *100, round, Decimal, /100 approach would, in this case, resolve it. Careful on choice of rounding algorithm, though.)
Very interestingly, the REPL hides the true magnitude of the issue.

>>> 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1
0.9999999999999999

>>> Decimal(_)
Decimal('0.99999999999999988897769753748434595763683319091796875')
But the REPL is also not rounding, it seems.
At least that's consistent with other REPLs.
@PM2Ring Heh, I appreciate the coin's resemblance to a Mandelbrot render, in that when the velocities get imaginary, you have a hole. Looks like a plot of the spatial curvature of a singularity to me! ;)
@amcgregor Luckily, Decimal contexts provide a whole family of rounding algorithms.
Sure but it's still true that Decimal(a) - Decimal(1) != Decimal(a - 1) with your a = 0.1 + ... and I don't think the standard lib should tell me which one I meant. That error has nothing to do with Decimal(), it's a floating point problem.
Or should we issue runtime warnings when a user adds floating point numbers? :P
the stdlib can only catch Decimal(float) which is more often than not a mistake
@AndrasDeak [citation needed], I guess is my point.
floating-point is "well-known" to be imprecise, while Decimal is "widely regarded" as precise, hence "Decimal with float inside" is a pitfall
20:47
@PM2Ring Heh. Imagine my surprise when I mention "choice of rounding method" in a team meeting, quipping "round towards the beneficial", and a more executive member of the team asked, agape, "… wait, really? We round differently in different situations?" To which the accounting member in the meeting and I both responded: "Of course! Round up to make more money, round down to spend less, you silly!"
@amcgregor Laurel
Well I guess agree to disagree then. Yes, it's a possible pitfall, but I don't think it's the library's purpose to give such a warning. I would rather the standard library either throw an exception, or do nothing. Saying "there are valid use-cases to keep this in" and simultaneously warn them that they "may be doing something wrong" is just weird, IMO.
Though Warnings have been built to be conditionally ignored. :/
(Contextually, or globally.)
@AlexanderReynolds Decimal(float) is almost always a case of "shutting the gate after the horse has bolted". The coder has learned that "Decimal is better than float", but they don't understand the true issue well enough to do things the proper way. OTOH, if they read the decimal module docs it tells you not to initialize Decimal with float, and to use integers or strings instead.
At a guess, I'd wager a high percentage of SO Decimal questions revolve around this issue.
Heh. I can count the number of questioners who join #webdev or the friendly Python IRC channel, who have attempted to RTFM before asking on no hands. Because I have yet to see one. A warning when actually doing the badness (which you can silence if you really do know what you're doing) seems… safest (?) to me.
20:59
Maybe making Decimal(some_float) an out & out TypeError is the clear-cut solution, but it does seem a little harsh to me.
@PM2Ring Well until someone calculates such a percentage...anyways, that's not necessarily a good metric, people misuse all sorts of standard datatypes :P
@PM2Ring right and then you're going to have people doing Decimal(str(some_float)) which would be even worse
@AlexanderReynolds Agreed.
s.webcore.io/d918d6183593/round-your-floats-people.jpeg — floats even bite the "big boys"; the ones you'd expect to know better.
And yeah, a warning explaining why it's a problem (or at least linking to a description of the problem) is superior to a TypeError developers would casually try to bypass without thinking. Gah, I'm going to have nightmares about that Decimal invocation, @AlexanderReynolds, so thanks for that. XP
Okay I've changed my mind. Issue a runtime warning, but the text for the warning simply says: "Read 'What every computer scientist should know about floating-point arithmetic' by David Goldberg -> itu.dk/~sestoft/bachelor/IEEE754_article.pdf";
I feel r"Unicode(Encode|Decode)Error"s should similarly link to kunststube.net/encoding
@amcgregor Nice. I occasionally link to Joel's article, but it is a bit light on technical detail.
Rhubarb, Alice.
wim
wim
21:18
oyaml released (for PyYAML v5.x support)
OY
...aml
(sorry)
21:39
...Acute myeloid leukemia?
I swear every combination of random letters is a valid acronym these days
not only valid, but maps to an array of disambiguations
even obscure seeming ones
XKCD seems to have survived
is that actually an acronym?
No, hence "survived" :)
I'm sure someone could make it an acronym if they really put their mind to it, but my Google Foo suggests that hasn't happened
"chronic disorder" might do as a clumsy ending, but I'm struggling with XK, stuck on Xylophones
00:00 - 22:0023:00 - 00:00

« first day (3070 days earlier)      last day (2103 days later) »