« first day (3275 days earlier)      last day (1672 days later) » 
00:00 - 17:0017:00 - 00:00

5:02 PM
@ReblochonMasque I posted it not long ago, but it always makes me laugh anyway: youtube.com/watch?v=5tP7Xrrxfxc
 
haha, yes, thank you @roganjosh
 
Has anyone made a Progressive Web App (PWA) with Flask?
I've found the guides etc. but actually, I don't know whether it meets my requirements so I'm curious if I can bounce some questions off someone
 
@roganjosh regenerates, while more correct, sounded weird
 
Put another way around, I'm trying to think about how to deal with the loss of connectivity. If someone inputs something on my webapp and loses connectivity, I want to be able to queue the action on the device
@AndrasDeak I still don't understand the sentiment, sorry mate
 
5:17 PM
@roganjosh ah, the sentiment. Being chased away by vampires and triviality
 
We have "blood is thicker than water" which makes little sense, but "blood" implies family
Ah, ok, now I follow. Sorry
 
Probably not your fault :P
 
"Blood is thicker than water" is doubleplus nonsensical if you consider the other interpretation of the phrase that means the exact opposite of its typical interpretation
> Modern commentators, including authors Albert Jack[7] and R. Richard Pustelniak,[8] claim the original meaning of the expression was that the ties between people who've made a blood covenant were stronger than ties formed by "the water of the womb".
 
we have "blood doesn't turn into water"
 
At a guess: you can't go back? What's done is done?
 
5:22 PM
Also "dogs don't end up in the bacon" but that's a different one
@roganjosh no, it's the non-Kevin version of blood is thicker than water
 
Kevin has posted an academic version of the phrase. At least where I am, "blood is thicker than water" means that your family comes before anyone else
 
Yes
 
That's also what it means in my area, and basically anywhere except online communities where people like to Well Actually each other
 
But apparently it also means "one can't forsake oneself", which is what the dog bacon one means
 
I guess the origins can be in the traditional film trope where people slash their hands and shake but... that's quite far removed from what it's become
 
wim
5:25 PM
what is a progressive web app?
 
@wim en.wikipedia.org/wiki/Progressive_web_applications Not gonna lie, that's kinda the basis of my first question :P
 
I declare that my previous message is not a Well Actually because it's not wrong to interpret it in the popular fashion. The "family > others" interpretation has been around since the 1300s, it's as legit as they come
 
wim
hmm, not seeing anything different from a regular old "web app" there tbh
 
My understanding is that you push the static content over to the device on first connecting
It is different because it will allow you to navigate offline
 
Sounds like HTTP with extra steps
 
5:28 PM
So, along with the static content, I guess you push templates too
 
Ok, I can kind of see the appeal.
 
wim
...that's still just a web app?
 
It's basically a web app with a single very large page
 
It has to be a single page?
 
wim
...it's a web app with some extra marketing BS from google?
 
5:32 PM
You might be right, which is why I'm asking
 
@roganjosh That's my assumption. If there's more than one page, then you're making an HTTP request every time you move to a different page, and that defeats the purpose. Right?
 
@Kevin My reading around this is that if can allow navigation offline
And by reading, imagine the dog with glasses saying "I have no idea what I'm doing"
 
wim
too late I already imagined the god
 
I can't believe I just typed "god" instead of "dog". I'll have words with myself
 
Well, you've done more reading than me on the topic.
 
5:35 PM
From the pair of you, though, I guess I'm going off course
 
Ok, since the wikipedia article says that pwas aren't supported by all modern browsers, that implies to me that they're capable of things that regular web pages can't do. Like seamlessly caching multiple pages for offline use, for example.
 
Scenario: Every machine operator has an ipod. They need to record the reason every time a machine stops. I can't guarantee that they will have connectivity. I need to catch the start time, stop time and reason. And sync whenever that device gets connection
 
Ok, sounds reasonable. Perhaps PWAs can do this, although they don't seem uniquely suited to it.
You could write a Python script that does the same thing
 
A python script where and using what?
 
wim
javascript
 
5:41 PM
On the ipod, using Python. If you are about to say "you can't install python on an ipod", then you've thoroughly dashed my plan
 
So hold the data in a JS loop until that loop connects?
@Kevin I'm probably saying that
 
"I want to use a web app because it would be very nice if I didn't have to manually install anything on 500 individual machines" is a perfectly cromulent requirement
 
Ideally, I just want my central server and it be accessed by browser
I'm glad that you agree it's cromulent :)
 
"Hold the data in JS until connection is reestablished" seems like a fine design as long as the machine operator doesn't close the browser window
 
What about when the device sleeps? I have zero experience here
 
5:47 PM
You could use cookies or web storage to keep a list of not-yet-submitted reasons that stays alive in between browser sessions. But of course it still won't push the data if the connection comes back up while the window is closed
Maybe PWAs have some cool always-on-even-when-closed ability, idk
 
Right, that's looking more promising. So pushing the data is just a case of opening the browser?
(if it happens to be closed)
 
Still wading through Google's marketspeak to figure out what PWAs give you beyond "the user can click an icon on their home screen and it will take them to your page"
 
I've been searching too but I've just come across: "The "DOM" in DOM storage does not literally refer to the Document Object Model. " when I flicked back to Wiki. Wut.
Aha, nm, it's the "storage" that salvages that
@Kevin it doesn't have to be PWA. How would you go about it if put on the spot?
 
Ok, unless I read otherwise later I'm going to assume that PWAs do not have a cool always-on-even-when-closed ability
 
6:04 PM
I appreciate your search
 
I've never had to write a web application whose target audience has less than 99.999% Internet uptime. If they try to go to my site and it doesn't load, that's too bad, and they should try again later. This is probably not suitable for your purposes.
If Google's marketspeak is to be believed, PWA does have the power of "static content is cached so the page loads even with no connectivity" and that's certainly better than delivering a 404
 
Ok, for a proof of concept, would a JS loop be feasible? Do you happen to know what breaks it? If the device goes to sleep, I know I have podcasts that will play on Android even if I deliberately sleep my phone
So I assume that the browser scripts can keep running
 
I know my home laptop runs scripts while it's locked, but I don't think it runs scripts while it's asleep
 
Maybe "sleep" isn't the right word. I mean when the device turns the screen off
 
Unsure. I think that would depend on the device.
 
6:13 PM
OK, well thanks for your input, my pile of things to look into in my plans to take over the world grows :)
 
Summary of difficulty assessment

Easy
 - while machine is active and page is open and there is no internet connectivity, periodically try to send data to the server

Medium
 - The user can open the page and fill out the Machine Stoppage Explanation form, even with no connectivity (requires PWA)

Hardware-dependent
 - the "send data to server" loop continues to run even when the machine is in an inactive mode of some sort

Impossible?
 - the "send data to server" loop continues even if the browser is not open
 
That is what you get paid for :)
It'll be a few weeks before I get to testing this, but I will be sure to report back. Thanks so much for helping me research this
 
I don't know if I said a single actionable thing, but hopefully I've added some interesting texture and volume to the cloud of confusion
 
You put a truly beautiful frame on it
 
wim
6:34 PM
for i in range(sys.maxunicode + 1):
    x = chr(i)
    if x.islower() and x.upper().lower() != x:
        print(i, x, x.upper().lower())
    if x.isupper() and x.lower().upper() != x:
        print(i, x, x.lower().upper())
^ wow look at all these jerks
 
It be like that sometimes
 
wim
8490 KELVIN SIGN you disappoint me
in my day, we used a LATIN CAPITAL LETTER K and we were happy with it!
 
Beware the SIGN of KELVIN, the sigil that heralds the arrival of He Of The Deviant Name
Luckily my protective wards filtered the symbol to "�"
 
wim
@Aran-Fey no public way possible, if Martijn is to be believed (ref: Given a dict iterator, get the dict)
 
6:49 PM
well that's lame
 
why's that lame?
 
wim
@MartijnPieters nice little ctypes usage example there
 
what use case do you have that requires you to recover the original iterable from an iterator?
@wim ta. It's been a while since I used ctypes, so that was fun to write.
 
isn't pickling/unpickling an iterator such a use case? Why does a dict_iterator turn into a list_iterator? They have different behavior
If I have a dict_iterator and I modify the underlying dict, it throws an exception. If I have a list_iterator and I modify the underlying dict nothing happens
 
wim
I agree it's kind of lame
I mean the iterator has a reference to the dict, as long as it's not exhausted - it may as well be public.
 
6:53 PM
Is it too much to ask that unpickling produces an object that's equivalent to the original?
why does it drop the reference when exhausted anyway?
 
wim
to make gc's job easier I guess
 
the assumption is that you want to send over the keys to something without dragging along the whole dictionary.
@Aran-Fey why keep the reference?
It's not as if it'll ever need it again.
you can't rewind the iterator.
 
wim
I wonder if there are other examples that fail this test
>>> obj = iter({})
>>> type(pickle.loads(pickle.dumps(obj))) is type(obj)
False
 
ah, it only drops it after it's thrown StopIteration. That's fair enough then
 
wim
PyDictObject *di_dict; /* Set to NULL when iterator is exhausted */
^ did this idea predate weakref ?
 
7:01 PM
@wim dict_iterators are the only one I've found thus far
 
@wim weakref has overhead too.
this is C code, why use a construct only really needed in Python?
Besides
>>> weakref.ref({})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'dict' object
The type must explicitly support weakrefs, and dicts don't.
 
@wim set (and frozenset) iterators turn into list iterators, too
seems to be a thing hash-based containers do
 
wim
7:21 PM
why don't dicts support weakref ?
anyone got a supercomputer lying around?
50 points bounty for the first person to post a random seed which makes random.uniform(0, 1) return a 0 on the first call — wim 4 mins ago
 
I wonder if there's an approach for that which is faster than just testing every possible seed in sequence. Does the RNG algorithm take pains to ensure that there isn't any obvious correlation between a seed value and the first number it generates?
 
wim
@wim because this
 
If you can make any observations like "all seeds that end in '5' produce a random number with bit count greater than 32" then that cuts down the search space
Not that reducing the search space by 10% is all that useful on its own, since the search space is like 2**53
 
wim
7:37 PM
if you like it then you shoulda put a RNG on it
@Kevin, you're on winblows right? Do you get 0.052363598850944326 from random.uniform(0,1) if you seed with random.seed(123)?
I think this PRNG is cross-platform but not really sure
 
Let's see.
>>> import random
>>> random.seed(123)
>>> random.uniform(0,1)
0.052363598850944326
Yes, I'm on Windows
 
wim
probably easier to reverse engineer the algorithm :)
 
github.com/python/cpython/blob/master/Modules/_randommodule.c only has one #ifdef so I imagine it compiles to nearly the same code on every platform
 
wim
wow looks complex
 
What is the difference between 5e-324 and 0?
Something in my head says that 5e-324 has already exceeded 64 bits
 
7:49 PM
> On a typical computer system, a double-precision (64-bit) binary floating-point number has a coefficient of 53 bits (including 1 implied bit), an exponent of 11 bits, and 1 sign bit. Since 2^10 = 1024, the complete range of the positive normal floating-point numbers in this format is from 2^−1022 ≈ 2 × 10^−308 to approximately 2^1024 ≈ 2 × 10^308.
Sayeth Wikipedia. I don't quite follow it, but if they wrote it down it must be true
 
wim
there's a cool module for this stuff ... bitstring
>>> bitstring.BitArray(float=5e-324, length=64).bin
'0000000000000000000000000000000000000000000000000000000000000001'
 
nm, I was being slow sorry
 
Key word in the wiki block is "normal". 5e-324 is denormal so it can go smaller.
 
Ok, so my dumb question is what is going on for python to be able to generate such numbers when floating point errors happen orders of magnitude above that value?
 
All the denormal numbers are evenly distributed between -1e-308 and 1e-308, and there are 2^53 of them, which means they're about 1e(308 + math.log(2**53,10)) apart, so the smallest abnormal float would be 1e-324ish
@roganjosh I don't fully understand it myself but I think an important observation is that the "resolution" of floats gets coarser the farther away from zero you get
 
8:00 PM
0.1+0.2 --> 0.30000000000000004, yet Python can handle 0000000000000000000000000000000000000000000000000000000000000001
 
Right. It can handle. 0.0000000000000000000000000000000000000000000001, but it can't handle 4.0000000000000000000000000000000000000000000001.
One way of thinking of it is, the precision of a float determines how many digits can go between its least significant digit and its most significant digit.
0.0000000000000000000000000000000000000000000001's least significant digit is "1", and its most significant digit is also that same 1. So it's got precision to spare. Meanwhile 4.0000000000000000000000000000000000000000000001's most significant digit is 4, and its least significant digit is 1, and they're miles apart
 
That makes sense
I was fiddling around trying to create a counter example but it seems not so easy :)
 
A while back I wrote gist.githubusercontent.com/kms70847/… which helped me get a handle on the internals of floats. I tried to show my work so it might be useful to others.
Or maybe it's whimsically opaque like all my other work, who knows~
 
wim
screams in IEEE 754
 
I wouldn't describe it as whimsical but I think I best not try to digest that output on a Friday night :P Thank you for the script, though
 
wim
8:14 PM
>>> my_set = {1, 2, 3}
>>> it = pickle.loads(pickle.dumps(iter(my_set)))
>>> my_set.add(4)
>>> {*it}
{1, 2, 3}
 
you expect something else?
 
wim
RuntimeError: Set changed size during iteration would have been nicer
pickle freezes the set
 
I'm confused about why you expected it wouldn't?
 
wim
it silently lost state
why bother to support pickling it at all if you can't do it properly
 
Ok, then what do you understand as the use of pickle?
Because I don't find this astonishing
 
8:21 PM
you assumed it(my_set) to iterate through a thing pointed to by my_set. So upon de-pickling you thought it would still be going through the same thing. But instead it creates a new thing that looks like my_set
 
I'm not gonna lie, your understanding of the internals is far better than mine, but I can't understand a use-case for getting the same object back here
 
wim
creates a new thing that looks like my_set previously looked like
 
However, that is the same behavior even if you didn't use pickle
 
wim
no big deal, I just would prefer an error here (or pickling of stateful iterators to be unsupported)
 
OR NOT!
it is different behavior
 
8:23 PM
as long as you don't pickle the set along with the iterator you really can't be surprised that the unpickled iterator doesn't care about you modifying the original set
 
wim
@piRSquared right, now you see the issue.
 
>>> s = {1, 2, 3}
>>> i = iter(s)
>>> s, i = pickle.loads(pickle.dumps([s, i]))
>>> s.add(4)
>>> list(i)
[1, 2, 3]
 
But still, I'd wouldn't have assumed that serializing something would preserve the something else it was pointing to.
Or I should say, preserve the pointer to the something else
Because that something else might not be there later
 
wim
exactly, so refuse to pickle the iterator in the first place
 
I'd accept that
 
8:33 PM
@Code-Apprentice I've never used it or had any need for it in ten years using R. But you could have found your answer from SO: Why is match.call() useful?. Answer: it captures the language of the call without evaluating it, and in this case it allows lm to treat some of the "missing" variables as "optional".
 
@smci yah, I've been reading the docs about it. And I think I read that SO post, too.
 
SE/NSE (non-standard evaluation) in tidyverse packages is more common. See all he articles on that.
@Code-Apprentice You definitely don't need unless you're writing your own package, and even then you might not need it either. So, ignore it.
 
I still don't get it really.
the problem is that I am reading code that uses it and trying to translate to Python
 
@Code-Apprentice Ignore it. Almost all languages have useless, outmoded, clunky features that we can skip. You ask what is the Python equivalent for passing function call args without evaluating them (in the caller's context) and matching optional args...? A lambda function. Beyind that it sounds like OO abuse to me, just declare classes already.
@Code-Apprentice If you're still stuck, post a specific code snippet as a question on SO, make sure to include a couple of examples of how it's called, then message me.
 
wim
What's a pirate's favorite programming language?
 
8:38 PM
@wim R, yes, very old... tReasure be buRied on thaR islands...
 
could also be C
 
wim
nayyyy... don't you know a pirate's first love is the C
 
@smci Yah, I'm probably too focused on some details that don't matter. In the end, I need to replicate the behavior. And that's the part I'm stuck on here cuz I don't know what this section of code is trying to accomplish, let alone how to duplicate that in Python.
 
@Code-Apprentice In general, just try to understand the spirit of the code, not how every last syntactical detail works. (Make the common-case work first) In particular, when code in language X is doing something that is bad idiom or not supported in language Y.
 
The function I'm translating is 1k lines of code, so it's difficult to get my head around it.
and I don't even care about the "common case"...I'm just trying to get it to work for the particular case I'm already using it for.
talking about "bad idiom"...I think that's spot on. Some parts of this code seem way more complicated than necessary to accomplish the task at hand.
 
8:42 PM
@Code-Apprentice Yeah but in R, separate the function body into a) the argument-matching/filling/parsing/defaulting part, and b) the function body proper that does the actual work. Try to pay as absolute minimum attention to a) as you can get away with, just start reading b) and don't get too hung up on how a) passes args into b). Start reading at the beginning b), and only look back to a) on as as-needs basis
 
even small things like x = len(y) and then z = len(y) for two lines in a row...
@smci ok, thanks for the suggestions. I been trying to do that but have yet to find where b starts...and I think a and b are intertwined in this code.
and I'm afraid of missing something in a that mutates the input values
 
@Code-Apprentice Yes, that's a common anti-pattern in R: arg-matching/filling/parsing/defaulting code interspersed with the function body proper.... ;-) Very often I find code that's too clever that I can't understand, so I just try to guess "Does this produce an integer? vector of string? 2D array?" then work ahead based on my assumption. Actually, you can hack some print() statements into the function, or single-step it with the debugger if you really want. But I've never needed to, so try not to
 
yup, I cloned the original repo and have been adding lots of print() statements to it.
 
@Code-Apprentice print() statement is your useful humble friend...
 
yah, I don't use print() statements in python much because I have very good source-level debugging tools. I don't have any of those tools for R. Even if they are available, I haven't figured out how to work with them.
 
8:50 PM
@Code-Apprentice Never feel intellectually intimidated from hacking print() and raise Exception into code (obviously a local throwaway edit, with backup). And run python -i interactive, or ipython.
 
For python, I use IntelliJ. I usually don't have to add print() statements because the debugger is so good.
But when I don't have that, I use lots of print() statements.
 
@Code-Apprentice It's also possible it's just bad code, and is implemented better by some other R package. (I mean, is anyone else using the repo? Has anyone referenced it on mailing-list, Twitter etc.?) Post a link here to the github if you want my opinion.
 
@smci yah, I think you're right that this code isn't well written.
 
@roganjosh "improve edit" option
 
8:57 PM
"changing "their" to "that" because the mod in question does not go by "they/their""
 
@roganjosh do I really need to read that link to answer that question?
 
@Code-Apprentice I, err, well I'll get back to my witty retort later :P
 
@roganjosh it's sort of relevant to the subject
but if you keep looking at it you'll definitely go mad
 
wim
looks like malicious compliance
 
Knowing that mod a bit, I don't think so. But I see how it looks like that.
 
9:00 PM
I think MadScientist must have "improved" the edit and it carried the same reason forward?
 
yes, so that it's a binding approve vote
 
wim
so is the 3 vote close/reopen thing coming back?
if so, when? not there yet.
 
it is, I think they're rehashing the interface first
> If no one can think of a good reason not to do this, I'd like to roll out the proposed changes as soon as we're done testing / iterating on the new post notice system that's currently under development.
only 6-8 weeks to go ;)
for once they are allowing time for feedback, because Shog
 
There is basically no moderation going on on SO right now. I have my reasons for not partaking, but is there some kind of official strike that will be called off?
 
wim
nice
the 3-vote close was the greatest thing since "🍞"[0:1]
 
9:09 PM
@roganjosh I think it's just people being pre-occupied with other things
I for once have been busy following the drama and arguing with people
 
hahaha
Right, I best pick a movie and hit the hay. We're going to a fancy steak restaurant tomorrow for my dad's birthday and I don't think my "I'm a programmer" excuse will hold for looking disheveled. rbrb
 
9:36 PM
@roganjosh have fun
 
9:46 PM
@smci this is the function I'm translating atm.
 
that whitespace use is giving me a headache
 
cabbage
 
wim
10:07 PM
oh damn, chainmap instances have a __dict__, how confusing
 
10:18 PM
@Code-Apprentice Is that actually the released code for a CRAN package, half the lines are commented-out blocks of code, that's not how we do revision-control... It needs bigtime cleanup. I'd pretty much ignore everything up to line 610. Anyway the package is dormant since 2016.
 
@smci what about those patchy intermittent lines of code that don't start with #? Does R have block comments? Or is there some syntax trickery?
 
MATLAB does and it can get confusing, especially if syntax highlighting is broken
 
@Code-Apprentice CRAN drc package seems to be totally dormant since 2016, it only has one maintainer, and I can't find a developer mailing-list. I recommend you ask on one of the R biotech lists "Is drc dormant/mothballed/what's going on with it? What package(s) have people moved to using instead?"
@Code-Apprentice Actually, faster to ask in the main R chatroom on SO, which is called GMTs.
@Code-Apprentice Best to move discussion over there, I asked them this for you: chat.stackoverflow.com/transcript/message/47492048#47492048
 
in GMTs, 6 hours ago, by Code-Apprentice
I'm a Python programmer trying to understand some R code. I've posted a couple of questions on SO and would like to get some input.
 
10:32 PM
@AndrasDeak Oh I hadn't noticed. But the way I phrased the question is much more specific and likely to get a response. There's a huge diff between "understand some R code" vs "the source for the CRAN package drc". The former just sounds like a timesink debugging some individual user's code for them. Not a released package.
 
just FYI
 
@AndrasDeak My FYI supersedes your FYI. Look, the blunt truth about the lifecycle of an R package release is many of them are PhD vanity projects and self-citation exercises. After the maintainer graduates, they get abandoned. But as long as they don't break outright (with future language version changes), they live on dormant in CRAN for years... even if they get superseded by newer better packages. It's widely agreed to be an anti-pattern in R. Python is more Darwinian...
 
@wim dicts don't support weakrefs because of memory. Loads of dicts in a typical Python process, and each adding yet another array of refs adds up really fast.
You can create your own subclass of dict and it magically becomes weakref-referencable:
class Dict(dict): pass
d = Dict({"foo": 42})
wr = weakref.ref(d)
 
...also Python has more churn in the language itself. So our constant background level of breakage self-selects out dormant packages. Different to R. R is way more academic than us crass commercial Python folks.
 
wim
@MartijnPieters yes I found the answer shortly after (chat.stackoverflow.com/transcript/message/47490340#47490340)
 
10:42 PM
@wim just saw that, sorry.
 
>>> weakref.ReferenceError
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'weakref' has no attribute 'ReferenceError'
 
2.7 has it
 
@Code-Apprentice Google for "Christian Ritz" drc 2018..2019 to see what's going on with that package; the maintainer still presents it at useR conference. But the code is dormant and quite likely superseded by lots of more recent packages; go ask R biotech lists/people if you want the specifics.
 
@Aran-Fey oh look, a documentation bug.
RefenceError is a built-in. There used to be an alias for it in weakref, but no longer.
In fact, the last release that had weakref.ReferenceError was Python 2.7.
 
@smci I've already been over at GMTs bugging them. And yah, the drc code is attrocious. Lots of commented out code. And even though I don't know R, the code seems much more complicated than it needs to be, even for as general as the library is trying to be.
@smci finding other R packages doesn't really help me. At the root my problem is that we built a system in Django that calls some R code using pyr2. I am trying to remove the dependency on R entirely while maintaining the same behavior.
 
10:51 PM
R is a small academic community and it's considered uncouth to write epic public takedowns of stale/broken/non-performant packages like Matthew Honnibal (spacy.io) famously did to NLTK Dead Code Should Be Buried – Why I Didn't Contribute to NLTK (spacy.io). Personally I think it's essential to keep a healthy ecosystem.
 
I like the idea of giving documented names to all classes (like ReferenceType), but I suppose that's not really necessary for ReferenceError, considering it's the same as the builtin anyway
 
I am actually gobsmacked that there isn't already an issue for that.
 
How I wish that list_iterator and tuple_iterator and odict_items and _collections._deque_iterator and all the other crap were official documented classes
 
@smci and for code like this drc package, people might still find it useful and it doesn't require a lot of maintenance any more since it gets the job done. I'm starting to understand that R programmers are probably not software engineers that futz over details like keeping a functions parameter list short and simple.
and all those other "clean code" maxims I try to follow
 
@Aran-Fey why should they be?
They are all iterators, and not re-usable by anything else.
So all you need to know about them is that they have an __iter__ method that returns the object itself, and __next__ to give you the next value. They have no other purpose.
 
10:57 PM
It'd improve compatibility between python implementations. If you pickle something like that with CPython 3.5 you can't even be sure if CPython 3.7 is gonna be able to unpickle it, much less PyPy or Jython or whatever else
The pickle protocol (specifically, __reduce__) even forces you to use iterators
 
@Code-Apprentice: ok, good luck with your porting. I have some generic advice about how not to ask a question. Like this: you don't ask any question at all until three screens down, then just the super-vague "What am I missing that the output is so drastically different?" ...
 
>>> iter(deque()).__reduce__()
(<class '_collections._deque_iterator'>, (deque([]), 0))
^ good luck unpickling that on a PC that doesn't have the _collections module
 
... And nobody in Python is going to google for the manpage for R's drc::drm(). Make it easy on your poor readers, give them all the information they'll need, don't force them to hunt for it, three pages down, or on Google. I wouldn't touch a question like that with a bargepole, and there aren't many people coding both Python and R to begin with.
 
11:12 PM
@smci yah, that's a good point. In part, I used posting a question as an excuse to create some MCVEs of both the python and R code.
 
@Code-Apprentice: the 'Minimal' in MCVE means please reduce your example from one page of Python to the absolute minimum that reproduces the issue (3 datapoints? 4?).We don't need the code for get_dose(), just give the x-coords already. Remove boilerplate like if __name__ == "__main__". Hardcode your function loglogistic4() inline, hardcode MIN_ASYMPTOTE, MAX_ASYMPTOTE. Reduce, simplify, reduce...
 
I think I've found the guts of this thing. Ultimately drm() calls optim() which appears to be using a different algorithm than curve_fit() in my python code.
 
@Code-Apprentice Yay. Perhaps someday you can buy me a beer...
 
@smci where would I hardcode loglogisitc4() exactly? Do you mean as a lambda in the call to curve_fit()?
 
@Code-Apprentice: apart from the two imports, your MCVE only starts at p0 = (-1, 1), which is halfway down.
@Code-Apprentice Ok instead of:
MIN_ASYMPTOTE = 0
MAX_ASYMPTOTE = 100

def loglogistic4(x, B, C):
    return MAX_ASYMPTOTE + (MIN_ASYMPTOTE - MAX_ASYMPTOTE) / (1.0 + np.exp(B * (np.log(x) - np.log(C))))
 
11:19 PM
oh...you mean to hardcode the min and max values?
I guess I could do that...but it seems like a minor nitpick to me
 
Hardcode the entire thing, inline it into your caller code as an expression:
 
but it's not an expression. I need a function to pass to curve_fit()
it has to be a function to even work
I'm not calling loglogistic4() anywhere. I'm passing the function itself to curve_fit().
 
from numpy import log, exp

return 100 * (1 - 1 /(1.0 + exp(B * (log(x) - log(C)))))
 
uh...that doesn't compile
if you mean hard code the min and max values in loglogisitc4(), yes, I can do that. I'm skeptical that such a minor change improves the qualitiy of the MCVE though.
 
And if B,C are related to the constant p0 = (-1, 1), then hardcode and simplify all that too.
 
11:24 PM
They are related, but cannot be hardcoded.
The values of p0, I mean.
 
from numpy import log, exp

your_variable_goes_here = 100 * (1 - 1 /(1.0 + exp(B * (log(x) - log(C)))))
 
that doesn't work for my purposes
plsq = curve_fit(loglogistic4, x, y, p0) # I need a function to pass here.
 
@Code-Apprentice Fine then def loglogistic4(x, B, C): return 100 * (1 - 1 /(1.0 + exp(B * (log(x) - log(C))))) like I said above. Plus the three import lines. My point stands that really your MCVE only starts on the p0 = (-1, 1) line. Please start fixing your MCVE now.
 
@smci my MCVE does what I need it to do. If I follow your suggestions it will do something different than what I need.
The loglogistic4() function is required. Removing it no longer makes my MCVE complete.
I could remove the get_dose() function and hardcode the values for x which might be easier to parse.
 
@Code-Apprentice Your code is nowhere near an MCVE, and I've just spent a page explaining to you why in graphic detail, scroll back to the The 'Minimal' in MCVE means please reduce your example from one page of Python to the absolute minimum that reproduces the issue (3 datapoints? 4?). At this point you either fix it or it gets ignored. Your choice.
 
11:32 PM
@smci but all of your suggestions so far fundamentally change the nature of the code.
or are small nitpicks
there are definitely deeper flaws with my question than the MCVE
and I'm thoroughly baffled that 35 lines of code aren't minimal enough
 
@Code-Apprentice No they don't change it all. Inlining and hardcoding don't change the output at all. Your question is three screens long and it doesn't even ask a question until the third screen, under two walls of code. And even when it finally asks a question the language is way too vague "What am I missing that the output is so drastically different?" and sounds like you could debug the slope, intercept by yourself. From 4 datapoints. Even if you only plotted graphs could probably debug visually.
 
@smci removing datapoints would violate the "complete" in MCVE. Removing the loglogistic4() function would also.
@smci hardcoding the way you suggest will cause runtime errors and not provide any output at all.
@smci "it doesn't even ask a question until the third screen, " And this is the most serious flaw of my question. Not the MCVE.
 
@Code-Apprentice No it wouldn't. You don't understand what MCVE means. Currently you have 8 (x,y) datapoints. Can you reproduce the issue with 3? 4? 5? Whatever that minimal number is, is your MCVE.
 
@smci no, I can't reproduce wth less data points. Less data points is meaningless in the context where I am using this.
 
@Code-Apprentice Wrong, it won't cause any runtime errors, I wrote you working code above for loglogistic4(). Making the other edits is your job.
 
11:41 PM
then I don't understand what change you are suggesting.
would you mind making a repl to show what you mean?
 
@Code-Apprentice Noone cares if it's not a realistic drug dosage. Please reduce your (x,y) to the minimum number of datapoints that numerically tickles the discrepancy between R and Python on the curve-fit coeffts: that's your MCVE. Also, remove the half-page of unnecessary boilerplate like I suggested.
 
it's not about being a realistic drug dosage. I have a set of input that I need to reproduce the exact same output in Python as the existing R code does.
you suggested removing one line of boilerplate, not a half page...
 
@Code-Apprentice No, I said We don't need the code for get_dose(), just give the x-coords already. Remove boilerplate like if __name__ == "__main__". Hardcode MIN_ASYMPTOTE, MAX_ASYMPTOTE.. That saves you ten lines incl whitespace. Also your y-vector alone is ten lines long. If you can round y-values to 2dp, you can save another 8 lines, for a total of 18 lines. If you can get by with only 3 or 4 datapoints, you save more lines.
 
all of those seem to be nitpicks to me
and reducing the number of datapoints will fail the C of MCVE
and probably the V
in part because for 8 out of 10 of my current data sets, I get the correct values
curating a dataset with 3 or 4 datapoints that leads to a discrepency will take much more time than it is worth.
 
@Code-Apprentice Noone in general will read a question that's three screens long. Why does "reducing the number of datapoints will fail the C of MCVE"? Well if you really can't make an example with 4 or 5 datapoints that doesn't repro the discrepancy in curve-fit coords. Better still, just plot both graphs, one for R, one for Python.
 
11:53 PM
my question isn't about graphs...
 
@Code-Apprentice Then at least round your 8 y-values to 2dp (or less), inline your x-values from get_dose().
 
yes, inlining the xvalues would remove some complexity from the example by making it obvious they are just input numbers
 
@Code-Apprentice I never said it was. I said your question is soliciting debugging help, with pages of overly verbose code examples. A fast visual way to both illustrate and find the bug is to compare graphs. Plot the R and Python curve-fit lines on their respective graphs. Put the graphs above the code cos noone's going to read three pages.
 
I agree that the question is asked poorly. I disagree that the code examples are "overly verbose".
I appreciate your suggestions. I need to get going now. Time to clock out and enjoy the weekend.
 
@Code-Apprentice If I was an average Python programmer, why would I care to debug whether the numerical error is coming from the R package drc::drm(), or the Python? You haven't even proven that the error is on the Python side, not the R side. I'm not writing testcases for Mr Witz's package (Herr Dr Witz, I suppose). Even Dr Witz stopped doing that three years ago, if not earlier.
 
11:59 PM
It's not about the error. I have an existing system that uses R code that I am replacing and I have to duplicate it's output exactly, including if it has any errors.
 
00:00 - 17:0017:00 - 00:00

« first day (3275 days earlier)      last day (1672 days later) »