« first day (3022 days earlier)      last day (1940 days later) » 
00:00 - 20:0020:00 - 23:00

That's the one :-)
 
It took me a minute to remember the name. :)
 
recbg
 
wim
^ The actual paper.
 
I have a bunch of linux wireless machines I'd like to have phone home when they come online, would AsyncIO be a viable solution for this??
or are there any others im missing, just plain old sockets?
 
8:09 PM
cabbage
 
it took me some time, but I've compiled a list of all my canonical posts and answers thus far on GitHub. Perhaps an RO can consider adding these to the SOpython canon list?
 
wat
 
IMHO, it's a reasonable law. For many properties of natural numbers, there's a small number that exemplifies that property. OTOH, each property of numbers can be identified with the set of numbers with that property. But the cardinality of the set of sets of natural numbers is larger than the cardinality of the set of natural numbers. So small numbers tend to be associated with multiple interesting properties.
 
wim
OTOH, each property of numbers can be identified with the set of numbers with that property <-- treading dangerously close to russel's paradox there
 
Halfway through that pdf and the theme seems to be more "don't make conclusions based on apparent patterns in the early parts of a sequence" and not so much "a number is more likely to show up in a lot of properties and proofs and stuff if it's small" but I suppose the former kind of implies the latter
 
8:20 PM
@wim the ex5 is nice :D
 
See the infinite cosine product integral as a recently discussed example
 
Maybe, but it's a fairly standard definition. Thus "evenness" is identical to "is a member of the set of even numbers". It only gets silly when there isn't a finite way to describe how to construct that set. Or if the construction process involves a Berry paradox style self-reference.
 
1, 2, 4, 8, 16.... how does it continue...
 
I'm going to go with "32"
 
im scared
 
8:22 PM
so what is ex5 instead?
 
its like i sense Antti having an evil chuckle to himself right now
 
Yeah, tl; dr
 
can't copy paste...
 
It's easy to come up with a polynomial that'll give you any value you like for the next term.
 
8:24 PM
I have a program that does that in my gists somewhere.
 
if you have a circle and you draw n distinct points and chords between each pair, how many partitions will you create for n?
 
Lagrangian interpolation FTW
 
oh ive seen that before i think
31 was the next
 
@AnttiHaapala neat, thanks
 
8:25 PM
@AndrasDeak oeis.org/A000127
 
Now we're talking
 
Hey, good afternoon. I have a question, could somebody help me please? I'll write it:
I have two dataframes like this:
 
Is it my imagination, or did they redesign gist.github.com? I feel like I had an easier time searching my own gists before.
 
@ParitoshSingh yes
 
wim
I really don't get gist
like, what does it do that a repo doesn't already do better
 
8:37 PM
it is a pastebin
you cannot have secret links to repo...
 
Easy workflow. Create Gist -> enter text -> Create Public Gist.
 
I have some questions about requests. A bit technical I think
 
I have

name group group_father
0 john H1 Design
1 keith H2 Manager
2 charles H3 Developer
3 john H2 Manager
I need to transform to:

name group group_father Design Manager Developer
0 john H1 H2
1 keith H2
2 charles H3
Sorry
I can't access to pastebin from my work
I'll try to look for another tool
 
@VictorMartinez can't access gist? ^
 
def download_bytes(url, from_byte, to_byte):  # stackoverflow.com/a/16696317/7886229
    local_filename = tempfile.NamedTemporaryFile()
    r = requests.get(url, {'Range', 'bytes=%d-%d' % (from_byte, to_byte)}, stream=True)
    for chunk in r.iter_content(chunk_size=1024):
        if chunk:  # filter out keep-alive new chunks
            local_filename.write(chunk)
            # f.flush() commented by recommendation from J.F.Sebastian
    return local_filename.name
 
8:38 PM
@JBis same thing^
 
wim
@Kevin it's same as creating a repo?
 
@JBis ok your problem is clear though
@JBis as soon as the reference to the local_filename is dropped, the file is deleted too
you must return local_filename (and rename it as local_file, as it is much more than a name)
 
To create a repo I have to click the plus icon and then click Create Repo. Already it's one button slower, and we're only on the first step.
 
the .name alone wouldn't keep the file alive
 
well that wasn't my question but lets deal with that first
 
wim
8:40 PM
maybe I am in a different ui
 
so the second I return local_filename will disappear?
 
wim
new repo is in the same dropdown as new gist for me
 
@AnttiHaapala
 
well that sucks
 
8:41 PM
@JBis another thing is {'Range', 'bytes=%d-%d' % (from_byte, to_byte)}, is a set.
you need {'Range': ... } to have it a dictionary
 
wdym?
@AnttiHaapala wdym by this too?
 
scusi?
 
whats wrong with the range thing?
 
headers needs to be a dictionary, you're not giving one.
 
Anyway, I found my polynomial fitter: gist.github.com/kms70847/9c0e31077fadba969298. Give it [1, 2, 4, 8, 16, 31] and you'll get oeis.org/A000127.
 
8:43 PM
sets and dictionaries both use {}. but sets are similar to lists in that it contains comma separated values, whereas dictionaries are key:value pairs. with the colon.
 
@AnttiHaapala Yeah, thanks!
This is the link:
 
oh I'm stupid
typo
 
@AnttiHaapala So if I return the object itself it will stay alive? And is there a way to return just name without deleting file?
@ParitoshSingh thanks
 
8:46 PM
I tried to do it but nothing works.
 
if you need the file to persist, dont use a tempfile
 
@AnttiHaapala What a surprise. I needed to read the docs.
@ParitoshSingh Its gonna be deleted soon.
 
Ok back to my main question. Where does requests store the content before its saved? In memory?
 
@JBis presumably
in your case since you're iterating then there is not much being saved anywhere.
 
8:50 PM
so what be the limiting resource of calling this function a bunch of times? Write speed? Network speed?
 
depends on the sizes and speeds involved, but you can generally assume the network speed
 
Whenever requests is involved in a performance problem, I'm inclined to blame Internet speed, yeah
Where's that article that says "if writing to a register took one second, pinging San Francisco would take a hundred years"
 
heh, sounds about right.
 
thanks. Ok next up: In a threaded environment, can multiple threads share the same variables? Kind of like a Master/slave relationship. The master will manage everything the slaves do, but the slaves need to be able to communicate with the master.
@JBis do a search:
18
A: Python creating a shared variable between threads

user3343166We can define the variable outside the thread classes and declare it global inside the methods of the classes. Please see below trivial example which prints AB alternatively. Two variables flag and val are shared between two threads Thread_A and Thread_B. Thread_A prints val=20 and then sets val...

top notch self help right there ;)
 
oof, that terminology. but yes, thats the premise of threading, having shared memory space. I personally do not have a lot of experience with it, but search "managers" and "locks"
heh, well done
 
8:56 PM
sorry about that terminology never used multiple threads or done anything like what im trying to do before
 
Oh, note that that link is an attempt to "reinvent the wheel" so to speak. great for learning, but not needed when you need to just use what exists
 
wdym?
 
There are already a wide variety of existing classes that facilitate communication between threads, so you almost never need to invent your own using primitive globals
 
^ This. i havent worked with threading, but that code seems to have too much of "raw guts" on display.
 
oh. Ok. Will look into.
 
8:59 PM
In particular, all the classes in docs.python.org/3/library/threading.html other than Thread itself, and also docs.python.org/3/library/queue.html#module-queue
 
Also is there a way to stop the request, mid request, incase the to_byte needs to be changed?
I was thinking breaking out of the loop or somethign
 
thanks man! I will check it.
 
hm, i dont think so. you can just make a new requests.get
oh, and i just noticed the stream = true. i think with that in place, you probably do not read/load all the data into memory at once.
 
@ParitoshSingh yes, this is the point
@ParitoshSingh hmm. That screws stuff up. But isn't it getting content as it iterates through the for loop?
 
well, it is.
 
9:08 PM
requests isn't the library you want to use if you need to do something fancy
it is the "http for idiots" library
 
what would you suggest? I am new to python/started yesterday.
 
no wonder i like it!
seems very intuitive
 
use requests and don't think you need something else :D
 
lol. am I gonna have to make requests manually or something
 
the point being if you need to "change things when the http connection is going on" then you pretty much cannot use requests
 
9:10 PM
httplib?
 
well... maybe, maybe not, or aiohttp or tornado or whatever.
chances are if you just started yesterday you're overengineering :D
 
@AnttiHaapala possibly, but I doubt it. I can tell you my idea. Prevent XY problem
 
cbg! QQ. How do I formulate this question properly... I have axes that already has a plot in it. I want to take an instance of this axes and other ones and "insert" them into some other subplots (i.e. other axes). Is this possible? Does this make sense? laurel
 
Most big companies cap download speeds. For example, I have 1Gbit speed but when i download from google drive i only get ~9MB per second. Obviosully google has more bandwidth than that, they are just caping it.
 
9:15 PM
@isquared-KeepitReal search for "inset"
 
Heres my idea:
Using range requests, split up the download. For example:
 
@AndrasDeak not a graph inside of graphs. I will give a code example in a sec
 
OK
(what you asked sounded exactly like an inset)
 
Thread A: Downloads 0-25%; Thread B: Downloads 26-50%; Thread C: Downloads 51-75%; Thread D: Downloads 76-100%;
But A, B, C, and D run at the same time.
Assuming the client has enough bandwidth, you would get 4x the download speed. And thats just with 4 threads.
 
what are you downloading?
 
9:19 PM
Heres the problem: I need to figure out the max amount of threads to use. Memory is static so thats easy, but since the limiting factor is network speed and that changes often, I need to be able to calculate, based on download speed, how many threads to create. Hence why I will need to stop requests prematurely.
@ParitoshSingh Anything. Files. But would prob work best on large media: Videos, Audio, etc.
 
also, i dont see why if a site bottlenecks you then you can assume that separate requests from same ip will be capable of downloading without hindering each other.
 
@ParitoshSingh This was the only conceptual concern I found, however based on testing, many file hosting sites don't go based on IP.
 
So this thing below produces some nice pretty picture. I want to take that axes object/chart object and then plot in a different axes object (when I redefine `fig, ax = plt.subplots(ncols=3, nrows=6)`, in a different place for example)

fig, ax = plt.subplots(figsize=(9, 5))
# https://matplotlib.org/users/colormaps.html
levels = np.linspace(np.min(Z), np.max(Z), num=25)
cs = ax.contourf(X, Y, Z, cmap=plt.cm.get_cmap('RdYlGn'), extend='both',
levels=levels)
fig.colorbar(cs)
ax.set_title(r'CAC Index lag-time $\rho$', fontsize=16)
 
still think over engineering?
 
@isquared-KeepitReal NameError on X, Y and Z
if I understand your question you might just be looking for fig.axes.append(your_axes)
apparently that doesn't work on newer matplotlib, here's how it does
> in newer matplotlib versions [...] axes cannot live in several figures at once
This will be an obstacle for you. I don't know if you can copy axes. Probably not easy.
 
9:27 PM
hello all
 
hello
 
@JBis overengineering yes.
@JBis and for that you'd not use threads at all but an async event loop
and definitely not requests.
 
i have 3 python modules. (1) driver1.py (2) a1.py (3) b1.py
 
@AndrasDeak I will just create a method to plot all at once instead lol. Thanks
 
9:28 PM
driver1.py can take 5 arguments and a1.py takes 3 arguments and b1.py takes 2 arguments
 
@isquared-KeepitReal that's probably simplest and most maintainable
 
i am trying to use argparse to do the argument parsing
 
@AnttiHaapala Whats the benefit of using async? Don't really under async entirely.
 
and hitting some roadblocks
because the other two modules take a sub-set of the arguments that the main driver module does
 
@JBis the benefit is the drawbacks are smaller than with threads.
 
9:30 PM
is there any way i can resolve this ?
 
@JBis Python doesn't do parallel execution. and with threads you'd have to figure out how many threads you would do.
so the async thing, you don't need to figure out how many threads you'd need.
 
Makes sense now:
 
at any given event you'd figure out whether or not start more requests.
 
fig 2 is not what happens in python though, it is like the fig 3, but less control and more overhead :P
but the thing is since these tasks are not separate then the threading makes it just more difficult
you don't have 3 tasks, you have 1 task that you do in chunks.
 
9:34 PM
interesting way to think about it
 
I am not saying any of this is easy :D
 
could anyone please help ?
 
or that I could do it outright
@SusheelJavadi not with these details
@SusheelJavadi so why are your modules reading the arguments at the module level
that's not how you should do it.
instead have the modules have functions or sth.
... anw, argparse does support partial parsers too
 
@AnttiHaapala - ok
i have pasted the code here - pastebin.com/WYdyhZxu
 
@JBis They may be lenient when you try requesting multiple files simultaneously from the same IP, and even from the same user agent, since browsers do that all the time. But if you try requesting chunks of the same file with range requests then they can rightfully deduce that you're trying to get around the rate cap, so they can do whatever they want to stop that.
 
9:38 PM
would you mind taking a look ?
i am using python 2.7
 
@PM2Ring you expect there to be a rate cap...
@PM2Ring but it might be there is no cap at all... only that one endpoint can give only so much.
 
@AnttiHaapala Fair point.
 
didnt even think of that. With CDN it would work even better.
 
@SusheelJavadi shouldn't you grab the remaining arguments from parse_known_args and give them to the other, or something like that.
 
alright guys enough thinking for now. thank for help, ill be back soon with more questions
XD
 
9:44 PM
This talk of rate capping reminded me of thttpd. Somewhat amazingly, it's still going, under new management. en.wikipedia.org/wiki/Thttpd
 
10:25 PM
Got a pandas question for anyone so inclined. I want to forward fill a column and I want to specify a limit, but I want the limit to be based on the index (a datetime index, if it matters)---not a simple number of rows like limit allows. Anyone know a good way to achieve this?
 
00:00 - 20:0020:00 - 23:00

« first day (3022 days earlier)      last day (1940 days later) »