« first day (3112 days earlier)      last day (1820 days later) » 
00:00 - 17:0017:00 - 00:00

12:20 AM
rbrb
 
12:31 AM
oh noes we're read-only
 
1:15 AM
noo
 
2:10 AM
yess
 
I have a script on an ec2 instance. I don't want to spend money to keep the instance on all the time. Do you know if there's a way I can use python and automatically start the ec2 instances once a week?
 
in search of the answer to life's biggest questions. I currently have an EC2 instance I'm dying to figure out how I can start/stop without having to do it myself
 
3:10 AM
cbg
@AndrasDeak Was this read-only 2 hours ago?
@cs95 Lol :-)
You're getting funnier and funnier :-)
 
3:58 AM
cbg!
@cs95 They had something that basically scheduled start stop times. timewatch or stopwatch or something?
ah. looks like cloudwatch. maybe one of these two will help. link1 and link2
 
rbrb
Just when people are joining
 
4:31 AM
@U9-Forward there was a hiccup in the Stack data center, it seems to be resolved now
 
 
2 hours later…
6:07 AM
@tripleee I think it was planned maintenance
 
@AndrasDeak true, the hiccup when they came back up didn't affect chat per se, but many chat bots etc
 
6:30 AM
stackoverflow.com/questions/55754234/… dupe in comments, op agrees. They just didn't press the button to accept the dupe it seems.
 
hey guys, is there a type of matrix product that takes an MxN matrix, another MxN or NxM matrix, and can generate a (MxN)x1 vector?
 
elementwise multiplication followed by a reshape perhaps?
 
i want to take an image (for now lets assume a square image) and then rearrange all the pixels along a hilbert curve (which maps it into a 1d list). I figure a good first test case that would also perform very well is to do this with an array of values.
so basically im sketching out the idea as (see if a transformation matrix can be made to rearrange everything, then merge lines, or see if a transformation exists that easily maps each element to the right spot)
 
6:46 AM
no idea what that is, im afraid. looks fancy.
 
the cool property of hilbert curves is that they represent locality of neighboring points in 2d/3d space well in 1d
 
@Arne still up, edited in the missing tag
 
Look at that, I didn't even notice. thanks
 
@Skyler matrix product no, general mapping probably, general linear mapping maybe. Define the operation first.
 
7:12 AM
@Arne closed
 
thx
funny, op did accept it as a dupe, I had the impression their vote alone were enough.
 
trying to debug, and I'm not sure what I'm looking for when I have an object. In frontend JS, I could print the object out, and the browser would give me a navigable tree
which is great when you have huge objects
I'm on a terminal, and python's json.dumps is being pretty slow and showing too much stuff
is there an alternative with a max_depth or similar?
 
if its an option, i personally prefer dumping to a file, and not bothering with terminal outputs
then theres some really nice plugins for notepad++ for example, but at that point the json is in my hands, and i can use whatever tool i prefer
 
Umm... do we have a specific list.append doesn't return anything specific duplicate or just the list.sort one?
 
well, I'm using vim over ssh
 
7:25 AM
Seems it's been asked many times over so I'd be surprised if there wasn't a preferred canonical...
 
actually, all of them seem a bit..weird
 
@ParitoshSingh thanks...
 
lemme see if i can find something nicer
 
Um... I've closed as a dupe of the first one for the moment... must be a better one out there...
 
yeah, i agree
weird, that it itself is marked as dupe, but seemed to be the cleanest one so far.
 
7:35 AM
@Paritosh agree... that's the most straight-forward direct answer to the question... so I've swapped it for the other one... which that one links to anyway for further reading...
Odd... in sopy it seems to do some ordering on the questions list so can't actually directly order them...
ahh... looks like it puts question urls as a set... so it doesn't matter what order you arrange them in the edit box... oh well.
 
8:41 AM
Morning guys...need your thoughts on how to deal with this situation : User submits a form that contains his/her address and phone number...on the other side there's 10-20 drivers that are interested to receive this data and pick up the package. Send 10-20 sms ? First come first serve ? How would you deal with this situation ? Wanna make this as smooth as possible for User...Now it's working the old way...calling 10-20 drivers asking who can pick up my package. Thank you
 
That's a business question, not a python question
 
@roganjosh hmmm...if you say so...thank you anyway for your time
 
There should be no need for SMS. If you're in that situation, you need an in-house app
 
@AnotherUser31 this sounds really similar to uber's model, so I guess you could copy their approach
 
^ dont disagree there. The sms might be needed, or some form of communication or indication to the drivers. But the communication should be to a "link" that lets them self-nominate on some criteria on your app. This app should be able to handle multiple users, and resolve multiple drivers trying to accept the job, perhaps on first come first serve
 
8:45 AM
which, incidentally, is also what roganjosh suggests
 
^ agreed :)
 
the app can be a phone app, a web based app, anything you see fit.
 
you guys are fantastic you know that !!!
forgot to mention that delivery location is always the same(i know that sounds weird)
 
I don't profess to know how Uber works, but I would guess it is a greedy algo for selecting the drivers. That's how I did it, anyway
 
i'm thinking more into a web app...
 
8:47 AM
Solving the routing problem is expensive and drivers may say no to the job, so pick them with a cheap algorithm
 
two step greedy really, first at distance ranges on which to notify drivers, second on who accepts.
 
Once a driver accepts, it shouldn't be a greedy solution
 
so let me explain you the whole picture and "business" idea behind it... there's around 20.000 eastern european expats in germany....from ukraine, moldova...that want to send something home...so there's around 20 drivers who drive weekly whole germany and pick up those packages...but for users it's a nightmare to find a driver...u gotta call them...that's why I thought to make something easier...they all use whatsapp or viber or telegram...
 
How far in advance do they book the pickup?
 
i thought to do a weekly thing
like..me as user submit the form....saying hey guys here i am - got a package...here's my address and my phone...you find me, and come pick it up
 
8:53 AM
So it's always a week in advance (solution-wise)? A pickup can be scheduled on a Sunday night but it goes into the weekly problem starting Monday?
 
no idea how to monetize that though...it's more a cash thing....later maybe when everybody's on board....
 
In which case, you don't give the drivers a choice
 
@roganjosh yeah, could be a weekly thing...
 
Well, you're gonna struggle there
 
no rush...
 
8:54 AM
Who are the drivers?
 
alright, suddenly it's a lot different than uber =D
just teaches us that you can't give good advice without knowing more about the problem domain
 
It's also where the industry is moving :)
But I'm curious who the drivers are
 
@roganjosh drivers are eastern europeans too :) never smiling :)))
 
Lol, I'm slightly less concerned about whether they do service with a smile
 
@Arne yeah :))) thing is there's a niche there :)
@roganjosh same here...as long as the process for me as the user run smoothly
 
8:57 AM
Not really a niche. You have the whole industry on your back
 
@roganjosh well...you can send something home using regular post...but it will take you up to 2-3 weeks
 
But your business model is unclear and tech alone will not fix that. Who are the drivers?
 
with the drivers it's about 2-3 days
@roganjosh don't really get you question...what do you mean by who are the drivers ?
 
Well, on the 1 hand you're saying they can decline jobs but on the other I guess they're not employed by you
But there's 20 of them, at least envisioned. I don't follow the business model
 
i'm not employing them...they run privately...bringing packages from eastern europe to germany and from germany to eastern europe weekly...
i was thinking to charge either the user or the driver for every package a small amount of money...but that's a "future music" as we say in germany :)
for now...just make it easier for the user...and maybe for the driver too...
but again i'm not sure what to do....i got the phone numbers from drivers...
 
9:04 AM
Ok, I think you really need to work this model out before going further. You're not gonna make any money as it stands; 20 drivers is too few and it's not clear why they want to do freelance anyway. I don't want to sound harsh but it's also the industry I work in and there's some crucial stuff missing here
@AnotherUser31 But keep in mind that this really is the way it's all moving. You've just got to get idea solidified first :)
 
@roganjosh the math here is simple say you have around 300 packages sent every month...if you charge 0.50 cents per package = u have 150 euro weekly - 600 euro monthly, 7000 euro yearly...by doing basically nothing...well offering a pain free service to users and drivers... might sound small money but...
*300 packages every week i meant
drivers are not doing freelancing....they want 2 euro per kg from the user... depends if you wanna send a fridge of course the price will be a different one...
 
9:22 AM
Im not sure I can be too much more help here. If you struggle with the vehicle routing, I'm a contributor to Jsprit and can perhaps help with that. I don't really have anything else to add to the idea itself
 
@roganjosh yeah, right, enough business :) thank you sir ! I just thought to get some ideas from you guys on how to deal with my situation :) you guys a re awesome !
 
@AnotherUser31 Sending packages in germany might be tricky. are you aware of the Verbraucherschutz regarding this kind of business? What happens if a driver steals all the packages or has a car crash, will you be liable for the damages?
I'm not trying to dissuade you btw, you should just be aware of nasty surprises
 
@Arne got your point...I won't deliver the packages...all I do is making the process of connecting the user with the driver smoother...from there on...driver is responsible...I'm more aware that the drivers won't want to pay any cent...so I would have to charge the user later....but again...for now it's making it work...I'm stuck with the thing...like I said...how to deal with the situation...send 20 sms and wait till first respond ?
 
Oh, man
 
@roganjosh :) alright...I'm off...thank you guys :) and sorry :)
 
Sam
10:01 AM
Morning all: does anybody know how to run a docker-compose file inside a Python script? I've tried using subprocess:
subprocess.run(["docker-compose", "up", "--build", "cd", "/Documents/Projects/app/app/docker"])
Can't find a suitable configuration file in this directory or any parent. Are you in the right directory?
The directory I specify is the parent directory to the docker-compose.yml file. If I simply change docker-compose up --build to ls in the subprocess command I can see the compose file listed
 
@Sam do you somehow expect docker-compose to run cd /Documents/Projects/app/app/docker as a shell command before attempting to build? Do you really have /Documents in the root of the file system?
I'm guessing maybe you want
 
Sam
I just removed my name from the directory structure
 
subprocess.run(['docker-compose', 'up', '--build'], cwd="/Documents/Projects/app/app/docker")
but this is pretty speculative at this point
 
Sam
ahh I didn't realise that cwd parameter, perfect
 
take out the slash before Documents to have it relative to the current directory
 
Sam
10:10 AM
Yup works perfect, thanks
 
very often you don't need to switch working directories at all if you just run the script where you want it to happen
 
Sam
That all seems to work fine, code after the subprocess.run() doesn't seem to be executing though, not sure if its because of how the containers boot up in the CLI
 
user7437554
10:41 AM
Hello guys, I'm reading a code with the following line:
 
user7437554
global max_score
 
user7437554
any idea what global might be used for? I havent seen that function before
 
@santimirandarp it's a statement... see: stackoverflow.com/questions/13881395/…
 
If you google "python global", you’ll find multiple Stack Overflow QAs that can explain it much better than we can in a chat message here
 
user7437554
that's perfect. I've tried with some docs but all seem too complicated
 
user7437554
10:45 AM
thanks guys
 
@Sam subprocess.run() will block until the subprocess is finished, if you want it to run concurrently try just subprocess.Popen() but be prepared to manage the resulting object yourself
 
or you can do stuff with asyncio... :)
 
11:16 AM
Im kinda embarrassed by how little I know about async def but I don't think I really understand the use-case vs. The other methods of launching processes
Am I right in thinking it's somewhat low-level and generally wrapped up by existing libraries?
 
async is a different way to achieve (apparent) concurrency, you have Python do a little bit of things and when the code would block it moves to a different task, sort of like how the OS does multitasking
 
I suppose that if I dig I to tornado or gunicorn I'll see it, but I've not found a problem I face that has directly pointed me to use it
 
so instead of muiltiple processes (multiprocessing or subprocess) or threading with multiple threads within a single process (thread) you have multiple async tasks within a single process (and typically -- I guess -- a single thread)
@roganjosh trying to understand Twisted was my introduction, though I have not examined either Twisted or the built-in async properly in a good while
 
@tripleee gosh, then I definitely don't know how that works. I'll have to do some research :)
 
11:30 AM
Thanks. Looking through that, it looks like I've been pretty fortunate that my use of it has already been wrapped up by libraries e.g. requests-futures for async requests and callbacks
 
Sam
@tripleee Thanks
 
12:03 PM
@tripleee That report caused our engineering at my last employer to switch from gunicorn + flask to asyncio
 
Did you search on Google for it?
@Arne I don't follow. What's going to make them drop an entire stack?
 
we had been having a bunch of problems with deploying gunicorn for about half a year and had someone on our team who was confident that he could rewrite our flask code to aiohttp in a couple of weeks
 
I can't say I've built a Flask app that gets battered, but isn't it the job of Nginx?
 
we didn't use nginx, azure handled setting up the proxy
 
Sam
If you structure the application correctly, it shouldn't get battered.. right? No matter the tech
 
12:11 PM
but it was bad and hard to configure, so we needed to handle at least basic async processing as lightweight as possible on our side
 
Battered by requests. That's never going to be in your control
@Arne did it end up better than a traditional approach?
 
what would a traditional approach be?
but the short version is that yeah, it ended up working out quite nicely
 
Part of my brain is screaming that it's like re-developing numpy
 
but only because we only had like 4 routes anyway, so refactoring and testing was a breeze
 
Nginx to do load balancing, and gunicorn to serve the app
 
12:15 PM
maybe some haproxy in there as well for good measure? :p
 
@JonClements can't hurt, right? something something behind 7 proxies :p
 
why not some cloudfare as well hey? :p
 
I need a pen to get this recipe down
 
How about if you just write down: "two large Gregg's sausage rolls" and "some doughnuts"? :p
 
Do they have jam in them?
 
12:24 PM
I prefer mine with jam in 'em... (doughnuts that is - not sausage rolls) - if you could get raspberry ones that'd be awesome - thanks mate!
 
It's all about the detail :) we also have a battered flask so I need to know how long that gets fried for
I've just had a scary realisation that I could probably find a library to support any cheap pun I want, which makes it all even cheaper :/
Speaking of cheap puns, where is Andras? :P
 
who dares raise me from my slumber
 
he's a very expensive pun I'll have you know! :p
 
12:41 PM
... The summoning is complete.
 
@roganjosh I don't see Cthulu or Tony the Pony anywhere!? /me is disappointed... :p
 
"Slumber". He doesn't even know the meaning of the word.
 
m8_
Slumber is a horror movie released in 2017 about a sleep doctor...duh!
 
If Andras is a sleep doctor then he should be struck off. I don't believe he's closed his eyes for more than a second
 
@roganjosh Sure I know. It's, uh, the comparative form of "slumb".
 
12:51 PM
@AndrasDeak Not gonna lie, I wouldn't know that and it's my first language :)
 
cbg
 
Cbg
 
m8_
Cigar Box Guitar?
 
cocoa bean grenade
 
m8_
chill black guy?
 
12:54 PM
c'mon be good
Kevin posted a thing about the Feynman Algorithm yesterday and I was reading through the link and found this:
"Winning a Nobel Prize is no big deal, but winning it with an IQ of 124 is really something." -- Feynman, referring to his tested IQ
Thought that was pretty funny
 
Isn't 124 on the upper end of the Spectrum though?
atleast for typical ranges?
 
I think it's 148 for Mensa
100 is average in the completely contrived test
 
Okay. So 124 is "above-average" then?
 
Well,every test is contrived. Inb4 that
 
Well. not wrong.
 
1:02 PM
124 is probably "smart dude"
 
I feel like if I were to take an IQ test I'd either get a low score and feel perfectly adequate or get a high score and look at the test and say its a load of junk.
 
They're interesting to do
 
that's just normal human behaviour
 
m8_
Hey, got a pandas question. I am changing the value in a column if it equals some value df.loc[(df['col_A'] == 'current_value'), 'col_name'] = 'new value'
Is this change occurs, is there a way to also note that in a new column? Like to keep track of which rows were changed?
 
Make a new column
 
m8_
1:07 PM
So the new column would say, changed if the value was replaced
or ok, you mean don't change it in the original?
 
mask = df['col_A'] == 'current_value' and assign that as roganjosh said
 
m8_
gotcha, thanks
 
Might be able to do it in one go - something like: df.loc[df['col_A'] == 'current_value', ['col_name', 'changed']] = ['new_value', True] or something... untested @m8_
 
m8_
cool, thanks @JonClements, I'll try it out
 
I'll say that I've never seen that and it's pretty awesome if that works (I can't test)
I assume the list must force it out of numpy, though?
 
1:20 PM
it all depends on whether you can create new columns with .loc
in this case even if it works I'd say that the two-step version is much clearer
 
m8_
Yeah, normally you can but this is giving me a keyerror, so Im going to try to create the changed column first
 
Hey so if using Openpyxl, is it possible to change the data format of a column? My Last Column should be dates but for whatever reason the repository I get my files from gives them out in random formats.
 
@AndrasDeak yeah... and more reliable probably... I know there's some way to do the above... to save scanning the DF twice but can't remember it... (that or I'm imagining it...)
 
why would the two-liner "scan the df twice"?
 
m8_
It worked @JonClements, I just had to create the change column before hand
 
1:24 PM
That must be in a python loop?
 
@roganjosh whom are you asking and what is "that"?
 
m8_
He is referring to Jon Clements msg about scanning the DF twice
 
@Andras I was thinking along the lines that you have to do the DF once to get the mask, then apply the mask against the DF to assign the value...
 
@AndrasDeak the list assignment suggested by Jon that apparently works
 
@JonClements the mask itself is the value, right? Or .astype(int) at worst.
I'm thinking df.loc[mask, 'old_col'] = 'new_val'; df['new_col'] = mask # .astype(int)
 
1:26 PM
Nice syntax but I'm not sure it's vectorized
 
meh... I'm barking and foaming at the mouth trying to get a magento cloud deployment working... I'm going to think I made this all up and try and focus on my rabies like symptoms and docker stuff :p
 
m8_
all I did was df["Changed"] = "" then df.loc[df['col_A'] == 'current_value', ['col_name', 'changed']] = ['new_value', True]
 
Makes sense to make the extra column but I'm not convinced that pandas can exploit this in any way. I could well be wrong on that. My gut feeling is that it goes back to a regular python loop
 
m8_
Maybe, but it does the job
 
So it may well be better to do in multiple steps
 
1:29 PM
a regular python loop over two iterations doesn't seem that bad
 
2:10 PM
"You forget that I live on my own estate" said at the bar. I think it's time to move on on my last holiday day :)
 
2:24 PM
sigh I just realized that on CFFI precompiled bindings is not possible in ABI mode if You have structs with various padding >1 :|
 
user7437554
2:46 PM
a = [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
b = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]

newlist=[name for name if name in a and b:]
print(newlist)

common_num = [a for a in list_a for b in list_b if a == b]
print(common_num)
 
user7437554
the previous code contains two versions which, in my view, should work the same. THey should produce a list with values found both in a and b.
 
[name for name if name in a and b:] isn't valid syntax
 
user7437554
Yes, but why?
 
@Aran-Fey could very well be, I'd like to check first
 
Because the logic doesn't work like spoken language
 
user7437554
2:48 PM
hmm well it should, logic is logic
 
because 1) x: isn't valid outside of square brackets like [x:] and 2) list comprehensions have to look like [x for y in z if w], not [x for y if w]
 
Python isn't natural language
a and b is not the concatenation of a and b
it's the boolean result of the the truthiness of a and the truthiness of b
 
user7437554
:(
 
user7437554
But it's strange...if fit is the boolean etc it should work. Anyways, I'll try to understand the right answer
 
then there is name for name if name Python doesn't know what to do with that unless you do name for name in a + b if name You can't invent arbitrary syntax
however I'd use filter
filter(None, a + b) or [*filter(None, a + b)] for a list
 
user7437554
2:52 PM
I'm too stupid so please do not change the syntax ha
 
You perhaps underestimate what gymnastics your brain has to do.
 
user7437554
it should go to the bin XD
 
@santimirandarp That is exactly what Python is saying.
 
user7437554
@Aran-Fey 1) do you mean x: is valid only outside square brackets? Or I don't know what you mean
 
No, it's only valid inside square brackets
>>> 'foo'[1:]
'oo'
>>> 1:
  File "<stdin>", line 1
    1:
     ^
SyntaxError: invalid syntax
specifically, square brackets that perform a __getitem__ operation
>>> [1, 2, 1:]
  File "<stdin>", line 1
    [1, 2, 1:]
            ^
SyntaxError: invalid syntax
 
user7437554
3:00 PM
yes I know that, it was an error because first of all I wrote it as conventional blocks
 
user7437554
ok the 2) is very helpful
 
user7437554
I didn't even notice that
 
user7437554
So the only possibility is the second way? @Aran-Fey
 
user7437554
I tried a third way but again errors
 
well, no. There's an infinite number of ways to write equivalent code. To be honest, I'm not really sure what your code is supposed to do. If you want "a list with values found both in a and b", you can do that with list(set(a).intersection(b)) - but that doesn't produce the same result as your list comprehension, so I'm confused
 
user7437554
3:08 PM
yes, that's exactly
 
user7437554
 
user7437554
(the second one)
 
user7437554
Do you think thats not the best way of writing it?
 
depends. It accomplishes the goal for the example but if i wanted a list of unique values common to both lists I'd do: [*{*list_a} & {*list_b}]
 
looks like hacks
 
user7437554
3:17 PM
no idea what's that
 
{*list_a} is just set(list_a) So it is list(set(list_a) & set(list_b))
 
user7437554
btw if you're bored here is the page with exercises practicepython.org
 
user7437554
but is for newbies
 
user7437554
okey I'll read what set is, thanks!
 
til you can take an & for sets. I dont get why though, it must have been specifically written that way, no?
 
3:20 PM
le cabbage
 
fwiw:
%timeit set(range(1000000))
%timeit {*range(1000000)}

# 60.7 ms ± 250 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
# 60.3 ms ± 362 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
@santimirandarp If you aren't aware of what a set is, you need to complete a python tutorial. Most will cover the basic data structures and set is one of them. It is essentially a group of unique things.
 
You have two presorted lists of integers a and b. You want to get a list of elements of a that are not contained in b. Can it be done in linear time and in one pass?
 
user7437554
Hmm, I'm not sure of that...@piRSquared
 
user7437554
haven't found any cool, free tutorial
 
3:33 PM
@ParitoshSingh the set class defines __and__ to be intersection See this
 
There are plenty of tutorials for this kinda thing.
 
nifty, thanks!
 
@santimirandarp room 6 list o' tutorials
With the official tutorial having a section on sets
 
user7437554
haha thanks, but I just want to learn the minimum for chemistry
 
user7437554
not sure if I should learn about sets...
 
3:37 PM
What is chemistry?
 
learn about sets. the basic datastructures are worth it.
 
Then you are putting yourself at a disadvantage and who is to say what the minimum amount is for any discipline?
 
Im at odds with "I need to learn the minimum"
 
user7437554
well I've never seen the word sets in simple scripts
 
user7437554
Yes, you're right but when you need to learn many different things the only way sometimes is to learn only what you're gonna use
 
3:39 PM
"I will learn the minimum of math. I will not bother learning multiplication, because I can do the same thing with repeated addition."
 
It says an awful lot about the asker
 
i get the idea behind it. Its one of "i just need to understand enough to use this tool, i dont need to know more about it, i just want to get some work done" . Eh, its not a bad idea. Might not always be practical, and you'll eventually find out you might have avoided some pitfalls had you spent more time with something. But i mean, if that's what gets you started on learning something, then thats better than nothing.
 
Every tutorial I've looked at has a section on set. If you aren't willing to go through a basic tutorial then I too will go the minimum route and do the minimum amount to help those who won't help themselves.
 
user7437554
So if you learn programming you also learn about what's the arrange of atoms in the laptop?
 
cabbage
 
3:41 PM
cbg
 
user7437554
Wait guys don't go too far...I think both extremes can be harmful, and I understand what you mean, but I'm studying ancient greek, mathematics, chemistry and programming and sometimes it gets really difficult for me
 
user7437554
Would you please understand this? If you can't help, it's ok, I will remind you not to help me
 
A lot of thought has gone into what should be included in tutorials. TRUST me, those tutorials are not comprehensive python references.
 
user7437554
okey, thats a reasonable sentence
 
user7437554
So I'll start the first tutorial :)
 
3:43 PM
Most of the regulars here are reasonable:)
If they're not reasonable, they can still be knowledgeable :P
 
user7437554
@ParitoshSingh thanks...yes sometimes you loose time, as you point out
 
I have a data problem. I have around 100 .csv files averaging 1.4M rows, and around 15 columns. I need to build an app that can visualize 1 file at a time, interact with it, view it in parts, have some customization in axes/labeling etc. How do I go about this? looking for keywords and theory to research. Which database to use, or h5? should I switch to javascript? how do I make it as seamless (in terms of speed).
 
The basic tutorials for any language are well worth the investment, you wont regret taking that little bit of extra time
and python is arguably one of the fastest ones to pick up anyways
 
user7437554
Is there any tutorial about how to learn faster or at least learn without forgetting? ha :(
 
They're csvs. The data is already in a format to be stored
 
3:47 PM
Nope. Its the same old formula. repetition and regular use.
 
user7437554
:P
 
@santimirandarp sure. Propose in chat and get shot down :) it's happened to me plenty of times
3
 
@pyeR_biz Dont worry about whether it will be too slow when you're starting out the work on it. If is has to be a web app, and youre comfortable with javascript, you can use javascript. But if you're comfortable with python, use that instead. First, get your work done for 1 csv file, regarding all your graphs and whatnot. Address speed if it becomes a problem.
 
user7437554
haha that's very kind @roganjosh
 
Frankly, python inherently can probably pull it off just fine. The real question would be how much data you're going to end up displaying on the final graph. And if that becomes a bottleneck, tone it down accordingly.
 
3:52 PM
@santimirandarp I'm a physicist. Learn sets.
 
user7437554
@AndrasDeak thanks, I'll do it
 
@ParitoshSingh I did do it for one well, using python. I wrote a .html that has embedded data of the plot that I want. The file is too large. I tested my plot by writing only a small portion of the data.
 
wim
Finally got to use list_ass_slice
 
Slightly concerned about my upvote on that btw
I'm actually pretty happy to have my understaning reshaped here
 
@pyeR_biz yeah, i dont see any page being capable of handling 1.4m datapoints just thrown at it. That's the real bottleneck worth addressing.
 
3:58 PM
@wim list.as_lice, the return value of which would fail itertools.islice
 
gets better and better.
 
My friend Peter Wang at Anaconda showed me once about using that package to view a data set with zillions of points in it, and that this was a typical problem in data visualization (that is, you have some large number of values, and you want to see the big picture without necessarily plotting every value).
 
@wim I presumed that was a typo, but apparently not.
 
wim
So, catching up, Martijn undeleted blatant hnq dupe ?
 
We even have a relevant message of his sitting quite organically on the starboard
 
wim
4:13 PM
the three undelete voters are the top 3 answerers, hah
 
@PaulMcG That might be datashader package. My case is a little different I think. I don't need to plot all 1-2 M points at 100% granularity. But when I'm zooming to say 10,000 points it should be 100% granular and, have around 10-15 columns/channels/parameters loaded at a time, on the same x-axis. So effectively 10,000 *13 = 130, 000 points (in separate traces). Then, if need to see next 10,000 points from the data, it should load fast (hopefully :-) )
 
In any event, you might give the Anaconda suite a try (bokeh, etc.). I know they work with large data sets a lot, including heavy lifting on the visualization concepts
It's more than just the conda package manager
 
00:00 - 17:0017:00 - 00:00

« first day (3112 days earlier)      last day (1820 days later) »