« first day (2956 days earlier)      last day (1987 days later) » 

1:03 AM
@holdenweb Nice to hear from you again, please don't be a stranger
 
 
6 hours later…
6:46 AM
@jpp a) It shouldn't have been closed, it's distinct. Voted to reopen. b) The solution is not necessarily regex, but for this one, regex is the simplest.
@AndrasDeak Ok well better to light a candle than curse the darkness: what are your top-5/top-10 recurring dupes that keep getting asked? Let's try to make canonicals.
 
7:29 AM
@smci opened
 
8:02 AM
@MartijnPieters Could you help me with stackoverflow.com/q/53369629/562769 ? I vaguely remember I've seen a FSM solution to a RegEx question which used a library for it... but I can't find the library / the question+answer :-/
 
8:22 AM
@MartinThoma Why are you using lists instead of strings for quotechars and ignore_chars?
 
9:06 AM
@PaulMcG Been pretty busy lately, but I drop by when I can. Lots of good people here.
 
 
2 hours later…
10:53 AM
Holy cow, got both kids to nap in half an hour. First time for everything.
Time for the daily SO fix...
 
11:07 AM
cbg
 
11:34 AM
I think this merely a duplicate of list-chunking recipe, and no need for any numpy/scipy. Does that sound right?
 
11:49 AM
@smci The OP did suggest Numpy, though. And a good Numpy solution will be much faster.
 
@PM2Ring Yes, but in general not necessary. I think it's good to show both approaches and let them choose. However, does that mean we should not close as dupe? If so, I would edit 'using numpy' into the title to make it clear why not dupe.'
 
the dupe is fine, the code in one of the answers is a verbatim copy of one of the answers on the target
 
@smci I'm not a fan of Numpy answers when the question doesn't mention Numpy. But since the OP did suggest it, I think they deserve a Numpy answer. But don't change the title, just add the Numpy tag.
 
OP thinks that numpy is magic pixie dust. If they were using numpy anyway it would be evident from the code and tags. If they get a numpy answer they'll import numpy for this single step which will cost them 10x the runtime
for instance they should use numpy.loadtxt or .genfromtxt while they're at it
 
12:05 PM
Fair point. And it's odd that they only read one number from each row; maybe the other columns have strings... But anyway, they currently have a couple of bad Numpy answers, and IMHO it would be good to see a proper Numpy answer.
 
there are numpy answers on the dupe as well
though I guess what OP wants is numpy.split rather than numpy.array_split
 
I guess we could ask for more info about the CSV, and what else they're doing, in case Numpy is worth importing.
 
or just close it
 
Why not pad the data to a a whole multiple of 100, load into an array, reshape, then sum the rows?
@AndrasDeak Ok, I'll have a look at it.
 
s/answers/answer probably, and it's not exactly what OP wants
 
12:16 PM
@AndrasDeak Yeah thanks for articulating my reason to PM2Ring. If there's no general need for using numpy/scipy, then it's just adding dependencies. Similarly we get users wanting to use NLP and NLTK like magic pixie dust. Or people who insist on using PIL for any image-processing, even super-basic stuff. Or plotting toolkits. etc.
Massive overkill, most of that.
 
Ok, I just had a look at both suggested dupes. The chunking one is a bit overwhelming, and only solves half the problem, since it doesn't the averaging. The Numpy one is ok, I guess, but it's probably a bit confusing, especially if the OP doesn't have much Numpy experience. But I guess a comment pointing at the most relevant answer(s) can cover that.
Using Numpy to process <100 numbers is certainly overkill. For >1000 numbers, I think it's borderline. OTOH, it is only 11 chunks, so unless they have other things they can use Numpy on, it may not be faster to use Numpy.
I'm happy to hammer it, though, if someone else can write a comment guiding the OP to make best use of the dupe target.
The OP's accepted a non-Numpy answer now, so I guess it doesn't really matter.
 
12:47 PM
then again some people don't quite know how accepting works, so we'll never really know. I've seen OPs trying to accept multiple answers many times, maybe out of confusion when noticing the other answer got the tick removed (and if they don't notice it, then they usually don't come back to check it)
 
hey guys, do we get distributable package / archive for python installation for linux, which we can simply extract and use?
 
 
1 hour later…
2:06 PM
Define "extract". If the user is tech-savvy enough to run pip install, then you just need to provide them the package. If they only know how to open .zip files, then you're going to have trouble getting anything to work on their machine if it has dependencies
If it's the former, consult python-packaging.readthedocs.io/en/latest
 
2:24 PM
TIL: time.strftime has a one-argument form that just uses the local current time.
Converting Outlook time to string would have saved themselves a debugging session if this was not the case, since their code would have crashed immediately instead of producing mysterious output
 
Cbg
@Kevin did you fix your optimisation problem in the end?
 
No, I couldn't find an approach I found satisfactory
It was for the Google programming challenge, and the time allotted for it may have expired by now... Not sure.
 
What went wrong? Would it not converge?
 
I wrote down "todo: machine learning approach?" in a notepad++ tab, then forgot about it
 
I accidentally threw my Google challenge away :/ I saw the screen sinking backwards and my instinct was "crap crap crap" so I clicked "back" ASAP
Nah, no hope
 
2:34 PM
As is often the case with programming challenges, I suspect the intended answer does not involve machine learning heuristics. I was probably supposed to discover an obscure property of coprime pairs, perhaps buried in a Wikipedia article somewhere, that makes the problem go from O(N!) to O(log n)
 
It's not a machine learning problem
Mmm ,I'm not convinced but it would be interesting if such a solution exists
 
The problem was something like:
> To pass the time, the guards on this space station like to gamble their bananas. Whenever two guards gamble, the one with fewer bananas always wins, and they receive that many bananas from the loser. The guards repeat this until they have an equal number of bananas. Sometimes the guards _never_ reach equilibrium; this is fine with them.

Given: a text file containing 100 numbers ranging from 1 to 1,000,000,000, representing the number of bananas each guard possesses. You wish to pair up the guards so that as many of them as possible enter an infinite gambling loop. How many guards can you
 
By the way, I tried to login to accept another challenge to do over Thanksgiving break and got the message: "Foo Bar has been closed for extended maintenance." I'm pretty sure you broke it Kevin.
 
Maybe we're talking about different problems here? Originally it was 100 guests to minimise tables whilst ensuring no enemies sit together. I'm not sure that problem you listed is on the same vein?
@W.Dodge How did you get that? Do you have to rely on the "you're speaking our language" result from searching or can you access the same challenge another way?
 
2:56 PM
It comes up from specific queries that relate to things like using lambdas and list comps in python. If you make it to level three you get an invite to share with someone. So those are the two ways that I know about. I think it's sort of a talent acquisition dragnet. It now appears to be closed.
I had been doing working the problems for an extended period of time only taking challenges when I had time to actually work them. I learned a ton. The last one I had to solve by finding the coefficient of the nth item in the Maclurian series of a generator function I found for an integer series on OEIS. I had no idea what any of that meant two weeks ago.
 
Ah. I only had it the once and closed it
 
@W.Dodge Maclaurin?
 
Mine was a search around polymorphism but, if you go back, there's no way to go forwards and get it back :/ it's a one-shot thing
 
A Taylor series centered at zero?
 
IMO they should have made it look less virus-y. By the time I went back, the screen froze to show me what I'd just thrown away lol
 
3:03 PM
yeah, Maclaurin
 
Oh spelling... yes Maclaurin
 
Please get rid: stackoverflow.com/questions/53377231/… close or delv I guess, I see it going down hill
 
What can we close that as? Probably needs a conversion to int. Dupe?
 
I'd just close. I'm not interested in interacting with someone who starts with that title
 
that's not how SO works
found a dupe
 
3:10 PM
Allow the content to persist then. I can't make a judgement on a close reason for someone saying they're too drunk to solve an issue
There are some things that surely can't be covered by the SO framework
 
I'm not sure what you're saying. If the drunken rambling bothers you you can remove that. Otherwise the question is whether the question can be answered
They have a debugging problem. Whether they can't solve it because they are drunk, or lazy, or tired, or stupid, or just missing something, is immaterial
though I guess it's technically missing example inputs so it was no MCVE
 
Fair enough. I voted too broad. You know I normally give a reason but I didn't feel I should dictate a reason to others in this case <shrug>
 
it's never about dictating, it's about accountability and making it easier to argue on either side of a request
 
I highly doubt "due to my drunkeness" will come up in searches
 
Again, that's just fluff. Same as "I'm new to python" and "I hope someone can help me"
it doesn't make a question more or less appropriate
 
3:17 PM
I see what you're saying, though
 
3:49 PM
morning cabbage
 
oh luckily it's not a morning anymore
that would be a nightmare
 
@roganjosh It's the same problem. Basically, I was trying to find patterns that could determine which number pairs have finite gambling sequences. For example, (1, 2^N - 1) for any positive integer N always terminates in N iterations.
I also observed that a sequence starting with (a,b) will behave identically to a sequence starting with (ak, bk) for any positive integer k. So I could investigate the behavior of values of a and b that are coprime, and WLOG those observations apply to non-coprime pairs too
The theory I was working on was that there were a finite number of categories, that each integer belonged to. If you had a gambling sequence, you could replace one number with a different number in the same category, while preserving some property of that sequence, such as "this sequence terminates". For example, let's say category Q was "powers of two minus one". We know (15, 1) terminates, and 15 and 31 are both in category Q, therefore (31, 1) also terminates.
If this sounds half-baked, it's because it is.
I wanted to use the enemy-table-assigner program to identify other categories.I was going to brute-force all the gambling sequences from (1,1) to (100,100), and get a boolean for each one indicating which sequences terminate. Then I'd feed that data into the table assigner. My hope was that each table would be equivalent to the categories I was looking for. So there'd be one table with (3,7,15,31...), and hopefully the other tables would boil down to arithmetic sequences or simple patterns.
 
4:12 PM
I have a large dataset (stored by ID in a dictionary) of which about 80% is not interesting, and 20% is, and I can only tell which of the two by plotting the data. What I'm looking for is some degree of interactivity (ipywidgets, holoviews perhaps) that would allow me to plot the data and then (based on my decision) add its ID to an 'accept' or a 'reject' list. Would anyone know of a way of doing this? Using ipywidgets' interactive function I achieve the plotting by ID part, but I would not know how to store the ID into the accept or reject list. Maybe something with True or False? The form
 
@user129412 If you can only tell if data is interesting by plotting it, then how on Earth can a computer make that determination? You need to define "interesting" in terms of something that can be computed
 
they want to be able to click on points to select them
(I think)
 
@roganjosh No no, my wording is off then. I will be making the determination. I just want my interactive plot to give me the option to store what my decision was. Whether to accept (add ID to accept list) or reject (add ID to reject list)
 
There's too much back-end logic I think. You can capture the point but what gets recorded about that point to give the code any meaningful feedback?
 
I plot the data for a specific ID, look at it, and decide. Then move on to the next ID.
 
4:23 PM
It's possible that I'm well-off but I still think you need to determine some metrics up-front that can be stored against your clicks
 
Maybe that visual distinction could be boiled down to a numeric relationship that can be used to filter the data without plotting. What is it that sets an acceptable plot apart from the rest?
 
In other words, you could capture the x, y coords of your clicks... And they correlate to nothing. You need some metric to compare against, surely?
 
I see your point. However, I'm not so much interested in coordinates of clicks and such. Its more of an old school method of pre-selecting the data that I'll run more sophisticated methods on (as @W.Dodge suggests) because part of the data will be just noise
 
Maybe the gradient of the curve on either side? You might be able to take a linear regression on either side of the click location?
 
Like they used to do in astronomy, they take thousands upon thousands of images of the sky and then click through them to look for ones that contain interesting features. The ones they do they store for further analysis
 
4:27 PM
Yeah, but the click event, alone, would not help them
 
In principle one could use machine learning techniques to do that for you, I fuly agree
I don't want to click on anything though
 
I'm not talking about machine learning
I'm saying that you need some metric to compare events by
 
I just want to plot the data of ID 0, look at the plots, and decide if its worth investigating in a more detailed way at a later stage. That decision I make myself. Then I want to store that decision, by either putting the ID in the accept list, or in the reject list. I then want to move on to ID 1 and continue
 
You're expecting magic to fix the void in between your two ideas
 
Well, the magic that is the filtering that goes into human vision
 
4:30 PM
You have to come up with some cost function that fits your criteria
 
I'm not intentionally being obtuse, I apologise if I come across like that
 
Nor I
But you're missing a step unless I'm really misunderstanding
 
This kind of data grouping reminds me of some of the multivariate analysis done for semiconductor processing. Plotting data points gave regions of goodness vs. ungoodness. Using Principal Components Analysis and Hotelling T² methods. I delved into PCA at one point, but never got into the Hotelling.
 
Yes, I see what you mean. That's a step more complicated than what I'm trying to achieve though, as at this stage the dataset is not sufficiently large to warrant me developing the skills/code to work with such methods
 
So I have a 4.5GB json text file. I want to add '[' to the beginning and ']' to the end because it's an array of json objects. I know I can easily append to the end, but how to prepend? I get memory error everytime I try to load it.
 
4:34 PM
My plan is that I basically plot the data, and this will either look like a bunch of noise, or a function that has been sampled with the wrong window, or something like that, which means I will reject the data. Or, it looks like a nice representation of the function I am trying to sample, and I want to perform further analysis on that. In that case it goes onto the accept pile
 
so your data is already separated into chunks, and you want to say "Yes/No" for each chunk?
 
But you need to identify what "noise" is
 
You want a widget that stores either "accept" or "reject" for each ID as you sift through the plots? And you don't care about the manual aspect of looking and each plot and making a decision? You just need to have your widget update a list. That's fairly straight-froward. Totally doable...
 
And the decision of which pile it goes onto is made fully manually, by me. I'm just looking for a degree of interactivity in python (ipywidgets, holoviews) that will allow me to store whether to accept or reject that ID
 
You surely have to have a metric to weigh against
 
4:35 PM
Yes, @W.Dodge and @MisterMiyagi
 
can you plot the data of each ID separately?
 
I can
 
@ex080 What are you going to accomplish with this edited file that you can't do with the current one?
 
I use something like
f = lambda idx: plot_dataset_complex_plane(data[IDs[idx]])
interactive_plot = interactive(f, idx=(0, len(IDs)-1)));
interactive_plot

where plot_dataset_complex_plane is just some subplotting and such that knows about the ID
 
And are there commas separating each JSON object?
 
4:37 PM
@PaulMcG I need to extract particular key value pairs from the JSON, but ijson throws an error because the data is not enclosed in []
 
Are we crossing wires here?
 
@roganjosh I see what you mean, and I will probably try and formalize that in the future, but the data comes in a ton of forms. Its experimental data from measurements that have different input parameters, variables swept over different ranges, and so on. Only a subset of them produces output that is worth looking at. And defining what is worth looking at is hard. So to start I will pick out those that are definitely worth looking at. But again, that criteria is tough to formalize
 
I think I might be better off using cat or tail
 
@user129412 don't you need something like stackoverflow.com/questions/22052532/…, and this linked therein?
 
@user129412 What kind of data is it? I reckon you could get some really simple heuristic rules to narrow the search down
 
4:41 PM
Is each JSON object on its own line? If so, look at this question: stackoverflow.com/questions/37200302/…
 
Cheap rules to just narrow it down - back of the envelope stuff to get you started
 
@roganjosh Well, that's something I should get into with the 'filtered' list in any case so perhaps it is a good point to start already.
But I need to think about how to make the problem as abstract as possible for that.
 
Data is data. You have an x and a y. It's already abstract, no?
 
@PaulMcG thanks.
 
It's only people that add meaning to graphs. At the end of the day, it's just an array of numbers
 
4:45 PM
You are definitely right. I'm just trying to think of a way to describe the different values the data might take appropriately.
Essentially my device is a function that has many inputs. All but two of them are fixed, and then I sweep those two, x and y. That returns z. z is complex. Only for specific values of x will the device give a response, for others it will produce noise. Those values depend on y, but also on the other input parameters. Simple enough so far.
But I don't really know the range of x beforehand, and the y-axis is not that stable, meaning that sometimes you get jumps. Based on your y=y1 trace you'd think you would now be measuring y = y2, but instead youre measuring at y = y19. But that's a detail already I think.
But essentially some of your datasets will have mostly noise, some will only partially capture the response of your device (improper x range), some will not sample all of the y values you need to sample due to jumps
So you need a way of quantifying whether the data you are looking at does not suffer from any of these issues
Moreover it is in the complex plane, and the 'angle' is different for each dataset, so your analysis needs to take that into account as well.
But I think that before I start bothering people with details such as these I need to sit down and think about how to make this more abstract/quantifiable.
 
I think you're over-thinking it, but we can't see the data. It seems to me that your "intestesting" points can be decomposed into a set of metrics than you can easily measure
 
It probably can. Someone who has more knowledge of these things could definitely do so.
My group doesn't have an expert on these subjects though, and I think it might cost me quite some time to figure that out. But perhaps it is what should be done.
For now my method was just using my vision and clicking accept or reject, essentially. I just haven't figured out how exactly (in python)
 
You don't need fancy maths to make the rules
 
That is true, but I at the very least need to figure out a way to rotate all of the data into a specific quadrature to apply the rules, or make rules that take the entire complex plane into account
But the rules I could come up with would mostly say something like, is there enough variation in z. It wouldn't tell me about if a function is only partially sampled.
But okay, something for me to think about.
 
5:01 PM
Yeah, I think we'll struggle to get anything concrete here,but I know there are better mathematicians in this room watching this, and they haven't piped up to tell me I'm wrong :P
 
wim
6 upvotes for copying the answer in the dupe, rather than suggesting OP figure it out themselves
 
Well, that's also because my descriptions are quite vague. Because I can't really copy paste my dataset, nor do I want to bother you with that
 
wim
@user129412 sorry to barge in, this was unrelated to what you were discussing
 
And for now, W.Dodge said it correctly, I want a widget that stores either "accept" or "reject" for each ID as I sift through the plots. And I don't care about the manual aspect of looking and each plot and making a decision. I just need to have the widget update a list.
No problem at all
 
@wim I think you could probably add to the convo tbh
 
wim
5:08 PM
It was just me being a grump because I think it's unfortunate if SO will become like free tutorial site for people that are unable or otherwise unwilling to help themselves. My vision was more for the site to be a helpful database of content, but still aimed to professional programmers i.e. all the important pieces are there but there's "some assembly required" (not 8086 assembly)
 
expecting that people think for themselves is not welcoming
 
requiring 8086 assembly would be fascinating, though
I tend to be a little hand-holdy when I comment and/or answer basic questions. I don't often write much code, though. Only just enough to nudge the OP in the right direction.
 
wim
@AndrasDeak it's not required to be welcoming to people that want your time and effort without being willing to commit some of their own
 
I feel your grump. Many questions seem to be just about limping from one error to the next, lately.
 
@wim ah, but most people on SO (especially those in charge) will disagree with you on that
 
5:18 PM
Never thought processing large text files would be this annoying
 
@AndrasDeak by "most people", do you mean those who are asking questions or those who frequently answer them?
 
wim
Really? I think the debate was mostly about "we should not be rude to those people", which I agree with
 
those who moderate
@wim it's even called the "welcome wagon"
 
@wim I agree. This is my take away from the recent changes, including the "new user" icon. If someone isn't willing to put the time into solving the problem themselves, being rude and condescending doesn't solve the problem and only builds resentment. On the other hand, ignoring them and moving along to help someone who is willing to put in that time is much more rewarding.
 
@user129412 I think interact is not what you are looking for. You need a simple "display this data and return on key-stroke".
 
wim
5:20 PM
I don't recall if the "welcome wagon" was a facetious name coined by detractors, or a legit thing coming from the ... protractors (?) :)
 
That could very well be! Holoviews might be more appropriate, although I'm not too familiar with it.
 
more like input, but showing a picture instead of text
 
@wim Tim Post mentions the welcome wagon initiatives
 
@user129412 You may want to look at raw matplotlib events, such as here: matplotlib.org/users/event_handling.html
 
I will have a look :) Thank you
 
wim
5:24 PM
@Code-Apprentice starred because more people need to see this excellent advice.
 
@Code-Apprentice the problem with that is that the site gets silently moderated. I don't believe that all people dump code of their first question to flaunt the rules
 
wim
@roganjosh it is still possible to help such users without being rude and without regurgitating content that already exists on site
 
You don't have to be rude in a response but, when the advice can be boiled down to "You did a bad".. well, people will lash out
 
wim
the trick here would be to incentivize the helping such users somehow (there's currently only really incentive for posting answer)
 
I ended up using cat and echo to solve my problem
but it still takes forever and uses tons of mem
 
5:27 PM
I see mostly the problem that even resolving such things civilised is a net loss. I've had quite some problem finding dupes because of all the junk questions piling up...
 
wim
check out Aran-Fey user script, if you haven't already seen it
it works pretty well for keeping a bunch of common answers within easy reach (helpful when the search turns up too much junk or the topic lacks obvious search keywords)
 
5:40 PM
@wim That's an XY problem. The Python 3 solution is to use float.fromhex
 
At times like these, I wish comments could be upvoted as answers.
 
wim
@PM2Ring I'm not sure float.fromhex works with their format
LMAO, there is a subtle burn in the docs:
> The form '!' is available for those poor souls who claim they can’t remember whether network byte order is big-endian or little-endian.
! is just alias for >
 
wim
6:04 PM
$100 says that was Guido sometime in the 90's.
 
@wim Oh, ok. I didn't look closely at their format.
 
The git history attributes it to Guido in 1996.
 
I used to use struct quite often when I was using Python 2, so I'm quite familiar with its docs, and I chuckled the 1st time I read that note about !. I don't often need struct in Python 3.
 
there's also an example that uses the _make method of a namedtuple
 
Today I am annoyed by chromatic aberration. Rainbows are nice, but on balance I think I'd prefer to live in a universe where light got refracted the same amount irrespective of its wavelength.
 
6:21 PM
but they are pretty
Photography?
We also wouldn't have cool stuff like achromatic doublets
 
My annoyance is 20% because it makes the math harder on a thought experiment I was working on; and 80% because it's becoming trendy in webcomics to apply a chromatic aberration filter, which gives me a headache
 
it is?
Do you have an example of such aberrations in webcomics?
 
No.
 
thought experiments are great because you can just modify the setup to only allow monochromatic light ;)
 
I didn't bookmark any of them, because I didn't like them
 
6:27 PM
ah
 
@roganjosh I assume that the OP is just in a hurry and unwilling to read the rules and other suggestions from help. I don't assume that they do this from willful belligerence. My approach is to gently remind them to read the help section. If they choose not to, then I will just move on.
 
wim
@AndrasDeak hmm, was *splat unpack unavailable at the time?
 
The one I remember specifically was kind of a Shen Comix ripoff, with an autobiographical/moralizing tone, and abstract concepts represented as physical objects. Ex. "my progress as an artist" represented as a set of stairs, and "self-doubt" is a ripped dude standing at the top ready to knock you back down to the first step
 
wim
opinions on a best canon for "textadecimal" (hex str) to bytes? my current candidates one, two and three
 
6:42 PM
@wim I didn't look closely at all the answers, but my favourite is #3.
 
@wim is "textadecimal" official terminology?
 
I doubt it
 
7:04 PM
Any idea why i cant get a progress bar this way using tqdm? repl.it/repls/AwfulMemorableChemistry
import tqdm
import os
import time

with tqdm.tqdm(os.path.getsize('./a.txt')) as pbar:
  with open('./a.txt', 'rb') as f:
    for l in f:
      #do something with f
      time.sleep(1)
      pbar.update(len(l))
 
len(l) is the length of the current line, so I expect that updating the progress bar with that value would make it jump around rather chaotically.
 
cbg
 
cbg
hmm then should I use tell
 
I'm assuming that pbar.update(x) just sets the bar to x% complete. I don't know if that's actually what it's doing.
 
Yeap
 
7:08 PM
Hmm, I think I'm wrong
 
I'm having a unittest.mock issue -- my patch isn't patching where I think it should. I have a package structure like pkg.a -> defines foo; pkg.b -> from pkg.a import foo. I'm putting b.some_function under test and doing from pkg import b in my unittest, then patching b.foo. That seems to be write (per Where to patch) but it isn't working.
Am I just having a case of the Mondays?
bleh apparently I have to patch pkg.b.foo. I'm not clear on why.
 
github.com/tqdm/tqdm#manual pretty clearly shows an example that just calls update(10) ten times in a row, so it must be additive in some way
 
@Kevin u r right
it adds to the prev value
 
In that case, I don't see anything obviously wrong with your code.
 
I'm tempted to post a question, but I think it's something obvious that I just cant see rn and dont want to get roasted by SO
 
7:26 PM
when I've seen tqdm being used, it was always a loop over tqdm(iterable)
 
@ex080: What do you mean by "cant get a progress bar"?
 
this
 
That looks like a progress bar to me.
 
Im gonna try to count the number of lines then do it that way instead of by bytes
@user2357112 I know lol, that doesn't show up
The code I posted above gives changing values and no bar
 
with open('./a.txt', 'rb') as f:
    for l in tqdm.tqdm(f):
        #do something with f
        time.sleep(1)
does that ^ work?
 
7:33 PM
@ex080: It shows up fine for me.
 
or perhaps that would consume the iterable or not be able to guess its length
 
The code I posted works for you?
Can u post a screenshot?
@AndrasDeak nope same problem with yours as with mine
 
I see
 
@oh I had updated the sketch when u ran it
:P
repl.it/repls/HastyJovialVendor Here is the original
 
7:39 PM
@ex080: You're passing the result of getsize as an iterable, not as the total argument. You need tqdm(total=os.path.getsize(...)).
 
Try changing tqdm.tqdm(os.path.getsize('./a.txt')) to tqdm.tqdm(total=os.path.getsize('./a.txt'))
Sigh beaten
 
yup, what they all said
it needs an iterable to decorate and a length hint
 
nice
my bad guys
new it was something totally simple
 
Curse you, repl.it, for taking a solid fifteen seconds to load the editor, delaying my fast-gunning by a critical amount
4
 
much thanks
to all
I am happy
now I can watch it go
and see the progress
 
8:17 PM
almost-end-of-the-day-realizing-i-didnt-cbg-Cbg \o
 
wim
8:38 PM
Anyone see anything obviously wrong here? downvoted but not sure why..
did I misunderstand the question or...?
 
I suspect the downvoter is the one who thinks that the list makes a copy.
 
````
pi@raspberrypi:~/Documents/rpi-rgb-led-matrix-master/examples-api-use $ sudo ./text-example --led-no-hardware-pulse --led-cols=64 -f ../fonts/4x6.bdf -y 10
Enter lines. Full screen or empty line clears screen.
Supports UTF-8. CTRL-D for exit.
````
 
perhaps someone thought it was not a question worth answering? It is basically a variant of "reusing a name in a function does not change the argument passed in".
 
Yup someone thought the whole thing was useless. You even mentioned that you think it’s a dupe.
 
hello I try to automate this command from terminal into python script.
When I write the command, terminal ask me to enter my text and press enter.
When I do the same command using python os.system(cmd1), python doesnt go to the next step to take the text
How can I press enter using python?
 
8:51 PM
add a newline to the text
"my text\n"
wait a second, it never asks for your input in the first place?
 
I tried \n and input() as well but python script stops in this line
cmd2= "sudo ./text-example --led-no-hardware-pulse --led-cols=64 -f ../fonts/4x6.bdf -y 10"
It asks me to enter my input just in Terminal
 
I'm interpreting this question as "when I run a command with os.system, the command is supposed to prompt me to enter something. How do I make Python automatically enter data for me?". I don't think system is capable of doing that. You need Popen.communicate.
 
os.system should directly connect to stdin, stdout and stderr - so basically defer to the terminal/user
do you see the prompt on your terminal after Python calls the script?
 
No I only see my text on RGB LED MATRIX which is connected to Raspberry pi. So There is no need for Terminal When I use Python
 
9:10 PM
but do you have a terminal connected?
 
After trying for ten minutes to make communicate work, I've decided that's not what you need. But I'm pretty sure the Popen object still has a way to automate inputs.
Smart money's on p.stdin.write() but the first three ways I tried it just made the program hang, so. Work in progress.
 
What do you mean please MisterMiyagi ?
 
C:\Users\Kevin\Desktop>type poll.py
track = input("Hello. Enter your name: ")
C:\Users\Kevin\Desktop>type test.py

from subprocess import Popen, PIPE
p = Popen(["python", "poll.py"], stdin=PIPE, stdout=PIPE)
print(p.stdout.readline())

C:\Users\Kevin\Desktop>test.py
[hangs forever, producing no output and ignoring KeyboardInterrupt]
I would expect this to print "Hello. Enter your name: ", and either terminate or maybe throw an EOFError... Is that a strange expectation? Is this a Windows thing?
 
Apr 19 at 20:12, by vaultah
Dec 5 '17 at 8:50, by Arne Recknagel
@AhmyOhlin If you want to format code in a post, press ctrl+k, or see the code formatting guide for more info: https://chat.stackoverflow.com/transcript/message/40291589#40291589
 
Well, input prints the prompt without a newline at the end, so...
 
9:19 PM
hello friends
I am back
 
that stdout.readline() will block indefinitely
 
cbg
 
I have Raspbian debian version for Raspberry pi and it has Terminal "Shell". But I don't want to use terminal because i want to automate all inputdata using python to display it on RGB LED MATRIX
 
@Aran-Fey I thought it might be something like that. I assume read() does the same unless I provide the exact expected length of the prompt. So how should I get the text of an input prompt if I don't know how long it is ahead of time?
 
@AhmyOhlin well, short answer is "use subprocess instead of os.system"
 
9:20 PM
How are you controlling the RGB LED matrix?
 
@Kevin ¯\_(ツ)_/¯
honestly have no clue
 
There are a couple stack overflow posts with wonderful solutions, that work if and only if the program you're calling has exactly one prompt. Because they all use communicate, which closes the pipe after it sends one input.
 
Yes I just saw kevin's answer. I have to read more about subprocess because I have never used it before.
 
@Kevin you don't see it print the "Hello. Enter your name:" because you also capture its stdout.
 
Hmm, maybe if I do something with seek?
 
9:23 PM
@ex080 I'm using raspberry pi for controlling
 
@MisterMiyagi I'm capturing it, sure, and then I'm printing it. Why isn't it printing?
 
you capture up to a newline, of which there is none
readline will not return before it hits \n
I think you need threads in this kind of situation
if you already know the answer before you know the question, you might be able to send to stdin before reading from stdout
 
Lot of answers on stackoverflow.com/questions/375427/… and they're all pretty untidy
 
Andras Deak this message was in dec 2017 hahaha. I forgot the code formating
 
notebooks r nice
 
9:36 PM
it's OK, I'm here to remind you
though I've yet to see a markdown flavour where four backticks works ;)
 
On my machine, popen.stdout is a BufferedReader, so I can use peek to see if there's anything waiting to be read, and I can then pass the length of that data to read to empty the buffer again. I have no idea if this is OS-specific behavior.
 
You can write to stdin in advance before reading the question:

from subprocess import Popen, PIPE
p = Popen(["python", "poll.py"], stdin=PIPE, stdout=PIPE)
p.stdin.write(b'"Foo!"\n')
p.stdin.flush()
print(p.stdout.readline())
 
Just spent ten minutes trying to get writing to stdin working. Could have sworn "send the input plus a newline, then flush" was the second thing I tried, but ¯\_(ツ)_/¯
There. A minimal prototype for automating a program that contains more than one interactive prompt.
I fully expect this to fail in every environment but my own, and in fact it might fail in my environment tomorrow. I am completely baffled that there is no idiomatic standard way to do this. I thought Python was supposed to be a scripting language?
 
this isn't so much a Python problem as it is a communication problem
 
inb4 "you should only be writing scripts that interface with good well-behaved programs; in other words, programs that can be controlled entirely with command-line arguments. No sane program will require prompts with no alternative"
 
9:47 PM
"do the right thing with this arbitrary stream of bytes" is awfully hard to handle correctly :P
 
Evidently sending more than one input is a valid use-case, or else the Linux utility yes would not exist
 
hello beautiful people
Im wondering if anyone here can answer a question question about panda
 
@Kevin the problem is mostly sending and reading at the same time. yes only does the sending, without regard to reading.
 
@ZacharyLucas they are surprisingly intelligent creatures, albeit lazy youtube.com/watch?v=pFI6Vyj6fU8
 
9:54 PM
side note: are you aware that your avatar seems broken?
 
how would one go about dropping a column by index? I have over 100 columns in this csv and they are not named. I just want to remove the last column in the csv
 
What are the names of the columns? Empty stings?
does that even work?
 
I am unsure of the names as I need this to work with multiple cvs files which have a different number of columns with different names
 
I see, so it does have names, you just don't know/care about them
df.columns should contain the list of all the names, I guess you could slice that or pick from that based on the index you have and drop/keep the rest
 
yes
4 of my csv files have no names listed, the 5th on has every column named on the first row
right now Im trying to drop the last index but using len(file)-1
 
10:00 PM
len(file)-1 seems like an index-like thing, i.e. it would drop rows
which one of us is confused?
 
but since I must use names with panda, it wont work. ya
I need to drop columns not row.
Sorry if I am confusing
 
also side note: it's pandas. Plural. It's right there on your import line :P
I can't even create a dummy dataframe with empty names
when you say "have no names listed", what does that mean in terms of df.columns?
 
let me find out
Interesting, My first row in the cvs files have data, not names, It is using the data as the names
Looks like I will have to specify my column names then
thanks for the help
 
no problem
 
 
2 hours later…
11:39 PM
lol how do i (format)[a_link] in comments?
 
11:54 PM
@JoranBeasley: Brackets around the link text, parentheses around the URL.
 
wim
square brackets first, parens second:

[XXX videoz](https://www.disney.com)
 
Also no space between the end of the link text and the closing bracket.
 
wim
and they don't work in multi-line messages. I wonder if link formatting could or should be added to the guide. It's not really code but it is useful.
 
It's in the regular faq or help
 

« first day (2956 days earlier)      last day (1987 days later) »