« first day (3293 days earlier)      last day (1638 days later) » 

3:14 AM
Why does PEP (or maybe it was Pyflakes, don't remember) consider it poor form to not use EVERYTHING from a star import (which doesn't take into account abstract classes and necessary globals)?
 
 
3 hours later…
6:37 AM
@JohnnyApplesauce star imports in general are usually considered poor form.
 
7:03 AM
Should Stack Overflow remove the regex tag? Finally a meta title I can get behind and it gets closed :P
I guess I can't dodge learning it forever :/
 
For what it's worth, you don't really need to "learn" it per se, just need to have regex101 open whenever you use regexes!
 
Cargo culting regex feels like I might accidentally start Skynet, though :/ regex101 could just be a cat giving feedback
<?!#<meow!?# <-- red alert
 
Haha
The good thing is, regex is not really an "open ended" problem per se. (Unless you use it like one, in which case you're probably doing something wrong anyways). So, it's not really possible to make it as bad as say, cargo culting a software. You can always have a vague description of "i want to match SOMETHING after these fixed symbols, and so on". The trick then is simply finding the vocabulary to represent that vague statement as a regex
This way, you don't have to feel burdened with learning the entire vocab of this cat language in one go
 
7:19 AM
Be under no illusion, I'm totally exaggerating :) The only place I could really see me being compelled to use it was in addresses for deliveries, and I'm not working with those atm. Tackling the cat language will require some solid project requirement I think
 
I think one of my most "complex" regexes was one for a custom tokenizer. And frankly it wasn't too bad. Having said that, my "first" regex was when i didn't know what regex101 was. That was..something else.
I'd restart a server and write a div with the regex output, just because i didn't actually know that there were javascript consoles on the browser, and someone had told me that regexes are amazing. (oh, and i didn't actually know what servers were too to be fair). It must have taken me 4 hours just to write what was effectively just a regex trying to find all special characters. :P
 
catastrophic backtracking?
@ParitoshSingh Those were the days, eh? :) I remember similar situations, just without regex
 
hah, it was definitely a catastrophe in any case. I will say though, i was extremely proud of getting through it
Hehe aye, good times. :)
 
Sam
Morning all, I have a list of dictionaries which I want to iterate over and remove a key value pair (of a certain key name).. I'm able to do this with a loop with uses del but was wondering if there was a cleaner more concise way of doing so? Made a minimal example here: repl.it/repls/ConsiderateUnluckyCones Thanks in advance :D
 
Other than getting rid of tmp = item, I don't think so
Why exactly are you using tmp = item anyway? No copy is taking place here
 
7:34 AM
cbg
 
cbg
 
@Sam have you considered what'll happen if the key doesn't exist?
 
Sam
@JonClements For this example i can infer that the key will always exist :)
@roganjosh good point
 
given the fact it's hard-coded yeah... never hurts to get into best practices about the real-world cases though :p
 
Sam
I take your point :)
 
7:47 AM
@Kevin Are we supposed to be editing the agenda ourselves? I just looked at this post again and there are no agenda points but I did suggest one a few days back about gitlab. I'm not sure whether that point is not suitable for the agenda or you haven't got round to editing the post to include it.
 
anyone knows how in grafana variables are used ,my variable is getting replaced by ^varname$
 
@roganjosh just edit it yourself :)
 
@JonClements oki doki :) While I'm at it, do you know if any other points were raised in chat? I've been offline almost all of the weekend
 
q:"SELECT "avg" FROM "mymeasur" WHERE ("param" =~ /^PROCESS_PACKET$/ ) AND time >= 1570561039724ms and time <= 1571350713419ms GROUP BY "/^hostIP$/""
 
Where does that string come from?
 
7:55 AM
in this my "hostIP" is the value of the variable that should just be hostIP and not /^hostIP$/
 
I don't know Grafana so I'm not sure what creates the string. Is it the library or you?
Well, part of the change is due to escaping I think. I don't know where ^ or $ comes from
 
@roganjosh I wouldn't - only starting to get back active here now :)
 
Having more puppy powah is definitely a boon for the room going forward :)
 
8:25 AM
give it a week (or less) and it'll be: "why.... why did he come back - dammit!" :p
 
Agenda suggestions in chat without RO pings don't really exist
 
Mine came with a ping :P
Oct 18 at 16:05, by roganjosh
@Kevin so my suggestion would be basically the 4 messages above. "Do we want to have a free gitlab instance where regulars can migrate/preserve good question + answer discussions that happen in chat, which can be given a named-and-searchable link on sopython?"
 
Ah :P
From his reply he clearly didn't take it as a formal request
 
Well, I'm not sure how much more direct I could be about it as an agenda topic, but then that's why I wondered whether it had been dismissed by not going on the agenda
 
I guess it's cleaner to add it yourself anyway
 
8:29 AM
The agenda gets reviewed before the meeting anyway... if it's blatantly not suitable, it'll get removed, but at worst if it's not a concrete discussion point, it can be moved to AOB or something
 
I do think we're throwing away a lot of good info in the room. Hosting gitlab shouldn't be too hard but we'd have to work out access rights etc
The requirement for posting big code blocks on off-site hosts pretty much guarantees their death
 
would we need to go to the extreme of a separate gitlab etc instance? sopython already has an import from chat feature which is currently used for keeping an archive of previous meetings (see: sopython.com/transcript for instance)
 
@roganjosh we could request github gists or gitlab snippets
 
mind you - that only works on start/end dates...
 
Ok, well that's cool if it can just drag out the lines :) We have to account for links that might expire. Plus, I think it takes some curation, not just a drag 'n' drop of what might be a drawn-out convo
 
8:34 AM
There'd be a lot of complaining
 
In fact... "If your code is longer than about 12 lines, use an external paste tool such as dpaste.com." - why do we not just recommend gist there instead of dpaste?
 
Morning guys, stupid question, how am I getting the text inside <strong> tag ?! [<span class="name"><strong>Nike Women's Spring Impact Sports Bra Black XS</strong></span>, <span class="name"><strong>Nike Women's Spring Impact Sports Bra Grey XL</strong></span>]
expecting : Nike Women's Spring Impact Sports Bra Black XS
Nike Women's Spring Impact Sports Bra Grey XL
 
@JonClements I never understood that :P
@Andie31 what library are you using?
 
I think that kills 1 part of the problem, but there's still the searchability aspect. I hate chat search and I'd have to remember some specific keyword that would take me to the specific discussion that could have happened 18 months ago
 
@Andie how did you get that list to start with?
 
8:38 AM
@AndrasDeak beautifulsoup
 
I'm also trying to be mindful of the implementation effort
 
@roganjosh okay... add it to the agenda with a summary of your reasoning - it (to me) sounds like something healthy to debate
 
@roganjosh to be honest I'd think long-term valuable discussions are rare
But yeah, let's continue at the GM
 
@JonClements I did edit it in. If it's not clear from what I added, I'll elaborate
 
cool
@Andie31 given your list spans, then: products = [el.select_one('span.name strong').text for el in spans] but looks like you should probably be doing that on the soup object itself with a selector rather than on a list of spans you've selected somehow...
 
9:13 AM
> I've looked at almost all of the regex solutions on StackOverflow submitted within the last 10 years.
Bold OP
 
Their brain is just some kinda psychedelia TV show now :P
 
is the next sentence... TO͇̹̺ͅƝ̴ȳ̳ TH̘Ë͖́̉ ͠P̯͍̭O̚​N̐Y̡ H̸̡̪̯ͨ͊̽̅̾̎Ȩ̬̩̾͛ͪ̈́̀́͘ ̶̧̨̱̹̭̯ͧ̾ͬC̷̙̲̝͖ͭ̏ͥͮ͟Oͮ͏̮̪̝͍M̲̖͊̒ͪͩͬ̚̚͜Ȇ̴̟̟͙̞ͩ͌͝S̨̥̫͎̭ͯ̿̔̀ͅ ?
 
9:30 AM
I'm actually curious now whether <?!#<meow!?# does something
 
it makes my ears prick up, look around the room and get ready for the chase? :p
 
:P
I think I screwed it with #. It looks like it's only used for "number" in English, like "Capturing group #1"
 
@roganjosh It's valid. regex101.com/r/XszqI6/2
 
@ParitoshSingh ... amazing!
"giving back as needed (greedy)". Yep, that's a cat alright
 
haha yep :P
 
10:01 AM
c.b.g.
 
'gbc'[::-1]
 
"".join(['c','b','g'])
 
print(''.join(chr(i) for i in np.array([99, -1, 5]).cumsum()))
 
Good one
 
"c.b.g." Sounds world-weary. I thought we were on Themes and Variations of Cabbage but Paritosh steals the show :)
 
10:19 AM
zlib.decompress(b'x\x9c\xf3THL/JM\x05\x00\x08\xcc\x02n') :p
 
It seems that like monotheistic religions the holy cabbage has many forms of address :p
 
<bows> "Oh, holy b'x\x9c\xf3THL/JM\x05\x00\x08\xcc\x02n', please rid me of tracebacks" </bows>
 
''.join(bool.__doc__[i] for i in b'\xad\x8c\xd0\x00\x9c)8')
 
@roganjosh My mouse is hovering over the star button :)
 
Someone did star it and it looked gross on the star board, then it got removed, so it's probably better left as it is :P
 
10:26 AM
oh that's one sneaky bytes
 
Haha i love it
 
I wonder what that says in older python versions... I only tested it in 3.7
 
TypeError: string indices must be integers, not str
 
that's a bit too old :D
 
it works in 3.6 and 3.8
 
10:29 AM
Imagine if someone constructed an entire docstring like that
...okay don't. i just tried imagining it, and i have a headache now.
 
float.__doc__ == ''.join(int.__doc__[i] for i in b'0LO3q?\x96\x86\xcfPt\xd4\x91\x87On#Z-\x89\x99`p\xf4\xf5\xd8\xac6\xd5\x80\xff\x8b\x83\xaeL\xd7\xba\x0c\xf1z\xd3\xb6\xaaf\xeb\x16\xd3i\x8d\xf3=\xd1\xf9X\x84\x14\x83\x9a\xb6Z\x1d\x1dG=\xaeV\x9c')
 
ugh
I'm leaving for lunch and there better not be quines here when I come back
 
Thank you Aran, haha
 
in other news, snakeviz really is awesome
 
hello, I was wondering if you could help me about urllib and sending files in a request object. I do not have any way to send it to another Flask server. Flask server expects the file inside the object "request.files", but the only way to send from a urllib client is in the "data" field (in the Request)
 
10:41 AM
Best advice is to move away from urllib and start using the requests library if you can... it'll save you a lot of grief
 
This doesn't make sense to me. Why/how can urllib be involved here if it comes from the front-end?
So you are trying to pass a file between two Flask projects?
 
the way i read it, there's no front end involved, or doesn't have to be available in any case. It's a simple request from one program to a server, except there needs to be a file tied to that request.
 
Flask is not a server, though, so I'm reluctant to build on shakey foundations
 
@AndrasDeak ''.join(issubclass.__doc__[i] for i in b'"$\xe6\xef\xb2Y\'\xef\x80\xda}4"$\xe6W}5\x03\x14\xb3\x19 $\xb8\x02\xd0\x9d\x865\x07j<\xbc\x07j\xbb$\x0e\x07\x04\xc0\xbb=7\xb1\x03\xc0Y\x01\x89')
 
@ParitoshSingh until it's clarified, I would guess that JSON is better for communication between servers.
 
10:50 AM
 
@Aran-Fey you had me there for a second
 
Where can I ask about logging into sites? I want to use requests to log into a certain website, but I can't find what authentication I need to send. I know an email, password and cookie needs to be sent back, I just can't find to which url and what form the data needs to be sent in. Technically it's not a Python problem but a request problem.
 
cabbage guys
 
open your browser dev tools, log in, find the login request, then send the same request with python
 
Yeah, that is what I am trying to do
 
11:00 AM
I have a weird question, imagine there is a for loop running N number of times, half way through the loop can I restart the loop in someway so that it starts again from the beginning of a range()
A for Loop, i might add
 
@Anarach I'd wrap the loop in another (perhaps infinite) loop and break out of the inner one
 
You don't... you use a while loop and count yourself...
 
or that
 
Will it always be a post request (I assume yes)? The site in question sends about 80 different get requests and one post request, ploughing through that lot could take me all day @Aran-Fey
 
You could do that, but why do you need to restart a for loop in the first place?
 
11:01 AM
@Simon the one POST request will be easy to see in your browser's dev tools - just filter by it :)
 
@JonClements This I know, but I was wondering If the I variable in the for loop can be changed or reassigned..
 
99.999999999999% of the time it'll be a POST. Not going with a full 100% because the internet is weird and made by lots of people who don't know what they're doing
 
@Anarach it can only be mutated, but it shouldn't be
 
@roganjosh some fun timer project I was trying out.. nothing important
 
@Anarach this actually got asked earlier today:
0
Q: In a for loop how to send the i few loops back upon a condition

Eshakais there any way to send the value of a for look back in few loops.. for i in range(10): print(i) if random.randint(0,9) in [5,6]: i = 0 what i want to do is exactly something like this. now here i know the range(10) generates the range [0,1,2,3,4,5,6,7,8,9] first the then trav...

 
11:03 AM
@JonClements melon
 
@Anarach I'm not questioning the project itself, only what you're intending to do :) Maybe there is a better way
 
@AndrasDeak It does allows me to reassign the I variable but then when it loops back it continues from where it left off..
@JonClements Interesting... I shall give it a quick read..
@JonClements Hah... That guy is also asking kinda the same thing.. I will probably use a while statement , was just curious.
Melon :-)
 
@Anarach what are you trying to achieve by changing the iteration variable?
 
@MisterMiyagi Its stupid :-D really
I wanted to make a simple timer on the command window
I know , I should have started with a while statement ,I just became lazy half way through and then got curious if its possible to reassign the I variable at all..
 
11:09 AM
I'm not sure whether a while is appropriate for your case, that's why I'm asking.
 
I don't think it's possible to say that you should use a while at this point with what we know
Ok, MM and I are in agreement. We're trying to understand the problem you're trying to solve at the root
 
Umm... when you say timer... I'm also now thinking you might be after something different than what it seems you're asking :)
 
I'm think I'm actually stupid, does the request header contain the form data? If not that explains why I can't find what I am after
 
pretty sure form data is sent in the request body, not headers
 
Yeah. In which case I'm in the wrong place entirely
*bangs head against wall*
 
11:21 AM
@Simon not necessarily. I think it depends on the back-end verification. How do they save a session?
 
does anyone have idea of using 1 variable in another variable in grafana
 
I'm pretty sure I commented on that and asked you questions earlier, which you didn't reply to
 
Is grafana even python-related at all?
 
about as much as mind-maps
 
I wanted to know how the query string is created.
 
11:33 AM
@roganjosh I think it sends over some cookies with a user_id and session_id
I will have to see if what I think works first
I do know I get redirected. it starts with www.somesite.com/login to www.somesite.com/api/login for the actual post request and then www.somesite.com/account once I am logged in
 
/api/login sounds good
 
sounds too good to be true... it might be a trap :p
 
@pythonRcpp If you're going to keep dropping this issue into chat, it would be nice if you could actually be around to follow it up.
 
unless I am seriously misreading the "question", doing that is actually pretty standard in grafana. It's a bit like asking the same for Python -- I wouldn't know an appropriate answer to "how do I use 1 variable in another in Python" either.
either you know how to use a variable or you don't. there is nothing special about the result being assigned to a variable or used directly.
 
11:55 AM
wolfram alpha's geoip local time is borked
it thinks that I'm in EST...but for some reason it also shows me Hungarian time
 
Maybe there's two of you in this world.
 
I hope that other one isn't leeching on my bandwidth then
 
Quantum something something superposition
 
Make sure you don't run into them. might cause a rip in the space time fabric.
 
yeah, that much cynicism in one place...
 
11:59 AM
Hah!
 
@AndrasDeak reminds me of poor Ron (sorry for the ads)
 
It's alright, I don't see a single ad
 
@AndrasDeak WA is normally fairly good... have you looked out the window to check just in case? Wibbly wobbly timey wimey and all that :p
 
Success `b'{"errorCode":"errors.com.somesite.accountportal.csrf_token_invalid","message":"It seems that you are trying to send request twice."}'`
It's not there yet, but I am getting closer since this time, it's not giving me an unknown error
 
@JonClements well I heard Hungarian swearing coming in from the street only a minute ago, but honestly that doesn't prove anything these days
 
12:03 PM
@roganjosh it comes from grafana only
@roganjosh ok
@AndrasDeak could not find any related chat group
 
@roganjosh Sorry for the ambiguity. My intention was for people to edit the agenda themselves. I marked the page as "community wiki", so I think anyone can add items -- let me know if there are any permissions issues.
 
@pythonRcpp if all the bakeries are closed do you go to the hardware store to try and buy bread? :)
 
@Kevin any user with 100 rep, yeah... so some here might still need to request it be added :)
 
@Kevin No permission issues, I was just wondering whether the point wasn't appropriate for the agenda. I've edited it in
 
@Kevin I think it's only the remark on the page about pinging ROs ("tell us in chat", to be specific) to add items
 
12:10 PM
Hmm, I think I'll put the "edit this page" option first, since I want that to be the default
 
sounds good
 
@AndrasDeak thanks
 
@roganjosh In the face of ambiguity, add it anyway. What counts as "on topic" for the meeting is fairly fluid in any case
 
At least for me, I always knew I could edit the page but I tend to err on the side of not meddling with the site, so my fallback assumption was that it might not be a worthy suggestion
 
@roganjosh if it was thought you going to "meddle" you wouldn't have edit access :)
 
12:12 PM
and we could just, you know, hamstring you and leave you in an alley if you do
 
... that escalated quickly :P
 
way to ruin the surprise @Andras... sheesh :p
 
Is there anything you're bottling up, Andras? :P
 
I'm fine :P
 
@roganjosh the teeth of his previous victims? :p
 
12:14 PM
@AndrasDeak One of the old hardware stores I used to go to actually sold snacks and drinks. It also had a charity book stall.
 
Everything is thrown into question at this point :P
 
just think... if only you'd fed the poor little puppy those sausage rolls :p
 
<-- no regrets <hides>
 
I guess I ought to go back and skim through the transcript for semi-proposed agenda items
In the absence of a message along the lines of "can you add this for me?" I figured they were workshopping the idea with the intention of adding it after mulling it over some more but that's looking less likely now
 
is anyone else having problems pulling metadata from PyPI?
"problems" as in "pip dies with 503 HTTP Error"
 
12:33 PM
@AndrasDeak Worked , I was making a pomodoro timer , ended up using a while loop
but then there is this website which has built in timer so facepal
 
okay, but that's what the others suggested ;) I'm glad it works
 
this might also be a use case for playing with threads
 
@Kevin Realistically, I think this just enforces point 2 - is anyone going to take responsibility for this? Objectively, I'm not sure what the meeting is going to achieve atm or what can/should be raised. That's not a criticism of any one person, but there has to be some agenda
 
@roganjosh the GM's agenda is usually largely crowd-sourced
what we're collecting on that link is the agenda
 
12:34 PM
Except my suggestion wasn't added
Nor any other
 
Working on it :-)
I've added Dair's and smci's suggestions so far
 
Again, I don't want an argument at all about who did what, but there has to be some impetus behind the meeting.
 
if you want something done right, do it yourself etc etc :P
(but of course the criticism is valid)
 
<insert Galadriel's trial scene here> I don't think I'm the person to lead this but I'm more than happy to help out where I can
 
@Anarach I'm guessing that's not a site that assists in boiling the perfect tomato so its yolk is still slightly runny?
 
12:40 PM
@JonClements Ha ha Ha pardom me but I thought yolks only existed inside eggs ?
 
well, people did warn what could happen with genetically modified foods - guess we should have listened :p
 
Also , Last week at Pycon we had a live coding demo by David Beazley in which he literally created a python stack machine live and ran a game made for web assembly which was written in RUST on this stack machine which he coded in about 45 minutes
Many minds were blown
 
I can understand not wanting to edit the agenda to add potentially contentious topics, for fear of being given the stink-eye, so I'll stick my hand in the fire and add a bullet point for "how come our meetings aren't doing all that much?"
 
that reminds me of meetings to discuss how to reduce the numbers of meetings...
 
It wouldn't be the first time I got shot down in here :) I'm not fussed, I just didn't want to edit an "official" page
 
12:46 PM
Clearly it's a topic on many people's minds since my joke about it got ten stars and I know I'm not that funny
 
perhaps we should've written "Do NOT under any circumstances touch this" with a big red "EDIT" button
 
@Kevin you're at least 12 star funny, then?
 
My hilarity is some number other than 10. Just choose an integer at random and that's probably it.
 
so 4
 
random.choice()
 
12:56 PM
Currently reading the previous transcript to identify resolutions that we passed, so I can add them as sub-items under this meeting's "review of previous meeting" point. In the interest of fairness, I invite any interested party to do the same, since we don't have a completely objective definition of what a "resolution" is
Oops, I think that's the Q1 transcript, hang on...
 
Question: neural network. If my model's "loss" is rising on test data, but my metric is still improving, should i keep training?
With the assumption that until my metric itself shows a drop in performance, i should keep training.
 
Isn't that overfitting?
 
that sounds exactly like overfitting
 
Yeah, that's how the blog posts talk about it so far. But i've never seen these blog posts address cases where the loss and the specific metric are showing different trends.
So, i just wanted a sanity check.
I should add, that my metric (non-weighted average f1 scores) are also on test data
 
Why are you not happy with the model before you push it over this edge?
 
1:07 PM
I went and picked two or three examples. My model performed really well on an unseen data example where a lower loss previous iteration didn't. Now, sadly this is just one example. I was hoping to trust my metric, but essentially, with the differing trend between my metric and the loss, i just want to actually ask, trust my metric or trust the loss curve?
 
I think your trust is misplaced
 
I suppose one assumption im making mentally is that i should see a point where my metric follows essentially the "overfitting graph"
However, finding whether it actually does or not would involve a few days of training, so i wanted opinions.
 
You want the model to work. Take a more objective eye on this
 
Is the objective eye the "metric" or the "loss"?
 
I'm gonna go "metric" but I don't know what that is here
 
1:12 PM
its non weighted f1 score averaged. I have big-time class imbalance, and i cannot change that. (building an ner "edit: essentially: or something similar in any case"). So, in my opinion at least, this metric should be really good.
Just that i've never had to face a differing trend between loss and my metric before.
 
What is the application (roughly)?
 
Like an NER, take in unstructured data, label relevant items in the document on our custom classes.
 
I think I better defer on this one mate, sorry
 
No worries, hopefully someone can give me some insights here. Uncharted territory for me. (This isn't the only issue im wrestling with atm, NERs seem like a different beast entirely).
Push comes to shove, i might bite the bullet and overtrain on 50 epochs or something, and just plot out my metric. That however, is a daunting task time wise, and Im worried about heating issues on our ghetto setup causing crashes. It's rough :P
 
sounds like you should revisit your loss function. either it doesn't match your requirements, or you should reconsider your requirements.
 
1:22 PM
hmm. This is where im severely lacking in knowledge. For multi class classification (such as in case of ner) im only aware of categorical cross entropy and sparse categorical cross entropy.
Are there other loss functions in tensorflow for multiclass classification? Everyone seems to be using the same cookie-cutter code in blog posts.
 
1:35 PM
Does it ever make sense to augment a web server (flask) with log-handling functions to parse logs (number of unique visitors to a given URL, etc..)?

For small logs, it seems quite reasonable, but I suspect it wouldn't scale. I'd rather be writing a cronjob in awk that sends daily emails. Yet that means maintaining a parallel system.

Ideas, please?
 
Flask is not a web server
I have to look into it myself, but I think you want Google Analytics to place a cookie on your visitor. Still, my previous comment stands, and you will want to look into that server issue first
 
My bad. I meant to say "to augment nginx acting as an edge server with gunicorn acting as an origin server for a web application written with flask".
 
Aha, much better :) I think you can configure gunicorn's output but I'm not sure it covers everything you want
 
lol..
You're right. Cookies are necessary to keep track properly. But just to get off the ground, we could count IPs and be content to double- (triple-, ..) count those hopping from IP to IP; no?
 
1:51 PM
I suspect that approach becomes quite hack-y
 
@Calaf seems as if you are looking for a tool like logstash
 
I think this might be the first time I'm thanking someone for pedantry.
Thanks, roganjosh, because searching for "parse nginx logs" produces much better results, including on SO.
@MisterMiyagi I am just realizing that the important thing at this stage is to resist the temptation to code it at all.
4
 
<dusts shoulders> Just doin' my unpaid job
 
It can spiral into a very intricate task. It's also entirely unnecessary, with tools like hping.org/visitors exporting to GraphViz.
 
@Calaf There is great wisdom in this.
 
2:08 PM
Hmm, what's a good way to losslessly encode an arbitrary string into a form that's safe to use as a filename? Anything better than 'base64, except replace "=" and "+" and "/" with less special characters'?
 
Remember the Windows file name length limit. If you are dealing with long strings be aware of that
 
@Kevin slugify it?
 
slugify would be lossy i assume, since you lose information about the position of your special characters
(if i understand it correctly)
 
As in, just remove all the naughty characters? That would probably be fine for what I'm trying to do, although for the principle of the thing I'd like the conversion to be reversible
 
that's true :/
 
2:20 PM
Avoiding collisions is 100% YAGNI but my heart yearns for it
 
@Kevin Depends how many you remove before you have length issues, but if you're only replacing a few, maybe you could replace the with an unlikely string eg spch01, spch01, spch02, spch03 for =,+,/ ?
 
I'm encoding urls, so if there was a character that's not url-safe but is filename-safe, I could use that as an escape sequence to denote special characters
So if, say, "^" was illegal in urls but legal in filenames, I could encode "foo.com/bar" as "foo.com^47bar"
 
angular brackets? are they legal in urls?
 
@ParitoshSingh I believe they are
 
In that case, i apparently found something exactly opposite of what we needed, great. Apparently they're illegal in filenames :P
 
2:29 PM
Illegal special characters in Windows filenames are:
:*?"<>|
So angular brackets are out
 
@ParitoshSingh In fact, I typed in chat.stackoverflow.com/rooms/6/python<1>; and it just stripped off the <1> and took me right here
 
Im not too familiar with what's legal in urls or not.
curved brackets?
 
I wager five quatloos that there's a specification that's politely ignored by all browsers and web servers, which all do whatever they feel like
Woe to the software that does reject illegal urls, as their users will howl at them that they can't access example.com/:*?"<>|
 
curly* brackets might do the trick
{}
 
Hi guys, if I have a regular stream of google sheets , is it better to build a data warehouse to analyze data in them ? If so , what are the options?(looking through the persective of a startup which can't invest in cloud at the moment)
 
2:32 PM
@Kevin haha
@RaphX Is the data itself well-behaved and tabular? And what kind of volumes are we talking about here? A database seems like the first option for storing tabular data.
 
ALthough a little bit of research finds that <> won't be used in a URL? Apparently #% are excluded by the RFC std...but we know they're used all the time? :S
 
user10984358
hey guys, if anyone here has used keras, then can you let me know if it is possible to see the metric values (accuracy, loss etc) at each epoch that I used while training the model?
 
url.spec.whatwg.org tells me that the "forbidden host code points" for urls are: U+0000 NULL, U+0009 TAB, U+000A LF, U+000D CR, U+0020 SPACE, U+0023 (#), U+0025 (%), U+002F (/), U+003A (:), U+003F (?), U+0040 (@), U+005B ([), U+005C (), or U+005D (]).
 
Yes the data is tabular , the volume is not that high as it is still a startup but is expected to rise quickly in the future @ParitoshSingh
 
But I don't think a "forbidden host code point" is guaranteed not to show up in the url; it just won't show up in the bit between "http://" and ".com"
Obviously "/" and "#" show up in urls all the time
 
user10984358
2:35 PM
the output that is like
Epoch 650/700 loss 0.3 acc 0.99
that line
 
@TheNamesAlc did you save the result of model.fit into a variable? It should be a history object. So something like history = model.fit(data...)
 
user10984358
actually I did train the model from scratch but it was on colab, so I have the .hd5 and json file of the model now
 
Ah. you shouldn't have access to the metrics anymore then, i don't think.
 
user10984358
dang, no way then huh?
 
the model save files only stores weights/architectures, unless you specifically add additional information in the h5 file.
 
user10984358
2:38 PM
assuming I train again, how do I add that information now?
 
Just save it separately, simplest option. Take a look at the history object you get when you use model.fit
Slice your dataset (so you can train one epoch in a couple sets. just 1 or two datapoints are enough), run 2 epochs, just so you can see how it looks.
 
user10984358
alright I will do that, thanks, I am in a bit of a time crunch so I am looking for the easiest way out, if only training happens a lot sooner
 
user10984358
basically I need to plot a graph of that, I cant do that without the history or is it possible ?
 
user10984358
that = how much loss and accuracy varied over the epochs
 
you can't do it if you don't have the "results at every epoch"
And unless you explicitly saved results, you don't have them.
#GIVEN
history = model.fit(your_params_here)
#a list with the loss value for every epoch
history.history['loss']
#a list with loss value on validation data for every epoch
history.history['val_loss']
#if accuracy was a metric used
history.history['acc']
That's roughly how the history thing will look. it will have a lot of keys in history.history based on your metrics.
After that, you just plot what you need.
#Quick example
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epochs')
plt.legend(['Train', 'Test'], loc='upper left')
plt.xticks(np.arange(0, len(history.history['acc']), step=1))
plt.savefig(os.path.join(model_output_folder, "Graph-Accuracy.png"))
plt.show()
Make sure you run a "mock" run with sliced data
There's absolutely no reason to run the whole thing, only to find out some code didn't work for your use case.
 
user10984358
2:46 PM
tysm I will do a sample run, also on a non python related note, is the 90 minute before termination in colab applicable if I start the model on a tab in my phone and close the page? or I should have the page open and not move my mouse or record a keystroke?
 
user10984358
which is considered "inactive state"
 
No idea, sorry. I haven't used colab for anything long running.*
 
user10984358
stackoverflow.com/a/47805171/10984358 this answers says its 90 mins for ideal cut off
 
user10984358
alright thanks anyway
 
user10984358
apparently my last colab run has the text only output saved
 
user10984358
2:51 PM
Train on 5884 samples, validate on 1472 samples
Epoch 1/700
5884/5884 [==============================] - 12s 2ms/step - loss: 2.7240 - acc: 0.1293 - fscore: 0.0000e+00 - val_loss: 2.6418 - val_acc: 0.1617 - val_fscore: 0.0000e+00
 
user10984358
I guess I will do string split and get nasty here
 
@Kevin What about the extended ASCII table? As long as you avoid the ones that are more likely to be seen in IDNs, you might be ok?
 
Hmm, possibly
 
wim
What should you do with answers that were valid at the time, but now are syntax error? And there is no replacement in current version of Python?
 

« first day (3293 days earlier)      last day (1638 days later) »