« first day (3607 days earlier)      last day (1327 days later) » 

6:27 AM
Eh? Quiet Sunday evening/Monday morning... where did everyone go?
@roganjosh Moreover that question has already been asked and answered multiple times on SO already, so it would be a dupe. OP should search SO first before posting yet another duplicate.
@inspectorG4dget Hi, can you give a link of where you discussed that, I can't find it. This is just some sort of unofficial hangout to discuss Python and packages, right?
 
7:01 AM
Hi,
Whats the best way to get the non null entry from a pandas series starting from the last index?
For example lets says I have a pandas series a = ['1','b',nan,nan] how do I ignore the null values starting from the last index and get the first notnull value from the series? In this case I just want 'b'
 
raf
7:38 AM
Hi, can you kindly tell me how to exact the day of the year (Julian day) from the date format %Y-%m-%d %H:%M:%S in pandas data frame?
Basically, I want to create a new column for `day of the year` in this dataframe:
```
```
0 2017-05-26 05:46:42
1 2009-04-16 20:06:23
2 2009-02-28 18:57:41
3 2019-03-07 01:45:06
4 2017-02-12 06:45:15
...
166651 2015-09-17 02:09:12
166652 2016-08-12 20:39:08
166653 2020-08-03 12:37:00
166654 2019-10-08 14:12:37
166655 2012-09-22 19:22:39
Name: user_created, Length: 166656, dtype: object
```
I found this one. But don't know how to apply it to my dataframe column!
https://stackoverflow.com/a/623312/7673979
 
8:31 AM
Hello All, is there someone here who was experience with pagination using the Requests model?
 
8:45 AM
@Stramzik you can try series.last_valid_index() with loc, something like: a.loc[a.last_valid_index()] provided there is no other value appearing after the nan
 
Morning cbg all
 
@raf I am not sure you want this but pandas has a dayofyear (not Julian) : pd.to_datetime(df['user_created']).dt.dayofyear , though this calendar returns me similar values
cbg
rbrb for a bit
 
9:19 AM
import requests
import json
import math

res = r.json()
token = res['access_token']
headers = {'Authorization': 'Bearer ' + token}
proxies = {'https': 'proxy.***.***.com:8080'}
mod_date = 'ModifiedOn gt 2020-01-31'
col = 'Field1, Field2, Field3, Field4'

params1 = (('$count', 'true'),)
response1 = requests.get('https://***-***-***.***.nl/odata/***', headers=headers, params=params1, proxies=proxies)

data = response1.json()

next_link = data['@odata.nextLink'] #<--- this is my 'next_link' link.
I'm trying to collect 250 records per 'slice'. In total I have 1429 records, so 6 slices of 250 should be enough to collect all my records. I tried to do this with above code but when I run it it gives me back only the records 251 to 500. My expected result was to receive the records 1 to 1429. Can someone tell me what I'm doing wrong?
 
10:08 AM
next_link.format(x) looks suspicious because next_link is a value you got from the server. Why would there be a placeholder for string formatting in there?
 
raf
@anky Thank you so much! This is what I was looking for.
 
10:41 AM
cbg all, bugrit.
@Mediterráneo I've used link header pagination. Requests makes that very easy, by adding a links attribute to the Response object which is a dictionary populated by some or all of "first", "last", "next" and "prev" keys.
Your extraction of "records per page" looks rather flaky as it appear to depend on the exact length of the inks being processes.
The logic with link headers is simple. Whether or not it will easily adapt to your pagination scheme I cannot say, knowing nothing about the server.
def read_from(url):
    r = requests.get(url)
    while True:
        for item in r.json():
            yield item
        if "next" in r.links:
            r = requests.get(r.links["next"]["url"])
        else:
            return
for i, item in enumerate(read_from ("http://localhost:8000/heroes/")):
    print(i, item)
 
11:16 AM
Hello guys I am trying to do some webscrapping from the following website: spitogatos.gr/search/results/residential/rent/r100/m2032m/… . However, when I try the following code I get an 405 error
my_url = r'https://www.spitogatos.gr/search/results/residential/rent/r100/m2032m/propertyType_apartment'
#opening the connection and grabbing the html
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
#html parsing
page_soup =soup(page_html, 'html.parser')
#grabs each product
could you tell me what's wrong?
 
@Vasilis 405 stands for unauthorised. So unless your HTTP request headers are set as a normal browser, then looks like the website blocks programmatic access.
It may also be against the site's Terms & Conditions to scrape it.
 
thank you for the response, is there a way around it?
 
Probably. Usually you spoof the request header "user-agent". See stackoverflow.com/questions/27652543/…
But as mentioned, if might be against the site's ToS.
 
12:20 PM
Thank you I saw it!
I have trouble installing the fake_useragent package, ModuleNotFoundError: No module named 'fake_useragent'
I have no idea why, I pip installed it, this is the first time I have an issue with a package
 
1:14 PM
morning cabbages, folks!
 
Hey guys, I am trying to scrap data from the site built in next JS. I am able to fetch data successfully with headless browser on my local. But its not working with pyvirtualdisplay code attached(pastebin.com/1p46d50Q) . Any solution
 
@MisterMiyagi Hmm, plausible.
 
@kevin
 
Hello Roshan. For future reference, you should try to only ping people that you're already having a conversation with.
 
@Kevin Apologize. We keep in mind for future.
 
1:23 PM
Anyway. When you say 'it's not working", how can you tell? What kind of output are you getting?
 
@Kevin I am only getting javascript code with pyvirtualdisplay. And if trying without virtual display working fine.
 
Strange. Display code usually doesn't cause that kind of problem.
 
yes strange, tried with time.sleep(5) but not helpful. shared pastebin too.
 
Are you sure the code in the pastebin is the code that you're running? Because as far as I can tell, it can only print "True" or "False". So it's unusual that your script is printing javascript.
 
raf
Hi, I want to draw a bar chart on s = df['user_followers'] for the range of user followers: <1000, 1000-10000, 10000-100000, >100000.
 
1:29 PM
Yes, In code I am checking if provided title exist in url then print true else false.
 
I wish I could forge /that/ kind of patience
 
raf
When I do:
 
Ok. So how do you know that you're only getting javascript?
Maybe you're getting the entire page and it simply doesn't contain the provided title
 
Puppy!!! Long time. Potato?
 
raf
1:32 PM
```
out = pd.cut(s, bins=[s.min(), 1000, 10000, 100000, s. max()], include_lowest=True)
ax = out.value_counts(sort=False).plot.bar(rot=0, color="b", figsize=(10, 5))
ax.set_xticklabels([c[1:-1].replace(","," to") for c in out.cat.categories])
plt.show()
```
I get this error:
```
TypeError: 'pandas._libs.interval.Interval' object is not subscriptable
```
 
printing source, in that source found javascript code not the actual texts available in DOM.
fetching this url and trying to find provided title in source
https://www.washingtonpost.com/opinions/2020/08/24/why-trump-so-often-says-quiet-part-out-loud "Why-Trump so-often-says-the-quiet-part-out-loud"
 
Interesting. When the source differs from the DOM, that's usually because the website populates most of its content dynamically after initial page load. A lot of people that use the requests library run into this problem. But you're using selenium, which is supposed to handle that kind of thing correctly.
Maybe the timeout isn't long enough...?
 
timeout is 60 sec
 
That's pretty long. Hmm
 
yes
 
1:46 PM
I wish I could test the behavior on my own computer... But pyvirtualdisplay won't work for me because Windows doesn't come with Xvfb :-I
 
elders of our little village: anyone got a minute to give me some professional advice?
It's been a month since I've joined newCo. My direct manager is supposed to be one of the founders. One of the senior partners is sporadically in touch with me and hasn't responded to external meeting requests he asked me to setup. Another senior partner (who is also Mr. HR) runs our weekly team meetings and hasn't said anything about my performance yet. I've asked Mr. HR for feedback (scheduled for later this week). I feel like I'm puttering around without much direction. Any thoughts on what I should do?
 
Asking Mr HR for feedback is good initiative, I think they'll appreciate that
 
Do you have something like quarterly goals? Sounds like they might not yet know where/how to use you, in which case waiting and signaling eagerness to contribute is a good course
 
I've been here for about a month now. So I'm technically still on probation. Also, not being in the office is a bit of a challenge - can't really do ad hoc meetings with folks
 
Puttering around isn't necessarily a problem in of itself. I know of companies where devs will come into the office and read the newspaper all day if they don't have any assignments. They literally get in trouble if they try to work.
Obviously this is an extreme example but it demonstrates what is in the realm of possibility
 
2:00 PM
They literally get in trouble if they try to work <- does not compute. Could you please elaborate? I've been trying to get ahead on features in future milestones and have had to stop myself, seeing as this would create bigger problems down the road
 
If there is nothing to do right now where you can add to the value of the business where you work in, you starting to work on something and getting attached to it creates more problems than merit for the firm
 
Hi folks, I don't know how to search for a thing but I think it is a pretty standard pandas operation.
I have this dataframe: df0 = pd.DataFrame(['A','A','B','C'])
how can I get this: df1 = pd.DataFrame([0,0,1,2])
from it? without doing replace() stuff & knowing how many things are there in df0 to replace
 
I read about the company in an article. They're a big game developer, like Ubisoft or something. There are occasionally gaps in development between the end of one game and the beginning of another. So during that period the devs come into the office and do essentially nothing.
 
understanding what "business value" is was the first hard problem I needed to crack when I left uni for corp work
 
They're not allowed to jump onto other teams' projects for organizational or legal reasons, which I'm not clear on
Maybe i misremember and it's something more like "you're not allowed to use work computers to experiment with personal projects (even though you have nothing better to do)", which is far more typical
 
2:05 PM
understood. Thank you :)
 
There's a possibility that Mr HR will say "oh, we've been underutilizing you, sorry about that" and he'll prod your manager to give you more direction
On the other hand maybe they expect you to take a page out of Arne's book and poke around for things that need doing. I think it really depends on the corporate culture.
 
thank you. I've already been poking around, but I suppose I could always do more of that. For now, I'll wait for Wednesday's meeting with Mr. HR

Thanks again :)
 
good luck, getting used to corporate culture and the speed at which it works takes some time and nerves =)
 
Working on future features seems relatively safe, if you're working in your own branch. Or maybe by 'bigger problems" you mean office politics. "Steve always works on UI stuff and he's mad that you called dibs on the September overhaul" is something that git cannot help you with.
 
maybe next release.. git rid-of politics
one can only hope
 
2:17 PM
lol! not what I meant. I meant more like dirtying up the git history/lineage
 
I suspect there is some git wizardry you could do to make things line up. keeping your work in a "future features" branch saves you the effort of disentangling it later
 
2:48 PM
@zabop Well, if all else fails, you could do:
>>> df0 = pd.DataFrame(['A','A','B','C'])
>>> d = {}
>>> for idx, item in enumerate(df0[0]):
...     if item not in d:
...         d[item] = idx
...
>>> df1 = pd.DataFrame([d[item] for item in df0[0]])
>>> df1
   0
0  0
1  0
2  2
3  3
Or, hmm, have I misread the question?
 
@Kevin No you havent , with pandas there is a builtin is pd.factorize(df0[0])[0] , docs say Encode the object as an enumerated type or categorical variable. :)
 
Will give you [0, 0, 1, 2] but that seems to make more sense than [0, 0, 2, 3] anyway...
 
I thought the question was asking "what is the first index of each element in df0?" but that doesn't match the expected output
 
I came across a reddit post about python GUI module that had 500 upvotes and a bunch of awards, so I checked it out because I'd never heard of it before. One of the devs had written a short summary of key features, which include 1) no dependencies and 2) cross-OS compatibility... Seemed interesting. Then I checked out their github and found code examples, and, well, my opinion on something has never done a 180 faster than today
from dearpygui.dearpygui import *

add_text("Hello world")
add_button("Save", callback="save_callback")
add_input_text("string")
add_slider_float("float")

def save_callback(sender, data):
    print("Save Clicked")

start_dearpygui()
^ kill it with fire
 
Maybe you want:
>>> df
   0
0  A
1  A
2  B
3  C
>>> (~df.duplicated()).cumsum()-1
0    0
1    0
2    1
3    2
dtype: int32
@Aran-Fey Ew, callback by string
 
3:02 PM
this is why I don't visit the python reddit all that often
 
Suppose one can get the 0, 0, 2, 3 with something like: df0.groupby(0)[0].transform(lambda v: v.index.min())
 
github.com/hoffstadt/DearPyGui/issues/103 suggests that the dev would like to support registering callbacks via actual function object, although he doesn't seem to be in a big hurry. He especially doesn't like the fact that you can't register a callback before defining the function.
"[callbacks are registered by string] because of earlier version in which dearpygui was standalone and included an embedded interpreter." Ok then.
 
@Aran-Fey hey, we have some php code that looks like that. I think refering to functions by strings is part of their core lib, so it's a pattern there.
 
@JonClements I'm scared to ask the OP what the output should look like if the input isn't sorted
 
Yeah, that's (unfortunately) normal in PHP
 
3:10 PM
PHP delenda est
 
I've misspelled a callback function's name on more than one occasion
 
in scrapy you can use strings as callbacks because of deferreds and all that...
and using strings for class names in Django ORM's is sometimes required - mostly if defining a self-referential field on the class itself...
 
github.com/hoffstadt/DearPyGui/blob/… is where callbacks get looked up in the global namespace, for the curious
I suspect most of the function would remain unchanged if he turned std::string name into PyObject* func. He could delete both if (pHandler == NULL) blocks and it would work more or less the same.
I suppose you'd also need to (in|de)crement the function's reference count in the class' (con|de)structors
 
Is there a way to pass frames directly to Userspace without going to kernel protocol stack(interface to Userspace)?
 
Nobody has ever had problems with object life cycles in C++, so I'm sure nothing could possibly go wrong
@Janith What kind of frame? Tkinter frame? CPython call stack frame?
 
3:26 PM
Ethernet frames
 
That doesn't sound related to Python
 
may be
 
it's not, you'll have better luck in a more appropriate room
 
"How do I do that thing I said... In Python?"
 
How can I reframe a question to be about Python... in Python?
 
3:28 PM
Independent of whether the question is on topic, I don't think we collectively know much about kernel protocol stack ethernet frame userspaces anyway.
 
Really,I want to use Userspace protocol stack....but,when received a data frame,it goes to kernel stack first.
 
Troubleshooting bad indentation is more my area of expertise
 
@Janith if it wasn't clear, since this isn't about Python you should ask elsewhere.
 
I know python is not using for system programming frequently... but,there is fnctl,ioctl modules also in python
 
Regardless, please go elsewhere for your question.
 
3:33 PM
ok
 
If a kernel expert happens to wander by, we'll send him your way :-)
 
    @Kevin
    i am using itertools module's combinations fn on a list. like
    aList = [list(x) for x in list(cmb(arr, k))] # cmb = combinations, arr is the list, k is an int.

    this works well for list lengths around 30 or so (i mean len(arr)).
But it doesn't work for lets say len(arr) = 50 and k = 8. {MemoryError}.

Is there an optimized way to do this 'cause I will need to do this for input with len(arr) above 10^7??
 
Consider using a generator, which is more memory efficient.
aGen = (list(x) for x in list(cmb(arr, k)))
You won't be able to get results by index, but you can still iterate through them
 
@PSSolanki Please don't ping specific people with questions unless you're already having a conversation with them. See our room rules: sopython.com/chatroom
 
You missed one: aGen = (list(x) for x in cmb(arr, k))
 
3:39 PM
Oops
Also, what davidism said
 
@davidism. I apologize. My first message in a chat room. Didn't know that.
@Kevin But will it give me a tuple or a list (i mean what will be type(aGen)) ?
 
generator.
 
@Aran-Fey Alright i'ma try that rn. Thank you.
I see. a new concept to learn now :)
Thanks
 
You may also be interested in stackoverflow.com/questions/1776442/nth-combination, which lets you get the Nth combination using a very small amount of memory
 
@Kevin This one looks interestin'
 
3:44 PM
The mathematics involved is only moderately brain-melting
 
@Kevin I have a maths major actually so I'ma give that a go :)
 
👍
 
Thank you :)
 
No problem
Unrelatedly, I'm having my own problems:
C:\Users\Kevin\Desktop>selenium_test.py
Traceback (most recent call last):
<snip>
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.

C:\Users\Kevin\Desktop>where geckodriver
C:\Drivers\network\gecko\geckodriver.exe
Selenium, why you do this ಠ_ಠ
Ah ha. Because I was passing a non-default argument to executable_path, it wasn't actually looking in PATH. Sensible behavior, bad error message.
 
@Kevin Well, I'm not an expert in selenium but from what I know, this happens when the browser executable are not in the path in which the webdriver is looking in.
 
3:57 PM
Exactly :-)
 
@Kevin oh man! that takes me back to my combinatorial algorithms class. We did gray codes and combination generators for... a while
 
@inspectorG4dget Too bad I didn't have any such classes :/
 
Uh oh, I ran my script and now Firefox won't load any new pages.
 
@PSSolanki if it helps, this was our textbook, which actually does a pretty good job of explaining the generating concepts you're interested in
 
Only selenium things :/

I wonder how chrome would behave under selenium (hvn't tried it though)
@inspectorG4dget That looks interesting. Thank you :)
 
4:04 PM
Guess I'm using Chrome for the forseeable future
 
@Kevin I would like to hear how it goes.
 
@PSSolanki I think you got the wrong impression when I told you not to ping people unless you already had a conversation. Also don't ping people on literally every message. They're here having the conversation too, they don't need constant pings. Again, please see our room rules: sopython.com/chatroom
 
The "I'm not an expert in selenium" ping was within reason, because you were trying to help me out :-) The subsequent ones were... Less necessary
 
My bad again. will take care of it.
 
I do try to read every message even if my name isn't in it :-P
 
4:08 PM
Got that :)
 
In any case, Firefox is working again. I think something went wrong because I was moving around the driver executable while Firefox was trying to install an update. I turned it off and on a couple times and that seemed to fix it.
 
Why Firefox cares at all about a driver executable that I downloaded myself, I'm not sure. Surely it has its own copy somewhere? Maybe I pre-empted it by putting it in the PATH.
 
What i believe may be reason is that the executable kinda works as a link between - firefox and webdriver (i'm sayin link 'cause I don't have a better word).
So i'm only guessing it would be the same with chrome (more or less maybe)
 
Yeah I imagine Chrome works roughly the same way
 
4:15 PM
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH.

As expected (just with a diff name)
 
4:48 PM
Could I get some comments on my edits here: stackoverflow.com/a/50681396/165216 In trying the suggested solution, I found I had to disable certificate verification, so I added the steps on how to do that. But I am concerned that I am leaving open a security hole. I added a warning, but who reads those?
 
I don't know anything about certificate security, but I think your warning is sufficient due diligence on your part
 
In principle, there are no trusted sites without SSL. In practice, people that know the distinction don't need to be told about it, and the pedants will find something else to complain about.
 
There are plenty of unlabeled footguns on the site, so you're already better than all of them :-)
 
Though I'm wondering why you added this to someone else's answer. There's no need to warn about something that isn't there.
Why not add your own answer?
 
This way is more aesthetically pleasing. #hahaOnlySerious
 
4:59 PM
I've heard that SSL is outdated and that everyone should use TLS instead. I've heard his from people that are much more wisdomous than am I. I really don't know what's up with any of this
 
Controversial opinion: trusted editors should do whatever they want with their powers as long as it's not for evil purposes
 
isn't that pretty much already the case? pretty much?
 
Yes, but a true anarchy has less second-guessing :-P
 
In this context, the warning is probably sufficient, and I don't think there is much MITM risk with this particular URL. I fear for the cargo-culters who stumble on this and think "Aha! I can use this to fix my stale certificate problem I've having with cute_kittens_clipart.com";, but there are only so many people I can worry about.
@MisterMiyagi I was adding a note to their existing posted Python commands, which gave me the certificate error when I tried them. 30 seconds research turned up this workaround. My note was just a qualifier to their answer, not really an answer to the original question.
 
Dunno, I just feel there is a difference between "do this (but fail)" and "do this with the safety off".
FWIW, people who think "I'll disable SSL to solve SSL!" is a good idea will find a way to do it. It might indeed be better to hand them the cargo with a warning label attached.
Either way, I'm just venting frustration and riding dinosaurs, so don't give too much thought to my musings...
 
5:11 PM
Hello, guys. How do I implement this double sum in SymPy? i.imgur.com/OWJl4Dp.png
 
This article kind of explains PyQt5 license vs PySide2's
It's kind of annoying how 2 packages are just almost the same thing.
Confuses people more lol
Was confused why some had From PySide2 import QtWidgetsand another reference had From PyQt5 import QtWidgets and there is some minimal differences in the methods for some which is something really odd and confusing
Also confused me when I was looking for PyQt docs and had one by doc.qt.io and one by riverbankcomputing.com luckily I found that article because I was really confused lol
 
5:33 PM
Currently trawling through wikipedia articles to figure out what bad stuff can happen to you if you accept a certificate from a website that hasn't been authenticated by a certificate authority
 
Welp, I wasted the whole day attempting to set up a Windows/linux cross-compatible backup software. I narrowed it down to Borg (which is nightmare to set up on Windows) and Duplicacy (which has an awful GUI and a pretty much undocumented CLI). You'd think backups would be a solved problem in 2020, but no
 
My guess is "the connection is secure and reliable, but you can't verify that the person on the other end is the owner of cute kittens clipart dot com"
If the FBI van outside your house splices into your internet cable and directs all cute kitten requests to their server, they can impersonate any https site they like if you don't cross check your certificates against a known-good list previously downloaded from a CA
 
Thanks all for the input - I'll add some more specific wording in place of "untrusted", since what could be more trustworthy than cute kittens?
 
I guess the upshot is that (AFAICT) it's only a vulnerability if the bad guy already has MITM access. If he can't redirect HTTP(S) or DNS traffic on your network, then they can't take advantage of TLS cluelessness
Looking outside for a poorly disguised FBI van is not a sufficient safeguard, because they might be parked outside the website's headquarters instead
 
5:57 PM
Hello, guys. How do I implement this double sum in SymPy? i.imgur.com/OWJl4Dp.png
 
Have you tried implementing a single sum first? That might be a good starting point
(not rhetorical or sarcastic, I really think it would be useful)
Sum and summation look useful
 
6:13 PM
@Stramzik a.dropna().tail(1) or else a[a.notnull()].tail(1) Please skim the doc for methods on a pandas Series and see the quickstart tutorials.
 
@Kevin the problem is more in the details, since there is no upper limit and initial values ​​for m and n.
 
@smci might want to look at Series.last_valid_index...
 
@Kevin the double summation itself is basically made by making two summations.
 
I don't know if sympy can represent "sigma with m,n under it" because AFAIK that's not a complete algebraic expression
 
@Marco Do you know how to do a single summation without bounds?
 
6:19 PM
It would be complete if there was an expression after it making use of m and n as indices to a matrix of known size
or a series-of-serieses of known size or however you want to call it
 
@MisterMiyagi no.
 
Does the use-case imply bounds?
 
If I'm reading this wolfram mathworld page correctly, "sigma with m under it followed by f(a[m])" means "the sum of f(a[m]) for m from 0 to len(a)-1 inclusive"
 
Usually, \sum_k implies "all valid k"
 
^
You'll have to use human cunning to determine the bounds because the computer can't figure it out with just the information given
 
6:24 PM
Sums that don't include all valid values customarily get a prime superscript
 
sigma with an asterisk and a footnote saying "for a list of valid values, subscribe to my onlyfans"
 
@Kevin Right...thanks.
@MisterMiyagi I understand, thanks.
@AndrasDeak prime superscript?
 
@JonClements Also
 
Is it possible with type hinting to specify which keys a dict is expected to have? not just the key's type, but the specific values.
 
Wild guess, based on no evidence: no.
 
6:40 PM
docs.python.org/3/library/typing.html#typing.Dict This is the only thing I can find which only specifies the types of the keys.
 
Maybe I am using the wrong tool for the job. I sometimes have dicts in my code, for example {'name': 'foo', 'value': 'bar'} where all uses expect the same keys.
 
"creates a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type"
 
that looks like it will work
 
Independently confirming ^^ TypedDict
 
6:42 PM
My particular use case is more for telling other developers the expected shape as for telling the interpreter.
 
Hooray for me being wrong initially
 
That is what I have started using type hints for also (telling other devs - or future me - what is supposed to be in a dict, or function args). Now finding that PyCharm is being especially helpful in keeping things straight.
 
> New in version 3.8.
Boo
this project is 3.7
 
I think there is very likely a backport on this one.
 
This features rustles my jimmies because {"x": "y"} and {"a": "b"} have identical types, strictly speaking. They should create a valuehint module and put TypedDict in there instead.
 
6:46 PM
That would just be a Dict[str, str]. TypedDict should wag its finger and say '"a" is not a valid key'.
 
My use case is where a particular dict is used, it is expected to have certain keys. If those keys aren't there, then everything just blows up.
 
Dataclass then, perhaps?
 
I agree that a backport is likely, because it mentions being compatible with "older versions of Python that do not support PEP 526"
Which came out in 3.6
 
And potentially, other programmers are constructing these dicts to pass in, so they need to know what keys are needed.
@PaulMcG What's dataclass?
 
6:48 PM
from dataclasses import Dataclass
 
A mutable namedtuple, kind of (I don't think you can iterate over its values or keys).
 
I mean obj.key notation is definitely more convenient than d['key'], if that's what it provides.
 
@NicolasGervais Their heart's in the right place, I guess :-)
 
there is a talk by raymond hettinger about dataclass usage/raison d'etre: youtube.com/watch?v=T-TwcmT6Rcw
 
6:50 PM
Nice of them to share information they find useful... Could do with another round or two of proofreading
 
@Kevin Yes he definitely had good intentions
It's going to break my heart when it gets closed
 
@Code-Apprentice And - bonus! - it is in Py3.7
Maybe better than mutable namedtuple might be a SimpleNamespace with specified names.
 
I'll have to take some time to learn more about that. It won't be going into my current work today, though.
 
@Code-Apprentice see typing_extensions for backports
 
speaking of typing, I just merged typing into ItsDangerous: github.com/pallets/itsdangerous/commit/…
It's already been added to Werkzeug and Jinja to some degree as well, although they're not complete yet.
 
7:05 PM
ItsDangerous becoming gradually less dangerous
 
7:15 PM
I thought typing ItsDangerous would be fairly straightforward compared to the other much more complex libaries, but I ran into so many weird unfinished corners of Mypy during it. There's a few places where I gave up and just used Any or ignore.
 
It looks like self.signer isn't annotated. Is mypy smart enough to infer its type? Or did you give up ;-)
Or am I reading the diff wrongly... [myopic squint]
 
@MisterMiyagi tnx
 
Nope, you're right, I fixed that and then forgot to push it before merging.
If you wan to get a PR into ItsDangerous, now's your chance.
 
signer (sans self) is annotated in the argument list, and self.signer is only assigned to signer, so mypy could, in principle, figure it out
@davidism Generous :-) but my plate is full this afternoon, so that race may go to someone swifter than me
 
I'm fairly sure it does, but I was trying to be consistent.
This was also one of the weird corners I ran into, in some places Mypy knew what serializer.signer was, but in some tests where I subclassed it, I was not able to convince it that it existed or was subclass-able.
Fixed it.
 
7:23 PM
It would be cool if you could feed un-annotated code into an algorithm that tries to annotate everything with an inferrable type. If x = 1 is the only assignment statement for x, it's easy to deduce it's an int. You'd need confidence values for stuff like a = b.format(c), which is like 95% likely to be a string
Only 5% of programmers have enough goblin energy to define a custom class with a format method and then use it in an ambiguous context
 
There are a couple libraries that do that. We used monkeytype on Werkzeug. monkeytype run -m pytest collects runtime information from all the tests.
 
Ooh, neat
 
We still had to review it afterwards since we don't have 100% coverage and it's not possible to figure them all out automatically anyway.
 
8:18 PM
hey wassup everyone?
 
Not much, just gradually slipping into my afternoon torpor
 
I've been coding a simple word game in python (PyQt5). Mainly as a distraction from my latest really project, which has been giving me some trouble lately
 
Highly relatable
 
I think i may need to to some serious refactoring. just the entire layout of the code is giving me problems
 
Wanting to refactor your entire project is a good sign that your design skills have matured since you started the project
If you know what you want to refactor it into, even better
 
8:26 PM
thanks for the encouragement ;-)
 
I like to tell people that their labor is constructive and meaningful, because I too wish to hear those sweet words
 
in case you were wondering, it's a bible verse memorization tool. right now it's able to look up verses from the kjv bible and print them to a QTextEdit. It saves the loaded verses to a .json file and persists them on the next start. the part I am having trouble with rn is the part where you actually practice the verse XD. it functions perfectly as a tool to lookup verses, but you cant actually memorize them using the mini games yet.
 
Nice. I've always wanted to make some kind of spaced repetition flash card app
 
cool
ive heard of an android (maybe ios) app that does that. i think it has an API, so you can just make flash cards and have it use its own algorithms for deciding when to show the cards. i might try doing that (with the API) when I'm done with this. of course it would be fun to try to implement something like it yourself too, I'm sure.
 
The kinds of things I'm interested in learning (katakana, morse code, rot13, rubik's cube solutions, the NATO phonetic alphabet...) can't always easily be represented with ascii text though so I'd have to throw in a couple kitchen sinks to get everything working
 
8:38 PM
lol
 
Rubik's cube algorithms are best learned by hammering the muscle memory into your fingers so I'd have to rig together a frankencube with, like, Lego MindStorm servos on the inside
 
8:59 PM
@Kevin Your work is meaningful and noticed, even if that which is noticed is not economically compensated. ;)
 
(◕‿◕)
 
@ThaddaeusMarkle One suggestion: is the Bible translation abstracted such that you could dropin replace it with a Spanish translation, SimpleEnglish/Kids translation, or Koine? This might be useful.
 
9:47 PM
cabbage
 
 
1 hour later…
10:56 PM
hi, im new to python and have couple of questions when trying my first python script. Im basically calling an API using python requests and writing the response to a csv file.
 
Ok? What's the question?
 
`import requests
import logging
import json
import sys
reload(sys)
sys.setdefaultencoding('utf-8')


url = "xxxx"

payload = {}
headers= {}


## Get Response From API

response = requests.request("GET", url, headers = headers, data = payload)
decoded_response = str(response.text)
data = json.loads(decoded_response)

## Write Response to File

file = open("xxxx.csv", 'w')
for row in data:
for i in range (0, len(row)):
file.write('%s' %row[i])
if(i != len(row)-1):
file.write(',')
 
Where are those exceptions being thrown?
 
Keyerror is at file.write('%s' %row[i])
 
A couple things:
1) The whole request thing can be simplified to `data = requests.get(url).json()`
2) `'%s' %row[i]` is a silly way to write `str(row[i])`
3) `file.write` only takes 1 argument, yet you're passing 2: `file.write(response.text.encode('utf8'),'\n')`
4) The file is opened in text mode, so there's no need to encode anything
Is row a dict?
Oh, I just realized you're writing a csv file. Use the csv module for that
 
11:11 PM
@Aran-Fey yeah row is a dict
 
import csv
import requests

data = requests.get(url).json()

with open("xxxx.csv", 'w') as csv_file:
    writer = csv.DictWriter(csv_file, fieldnames=['foo', 'bar'])
    writer.writeheader()

    for row in data:
        writer.writerow(row)
^ that's all you need
No clue where you got the reload(sys) thing from, but forget you ever saw it
 
@Aran-Fey Looks like this is straightforward. Let me try this and will update on the outcome. Thank you
 
Hi, didn't realize you have a chat here, I've made a post, maybe I can link it?
 
We ask people to wait 48 hours before linking their SO questions here, see the room rules
 
11:37 PM
@Aran-Fey, i got the same ascii error after trying your version.
Traceback (most recent call last):
  File "so_test.py", line 13, in <module>
    writer.writerow(row)
  File "/usr/lib64/python2.7/csv.py", line 148, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 8: ordinal not in range(128)
 
Huh, looks like your default file system encoding is ascii? That's super weird. Try with open("xxxx.csv", 'w', encoding='utf8')
 
:50352678 Traceback (most recent call last):
  File "so_test.py", line 8, in <module>
    with open("xxxx.csv", 'w', encoding='utf8') as csv_file:
TypeError: 'encoding' is an invalid keyword argument for this function
 
Uh, what's the output of print(open.__module__)?
Oh lord, python 2
 
@Aran-Fey builtin
 
Uhhhh... upgrade to python 3?
 
11:46 PM
@Aran-Fey may be, but not right at the momemt
 
@Aran-Fey ok
 

« first day (3607 days earlier)      last day (1327 days later) »