« first day (3790 days earlier)      last day (1383 days later) » 

00:52
Any idea why: self.assertEqual(len(get_count(['','','somevalue'])), 1) is expecting 0 instead of 1?
get_count returns list(filter(None, string.split(',')))
01:47
@Aran-Fey does this support tab to complete in python repl? the only alternative for that I know in windows it to use ipython
 
2 hours later…
 
1 hour later…
04:29
I have values like [105.32043301]
[-71.08949795]
[62.54161414]
[114.1538528]
[-16.35550116]
[25.38564488]
[-47.17892941]
[-47.82122656]
[25.6134267]
[178.07654254]
[23.77805946] and I need to store it in an array but whenever I do this array = [] and append to it I get [array([105.32043301]), array([value1]), how do I get a simple array, I believe `array = []` is a list I suppose?
user11282064
05:12
Hi all, I need an urgent help on executing Hive query using Spark. The query is "SELECT MAX(date) FROM schema.tablename". Here, the table is not regular table, it is a view table. when I run this query in my Python script using Spark engine, I get an error. Kindly tell me how this can be solved.
05:37
Hi Beginner, I noticed you are new here, a log dump like that could instead be in pastebin or dpaste
that is a huge log, you will have better luck if you can narrow the logs to what you think is relevant
that has a username password in the log as well, a mod can help you delete it, unless its a placeholder, idk if it is accessible outside
 
1 hour later…
06:41
@AndrasDeak I want the true answer
 
1 hour later…
07:48
@python_user No, but it should be possible to enable that with rlcompleter
08:03
that is neat, thanks
user11282064
08:26
can anyone help me with deleting my message? whom should I contact? I couldn't delete it
@Beginner done - although it's been around a while by the looks of it...
@AshwinPhadke Can you clarify what you want? Do you need a array([105.32043301, -71.08949795]) (numpy array) or just a [105.32043301, -71.08949795] (list)?
user11282064
is there any way to get help/solution for my question posted here? I am struggling with this for past 2 weeks, which has to be completed soon. Please help me on this
@Beginner It's a tad difficult to help with a question that hasn't been asked, on an error that hasn't been shown.
08:56
cbg
So he claims that having the proto files inside allows you to just git clone and then you dont have to compile the things again. I mean I see the appeal kinda, but it also feels wrong. The artifactory thing somebody posted yesterday is a much better way to go, but I don't think we have the scale to justify it. Well luckily there are different changelists in pycharm
Ugh, pycharm automatically display != as another symbol. It's weird
@Hakaishin Still sounds a job for LFS at least.
But it's probably not worth pushing.
09:33
cbg
@Hakaishin I'm not a fan of that beautification. I think that new char is also only 1-char wide, so visually I have to re-look at it to make sure it's what it's supposed to be
@Hakaishin does it also beautify >=, <= and == ?
(beautify is the wrong word here, coz normally beautifying/prettifying code is a good thing)
ugh, it does and it is honestly less readable
<=< this is just a weird looking arrow
Hey guys so I am busy working on my first scraper and working through a tutorial but found that it isnt adding the items to the dataframe. I have played withit individually before writing the entire scrape code which works with taking the correct items, but i think it is how I am saving each to the df at first i tried
df.loc[num] = job_post
then went on with this dpaste.com/8PVUAH6GN
i think perhaps I am appending to the df incorrectly?
@aneroid @Hakaishin they are called ligatures, there should be settings to turn them off, or fonts that don;t support them. I personally like them a lot, in particular because I let black do my space formatting anyways
09:54
@Kwsswart good self-catch. df = df.append(...) since .append() returns a new dataframe. it's not like Python's list.append which appends to the list in-place.
@Kwsswart consider enabling syntax-highlighting when you upload the paste - under "syntax" select one of the Python options
@aneroid thanks mate changed that seems I have got i something else as went placing prints throughout here dpaste.com/49NNP5FPQ and the start and links printing but not posts... is it possible that the site itself may be blocking the cumulative scrape? as when scraping a single time with the functions dpaste.com/2KQ9D2L3M it worked but now isn't
@aneroid Will do mate
@Kwsswart check out these two links you posted and let me know if you can read the code in them
@aneroid didnt realize here fixed with the functions at the top and current scraping look at the bottom dpaste.com/857N3H968
@Arne Thanks, I learned a new word :) I like them for pretty writings like books and labels in buildings and all decorative stuff, but def not in code
@Kwsswart is print(job_post) printing the correct information?
10:07
no thats what I am trying to figure whats going wronghere when printing page im recieving <Response [200]>
@Hakaishin fair enough. btw, in pycharm:
ahh I see
when printing print(soup.prettify())
It's already off, but I still have them :D
but only in 1 project, it's weird
is there any way to get around
hey, I'm facing an issue with socket io , could anyone help?
I keep getting this error even though the version is satisfied.

`The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)`
@Arne when in code, hard-pass for me. going through this only helped strengthen my opinion - betterwebtype.com/articles/2020/02/13/… the only one which is okay is the => arrow on the first line of each snippted. hard pass on the rest
@Kwsswart no*, that's literally what captchas are there to prevent
@Hakaishin maybe an .idea folder that overwrites global settings was committed to the repo? if it's not that, I'd just accept it as one of life's mysteries
10:13
*no == one could write an ML/AI prog that defeats captcha and integrate that into your code
@aneroid Damn well atleast I have an idea of how web scraping works ^^ got my second interview later for a junior python developer position and within the job description one of the responsibilities would be to build scrapers so thought would try
@Arne the weird thing is it was not like this I opened it last time and I did nothing with git. I think it will fix itself on the next restart :P
first one so pretty happy that managed to get working initially before being hit
@Kwsswart consider lowering your scrape rate. wait 10-30 secs between each. and before that, wait 15 mins to an hour for your IP to probably be un-blocked
@Kwsswart probably also worth checking out scrapy.org at some point
10:15
@aneroid cheers mate will do, before hosting this code on github is there anything I should be aware of or remove?
@JonClements Thanks mate will take a read through that now
@Kwsswart i didn't notice any passwords or confidential info. so if it's to demo your ability, should be fine
Yeah no confidential info always nervous though as didnt realize my mistake on that note prior ^^ and don't want to repeat mistakes
as long as it's "working", it's fine for a junior position
 
1 hour later…
11:25
@Hakaishin No way to switch the alternative renditions off? That sounds intrusive enough to need user control.
@python_user it's better not to make that observation publicly if the credentials seem real. Raise a mod flag yourself next time.
@Hakaishin well duh
@AndrasDeak it didnt occur to me, my bad, wont repeat it, I did raise a flag, but should have not said it
It's alright, it's not your responsibility. I meant to use a more FYI tone, sorry :)
but if you're worried for the user it's better to be overt
@Mina sorry, that message wasn't aimed at you
11:44
all cool :D
@Kwsswart Knowing something about the techniques is unlikely to hurt.
@holdenweb That was my thought process something is better than nothing and even if it doesn't work out well I have learnt something new
 
1 hour later…
12:56
@Arne I solved another one of life's mysteries :) intellij-support.jetbrains.com/hc/en-us/community/posts/…
the last comment was it for me
to be fair, an IDE is downright boring if there is only a single place to set a setting 🙄
for all its failings, the search-function in pycharm's project settings is quite good at finding stuff.
It's like if my files start wit foo_xxxx.py I want these set of settings if they start with bar_xxxx.py I want completely different settings... I don't see why this was split into two places for library files but whatever
@Arne it's amazing yes
Yam, Python is only #4 on codecov's per-language coverage. Beaten by Clojure, JS and Ruby!
Keep on writing unittests, folks!
link?
13:08
damn 13.5% 100% coverage is quite high
fortran is still growing... :O
Objective C is growing? thought Swift was there to replace it
13:25
can anyone clarify this for me, formatting sql queries with with the + operator is "sql injection" or is there more to it?
the concatenated string has inputs from user
using + with your own stuff ok, using + with user or unknown inputs = bad
then yes dont use +
ok that is what I needed to know, thanks
14:19
@python_user if it's not the database engine who puts the value in the query it's an injection vulnerability
whether it's + (string concatenation) or str.format or % or f-strings is irrelevant
@AndrasDeak but prepared statements are the best defense right?
I am going to use your first point when I bring this issue up, that seems like a definition without too much technicality
@Hakaishin if the prepared statement uses user-input values in the "wrong places" then it's still vulnerable. what you need to do is have placeholders.
also, obligatory xkcd - xkcd.com/327
Just use template strings. No one remembers how they work. Perfect injection defense.
I was told not to use a library and construct queries by formatting, reason = new update might break the insert() or update() function:/
14:23
That is... unlikely.
I couldnt really tell anything, library in question pypi.org/project/python-arango
they use the library but only use the .execute method that expects a query, it seemed unproductive from the start, might as well call the REST endpoint and save a dependency
14:46
morning cabbages, folks
cbg
@Hakaishin well that's a bit like saying that the best defense against injuries related to juggling chainsaws is not to juggle chainsaws
refuels chainsaw of +3 against chainsaws
@jamjam46 please try again
14:59
.strip(y) :p
anyone use requests here?
I do to some extent, but you can go ahead and ask
@jamjam46 hello. If this is about your fresh question on the main site please read our rules before asking.
@AndrasDeak nope its something else
OK then.
15:02
if i log in using a session by using a post request. then get another page by using the get method. will I still be logged in?
like all on the same website
15:19
Oh what a goose chase, a pc couldn't get to the internet, after some debugging it turns out the ethernet cable was unplugged from the other side :D
sounds wild
first wrong wifi, then internet over iphone charger cable, didn't even know that's possible, then unplugged cable :D
wrong wifi and ethernet...?
yes :D
he was in the guest network, which ofc doesn't have access to internal resources :D Neither does the internet(the iphone connection) :D
"internet over iphone charger cable" is everything wrong with the 2020s
(just kidding, tethering has been a thing for a long time)
15:22
xD I do agree though :D
I remember that I tried this years ago and it didn't work
am i the only one that gets spammed by friends to "Hack Instagram accounts" and "fix router problems" just cuz I code
most of my friends can also code
yes, I never had such friends :) And that too :P
Also I just say no
My friends need help accesing their emails : (
My high school didn't teach us anything about computers/coding. All I learned from high school was how to download photo shop ;p
15:37
I think my friends suspect that I don't know how to code
When anyone has tech problems, they prefer to ask, in order, 1) Friend 1, who does IT; 2) Friend 2, who went to med school; 3) me
I can understand Friend 1 having top ranking, but I don't think network security was in Friend 2's curriculum
@jamjam46 not at all. I sometimes even get the "can you please call customer/tech support for me because I don't speak the [(techie)] language?"
@inspectorG4dget well they do ask me that sometimes as well. Even for the simple stuff. For example the other day my friend's PSN account got hacked and he didn't know how to start a live chat with Sony. I sent him the link and everything but I ended up doing it for him
but tbh i do actually like it. It makes me keep in touch with old friends
15:53
yeah, I'm not exactly complaining. I have some technically capable friends, and only a VERY select subset of the remainder have the "ask me to call customer support" privilege
The trick to not getting these requests is to have more learned helplessness than the other guy
or going into "let me teach you" mode immediately
"hey Arne, how do I install drivers for this printer?" "Well, first you need to know what a transistor is."
3
so I usually go the teaching route - "you're going to learn this now so you don't need to ask me next time". But I do have some friends who can't grok tech (beyond click on this series of buttons). For them, I either explain the recipe (and am never bothered again) or I just do the thing (which I'm called to relatively rarely)
@Arne xD I do this to my mom all the time. First you need to know what a bit is :D
16:49
hey guys, stackoverflow.com/questions/66412420/… its my question on main, eligible for bounty in 1 hour, I dont think it is breaking rules by too much, can anyone suggest why it got no attention?
Looks like a fine Q, maybe some readers got spooked because it requires a third party library
cabbage
@python_user You need to add "give me the codez!" to get more engagement
lol, but I guess Kevin has a point, I was afraid people would not install PRAW let alone the async version of it
Is 'praw_ini_name' a placeholder? If the reader has to provide their own credentials, that can discourage answerers too
Not much you can do about it, unfortunately
yeah it needs an .ini file, api keys from reddit, its free though but user has to go a step to get one
that is as much Minimal I could get :(
one could also pass the keys as parameters, but your point still stands
whoever upvoted it, Melon much :D
17:01
Guys, is there a clean one/two command way to take a variable and: if it is a list (of strings here) return itself, if it is a single string return a list with that string in it. It's easy to see a method for doing this but I figured there might be a more pythonic way
if isinstance(variable, string): variable = [variable]?
@python_user yes, you have a reasonable MRE with some additional steps needed by anyone who is curious to set it up. It seems like you can't do much more really.
is it a bad idea to create a throwaway account and API linked to the Q?
I don't think there's a rule against it, but readers will probably think it's silly to do that
@python_user i mean moreso something that can be used at the time of inputting to function, cant do that with conditionals
17:04
@Skyler can you show a more fleshed out example of what you have in mind?
how about the function itself does something similar to what python_user suggested after the call?
@skyler perhaps you would be interested in the expression form of a conditional, a if b else c, which in this case would look like my_function([variable] if isinstance(variable, str) else variable)
If I understand what you're asking for
well, I have to say Poetry is my new found friend
so much better than vanilla pip
That said, ideally you would not have a variable that is sometimes a string and sometimes a list of strings. Even though Python is dynamically typed, most of the time you want a variable's type to stay the same throughout its lifetime, across all branches of your code
17:09
we do see pandas kind of regularly do something like this, for columns to do operations on it accepts a string for a singular column or a list of strings
Yeah it's one of those rules of thumb that you can ignore if you're confident you have a good reason
so I'm curious what the best way to follow that was, seemed common enough that I was wondering if its almost as easy as [] or list()
I see this kind of thing most often as a parameter that will allow multiple types. I assume that under the hood it has code that massages all values to a uniform shape, similar to the suggested if statement earlier.
For example there's a good reason that json.dump accepts a list or string or dict or number, and that reason is "because it would be a pain in the butt if there were four functions"
[variable] if isinstance(variable, str) else variable is about as easy as you can make it
yea, i wonder if that's what pandas does under the hood tbh
17:15
"seemed common enough" -- perhaps it is in beginner-to-intermediate code, but not so much once you get in the habit of strong typing best practices
I'm also mildly curious how Pandas does it. Do you know the name of a specific function that accepts a string or list of strings? I could look up its implementation.
how pandas groupby describes its function inputs
Parameters
----------
func : callable
A callable that takes a {input} as its first argument, and
returns a dataframe, a series or a scalar. In addition the
callable may take positional and keyword arguments.
args, kwargs : tuple and dict
Optional positional and keyword arguments to pass to `func`.
kinda tough to follow
@Skyler I don't follow. How is this related to the earlier conversation about a variable that may be a string or a list?
@Code-Apprentice the pandas groupby function is an example of a function that takes a column (as a string), a list of columns (as a list), or even a series (of length equal to the length of the dataframe)
the third case it does probably have diverging behavior, but the first two are almost identical in many ways so it probably just treats the string as a list a somewhere early in the code
@Skyler ok, thanks for the explanation. I didn't look very closely at the github link and your quote from the docs didn't include those details. So I was confused.
and yah, turning a single item into a list with a single item seems easy enough so that you can treat both cases similarly.
Slowly wading through the groupby source... It's messy in here
17:25
head feels like spinning whenever i try looking into pandas source
most other source code is instructive for learning the library
Ok uhhh I'm 60% sure that github.com/pandas-dev/pandas/blob/master/pandas/core/groupby/… is responsible for groupby's ability to take lists and nonlists
if not isinstance(key, list):
    keys = [key]
    match_axis_length = False
else:
    keys = key
    match_axis_length = len(keys) == len(group_axis)
As you may have suspected, it uses isinstance and a conditional that sometimes wraps the object in a list
so basically the same as python_user's earlier example
Yeah
lol at line 730:
    # what are we after, exactly?
I ask myself that every day
17:29
developers getting existential on us all of a sudden
a byproduct of working with pandas source code no doubt
@Skyler any_callable :P
but man pandas can be convenient sometimes, like I was asked to write a data splitter/merger/updater script for several datasets, and they waited until after I finished everything to mention that some of the datasets have no unique identifier/key columns
pain
@anky I guess at least that programmer has an easier life then most
I am not a programmer so can't say :)
17:34
luckily though I had just 1 function that I had to do anything more then a minor rewrite because pandas can pretty gracefully use a list of columns to create a unique descriptor
the one thing I despise about their codebase though is that so much of their code is built to fail silently
just add a log entry somewhere and chug on, and its not even any kind of online application
looks like reduce+merge
which would be fine if they were doing a ton of parallel operations, but so much of the code operates in serial that having to fish for a failure that happened 80 steps ago and may not even be where the error log spots it is joy inducing
enough time spared for a mcve
they don't wanna learn
@Skyler you spelled headache wrong
Laurel that was good :P
17:45
@Code-Apprentice just crack the head open at that point
basically any time I adopt a bit of their conventions to their code it takes 5-10x the time to debug
so now i build everything that can be reasonably done out at the module level before pulling into scripts where they can just nod and accept my test code coverage as a verification of it's functionality
18:01
so you contribute to pandas?
on another topic...anyone here familiar with Swagger UI? I'm using it to generate docs for my rest endpoints in a Django app. The "Try it out" buttons aren't quite working because every response comes back with "Authentication credentials were not provided." So now I'm trying to figure out how to include a token in the example requests.
@Code-Apprentice hard pass, I think I reported issues and suggested improvements to a spin off library of sklearn once
Who is the "they" that are nodding in your previous comment?
coworkers, those are more in-house codebases for data science/analytics projects
oh gotchya...so code you are writing for your custom work, not for pandas itself.
yea
18:07
@python_user if you are just interested in knowing why the loop = ... works and the asyncio.run does not, that is straightforward to answer. If you are wondering how to fix it, that is probably more complex – my hunch is that the library is simply bugged for that invocation. Deterministic closing with asyncio is a pain.
tbh you can see a big difference between new specifications they asked me to create vs stuff they asked me to refactor. A lot of tied hands on design choices and way too much hardcoded stuff for a "refactor"
18:28
hey everyone
I have encountered a python behaviour which is surprising for me
@MisterMiyagi just a list
@AshwinPhadke Where does the numpy.array come from, then? Neither your input nor desired output seem to have it.
@MisterMiyagi thanks ! it is highly related indeed.
@MisterMiyagi need_the_result_list_here = np.dot(abc, Xyz[i, :])
18:42
@zabop please don't ask for help here with fresh questions on the main site as per our rules
@AshwinPhadke I don't see how this relates to what you showed earlier. Was this your attempt to concatenate the lists?
I give it 75% chance that numpy.matrices are involved
@MisterMiyagi the result I showed you is the one I need to store as a list in the variable "need_the_result_list_here" which comes from doing a dot product of abc and xyz[i, :].
@AndrasDeak matrices are involved indeed
@AshwinPhadke there are too many moving parts here. Please show actual code that corresponds to your issue.
preferably minimalish, definitely self-contained
@AndrasDeak is numpy.matrices the data science equivalent of asyncio?
18:50
Unless I'm missing a pun: it's just the plural of numpy.matrix.
I refuse to not pluralise that word properly.
@AndrasDeak pastebin.com/XfLCSvJJ here ya go
@AshwinPhadke np.shape means the contents can still be anything. I need better.
I think MM is referring to the headache inducing complexities
actually, hold on, np.array(w) is OK
so I only need to know what X_test is
type(X_test), X_test.dtype is a good start
X_test.shape == (1, 64) but then you have Xtest[i, :] where for i in range(0, len(w)): with len(w) == 242. Hard pass.
@AndrasDeak <class 'numpy.ndarray'>
18:52
@AshwinPhadke Let's continue when you have a snippet of code that I can run and see what you're seeing.
@AshwinPhadke It sounds like you need to learn the difference between a list and an array in python.
@AndrasDeak I did not get what you mean by hard pass
@Code-Apprentice I did read on it yesterday
@AndrasDeak sorry it's len(xtest[i, :])
@AshwinPhadke it's OK, but I no longer believe stories. Runnable complete example, nothing less.
I don't have 15 minutes to look at unreliable code only to be told that "oh, that wasn't actually right, sorry"
Come back with code that I can run and see what you're seeing, and I'll help.
you may even ping me when that happens
Yeah sure , I'll make some runnable code and get back
 
1 hour later…
20:21
anyone have a favorite way to split a 'CamelCase' string to substrings? ['Camel', 'Case']
sounds regular to me
Does re.split support splitting on zero-length matches now?
I'm debating amongst myself if I'd rather loop and str.isupper before I go re
@AndrasDeak Yeah, but there's no regex for "upper case unicode letter"
@Aran-Fey sounds like identifiers to me. But fair enough.
In [23]: re.split('(?<=.)(?=[A-Z])', 'CamelCase')
Out[23]: ['Camel', 'Case']
USA-compliant version ^
Make sure to consider what you want to happen for inputs with multiple sequential capital letters, like "FlaskWTForm"
20:27
edge cases are best left ignored
all my cases are squarely in the middle of case-space. so ignoring is the preferred option
also AD thx, that is too pretty not to use. I was going to use a generator but nah
21:10
@MisterMiyagi In your answer, you say "the event loop is not available for cleanup once main finishes". What I don't understand is why anything would need to be executed after main finishes. The Reddit object was already close()ed, so what's left to clean up? Is that just a bug in asyncpraw?
21:42
elders: I come seeking best-practices wisdom for maintaining a database.
When a live change is made to the database, should Option1: the original schema.sql be changed and pushed to VCS, or Option2: should the changes be saved in updates.sql and pushed to VCS separately?
22:10
alembic <waves hands wildly and flies away again as it's probably not what you're after>
@inspectorG4dget How are you connecting to the database? Are you using sqlalchemy or some kind of ORM?
@Code-Apprentice psycopg2
I ask because with the Django ORM, you generate "diffs" in the schema and commit each of these diffs.
I wish I'd thought to do something that sensible
You can still define the database schema with sqla and not make any use of it other than with alembic
22:20
In raw SQL, these would be the original CREATE TABLE then each UPDATE TABLE statements.
yes. I have schema.sql that has all the CREATE TABLEs and I have update_schema.sql that has all the UPDATE TABLEs. This is the Option2 I mentioned.
Option 2 would probably be the way I would go then. As well as a way to manage which ones need to be applied when you deploy.
assuming you are deploying this to some kind of "production" database
correct. My argument (at work) was that a bash script could execute schema.sql and update_schema.sql in sequence should we ever need to replicate it
thanks for the confirmation
Are you planning ahead for any future changes? Say update_schema01.sql through update_schema10.sql for each of the next 10 releases?
If so, I suggest leading digits for sorting and more descriptive names: 01_initial.sql, 02_add_foo.sql, 03_move_bar.sql, etc.
A bash script like you are thinking will be helpful in rebuilding a database from scratch. You also should consider how do you know which updates need to be run against the current production that already has some of them applied.
and what if you need to roll back one of the updates

« first day (3790 days earlier)      last day (1383 days later) »