« first day (3175 days earlier)      last day (1998 days later) » 
00:00 - 20:0020:00 - 00:00

00:39
I guess paritosh has a bengali origin, cuz it does sound like a word in Bengali
00:51
cbg
 
3 hours later…
03:42
cabbage
 
2 hours later…
05:29
recbg
recabbage
 
2 hours later…
07:20
hi, im trying to use the ? (help) functionality in jupyter notebook - does this work with pandas syntaxs? e.g. ?pd.sort_values()
it should i believe. help should not depend on package.
i get the error Object pandas.sort_values()` not found.`
Make sure you aren't using the help with the brackets. it should be pd.sort_values
same error
did you import pandas as pd first?
and run it?
07:26
yes i am currently using pandas as pd
ok maybe cause im getting using the syntax as i want it
okay, in that case pd.sort_values really doesn't exist i'd assume
you made me google. :/ link
it's a method on the dataframe. use ?pd.DataFrame.sort_values
sorry - saw that and i think because i was using the name of my dataframe it didnt work
thank you
no worries
07:30
i dont see why pd works but my syntax im using doesnt... makes help function less effecient
what do you mean?
ok thanks i think i understand now.
07:38
guys, can I ask a question in the chatroom?
yes, though take a second and make sure you're familiar with the room rules
08:05
Thanks
08:16
What is the input shape of the lstm if the data frame is a one hot encoded vector format(288*74)
08:42
i am trying to convert a column of dates to a string so i can remove the - or / in some of the values
so i first do .. astype(str)
then a replace('-','')
but the value still comes through with -
gonna try to get only digits
got it
good chat :)
09:05
@ThelurkerLurker XY problem, use strftime
09:15
rbrb
10:01
if my bashrc sets 2 env variables like $a and $b, how can i source .bashrc in my python script and then do os.getenv("a")
sounds like you're mixing "variable names" and "data".
example if you do os.environ['HOME'] it shows home directory
sorry my bad
suppose i set 2 variables in bashrc
how can i access them in python
i have tried the solution on a question for removing items from a pandas df from another one but my results do not show as expected. I am hoping someone can point out my mistake?
16
Q: How to remove a pandas dataframe from another dataframe

176codingHow to remove a pandas dataframe from another dataframe, just like the set subtraction: a=[1,2,3,4,5] b=[1,5] a-b=[2,3,4] And now we have two pandas dataframe, how to remove df2 from df1: In [5]: df1=pd.DataFrame([[1,2],[3,4],[5,6]],columns=['a','b']) In [6]: df1 Out[6]: a b 0 1 2 1 3 ...

exclusions = pd.concat([corrections,data]).drop_duplicates(keep=False)
data.shape
(348118, 26)
corrections.shape
(9313, 26)
exclusions.shape
(343392, 26)
348118 - 9313
338805
@pythonRcpp If you can't access them through os.environ then your bashrc wasn't sourced. That's not a python problem, that's a problem with your linux
my exclusions df is 4587 rows different
10:11
You're basically asking for guesses here because we can't see the data. How do you know that every entry in data is exactly in corrections?
^ this is most likely the issue.
i see. thanks
although i have found at least a few examples
A few examples of what? We really can't help with vague statements.
i think he means he found a few cases of mismatching rows
aka good to go, crisis averted
10:26
I assumed as much, but I think we'll struggle to pin down any issue to help with
Oh wait, I actually read it the other way, that there's a few examples that should have been removed. I'm a bit discombobulated; I had the bright idea of "I'll just fire up that server we wrote in Java a few years ago" this morning. Good lord the code is awful and I haven't learned a single thing in Java since it was written
sounds like a typical Java session
Good thing python exists :)
This was when I was assigned a "software engineer" to help me on this project. It's his code and my logic.
But looking through it now, just knowing Python OOP better than I did then, I haven't the slightest clue what he was thinking. Half tempted to try cobble something together myself but... Java
Fwiw, i presume almost everyone looks back at their own code and feels the same way. Maybe he would too if he saw it now
It's pretty amusing revisiting old code
Probably true
Eclipse is lit up with warnings about variables that never get used. It's like Blackpool Illuminations on my telly
10:41
but roganjosh, i swear i might need to use them in the future!
int[] storeActivity = new int[1]; never used, but looks like it might be to get around some scope issue, so probably for the best :P
is that an array of int declaration?
Of size 1
yeah nope, i think python has spoilt me. i like not having to specify datatypes
Since it was never used, I don't know what it might have been an attempt at. I also don't know the scoping rules of Java very well (though I may have to learn now) but it seems the equivalent of putting an int into a list just to get a mutable type in globals
10:52
sourcing means *executing* a bash script, so you would basically need to run bash code from python
that's not generally possible
what you can do is spawn a bash process that sources the script and writes the values back to you
e.g. by calling env to write them to stdout
Max
Max
11:10
Hey peops, I have a short question, that I couldn't find a proper answer in the search, but I'm not sure wether it's worth of posting a question: I'm new to python and trying to run the debugger in pycharm, but nothing happens. My IDE just shows "connecting to console..." in the bottom where they say which processes are running. What is happening or why is it not working?
11:36
Does anyone working with numpy have insight into these timings? np.where is dropping below a list comprehension?
11:54
@roganjosh just a guess, but tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp'] might take a while to build
the list comprehension only builds strings as required
basically the comprehension is streaming once, whereas the np.where builds everything and then filters it
@MisterMiyagi but shouldn't np.where also be building as required? The link from jezrael kinda suggests that both possibilities need to be examined?
no, numpy where builds the full array i think
and i imagine it does not like the fact that the rows that it has to deal with have NaNs in it
that alone probably can explain the added cost of building the whole array
Ok, then my understanding is completely off
let me just confirm..
it's part of how numpy does most its operations though, you can't vectorize something that you have to process on-the-go.
In what situation does np.where require both series and then cherry-pick the results you want?
I kinda assumed it was syntax pushing the if check into C that just builds a single array
12:00
that statement confuses me. np where* in 3 arg form is doing the "cherry picking" from 2 arrays that are fully constructed
But the equivalent Python loop would only build one list, wouldn't it?
indeed.
if there's anything i've understood when it comes to numpy, it's that you're almost always making a space tradeoff to gain speeds
But this suggests you get neither? Let me re-read the comments
loop vs vectorization can essentially be considered two paradigms at odds with each other.
mmm, I think all 3 of us might be on different pages here
12:04
%timeit tpl_subset['Tmp1'] = np.where(tpl_subset['Infrasp'].isnull().values, tpl_subset['Fullname'].values, tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp'].fillna(""))
470 ms ± 8.34 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit tpl_subset['Tmp1'] = np.where(tpl_subset['Infrasp'].isnull().values, tpl_subset['Fullname'].values, tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp'])
561 ms ± 5.09 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
it;s the Nans
i think.
Sorry, but those timings seem even more backward!
give me 1 sec
I'm not disagreeing with you btw, but doesn't that outcome seem nuts? :P
here's the bottom line issue
tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp']
runs fine, but is probably jumping through some overhead. because of this:
tpl_subset['Fullname'].values + ' ' + tpl_subset['Infrasp'].values
This will blow up. The Nans are probably being specially handled in the series concat/string sum or whatever you want to call this operation.
considering not a whole lot else happens in the call here, this overhead is probably enough to account for the difference on its own
But they needn't exist, because that's the else branch. Which is why I questioned why np.where needs to make two arrays
12:14
think vectorization. I mean, let's pause for a second here and i should mention
on my machine, with fillna, this is the fastest operation out of the three
%timeit tpl_subset['Tmp1'] = np.where(tpl_subset['Infrasp'].isnull().values, tpl_subset['Fullname'].values, tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp'])
683 ms ± 56.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit tpl_subset['Tmp1'] = np.where(tpl_subset['Infrasp'].isnull().values, tpl_subset['Fullname'].values, tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp'].fillna(""))
464 ms ± 13.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit tpl_subset['Tmp3'] = [f'{a} {b}' if b == b else a for a, b in zip(tpl_subset['Fullname'], tpl_subset['Infrasp'])]
Hi all,
Few days back I have been asked a question related to which option will I choose if this is the problem statement and why?
2?2 = 4
options
1. + 2. * 3. ^ (power of operator)

I still did not get the right answer.Actually I had to choose only one and question is more related to execution time and compiler specific..
@ParitoshSingh can you time tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp'] in isolation?
sure, lemme do that
%timeit tpl_subset['Fullname'].values + ' ' + tpl_subset['Infrasp']
491 ms ± 26.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
%timeit tpl_subset['Fullname'].values + ' ' + tpl_subset['Infrasp'].fillna("")
384 ms ± 26.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
oh, i should remove the .values call too. sec
that's what I assumed - it builds the entire array
%timeit tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp']
502 ms ± 22.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit tpl_subset['Fullname'] + ' ' + tpl_subset['Infrasp'].fillna("")
341 ms ± 7.44 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
12:17
np.where just removes from that
aye. I do not know the internal working, but im willing to bet it's essentially like boolean indexing a 2d array to get a 1d array.
that kind of indexing is fast. feed a 1d array of 0s and 1s, and you're done
^^ yep and that's the bit that's now confusing me, because it fails in the timings and it can't possibly win on space.
it's only failing in timings because you're delegating work of summation to pandas series that's trying to figure out how to concat a string with a Nan.
once we eliminate that overhead, amusingly enough by adding a fillna operation, it's fast.
in the case of np.where, you build two arrays - the output and the temporary, full concatenated series
in the case of the list comp, you are building just the output
^ you essentially get to bypass the Concat for all Nan rows when looping
12:21
yeah
Hi guys
I have a pandas code which is taking like 20 -25 mins to process 60 million records?
Can it be optimized further? If so how?
columns = list(adr_data)
for i in columns:
    adr_data[i] = adr_data[i].str.strip().str.lower()
adr_data here is a dataframe
I think i understand where you're coming from @roganjosh My understanding could be a bit flawed, but here's how i've understood vectorization. the thing with numpy and vectorization is that they do not need to "loop and operate" one by one. (well, logically/semantically sure. but internally, it's just "prepped and fired" at once on every value in the array) . To the point that you still gain time even when you calculate "Extra" things. Only because you calculate things "At once".
It's a different paradigm compared to looping, keeping as few things in memory as needed, operating one by one.
I think your assessment is reasonable. It's just that the approach seems to fall apart with strings every time.
The only issue here being that the vectorization was hampered by the Nan values. It's like throwing a wrench at the system.
indeed. i've seen issues with strings in the past too
the problem is that you are doing things that you don't need
12:27
i have not spent the time to understand why that could be, but i've just intuitively assumed that the cost of operations when it comes to strings is much higher than just numbers
and then throw them away
instead of not doing them at all
my barebones assumption is that the array sizes get super mucked up when it comes to joining strings
I've seen list comprehensions beat .str methods on dfs before, so that part wasn't too shocking. But to find it in numpy... I think I need to re-evaluate my thinking on this
that kind of adds a lot of "on the fly" overhead with many string operations.
So, the approach of doing "Extra" work can bite you a lot quicker with strings than with numbers. Now, everything i'm saying at this point is just assumptions so take it with a grain of salt
but atleast to me, that makes sense.
Can anyone help please?
12:28
@RaphX How many columns do you have?
Let me check
is there a reason why you use .str twice?
@ParitoshSingh I'm on-board at this point, and it's backed by the timings. Doesn't mean that I'm not left slightly confused by the whole thing, but I think you're probably right :)
I think I can't call lower before str is called, also I guess the data is not getting imported as a string @MisterMiyagi
There are 42 of them @Kevin
@MisterMiyagi you get back a series object after finishing up the first operation. i've done something like this in the past
12:30
hmm
So, if there's a better way, i'd also like to know too. o.o
So topical :P Build a new Series with list comps for each column
@ParitoshSingh have to admit, the pandas API always confused me if I haven't used it in a while
12:32
i remember that trying to do a concat of 3 numpy arrays of strings was flat out slower than doing the same with looping. With numbers, you wouldn't even bat an eyelid trying to sum 10 arrays at once or so.
@MisterMiyagi oh same here haha. :P
cbg
@RaphX for future reference: "Can anyone help please?" is implied when you're asking for help. Repeating this question with no extra information comes across as needy and is much more likely to make people stop helping you.
Sorry @AndrasDeak
I notice that DataFrame has an applymap method that lets you call a function on each element of the df... I have a feeling that df.applymap(lambda s: s.strip().lower()) is not going to be faster than .str.strip().str.lower() though
apply is typically the slowest option for dataframes, but for some reason string accessor methods can also be very slow. Sometimes native python even outperforms df.str solutions.
@ParitoshSingh do you know if it makes a difference whether the strings are fixed-length or arbitrary size?
12:37
The fastest way will be to rebuild the columns in list comprehensions and assign them back. I think.
my strings were arbitrary sized when i tested it. That's probably a pretty good question to consider. i don't know offhand
If there isn't substantial magic going on under the hood* then I'd expect list comps to be about as fast as Raph's current code
(* but numpy/pandas is all about magic under the hood, so I look forward to being proven wrong)
also, related read, might be interesting. something called Vaex promises to handle strings really well. It's on my "want to explore" list.
@Kevin the .str methods invoke all sorts of magic
I'd assume that a fixed-size string is just a byte array, so it can be embedded directly into an np.array
whereas an arbitrary-size string is a pointer to some rich data structure
12:39
But if Raph's current code calls str the same number of times as the list comp based solution, then they're equally magical
@MisterMiyagi my assumption was t hat arbitrary sized strings are stored according to the largest string in the array, and require reallocation during operations as needed. (atleast that was my assumption when it came to concats)
i would imagine it's still in the realm of char arrays, and not a wrapper around it.
Numpy has max width string arrays. Pandas uses object dtype.
makes sense.
But still string ndarrays are often weird.
If we're calling strip on a series, then I don't have much hope of it being fixed size
12:45
The space allocated to each entry cannot increase, though
You could .str.replace to grow it for instance
Hmm, can lower cause a string to get longer? I think there are a couple unicode letters with peculiar behavior when it comes to casing
I don't think ß should decay into ss under lower
it's the same byte length either way
i think that would be casefold that is more aggressive with unicode
12:50
>>> s = chr(304)
>>> len(s), len(s.lower())
(1, 2)
Curse you, LATIN CAPITAL LETTER I WITH DOT ABOVE!
AFAICT that's the only unicode character that exhibits this quirk
On the other hand, a substantial number of characters get longer when you upper() them. pastebin.com/tkFiUccY
ah, scharfes s decays with upper
I wonder if Raph's code would be faster if he used a bytes-like type?
Then the backend wouldn't have to worry about unicode weirdness such as the last half a page of conversation
lower() is going to need unicode for correctness
13:02
I'd also try using a native python reconstruction of the column, in case that hasn't been tried yet (hence my remarks that sometimes native python is faster). adr_data[i] = (s.strip().lower() for s in adr_data[i]) or the same in a list comp if the genex doesn't unpack right
coldspeed claims as much in one of his famed canonicals stackoverflow.com/questions/54028199/…
@MisterMiyagi Hmm, true. If his code is unicode to begin with, then converting to bytes and calling lower and converting back to unicode is going to produce some junk in certain cases. It should be OK if his data is strictly in the ASCII range though.
13:20
Here's my hacky benchmark. pastebin.com/rhcD3wLu. Conclusion: bytey dataframes are a little faster than stringy ones, but not by much.
I don't know if this is a useful observation, since both of these dataframes have a dtype of "object". Is there a way to coerce that to a more specific type?
I don't see any stringlike types in pandas.pydata.org/pandas-docs/stable/getting_started/… so I guess not
Or is that table a listing of pandas dtypes, and there are more numpy dtypes elsewhere? More googling is required
Well, pd.Series([randstring() for _ in range(num_rows)], np.unicode_) crashes with ValueError: invalid __array_struct__ and I don't feel like deducing what that means, so I guess I'm done.
I choose to believe that my awkward fumbling has been of tremendous use to RaphX. Another job well done.
13:53
cbg should I flag for an unfortunate typo stackoverflow.com/questions/56773556/…
On the basis that it may be considered offensive? I reckon you should flag it if you personally find it offensive, but don't flag on behalf of a hypothetical reader
There might be unicode characters @Kevin
I don't find it offensive. But my 7th grade mind finds it distracting.
I came across some of them in one of my columns
13:56
@RaphX Oh, darn. Well, you're not missing out on much, if my benchmark is accurate (confidence: 40%)
If you used bytes then your program would run in 23 minutes instead of 26
:D
Thanks @Kevin
I will also try using a list comp as suggested by @AndrasDeak
I'd first try the generator expression since you have a lot of rows
Ok
creating an auxiliary multimillion-element list will probably hit your performance
14:16
I am currently trying to find a mixed strategy Nash equilibrium for a rock-paper-scissors game where the payout for winning with Rock is twice as much as winning with any other choice. I think I need to find the maximum of a function with two variables, and I learned how to do this in college but the class was right after lunch in a dimly lit room with no windows, so I slept through a lot of it.
I remember "if all the variables lie within a finite range, you can imagine all possible solutions as an N-dimensional solid, and the maximum is usually (always?) at one of the vertices (edges? faces?)"
Then a great deal of math ensues in order to find the vertex in less than O(vertex_count) time
I'm not sure we've discussed them yet, but RPS creates very nice spiral patterns due to the rock beats scissors beats paper beats rock cycle youtube.com/watch?v=vqnpl5G17Ws
There are only three variables* so I guess I could just brute force them all. O(8) isn't bad.
(*two, if you derive "probability that I will throw scissors" as 1 - p(rock) - p(paper))
14:36
Need help with this question - stackoverflow.com/q/56774557/8121719
15:10
I've now gotten as far as writing a brute-forcer that tries all 100^2 mixed strategies whose probabilities only have two digits past the decimal point. It says that if your opponent knows your strategy when he chooses his strategy, you should throw rock 26% of the time and paper 74% of the time. He'll choose paper 100% of the time, and you'll lose an average of $0.36 per throw. This strategy has the smallest losses among strategies I looked at.
hi I am new to Python but experienced .NET developer. Can you please suggest a roadmap I should look? For example, in .NET, I would use ASP.NET Core for Web App (backend) including DI, Entity Framework for ORMDB. What are the equivalent in Python? Are an app structure/design similar (say, Controller for backend, Views for front-end HTML, Entities/ViewModel/Service for business logic)?
This is surprising to me because I expected to find a strategy that has an expected value of 0, or very close to it. "rock paper scissors except winning using rock is twice as good" is a symmetrical game so I thought there would definitely be a mixed strategy nash equilibrium.
Granted I did not check every strategy, since it's hard to exhaustively iterate over every pair of real numbers, but I was hoping I would come close
Oops, I misread my program output - the actual loss is not $0.36 but rather $0.26. Still surprising though.
[End of boring game theory chat]
----------------------------------------
[Beginning of Python chat]
Hmm, it bothers me that if I do def f(*x):, I can't later do f(x=something)
15:30
I don't see why you'd want that
When I'm in a paranoid mood, maybe I like to use named parameters at all possible opportunities.
If you ever want to use a named argument, chances are the function shouldn't be using positional-only varargs in the first place
As soon as I submitted my last message the Good Design Fairy appeared on my shoulder and said "your vargs and kwargs should not have names related to your business logic; most of the time, calling them vargs and kwargs is sufficient"
I think Model View Control is a pretty standard design regardless of language
as for the respective framework, Google will give you much more information than what we could dump here
Thanks. I am quite surprise there are a lot (Django, Flask, Bottle etc). For .NET, it's almost ASP.NET is the to-go. I am looking at Django now.
15:42
Is .NET open-source? I thought it was a microsoft thing.
the latest .NET Core and their stack/frameworks are, and cross-platform as well.
this includes their C# language, .NET API design and implementation.
morning cabbage
@AndrasDeak Microsoft release .NET as open source several years ago.
@LukeVo Django is pretty good. I use it every day at work.
@Code-Apprentice thanks :) hope I will get a hold on Python soon!
@Code-Apprentice I see. Also open-source in the "open source community" sense?
@LukeVo as an experienced programmer, most of the basics should come quickly. Although there are some gotchyas here and there. And Python programmers use certain techniques that may not be the same as how you would do it in C#
@AndrasDeak Open source as in you can clone it from GitHub.
15:51
cool, cool, thanks
I don't know the exact license...it might be a custom Microsoft license even...
@Code-Apprentice thanks for the advice, I will surely come across these in time.
<brief return to boring game theory chat> Ok, so apparently P(rock) = 0.25; P(paper) = 0.5; P(scissors) = 0.25 is the best strategy in biased RPS with an expected value of -5e-17. But it's not a nash equilibrium because your opponent replies with [0.01, 0.12, 0.87] instead of parroting back your own strategy. Maybe not all symmetrical games have a mixed strategy equilibrium?
Here's a heat map of all possible strategies. the X axis is P(you throw rock), Y is P(you throw paper). Black means you lose 0$, white means you lose $2. Pink is the impossible zone where the sum of probabilities is greater than one. There's clearly only one peak of greatest expected value, so I don't think an equilibrium can be hiding anywhere else. </brief return to boring game theory chat>
16:42
<last one I promise> Oh, my code only picked [0.01, 0.12, 0.87] due to float arithmetic imprecision. With the strategy [0.25, 0.5, 0.25], any conceivable counterstrategy has an expected payout of exactly zero. And I can prove it symbolically: pastebin.com/LSnxE2EZ. So [0.25, 0.5, 0.25] is indeed the mixed strategy Nash equilibrium for biased RPS.</last one I promise>
@Kevin - it would interesting to turn this into a Monty Hall problem. After making your choice, the other player says "Well it's a good thing you didn't pick <other choice, either the loser or draw>. Do you want to change your choice?"
@LukeVo GL and welcome to the Python world
17:02
SO has reached a new low. I'm used to people turning other people's comments into answers, but apparently nowadays they choose to post worse or blatantly incorrect answers instead.
That's what downvotes are for!
17:35
self._i_hate_double_underscore_attribute_names = True
@PaulMcG did a coworker's code prompt this?
self.___trunders_are_better
Here's a formula for the mixed strategy Nash equilibrium of any RPS game with positive payouts. pastebin.com/Z5D9AEX2
I probably could have skipped like ten steps if I assumed without loss of generality that the payouts all add up to 1. Oh well.
17:50
@Code-Apprentice I'll have to plead the 5th (which I guess is just another way of answering your question). Although recently I've been doing more work with mixin classes and MI, and I can appreciate the value of dunder attribute names in a mixin class, to avoid stomping on mixed-in attrs. So I just searched my mixin classes, and none of them implement __init__, only inherit from a common abstract base class, so there are (so far) no mixin-specific attributes that need to be dunderized.
^^ (Using PyCharm to flag introduction of attributes outside of __init__)
I wonder if this implies that for any symmetric zero-sum game, there's a mixed strategy that has an expected value of 0 no matter what strategy your opponent uses. Or maybe things get wonky when there's more than three choices? Who's up for a couple thousand rounds of Rock Paper Scissors Lizard Spock?
I would think that three is wonky enough
I suspect wonkiness is exponential with respect to choice count. All the pages I found online explaining mixed strategy Nash equilibria used two-choice games as examples, and they all needed about three lines of work to prove that you can choose a strategy with expected value 0 regardless of your opponent's strategy. It took me 184 lines to prove the same thing for a three-choice game. Therefore, a five-choice game would require ten quadrillion lines of proof.
(I made that number up because my mental arithmetic logic unit has melted into slag)
math.stackexchange.com/questions/2452520/… contains many scary symbols I don't understand but I think it proves the principle for games of arbitrary numbers of choices
18:06
@Kevin 2 to 3 is probably a much larger gap than 3 to arbitrary
Even without the above link I'm pretty sure it generalizes. The formula for expected payoff never has any exponential terms, so you can always choose values for your strategy such that the remaining variables represent a horizontal slice through the N-dimensional solution space.
There's like five handwaves in that sentence. A personal best!
18:43
Well I am starting to reapply for jobs again, and was wondering what's the expectation like in the industry in regards to python knowledge
especially how much python is expected to be known in a job interview
If you determine the answer, let us know.
haha, well I am the one looking for an answer :)
Isn't that going to depend entirely on the job you're applying for?
my roles are primarily open ended, they look for general coding skills in the language you want to interview on, plus some language knowledge, based on the experience
Shouldn't take more than ~5 interviews for you to get a representative sample of what kind of Python knowledge the average interviewer is looking for. (which is not to say that you will have five interviews without getting an offer -- naturally you'll want to get several offers and take the best one)
18:48
if they focus on technologies or frameworks, it will be on a more general level, rather than language specific, atleast that's how it is in india
@Kevin yes testing the waters with interviews, but I was thinking what ground zero knowledge should I prepare myself with, before giving the first one
perhaps what i was getting at too is that lot of time in coding interviews, we can also make use of a certain module (collections, itertools), or a builtin function which can reduce the problem more easily rather then using the basic datatypes like list, str, tuple etc
So I am confused as to which of the two options to take
19:06
@DeveshKumarSingh Sounds like, at this point, the best option would be to put all effort into landing an interview. Then you'll have a baseline for what might be expected, as well as what the interview process is like. You clearly have what it takes to be a programmer. You just need to convince someone who is hiring Python developers that you will be useful.
actually I do have a few lined up but they are big firms, where once you screw up an interview, you have to wait 6 months for giving another
If a stdlib module lets you do something more efficiently, use it. If they want you to do it with just the builtin types, they'll specify that.
hence I was planning to interview for smaller firms to practice more, and improve my skills and then give these big firms
I dunno, if I'm in your shoes I do one purely as a test with no expectations. One of the smaller firms in this case
@Kevin okay, so it's a good question to ask before an interview, just builtin types or can use stdlib modules , which I have seen from SO can be used pretty neatly in lot of cases
19:10
It's a good idea to ask questions all through the interview as long as they're pertinent to the problem
Ideally, it should be obvious whether you're allowed to use the stdlib without you having to ask.
If it's ambiguous, subtract a point from the company's score for writing bad problem descriptions. Interviews go both ways.
Ask questions that can differ from one place to another. And keep the questions to things you actually care about so you can converse with passion. Ex. What version control do you use? What is the culture like?
^^^ Disagree. I've been to several interviews where the problem is unclear and it's deliberately set up for you to either ask for clarification or invent your own understanding and go off-course. That's actually a good type of question
@roganjosh yes I always encounter such types where you need to question everything, and they sometime drop hints which you can take and run to solve the problem
Ok, then only subtract points if they're being unclear because they're clueless, not if they're playing n-dimensional-mind-games
19:14
@piRSquared yes those are manager rounds where you ask team culture, which can go both ways, I was more asking about algorithm questions like trees, stacks, queues, hashmaps etc, or things like dynamic programming, greedy etc
An easy distinction to make if you possess an N+1 dimensional crystal ball
whatever leetcode is full of
@DeveshKumarSingh yeah, I realize I answered too soon /-: my bad
@Kevin I have a balloon in the back of a van and the van turns right. Which way does the balloon move inside the van?
@roganjosh relative to what frame of reference? Over how much time?
19:16
is it a filled balloon? what is the balloon filled with? has to be asked first up
Devesh has already asked the right questions :P
I know that a balloon moves towards the front of a car when you accelerate. Something to do with bouyancy. I suspect this generalizes to "a balloon in a moving car always moves in the opposite direction of an object that's heavier than air". An object heavier than air moves to the left when you turn right, so I'd expect a balloon to move to the right.
Unless it's filled with physics-defying matter, it'll go left and back
@Kevin if the balloon is filled with something lighter than air (which I see is an implicit assumption of yours)
But you invented the fact that the balloon is inflated, and what it's inflated with :P
19:18
same as a cork floating on water
In the specific case of "giving requirements that don't mention whether or not you have access to the stdlib", I'm skeptical that an interviewer would establish this ambiguity in order to test whether you would ask for clarification to ambiguous requirements in a real-life situation, because there are very few real-life situations where you don't have access to the stdlib. Unless you're working on a very small Python distribution designed for an embedded system, I guess?
I know that after enough time, that balloon is going right relative to anyone standing outside the van.
If you're applying for a job in the embedded systems industry, ask whether you have access to the stdlib.
what about questions that want you to build a simple linked list and don't want you to access deque. Then that's a valid question.
@Kevin but lot of times, interviews might feel that the whole thinking portion of the question goes away on using a stdlib module as a crutch, I mean a frequency counter problem is just a single call to collections.Counter
@piRSquared yes linked lists, and trees are not really available in python as builtin types, so I will need to come up with a small class for those
19:21
make a heap
If "the stdlib is a crutch" is a principle that the interviewer also applies to people already working for the company, then I don't want to work for that company.
If the interviewer applies double standards of what is acceptable for an employee vs what's acceptable for an interviewee, I don't want to work for that company.
^ True but... you often want to figure out how much a programmer understands under the hood.
If the interviewer says, "well done. The Counter class can indeed be used to count the frequency of numbers. Now please do it with only standard types", they have escaped my ire.
Build a heap and Use a heap are two separate matters.
@piRSquared yes my point exactly, using a library is one thing, and doing it from scratch is another, which is what the interviewer asks, because anyone can read up the document and understand the stdlib, but actually doing it in half an hour timeframe on a whiteboard is another challenge
19:25
Is it not common for companies to simply offer one a lower level position if they know how to use a heap rather than build a heap?
I remember writing a comparator class to implement object wise sorting in one of my interview problems in Java, but in python we just define the key appropriately and voila
I am 100% in favor of interviewers asking questions that gauge your ability to reinvent algorithms that are already in the stdlib. But they have to ask you to do that. You don't ask someone to do X and then ding them points when they give you X because you secretly wished for them to do Y.
I'd find it fun if an interviewer ask questions like "Here is a scenario, what is the most logical choice for a data structure?"
^yes that's also a standard question in system design, like a deck of cards, or a parking lot design
Keep in mind, I'm not a programmer. And, I've only ever been on one programmy type interview.
19:28
but you do use pandas right in your day to day job? but I guess more from a data-science/modelling perspective closer to your real non-programmer job
What is the definition of a programmer if @piRSquared is not one?
*looks around*
A person who can write a line of code without an asterisk :P
*ducks*
yes if the interviewer is a cat, that makes things harder :(
Then it's just gonna be regex
19:39
^^^ Me lashing out at @Aran-Fey in kitten fury.
If the interviewer is a cat then you can easily get the job with some string and catnip
@Dodge also, I'm totally with you on this
A programmer is a person who is a practitioner of The Art, and who writes programs. Not all people who have written programs are practitioners of The Art, and not all practitioners of The Art write programs.
what is The Art ?
The Art is undefinable. If you work in Science, Technology, Electronics, or Mathematics, you may be a practitioner. But The Art takes on other guises than these.
19:42
not even sure how to respond to this Kevinism :) I do see this word used in spatterings in the past
At the one programmy interview I had, I got asked by the head dev guy how to write a partial regex parser. He wanted an answer that involved recursion. My mind didn't go to recursion. That was enough for him to conclude that I wasn't the right fit. And it was enough for me to be glad I wasn't the right fit.
yeah that's a good interview problem, some times the recursive version is way easier than the iterative one, e.g in tree traversals (inorder, preorder etc)
@Kevin intentional change to STEM?
I enjoy programming but I haven't studied enough about it to be an authority of any kind. If you need to munge data, I'm your guy. If you want to posit on how to model stocks, give me a ring. If you need someone with all the common algorithms at their disposal... don't call me. Don't get me wrong, I'm happy to learn new stuff. I just don't have all of it at my disposal at any given time.
@piRSquared diamond ring?
19:46
pi is a programmer under my definition. It's hard to pass 100k rep without practicing The Art. If a non-practitioner passes 100k, people will grumble about underhanded rep hunting techniques. (people might grumble about genuine 100k practitioners too, but that's just because we're a catty bunch.)
and he doesn't have pandas in his ignored tags :) That's also one important asset of practicing The Art
@AndrasDeak Just not a blood diamond. Otherwise yes.
@AndrasDeak No, I just can't ever remember what the E stands for.
Engineering is also a hotbed of The Art
There's a few STEM guys laying waste to people who apparently "know The Art" but who don't class themselves as programmers. The revolution is well underway
Whoah! There are a ton of STEM guys I know that are FFAARRR from practicing the art. I worked at a place where 15% of the work force was from CalTech. Some of them were terrible coders.
19:50
SO down for you guys as well?
Unfortunately for interviewers, it's impossible to determine whether someone knows The Art, because it might only be obvious when they're talking excitedly about their model train set at home. Interviewers: ask about the interviewee's model train set.
@Erfan yes
@piRSquared and people avoid asking stdlibs to weed such people out as well! Who know how things work, and not just that it works using xyz library
Can't answer questions.. What am I gonna do with my life now? Hehe
@DeveshKumarSingh Don't forget that some of this is dumb luck.
19:51
sleep?
@Kevin Are you starting a cult? :P Do you have any specific references for coding style, git practices, testing practices or other developer best-practices? Otherwise this all sounds hella vague, sir...
@piRSquared as in? right place at the right time
@smci hella, now that's a Cali word if there was one :) especially in the Bay Area
@smci "Makes an informed decision about coding style, git practices, testing practices, and other developer best-practices" is a good quality, regardless of what one actually decides on
anyone having trouble with the main site? pages are loading very slowly for me
Maintenance @Code-Apprentice
19:58
I am not starting a cult. Do not send me money or buy my personal lifestyle brand of healing crystals, no matter how much I implore you.
@Erfan they didn't send me any notice
00:00 - 20:0020:00 - 00:00

« first day (3175 days earlier)      last day (1998 days later) »