« first day (3742 days earlier)      last day (1209 days later) » 

12:03 AM
Could someone explain how I can solve this without a lambda?
# Lambda may not be necessarypylint(unnecessary-lambda)
strategies = {
	'vix': lambda trader: VIXStrategy(trader),
}
ah maybe like this?
strategies = {
	'vix': VIXStrategy,
}
feeling a bit stupid ngl haha
 
 
1 hour later…
1:20 AM
@ChristophBühler that. Otherwise the lambda would be fine there.
 
 
1 hour later…
2:23 AM
cbg!
I am getting this error. Is it relative to memory depletion?
/usr/local/lib/python3.6/dist-packages/GPA.cpython-36m-x86_64-linux-gnu.so in View.MemoryView.pybuffer_index()

IndexError: Out of bounds on buffer access (axis 0)
 
 
3 hours later…
5:32 AM
cbg all, should I prefer the first way or the second way in the code snippet?
def converter(cost, factor):
    return cost * factor


def currency(factor):
    def converter(cost):
        return cost * factor
    return converter

print(converter(2,50)) # 100
print(currency(50)(2)) # 100
I am learning how to use closures
or is closure more suited for complicated tasks?
 
6:10 AM
Thanks anyway! *
 
@python_user you should prefer the first, since it's much simpler in this example. Closures definitely come in handy for certain reasons, but this isn't really one of them. FWIW you can use Python's functools.partial to easily make a closure (in this case a "partial function application") out of the two-arg function so that you can call it later with one arg.
Partial function applications are useful when you need to keep a function of fewer args around, even though the function requires more args. Pretty common in a callback scenario for example, a library might want a function of one arg, but your function might take multiple args. So you can create a closure that freezes some args, returning a function of one arg so you can pass it to a library.
 
I am aware of parital but havent really used it, thanks for the explanation
though I mostly confuse myself why a lambda wont be enough
 
lambdas are fine too
lambdas just usually have more overhead than a standard function call
so if it's used once, it's w/e, if its used 10k times, you should probably use an actual function
at least....I think that's true.
 
I will stick with the first one, what I am doing is not even close to 10k calls
 
oh and also lambdas can only contain an expression, they can't have multiline statements etc so they're more limiting
 
6:23 AM
hi there guys.
I need a little help from you guys
can any one help me?
 
Maybe. That's not really how it works with the chat. State your problem to the void and it might respond, or not.
Also, check out the room rules on the sidebar if you're new here :)
 
@alkasm := makes this a lot more workable I guess
 
oof. i would not condone the usage of mutations inside a lambda in general.
 
@MuhammadAbubakar you can ask python related questions and people will get back
 
but I haven't even thought about an assignment expression in a lambda
 
6:26 AM
ok can I ask here then?
 
@alkasm it is possible I am yet to use it in a meaningful scenario
@MuhammadAbubakar if users here are not aware of the library you wont get much help
 
okay. got your point.
so my question is realted to integrating third party apis
 
as alkasm pointed out, if you are new read the rules
 
I am reading it
 
7:14 AM
@MisterMiyagi I'm sorry for the long delay in answering you, I was busy, and I couldn't answer you before. Yes, that's what I ended up concluding, plotting a 3D graph in this situation s the best option, I've even plotted it that way. Thanks for your attention!
 
8:02 AM
cbg
 
@alkasm I'd be surprised if lambdas had more overhead. As far as I know once it's created it's just a function. The issue is more about adding an additional level on the stack in typical lambda use cases. And another is using built-ins implemented in C (like partial and itemgetter) are probably faster than a pure python lambda
 
yea i suppose that makes most sense
 
I hate this big corporate's overhead, we talked a lot more than we do things. Is it same for all big corps?
 
hard to make sweeping generalizations among very different companies but generally yes, i think thats pretty typical
the opposite extreme side of the coin being startups where things are built quickly with design decisions that severely impact the future of the company/product being made without much consideration or without much experience.
 
People wrote long e-mail's about why they do not do certain things, they would do it at the same time they wrote that e-mail, lmao. I am giving technical advice yet no one cares because of my place in the hierarchy(which is down bottom). Basically, we are doing monkey wrestling. New age is not about hierarchy, they ll learn this in the hard way. I have less experience than almost all of them but have more technical knowledge than almost all of them.
 
8:12 AM
after a couple years there will be the some guy telling the same thing about current devs lacking expertise, the cycle goes on
I had to learn this the hard way, so I just worked on stuff that was expected of me as an intern and learnt new things that I use in my personal projects
 
Next-generation is not bright as you said so I see no challenge in near future, lmao. But I got what you mean. I born right in the internet era, we are very superior because of the internet. I am y generation(Millenials). I believe most of the Z generation defeated by the internet and social media. They just watch videos on the internet instead of working on some things.
 
Sounds a little cynical. I've been really impressed by a lot of zoomers that I've met. I know more millennials that meet your description than Gen Z-ers, personally. But who knows which is the norm. (I am also a millennial fwiw)
I am also literally watching a twitch streamer while I'm programming right now, too :P
 
something something sokrates "the youth of today is useless" .. being disappointed in the next generation is a custom as old as time, yet humanity still prevails
 
8:27 AM
@alkasm We all are cynical. My caring for them changes nothing, you can't change others. I am also listening to music at the same time. It is okay to do something else at the same time. Doing nothing productive all day is a problem for the person. I am stating this fact. Yet I am liberal, everyone should do what they suppose to do. I am saying they are not creating any challenge for others. At least most of the people I saw.
 
Cbg-ning
 
if I have a context manager that was implemented as a function with contextlib.contextmanager, is it possible to call enter and exit explicitly somehow?
 
Of course, it's not like calling the function calls __enter__ automatically, it just returns a context manager. So just do mgr = func(); mgr.__enter__(); mgr.__exit__(None, None, None)
 
ohh, stupid me
 
>>> from contextlib import contextmanager
>>> @contextmanager
... def f():
...   yield "blah"
...
>>> f().__enter__()
'blah'
beat.
 
8:36 AM
I was just writing out a completely silly MCVE, good thing you guys are quick
 
cabbage, Andy
 
@alkasm lambda and def are just different syntax. Functionally they are 100% equivalent. The only observable difference is that lambda creates functions with bogus __name__ etc.
 
Yeah I read up on that more since, not sure what I was thinking really. Thanks for the corrections
 
9:34 AM
can you type annotate the static size of a list for any benefit in Python?
 
tuple?
 
mutable containers are always of arbitrary size as far as typing is concerned. only tuple supports a fixed size.
 
Yeah that's what I'm going to change to using, but was wondering in general (maybe if its big and homogeneous)
That's what I figured, a fixed size is not part of the API so it'd really be a different type if it were "truly" static.
Have you guy seen the typing.Annotated? One of the examples in the docs actually is a list with a max length. kinda interesting to see how tools will try to use this
Also is there a way to exclude a specific type with annotations that otherwise meets the interface? common example (and the one I need) is a "non-str sequence"
 
@alkasm I find some of the examples intriguing, such as ValueRange(-10, 5). But ultimately it feels like something that should be part of the actual type system. Not sure if it will see much use.
@alkasm Sadly no. :/
 
@MisterMiyagi yeah, it looks more like a runtime way for plugins to use the type system
@MisterMiyagi I guess it'd need a static_assert thing going on. Does an assert work, actually?
no, it doesn't. lol
 
10:00 AM
@Alper This is an interesting take that might explain things: leadingagile.com/2018/02/applying-brooks-law
@Arne ... for the moment.
 
@alkasm Might be relevant: MyPy issue on this github.com/python/typing/issues/256
 
Yeah that's a little diff, but related. I was just reading those issues, too.
lol now that I think about it my code is overly generic anyways. I have this as a function:
def _is_sequence(o):
    return isinstance(o, (Sequence, np.ndarray)) and not isinstance(o, str)
and this is used like "if its a sequence unpack it, otherwise treat it as a singular value" but my lib cares nothing about singular strings and it'll just error wherever that is used, anyways...
 
You might be interested in PEP 647 -- User-Defined Type Guards
Alas, all the nice toys are in 3.10 at earliest...
 
yes, yes i am interested in that. thank you!
3.10? you mean, 4.0? :P
 
11:06 AM
@holdenweb I knew you would have a say for what I said. That is a great read. We have 5 days for 1 man work to do and people keep e-mailing each other for quite some time. If we calculate everyone's communication overhead on this matter, it already passed 30 days of work till today. The good thing is, it is still not concluded and it is ongoing.
 
Unfortunately you may find some resistance to the idea of behaving rationally. :D
 
 
2 hours later…
1:05 PM
Cbg
I wielded the exec hammer at work today. Never have i felt such power doing something so wrong before!
 
 
1 hour later…
2:32 PM
morning cabbages folks
 
3:12 PM
Cbg.
 
3:25 PM
Hi folks,
Does anyone has experience with Celery and Selenium chromedriver ?
 
3:51 PM
Put in a "one-time budget application" for a couple of new monitors and dock. CEO knocked it back "Under $10,000, speak to X". Forwarded message to X who replied with "Sure, I'll let you know when delivery is expected." I'm starting to love HoldenCorp.
 
@holdenweb haha what a response
 
4:08 PM
@holdenweb fancy tagging an extra one on the order? :P
 
Sorry, no way to do that. If you want to inherit my ageing acer I'd be happy to shi it, but it's been fritzing a bit lately.
 
4:30 PM
    def get_formatted_name(first_name, last_name):
        """Return a full name, neatly formatted."""
        full_name = f"{first_name} {last_name}"
        return full_name.title()

Morning all - I'm a beginner in learning python and I'm having a hard time understanding the return statements. Is return required if you are combining the variables in the function? Just having a hard time grasping the purpose from the book and looking online.
 
4:43 PM
@TylerEast return is what lets you... well... return a value from the function to the code that called the function. With no return, you can't do name = get_formatted_name('foo', 'bar')
Well, you can, but name will be set to None
 
Do I need to do a return value for every time I do XXX = in the function?
 
No, you only return the final result
 
Okay makes a bit more sense. thank you
 
5:03 PM
@holdenweb was only joking but thanks :)
 
5:36 PM
I need to make users,roles tables in this database I'm managing for a project at work. I have a schema, but would love a smarter set of eyes to tell me if I'm footgun'ing myself
users
    id
    username
    password (really, it's the hash)
    -- other stuff

role
    id
    -- other stuff


user_role
    user_id (FK to users.id)
    role (FK to role.id)

role1
    user_id (FK to users.id where role.id=user_id and role.name = role1)

role2
    user_id (FK to users.id where role.id=user_id and role.name = role2)
I feel like there's a better way than to use role1 and role2 tables
 
@roganjosh I had an idea ...
 
role1 and role2 are effectively just lists of user_ids for uses that have that role
 
So role1 and role2 are just there for convenience?
 
@inspectorG4dget You can synthesise the role1 and role2 (and role3, and ...) using SELECT DISTINCT user FROM user_role WHERE role = ?
 
you're both right, and I just realized I forgot to include one other table that demonstrates why I wanted that role1 table:
project_user  -- maps which users are on which projects
    project (FK to project ID in another table)
    user  (FK role1.user_id because only `role1`s can be associated with projects)
    -- other stuff
 
5:45 PM
Also, the DISTINCT is probably unnecessary if the PK to role_user is (user_id, role)
 
I have another table that has a similar FK to role2.user_id
@holdenweb That is indeed the PK, so the DISTINCT is indeed redundant
 
The problem with the role* tables is that you would then need triggers to update them, or run the risk of inconsistency, and you don't want a trigger that's essentially a long list of elifs.
 
the reason that I thought I needed the roleN tables is that I don't know how to write a constraint on the project_user table to ensure that only users with the appropriate roles can be FK'd against
@holdenweb totally fair. That's in fact the exact problem I thought of after I designed this, and I now fear I've painted myself into a corner
 
I'm a database n00b, so I wanna ask, will role1 and role2 automagically update themselves whenever user_role is modified?
 
@Aran-Fey yes, with the trigger that @holdenweb alluded to
 
5:49 PM
Bleh, should've read holdenweb's message before I asked
 
tee hee. I did not specifically mention that, though it is indeed implemented
 
Why? You could use a stored procedure or define a set of views on user_role for each of the roles, if you are forced into using such a denormalised representation.
 
I am not forced into this representation by anything other than my own n00bity. So I'd absolutely love it if you could show me the ways of the Jedi
 
I maintain the need for actual tables is zero unless you can come up with better reasons for it. CREATE VIEW role1 AS SELECT user_id FROM user_role WHERE role='role1'? Then just use role1 like it was a table!
 
does the view automagically update on user_role insert/update?
 
5:52 PM
But I still don't see why you need the views/tables at all.
 
This looks reasonable to me. I'm sure there are other ways to do it, but if you have a lot of tables like project_user where you need to reference users with specific roles, doing it this way will likely be more convenient and make the db schema easier to understand
 
@holdenweb what would be a better way to do this? How could I implement the necessary role1 contraints in project_user?
 
The view is just a select on the table. But you'd be better off (in SQL, anyway) just using the subquery: SELECT ... FROM ... WHERE some_table.user_id in (SELECT user_id FROM user_role WHERE role='role1'), for example?
What are "the necessary role constraints"?
"The view is just a select on the table.": should have added "and therefore always represents the current content of the underlying table".
 
Only users with role1 (really, it's a list of roles, but let's just go with role1, without loss of generality) may be allowed in the project_user table. Therefore, if I try to update/insert project_user, the check/constraint needs to ensure that the inserted user_id has a role1 role in some row of the user_role table
@holdenweb ahh, thank you. That answers what would have otherwise been my next question :)
 
As you might expect, you can generalise to WHERE some_table.user_id in (SELECT user_id FROM user_role WHERE role IN ('role1', 'role2', ...))to handle multiple roles.
 
6:00 PM
I totally understand how to generalize the definition of the view to accommodate the multiple allowed roles. I'm still curious as to what "the correct" way would be since "I maintain the need for actual tables is zero". Or have I misunderstood you?
 
If there's an actual purpose to those then I'd recommend the views to keep the subqueries under control by encapsulation.
Then the check constraint would be user_id IN role1 rather than the more complex query above.
 
The only reason that this (now) view exists is so that project_user.user can FK against this view. However, if that denormalized, I'd love to know how to implement the constraint in project_user that this view otherwise is an implementation of
create table project_user (
    user INT CHECK(user in (SELECT user_id from user_role WHERE role in ('role1')))
    -- other stuff
)
is that basically it? I didn't think CHECKs allowed that
 
Sadly, CHECK constraints don't allow off-table references.
SQL is a declarative language, so any kind of procedural programming sits uneasily in pure SQL (which is why the vendors have all defined procedural extensions).
 
would that therefore constitute "an actual purpose" to having the role* views?
 
Wouldn't do any good - you'd still have to query the view. I think the solution you are looking for is a trigger that aborts the transaction if the user doesn't meet the required conditions, but I'm then unsure about how you would apply that trigger selectively.
 
6:12 PM
I see. Thank you. I think I'll stick with the view then, and FK against it :)
 
6:22 PM
it turns out that I can't make that a view, either, since it is impossible to reference a view as a foreign key. As a result, I'm going to have to maintain those tables with triggers
rapidly switching between vim over SSH, PyCharm, Slack, and room6/markdown is causing my brain to meltdown from the rapid macros/syntax changes
 
Obviously I don't know much about databases, but I kinda get the impression that they're bad at their job...
 
6:45 PM
@inspectorG4dget So does that mean that you'll have a table for each new role?
Ah, nm, I should have re-read the subsequent comments
 
@roganjosh yup. I would have loved to do with multiple views instead, but apparently I can't FK against them D:
 
I wouldn't set things up as a single query myself
 
how would you recommend I do it instead?
 
If the role table is just user_id | role_id and both are indexed then it should be easier just to query that and get a list of role_ids that a user can access
You could also have the same with things_you_can_see | role_id mapping. If there's no intersection, abort the next query that actually gives the data
I mean, that would be 2 queries minimum, and 3 queries (if they pass the auth test), but there's no ambiguity
As a baseline that would work, then you can look at optimising it, but I suspect it would involve joins, so I'd prefer multiple super-cheap queries. But I'm not a database guru
 
(just to ensure we're on the same page, because I don't understand how this solves the problem as I see it):
the issue is not querying something on the application side. Rather, I want the database to restrict table1 from referencing foreign key in table2 when there does not exist a row in table3 whose values are (the_value_of_foreign_key_I'm_referencing, one_of_the_roles_I_care_about)
if that is indeed what you were talking about, then my mere mortal brain falls just short of comprehending your suggesion
 
6:58 PM
recbg
@inspectorG4dget I tried to read what your problem is but... TL;DR...
of course you'd make role1 and role2 into views... (though why bother because that is a simple projection); but to ensure referential integrity, you could have a composite foreign key of (role1, userid) to the role table; with check constraint checking that the role == role1 in the authorization table...
 
I think I understand it but I've got to pop out. I'll have a look when I get back. But yeah, I think it would be easier if there's a precis
 
BUT ... the solution is to throw in more tables...
actually I hate the word table. These are relations. They're predicates.
 
7:57 PM
@AnttiHaapala sorry about the delay - I was in a meeting. This sounds really interesting. Does this capture your recommendation?:
table project_user
    user int
    -- other stuff
    FOREIGN KEY(user, 'role1') references user_role(user_id, role)
if that's correct, that's exactly what I'm looking for
 
yes but the only problem might be that ...
 
I'm sitting on the edge of my seat with bated breath
 
you cannot use a literal in the foreign key :P
hence you must have a column...
 
oh dang! I got too excited too fast D:
... and I'm back to my trigger
 
8:24 PM
cabage!
How do I use pypy in Google Colab? I followed this code (stackoverflow.com/a/61200280/2369957) but the error bellow occurs when executing the magic line "%%pypy":
%%pypy
    ^
SyntaxError: invalid syntax
 
8:56 PM
hi guys. i have another question. is there any tool to preview pythoon variable dependency. like, its gonna list all variables and mapped their dependency between other variable?. because I'm currently try to read my coworker code. and he forgot what he just write :/
 
Can you give a definition of "dependency between variables" that would satisfy a computer?
 
like for example. x = y + 1. means x is dependent to y. sorry if it sound stupiid
 
It just sounds awfully vague
 
im working with dataframe code. he did so much slicing and keep creating new variable everytime he slice something from dataframe
thats why i need it hehe
 
Some of the problem is probably solved by type checkers (where type information can be deduced, so can "dependence", but I suspect the question is ill-defined to begin with (but I'm not in a mental state to try and come up with counterexamples)
@lone_coder let me guess: you only have state in a jupyter notebook with history edited/removed?
 
9:01 PM
no. its a single file. non-OOP sequential code with 2000 lines
 
I recommend an IDE, lots of coffee, and extensive code refactoring.
 
Ah, so it's not hard to figure out, but you want to omit it
 
yes hahaha
 
yeah, renaming those plentiful auxiliary variables would be a good first step
 
its gonna take a long time
 
9:02 PM
there's nothing wrong with creating new names for intermediary results
 
its contain heavy data per dataframe variables. i really want to reduce it. about 1.5GB of csv rows
also turning it into OOP is also a challenge
well im gonna start with refactoring it with F2
thanks for insight
 
I'm gonna go ahead and say that that code most likely won't benefit from OOP
 
probably yes. it will not do much and just add more time to work on it.
 
@lone_coder are you aware that x = df; y = df doesn't give you additional memory overhead?
And that x = df; y = df.col1 doesn't give you additional memory overhead?
Specifically slices play nicely with memory (in numpy, and probably in pandas for the most part)
Of course if you have advanced indexing (array-valued indices, rather than slices) you get copies. But you'd have to have those copies even if you don't use temporary names. And if you use temporary names you can still del them if they are the only reference to a large object.
In other words, make sure you're not tilting at windmills
 
9:19 PM
hmmm. I'm not aware of that before. thanks. I thought it gonna "slow down" the code pretty much
I also want to make it more readable. it's really hard to read in the current state. I'm preparing for future changes. also trying to add thread and chunk size when reading and writing that df.
 
 
2 hours later…
10:55 PM
Cabage!! To your knowledge, I have already managed to resolve this (chat.stackoverflow.com/transcript/message/51363893#51363893).
I really need someone to help me. I have a loop in which I use an instance several times, but I end up getting an error due to, I believe, lack of memory. What is the best way to deal with this situation?
 
1. debugging the situation to make sure, 2. reducing memory need
 
I was about to suggest the same thing! What a coincidence!
 
Recently I ended up sending a message regarding this problem, in the previous situation I was creating a new instance inside the loop, but after I removed it from the loop, I solved the problem (although this error did not previously occur). In this loop in which the problem was solved, I used the same instance 1024 times, but in this other loop I have, I need to use this instance 2048 times, and this problem occurs, remembering that I am creating this instance only once, outside the loop.
@AndrasDeak The problem is really that. I already debugged. How do I reduce the memory requirement?
 
how cryptic
 
So what's all your memory being used for?
 
11:08 PM
probably filling the same scatter plot with 2048 copies of many data points
 
I use the instance to call a function that returns a value, and I store that value in an array.
Each time I enter the loop I need to call this function 3 times, so there are 3 arrays.
At the end of the loop, there will be 3 arrays with 2048 values in each.
 
Okay, but this all far too vague for us to help you. We don't know your code, we don't know where your memory is going. You'll have to give us a lot more info here
 
it's probably in the instance
 
Okay, I'm going to do a simple example.
object = script.class()

for i in range(2048):
    array[i] = object.function(1)
    array2[i] = object.function(2)
    array3[i] = object.function(3)
This example goes well to explain.
 
11:25 PM
mhmm
 
I tried to use garbage collector (gc.collector) and delete the instance (del object), creating the instance again afterwards, calling the function again, before each new use of the instance, but it also didn't work. :(
A part of the error.
(...) cpython-36m-x86_64-linux-gnu.so in View.MemoryView.pybuffer_index()

IndexError: Out of bounds on buffer access (axis 0)
 
unfortunate
 
That is really no help at all. We don't know the purpose of your code, we don't know what kind of data you're working with, we don't know what's using up how much of your memory, and to be honest I'm still not convinced that running out of memory is what's causing that error
I was going to ask for the output of a memory profiler, but even if you provided that, we'd still be missing a lot of crucial info. So I think you're gonna have to figure this out on your own
 
@Aran-Fey it's not the cause
Just an indexerror with maybe something gpu-related
 
@Aran-Fey Ok, thanks :(
@AndrasDeak What do you think it can be? GPU-related? Could the fact that I run the code using GPU or TPU from Google Colab be interfering with this?
 
11:36 PM
unfortunately it's bedtime here
 
@AndrasDeak Ok, some little hint?
I don't know if it's relevant, I don't know if you noticed in the error message, the function is executed in Cython.
 
@Marco OK, hint: did you read the error message?
 
that says cpython, not cython
 
Good night
 
@AndrasDeak Yes, I already searched on google but I couldn't find any answer, I didn't understand what this error means.
@Aran-Fey I thought Cpython was the same as Cython. Sorry.
@AndrasDeak Thanks anyway! Good night!
CPython*
 

« first day (3742 days earlier)      last day (1209 days later) »