« first day (3852 days earlier)      last day (1096 days later) » 

1:22 AM
cbg :)
@12944qwerty you can get 100 rep (once) if you join other SE sites, not sure if you have already got that
 
1:50 AM
@python_user already have :D
oh wow, i just got 25 points while I was eating
 
 
5 hours later…
6:26 AM
Hi
I am a newbie to Python
Can anyone tell me why in-built fn id() is giving me different values each time for the same variable.
I thought it should have fix location in the memory (RAM) but it is not..
 
@Exception what makes you think they must have a fixed memory in RAM?
how to deal with an OP who reverted my edit for removing the python tag? question has nothing to do with python
I have left a comment explaining my rationale behind it, should I go ahead and edit it again?
 
7:29 AM
@Exception Python names are effectively references to objects (what some languages call pointers), and the (C)Python id function therefore shows you the memory address of the current value. Therefore when you bind a new value to a name its id changes to the address of the new value. If you merely mutate the value (e.g. my_list.append(value)) it stays the same.
 
@python_user I was not sure.. but with @holdenweb's explaination, I understood it.
 
@python_user Little value in entering a p*ssing contest with a determined OP. If you want it changed, flag for moderator attention?
 
morning cabbage
 
7:48 AM
@holdenweb they apparently removed it later, not even a year of SO and I am already tired of this, how did regulars manage this :D
 
8:23 AM
Hi I'm having an issue saving a dataframe to csv.. I want to save it without the column indexes and the row indexes.. I used df.to_csv('myData.csv',index=False), but it only removes the row indexes, not the column ones.
how to remove the column indexes?
Just to clarify..
for example my dataframe is a 5x5 array.. When I save it to a csv, I get the 1st row to be to column indexes going from 0 to 4 and the first column to be the row indexes going from 0 to 4.. How can I remove both when saving to csv
 
header=None should do what you want
 
oh, thanks a bunch, it worked
 
 
1 hour later…
9:52 AM
@python_user never edit twice. Don't engage in edit wars.
 
10:34 AM
Hi again,
I'm having a small confusion with numpy's np.sum... basically I was using np.sum to carry out a softmax normalising function where for my 200x200 matrix, I normalise each row value by dividing each element by the sum of the row elements.. When checking the output through np.sum function, it all adds up to 1, but when exporting to a csv file, when checking the sum for each row, they turn out different..
is there a limitation on np.sum? or am I missing something here
could i attach a ~300 kb csv file for your reference here?
 
@cmk101010 no
How large is the difference?
And how precisely are you printing values for the csv?
 
Sounds like csv serialisation drops some digits.
 
So i first used a for loop to do the normalising and then output a csv..
when checking through the csv file, for example the first row's sum shows 1.12...
Then I read that csv file again to double check and used row sum, but that comes to 1
i meant np.sum*
rowsum = np.sum(TPM[0][:])
 
Since you say the issue occurs for each row, why not make an MCVE of one row?
 
sorry, how to do that.. Can i just copy and paste the csv values for one row here?
hmm, ok, i tried pasting it and clicking fixed font, but I get the error that the message is too long
0.814998443	0.303523385	0.001564341	0.000179323	9.47E-05	3.46E-05	3.07E-06	0	0	0	3.11E-05	0	3.00E-06	0	2.73E-06	1.62E-05	2.99E-06	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	8.42E-06	0	0	0	0	0	0	0	0	0	0	0	0	0	1.23E-05	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
ok, i deleted a bunch of zeros
this is all just one row btw
 
10:54 AM
So, does that match the data you expected?
 
adding it up myself, i still get it higher than the rowsum
 
And yes, dropping the zeroes should be fine.
 
when using np.sum(TPM[0][:]), I get 1
 
>>> z = "0.814998443	0.303523385	0.001564341 etc. ...
>>> sum(map(float, z.split()))
1.1204746019999998
 
@AndrasDeak googling for edit wars actually gives me useful results
 
10:57 AM
@PaulMcG, when u tried np.sum, what does it give u?
 
Sorry, I don't numpy
 
@cmk101010 why not answer my question?
 
I dont know what you mean by precisely printing into csv. I just used
DataFrame.to_csv
 
21 mins ago, by Andras Deak
How large is the difference?
 
~0.12 for the first row
 
11:04 AM
I'll just have breakfast
 
In [38]: np.sum(np.array(
    ...:     ['0.814998443', '0.303523385', '0.001564341', '0.000179323', '9.47E-05', '3.46E-05', '3.07E-06', '3.11E-05', '3.00E-06', '2.73E-06', '1.62E-05', '2.99E-06', '8.42E-06', '1.23E-05'],
    ...:     dtype=float,
    ...: ))
Out[38]: 1.1204746019999998
 
hmm
 
I suspect TPM is not the data frame you're looking for.
 
No idea how this should go down to 1. The first two are already ~1.12. Compare 0.8149 + 0.303 == 1.1179
 
thanks, I'll check my codes again
 
11:12 AM
What kind of object is TPM?
Ah, a dataframe
@cmk101010 either you didn't normalize properly or you exported the wrong file. Please post an MCVE in a code paste site.
 
is it because i am using numpy operation on a dataframe?
i just tried df.sum(axis =1).loc[0] and got the correct answer
 
11:29 AM
We'll never know
 
sorry, this is the codes pastebin.com/kVmzrZNQ
 
The input csv is just a matrix with no headers, right?
 
yes, sorry, just copy it again, i pasted a wrong one initially
it comes out as a 200x200 matrix
 
So you don't need pandas. And no loops. This is a few lines of numpy.
 
yes, just to check
 
11:40 AM
cbg
 
cbg
@cmk101010 the first two values of the output dataframe in that pastebin is 0.8149984434087949,0.30352338549250074. I want you to understand and acknowledge that for a second.
 
In [17]: import numpy as np
    ...: from io import StringIO
    ...:
    ...: # create dummy data
    ...: f = StringIO('1,2,3\n4,5,6')
    ...:
    ...: # load data from csv
    ...: dat = np.loadtxt(f, delimiter=',')
    ...:
    ...: # divide by the row sum
    ...: dat /= dat.sum(1, keepdims=True)

In [18]: dat
Out[18]:
array([[0.16666667, 0.33333333, 0.5       ],
       [0.26666667, 0.33333333, 0.4       ]])

In [19]: dat.sum(1)
Out[19]: array([1., 1.])
 
This will be the room where cabbages are given the most importance :p
 
@cmk101010 numpy ^
 
morning cabbages, folks
 
11:50 AM
@AndrasDeak, so, it rounds it?
 
What?
I'm saying you should drop pandas and python loops and do what I posted
 
@AndrasDeak oh sorry, i didnt enter the whole number
 
This is all I can say, it's probably more than merited. Good luck.
 
thanks
@ParitoshSingh yes, i saw that, but when I did np.sum for that row, I kept getting 1.0
 
@cmk101010 part of acknowledging this fact means that you have to realise, it is impossible for that row to have a sum of 1.
 
11:55 AM
@AndrasDeak Not using pandas? But there are numbers in this problem!
 
12:08 PM
@cmk101010 The inference we were all hoping you might draw is that since the first two numbers sum to more than one, it is obvious that whatever you are taking the np.sum of, it is not that row. When someone asks you a question it's considered polite to give a considered response, rather than galloping on and reporting symptoms form further misguided attempts.
 
Yes i understand, I just have to find what I have done wrong in my codes
 
So this data is being normalized? In that case, it is supposed to add up to 1.
 
@PaulMcG yes, that's the goal
 
@cmk101010 Believe it or not, we ask questions to try and help you solve your problem. Without answers, how do you expect us to help? A little cooperation assists the process enormously.
 
And it does, and it doesn't, depending on what cmk is doing in secret ¯\_(ツ)_/¯
I wouldn't try debugging that loopy pandas thing anyway, just replace it with something idiomatic
 
12:10 PM
no worries, I'll find out where i went wrong
 
@cmk101010 Besides which, Holden's First Law of Software Development states that any statement of a programming task beginning with the word "just" is to be considered bogus. Your statement suggests you underestimate both the nature and the magnitude of the task.
 
Trying to reverse engineer all the matrix math I forgot from college... I'm back up to "basis vector".
 
@Kevin that's actually more fundamental than matrices. Congratulations, you've reached linear algebra bottom :)
It's like rock bottom but with an embrace.
 
@Kevin matrix math is very weird. It's kinda similar to normal math, but with extra special rules what's allowed and what not. And I have an intuitive understanding of the rules, but I'm never quite sure if what I'm doing is allowed :P Or just a special case working atm but not in general
 
I don't know if I should be going up or down the abstraction ladder. I'd be happy if I gained a better ability to solve matrix/vector problems symbolically. I can get pretty far by rewriting everything as a system of equations and hitting it with my algebra mallet, but it's quite time consuming
@Hakaishin I know what you mean. Earlier I was trying to figure out if I was allowed to assume M * (c*V) == c * (M * V) for matrix M and scalar c and vector V. The answer seems to be "... sometimes"
 
12:26 PM
@Kevin riiight :D
 
@Kevin for scalar c, always, as much as it's true for floats
 
@Kevin Exactly, I hope I will surpass this stage of doing math at some point. The next time I need it maybe :P
 
@AndrasDeak I believe I blocked out an entire semester of choosing ways to multiply matrices that minimizes accumulated float error, so I understand the gravity of your disclaimer there
 
Nah, this is mostly just float associativity
choosing between (A * B) * C and A * (B * C) is a lot more tough and I would rather not touch that
Even the number of operations is a mess to count
Apr 26 at 17:35, by Andras Deak
In our numerical analysis course there were a few entry questions for the exam, a list of about 10 short questions handed out before exam season. One of the questions was "what is Kutta's first name?", because one of the pet peeves of the lecturer was people thinking that "Runge Kutta" is one person. (To this day I know that the gentlemen were called Carl David Tolmé Runge and Martin Wilhelm Kutta.)
 
I think... Triangles are involved
 
12:30 PM
Another intro question was "what is the (asymptotic?) number of operations for Gaussian elimination"? I just had that memorized.
I'm actually not very good at computational linear algebra.
 
I remember Gaussian elimination. I liked the puzzle of solving matrices with it, although I wasn't crazy about the amount of numbers I had to write down if I wanted to show my work
Whatever the asymptotic number of operations is needed to solve an NxN matrix, the amount of writing is N^2 times that
Fortunately N will remain exactly 2 for the duration of this project of mine
 
It's actually useful for brute-forcing equations. Like in physics if you have 4 degrees of freedom in a mechanical system, you typically have a 4x4 linear system for the forces/accelerations from the equation of motion. If you don't have the experience to look at them the right way you can just do GE and get a result in finite time.
 
@Kevin "check the commutator" is probably not helpful, is it?
 
Any time I see a math word that I don't already know, I find that googling it is usually at least a little helpful
 
this is a trick question because you'll be hard-pressed to find information about the commutator of a scalar and a matrix
 
12:46 PM
Is M*c even defined for a matrix M and scalar c?
 
yes
It multiplies each element of the matrix by c.
Which means (M*c)*V is the same as M*(c*V)
 
cbg
 
Ok, makes sense
 
@AndrasDeak "Here's the formula. Just insert numbers and you have an answer." – the professor I did not deserve but needed back then
 
numbers are overrated
 
12:56 PM
all i need is 0, 1, and 1 more than 1, and im done
 
@MisterMiyagi Was that the one who also said "Why do you need an example? Just insert numbers into the equation and you have an example."?
 
@Dodge Yeah, he liked that line of thinking.
 
@Kevin Why are you doing matrix math? Is this part of your OCR project?
 
Nope. I'm trying to prove that you can emulate almost any linear transformation in classic MS Paint by using the stretch/skew window exactly once
I already have a working algorithm for emulating rotations (except for rotations by multiples of 90 degrees), so I'm halfway there
The actual practical problem that this solves is, I want to transform a rectangle into a perspective-correct* parallelogram so I can paste it into the Harvey Birdman calendar reaction image
(*ok, real perspective doesn't turn rectangles into parallelograms, but getting something that looks 99% right is more than sufficient)
 
1:14 PM
OpenCV does that
But that doesn't prove your point about MS Paint, so not as fun
 
Right.
It's encouraging to know that an algorithm exists. It means I'm not on a wild goose chase.
 
@Kevin "almost any", except that zero-determinant thing. Which means that at least one of the eigenvalues is zero. I think every projection is this category.
 
Yeah, projections are especially doomed because Paint won't let you stretch with a factor of 0. Closest you can get is 0.01.
My practical case guarantees the input and output vectors are basis vectors, which I think removes some of the scarier zero determinant cases.
 
Yes, transforming two non-collnear vectors into two other non-collinear vectors is very well-behaved
This is probably an affine transformation.
 
Yeah
I suspect the final algorithm will actually be fairly simple. During my journey through Wikipedia I've seen at least one article describing the matrix that turns one basis vector set into another. But I'm taking my time deriving some things myself because I don't want to hit an off-by-one error like yesterday's code, and not have enough conceptual knowledge to debug it
 
1:29 PM
If you like brute force: 1. check what the matrix is if you know what [1, 0] and [0, 1] are transformed into, 2. invert the matrix to go from (v1, v2) to [1, 0] and [0, 1], 3. multiply this from the right by the matrix that goes from [1,0] and [0, 1] to (w1, w2) (your result vector pair)
I don't know if this helps you any, but I like to think of matrices as representations of linear transformations (abstract operations). So when you see M*v it might help to think that it's some T(v) function that transforms the vector v in a given way. I find this picture more intuitive when composing transformations.
So this gives you a (v1, v2) -> ([1, 0], [0, 1]) -> (w1, w2) transformation
 
Yeah, I've been trying to visualize M*v in geometric terms. I think it helps a little bit, but I'm not quite at Feynman levels
 
Matrix multiplication is defined the way it is exactly to match the composition of the corresponding linear transformations.
 
I don't mind making a pit stop at ([1, 0], [0, 1]) to get from v to w, once I digest the concept more
 
The fun part is that the transformation is unique so you can get it however you want.
Super brute force: W = M * V -> multiply by V inverse from the right
 
Hmm! I was wondering earlier if there was some way I could divide by V, but I couldn't figure it out because I couldn't get the dimensions to match up.
In other words, I had w1 = M * v1 for matrix M and vectors w1 and v1, and google wouldn't tell me how to find the reciprocal of a vector.
But if I let W be the 2x2 matrix characterized by w1 and w2, and likewise for V, all the dimensions are the same, and inverting is well defined
 
1:44 PM
Yup
Just watch out to choose rows vs columns well
 
2:18 PM
Progress report: I did not choose rows vs columns well, and I have a mysterious off-by-one* error
(*in the sense that three of the elements of my inverted matrix are correct, but one is off)
I think I'll recruit sympy to check my work
 
2:39 PM
@Kevin curious
 
95% chance that I did the arithmetic wrong, 5% likely that my answer is actually correct but not obviously equivalent to the textbook formulation unless I apply some identity that I don't know that I have
 
You can put in numbers to test
Does M V give you W or not?
 
I can answer that question after shaving 1.7 yaks
 
How so? Generate arbitrary W and V, compute M, check result. Or does this involve all those yaks?
 
I'm printing V's inverse to stdout rather than storing it somewhere, so to verify the result, I have to either refactor or do some hand calculations
Simple enough to do, just gotta take a minute to move some things around
 
2:55 PM
I see
 
I know that usually when help-seekers say "I'm not sure, let me get back to you on that", they're actually saying "Your question takes a lot of work to answer so I'm just going to quietly fade out of the room now", but I will get back to you on that and I am not going to fade out
 
I wasn't worried :P
I rarely am, actually.
 
Either the help seeker gets back to you, which is good, or they stop bothering you with an ill-posed problem, which is also good. No reason to worry in either case :-P
 
yup :P
 
3:27 PM
Are there any good exercises for first-time pandas users? (or numpy)
 
you'd start with a tutorial
 
cbg
 
I'm trying to train myself to learn from documentation instead of tutorials like that....
 
@12944qwerty that's a silly thing to do
train yourself to find the right tutorials instead
 
or get a lot of bamboo... I hear pandas like that...
 
@piRSquared that's an edit link...
 
thx
 
It might be more educational to point to pandas.pydata.org/pandas-docs/stable/getting_started/…
 
that is what the 10 minutes link should take him too.
 
Wonder why the 10 min is not there
 
3:36 PM
wanted an opinion on best data structure to approach this problem with. I have data from alot of different sources about company names, contact name/phone/emails that are potentially out of date/company name updated/differently formatted/not individually one-to-one (same people own diff companies) and I want to make a structure that I can use as an entity tagger so that every businesses in every dataset can be stamped with an entity ID to make this and all future comparison easy.
What type of data structures would make sense? I'm picturing being able to mostly use this structure to programmatically tag all the old data, and then providing a little utility to quickly cross-reference contacts (or update latest contact) at any stage we are populating new data
An atomic table with some kind of compound key for entity+source+type seems like it could work here but I'm a bit worried about if I could mess up with the people who own multiple companies, and the fact that to do that cleanly you almost need the entity id already to then create the entity id
 
Ok, I still haven't put in example values for M, V, and W. But I do have a (!M)CVE that very thoroughly shows my work and the surprising result. I think I'm two minutes' worth of grade school algebra away from solving this on my own, but I figured I would share what was confusing me in my last message. pastebin.com/eJmBXWrz
 
you know that sympy has matrices, right? :P
 
No ∠( ᐛ 」∠)_
 
If you feel like giving up/checking, replace that Matrix with sympy.Matrix, then do system[:, :2].inv()
 
3:53 PM
Definitely useful for double checking if nothing else 👍
 
I have proof for your result being correct
 
I think I do too, although it's in plain text rather than sympy-verified operations
 
same
Want the sympy proof? I've got one.
 
I've cleaned up my text proof enough that I think I'm satisfied with its correctness
c*(b/det + 1/c)/d           #prove this equals a/det
bc/(d*det) + 1/d            #factor
(bc + det)/(d*det)          #common denominator
(bc + ad - bc)/(d*det)      #substitute det for a*d - b*c
ad / (d*det)                #cancel
a/det                       #cancel
□
 
Yup, but now you'll get it anyway:
>>> a, b, c, d = sym.symbols("a b c d", real=True)
>>> det = a*d - b*c
>>> sym.simplify(c*(b/det + 1/c)/d - a/det)
0
(the reals are not really needed, those were left over from earlier attempts)
 
4:06 PM
Simple and clean :-) You've convinced me that I'm not using sympy's full power. A valuable lesson, even if I stubbornly continue to reinvent wheels for my current project. I'll remember for future projects.
 
if you don't want any specifications for the variables you can also do from sympy.abc import a, b, c, d which is fun
 
Ha, useful.
 
It has Latin and Greek letters (minus those that would clash with attributes in the main sympy namespace, such as pi or gamma). Greek ones spelled out in Latin, of course.
 
Just a little disappointing that I can't from sympy.abc import det. I wonder if it would even be possible to make a module that can dynamically support imports of arbitrary identifiers... Maybe if you turn globals() into a defaultdict somehow
 
I'm sure there's some import magic that would allow to do that...
If there's an answer on SO hopefully it's behind three layers of red warnings
 
4:13 PM
Yeah :-D
 
4:23 PM
@Kevin we got an MP smack down... stackoverflow.com/a/35436756/2336654
 
I think you can define a custom Loader that lets you do fun things to a module object, perhaps including altering its getattr implementation. But I don't think that alone is sufficient to get import-any-name behavior without any outside setup
 
modules being loaded once can also be a problem
 
@piRSquared Yeah, I figured that a globals() based attack would be foiled for a reason like that
At least we know for sure that it can be done for namespaces that you have metaclass control over
 
Say I have a class Company and it has an attribute named CEO which is also an instance of a class named Person. Suppose I have this structure "my_list=[] for (..): company = Company(...) my_list.append(company.CEO)". My question is, will the memory allocated by the object company be freed at the end of the loop iteration? Since I am appending one of its attributes (CEO), which is also an instance of a class, to a list. (This is of course a toy example).
 
What is the type of company.CEO?
If it's a simple native type like a string you're good. It doesn't know where it came from.
 
4:34 PM
It's not, it's a user made class (Person)
 
If CEO does not contain a reference to the company instance, then holding onto CEO should not have an impact about whether the rest of the object is collected
 
Then the question is whether a Person class instance has any information (reference) about the Company it's assigned to.
 
I want to say "INCREFing a class instance also INCREFS all its attributes, but not vice versa" but I'm worried about corner cases. I'm pretty sure your case is not a corner case though.
 
@ihavenoidea but note that the last company will stick around in memory, because loops don't have their own scopes in python
@Kevin is that even remotely true?
 
Gotcha! So if Person has no reference whatsover to Company (only the other way around), the memory from company should be freed. Is that correct?
 
4:38 PM
I'd expect that the attributes have one guaranteed reference from the instance, and these get cut when the instance runs out of references
 
@AndrasDeak Epistemic value: educated guess, based on the education of "I occasionally see INCREF when I am source diving for something else"
 
@ihavenoidea when there's no other reference left to the Company, then yes. This happens in the first n-1 loop iterations when the next assignment to company removes the reference to the last Company, so you'd have to add del company after the loop or just wait until the function/script finishes
@Kevin weird
 
Understood, having only the last object not freed is not a big deal in my case. Thank you @AndrasDeak @Kevin
 
I hit my mental model with the Rigor Crowbar, and it started making a funny noise. Please downgrade my guess' epistemic value to "seemingly correct in some cases, but perhaps only by coincidence" until the machine is repaired
 
In [10]: import sys
    ...:
    ...: class Foo:
    ...:     bar = 42
    ...:
    ...: foo = Foo()
    ...: bar = foo.bar
    ...:
    ...: lst = [foo for _ in range(1000)]
    ...: print(sys.getrefcount(foo))
    ...: print(sys.getrefcount(bar))
1002
307
don't ask me
 
4:43 PM
Ok, useful data for my recalibration
 
@ihavenoidea of course there could be any kind of side-effect when instantiating a custom class, so references could stick around despite the attribute not having references. For instance classes might keep track of their instances.
If you're being paranoid you could leave a weakref on the last company then del company and check if the weakref gets cleared.
@AndrasDeak hold your horses
I had a stale interpreter
 
weakrefs, good suggestion. Nice way to test in general which objects drop to refcount zero, and when
 
In [1]: import sys
   ...:
   ...: class Foo:
   ...:     bar = 42
   ...:
   ...: foo = Foo()
   ...: bar = foo.bar
   ...:
   ...: lst = [foo for _ in range(1000)]
   ...: print(sys.getrefcount(foo))
   ...: print(sys.getrefcount(bar))
1002
28
 
import weakref
class Company:
    def __init__(self, ceo):
        self.CEO = "Mr. " + ceo + " the third"

my_list = []
weakrefs = []
for name in ["Kevin", "Andras", "ihavenoidea"]:
    company = Company(name)
    my_list.append(company.CEO)
    weakrefs.append(weakref.ref(company))
print(weakrefs)
#result#
#[<weakref at 0x005D0D48; dead>, <weakref at 0x005D2AF0; dead>, <weakref at 0x005D8690; to 'Company' at 0x00577B38>]
As Andras described, after the loop, the first two company instances are gone, but the final one remains
 
Hey. I have a simple canvas used in tkinter app. How can I update elements on the window in a loop?
smth like:

gui = GUI()
gui.redraw(STATE)
gui.root.mainloop()
while ...:
gui.redraw(STATE)
 
4:57 PM
Lol, love the code :) Thanks again!
 
If you're like me and love gratuitous status messages being printed to stdout, then ref's callback argument gives you even more power in determining the time of collection of objects. Example: pastebin.com/fRhdW1Rj
I hit an interesting race condition while writing that code: if you don't return weakrefs, then they get collected at the end of the function, at the same time as the third company instance. So the callback might cease to exist before the company instance does, and it will never fire the third time. I wonder if the order of deallocation is well-defined...
@entithat A while loop appearing after mainloop will not execute until the user closes the window. To make something happen periodically in a tkinter application, use root.after
 
don't I need to use two threads to make it works?
because I'm calculating smth in this while loop and then I want to print it on the window
 
Using two threads is one possible approach. The nice thing about it is, your existing while loop can probably retain its overall structure, and you just need to paste it into the function that's being run by the child thread. The bad thing is, it takes a moderate amount of boilerplate to get that thread to communicate properly with the main thread where tkinter is running.
It is often easier to use a single-threaded approach where root.after eventually calls a function that does a little bit of work, then calls root.after again and returns. Eventually the little bits of work add up to a complete task, while still giving tkinter the idle time it needs to repaint the window and respond to user input and such.
If you're still stuck after I get back from lunch, I'll go into more detail
 
hm ok, I will try to fix it. thanks
@Kevin, hmmm it's just stuck. Nothing is happening. (look at the end of the file) pastebin.com/3gBE6Kgm
oh, I forgot to call the function. Fixed it. Thanks a lot
 
AAB
5:34 PM
cbg all,
Need some advice on django webapp
a query set is not evaluate till its sliced or length is called
So when I use django pagination with page length say 30
The Database is called a sort is done and top 30 records retrieved
when user moves to page 2 db gets hit again sorted 30-60 records gets retrieved.
In terms of design is it better to just retrieve all 700 (max not more than 3k) records at once and do client-side pagination using datatables?
the table will have 4 columns only
What are the disadvantages of retrieving all records at once?
if many users are logged in will retrieving all records hurt performance more?
 
@AAB Are you implementing your own pagination? Is this for a REST framework or something? If so, I suggest using django-rest-framework and its built in pagination implementation.
but to answer your question more directly...the disadvantages of retrieving all records at once are many. There is a performance cost up front for fetching them instead of fetching those for the current page. Also, if any of the records are modified by another process (such as an update from another user), you have to do another fetch to update the frontend. On the other hand, if you fetch one page at a time, you will get the most recent data for that page.
 
AAB
6:15 PM
@Code-Apprentice Not using Rest calls as of now user requests a page and in the django view I get all records. Use datatables client side.
I want to use djangos pagination, but I see how datatables serverside flag makes things look so nice and pretty.
I was planning on creating a view for this and return JSON response.
 
@AAB there are definitely advantages and disadvantages for server-side vs client-side pagination. I've done both. It just depends on the constraints you are in.
 
AAB
do I need to use django-rest-framework?
for this
@Code-Apprentice I just can't make up my mind I feel the load will be high and user may find what he is looking for in the first 2 pages
 
I have a strong preference for a separate backend/frontend. I'm most familiar with doing this with DRF and a JS framework, such as Reach.
at the end of the day, the most important thing is to implement something that works. Then you can measure performance metrics to see if there's anything you can optimize.
if you have time to prototype two different solutions then you can run a profiler on each and compare
 
AAB
@Code-Apprentice I have not used a profiler so far, will need to learn about them :/
are they like Apache Jmeter?
 
@AAB I haven't much either, I just know they exist.
I'm not familiar with Jmeter
after googling it just now, it looks like it is intended for a different purpose than what we are talking about here.
In Python, I'd check out newrelic. I've only heard of this recently and I'm not familiar with it at all.
 
AAB
6:30 PM
@Code-Apprentice I dont know what a profiler does I assumed it tells how much load your app/function can handle or works in
 
if you use pycharm, I think it has a profiler built in.
 
AAB
@Code-Apprentice thanks I will look into it.
 
@AAB not quite...it measures how much time each section of code takes...and I think it measures resources like memory, too.
 
AAB
@Code-Apprentice Nice :)
@Code-Apprentice thank you for ur time will look into DRF
 
7:03 PM
@AAB Where do you plan to store the queryset between client requests for the different pages? Putting that kind of data volume into web sessions isn't terribly good practice.
 
AAB
@holdenweb I don't plan to store it.
 
So you plan to serve them all in a oner and then let the client sort out pagination?
 
AAB
@holdenweb yes I am doing that now retrieve all and use data tables on client-side for pagination and search.
I am planning to switch it to data table with server side processing
 
And the database doesn't handle LIMIT queries the way you think it does.
 
AAB
@holdenweb my query uses order by on a column(desc), so each time I request a page that query will be run only the start, end change right?
 
7:11 PM
 
AAB
@holdenweb all rows are computed even if I mention offset
 
Not all rows, but it will have to construct an index table internally in order to know which records to put in the result. So indexing will potentially make a difference to performance. DBAs typicalyl know a lot about the storage methods of their chosen DB
If you index the columns in the ordering with the key columns in the right order that will be optimally efficient.
 
AAB
So pagination may not be as bad as it seems
@holdenweb not sure if the data I have is less but order by does not always use the index
when I run explain analyse it shows seq scan for one of the cols
 
7:27 PM
yesss, finally 500 rep. This thing is gone now....
 
AAB
one such query is select name from table order by similarity(name, 'query')
I am using pg_trgm and have created a gin using it as suggested postgresql.org/docs/12/pgtrgm.html#id-1.11.7.40.7
It never uses the index for this
whereas for some other column like price its does use it
 
@12944qwerty wait what... it's still there? (in the review queues)
 
You need 2000 rep for edits, don't you?
 
Oh yeah.... I remembered incorrectly
5
A: Edit button tells me that suggested edit queue is full, but it's empty, what's wrong?

user4639281 I've noticed that the edit button is disabled for me ... suggesting that "suggested edit queue is full". This means that the review queue for suggested edits has reached 200 suggested edits and no edits can be suggested by anyone (users with less than 2000 rep, as users with > 2000 rep do n...

 
7:50 PM
Daily reviews don't have any perks or any type of consequence right?
 
no, it's mostly altruism
They need patience and meticulous attention so I don't recommend them.
 
I mean, I get 2 rep
there are so many questions I see now, that have incorrect grammar and formatting and they just bug me
 
@12944qwerty 20% of your suggested edits in the last 24 hours were rejected, and another 20% is still pending (so they could go either way). I suggest revisiting your rejected suggestions to see what you should do differently.
 
How do I know if they're rejected or not? I couldn't find a place where I can see the edits I've made
 
8:05 PM
 
Why was this rejected? I had done the same thing that the person who edited after me did...
 
I was just about to link that as a non-helpful edit
The editor also removed the Thanks from the end. But most importantly, that question has code in a picture. We don't want that. It's a low-quality question. Don't suggest edits on those, they are a waste of resources.
 
Yeah... I guess
This edit conflicted with a subsequent edit. What does this mean? The edit after mine didn't look like it conflicted
 
There's a broad category of trivial edits that are fine from a 2000+-rep user, but are problematic when added in an edit suggestion. If you make people review your suggestions, make it worth it.
@12944qwerty if a high-rep user makes an edit while you've submitted your suggestion yours automatically gets rejected. It's a race condition, don't worry about it.
If you can submit your suggestion before any high-rep user can open the edit dialog, they will be sent to your suggested edit review item to handle first.
 
So in this instance, they opened the edit page while I was editing myself?
 
8:11 PM
yup
 
Is that also why it said that Community reviewed mine instead?
 
Community is responsible for binding votes made by people who don't normally have binding votes. For edits this means 1. when edits conflict like this, 2. when OP rejects your edit, 3. when a high-rep user makes an "improve edit" or "reject and edit" choice in review.
so yes
 
I feel like I just joined meta chat instead of python. :) Useful stuff to know, though.
 
Welcome :)
There's usually less meta. Not necessarily more python.
 
Hee.
 
8:19 PM
It's been... uh... *checks transcript*... a while since the last one, so here's a riddle even though it's not all that interesting: What's the output of this code?
class Foo:
    def get_class():
        return __class__

    print(get_class())
 
hmm
the only reason I'm not entirely surprised is the context
 
(Obviously you can just run the code to find out, but that's cheating, and the punishment for cheating is a slap in the face with a wet fish.)
 
Does this mean view spoiler?
 
(Since we don't know where you live, you will be expected to enact the punishment on yourself)
 
My gut tells me something like __main__.Foo. Checking my gut.
hmm, There is all sorts of wrong about that.
 
8:27 PM
Ugh, my spoiler is too long
@AndrasDeak view spoiler
 
Neat, thanks
 
9:04 PM
@Aran-Fey Aren't you missing a self parameter?
 
Nope
How would I call it if it required a self argument?
 
Habit I guess... putting self in all (except class and static methods) methods in the classes...
I didn't even know you could do call methods in the body of a class
@AndrasDeak I probably should've gone in that chatroom I guess 😂. I forgot about all the other chatrooms
 
9:23 PM
 
Oh wow, it's empty...
 
Did you just ask there what I already told you?
 
It's different..?
 
right
 
@holdenweb with django, you rarely write database queries directly. The ORM takes care of translating your python code into SQL.
 
9:40 PM
@Aran-Fey How often do you plan to do these?
 
Planning? One per day. But in reality? Maybe one per year. We've already used up all our good ideas: sopython.com/wiki/Riddles
Hmm, looking at some of those, I've used up plenty of not-so-good ideas as well...
 
These are cool... Imma take a look at them later
 
How can I call an instance method with parameterize? I tried the following, but it doesn't work.

class Test:
def data(self):
return [(2), (4)]

@pytest.mark.parametrize('val', self.data())
def test1(self, val):
assert((val % 2) == 0)
 
@user2420374 don't call the method, refer to it as self.data
@user2420374 and please see our code formatting guide to chat and practice in the sandbox if necessary
 
I'm on my phone, how do I indent?
 
9:51 PM
good question
 
ok will try with self.data
 
Wouldn't you do extra spaces before each line?
 
ctrl-k works on my phone but I have an extended keyboard
@12944qwerty that works but also very hard to do correctly on mobile
 
True
 
9:53 PM
What? I was helping....
Can someone flag an answer for me? I've run out of them...
 
with self.data I got the following error:
NameError: name 'self' is not defined
 
@user2420374 ah, yes, of course
 
self is defined on a local scope (which is the methods). You cannot call it from outside the scope (which is the decorator)
 
Is the method really an instance method? You could make a classmethod or staticmethod work there. But I suspect this is an XY problem, but I don't know much about testing.
I didn't even know you needed a test class for pytest, the uses I've seen used regular functions.
 
10:06 PM
should I have to move that method outside of my class to achieve this? doesn't work as staticmethod
 
Oh, right again.
The decorator gets called while the class is being constructed, so it doesn't exist yet
 
it's easier to store variables coming from autouse fixtures with classes
no issue, I think I'll move that method outside of the test class
 
let's hope someone with more expertise can eventually help
 
What's the X in this XY problem again?
 
Hello
 
10:12 PM
hello
 
Cbg
 
I am need some help on how I can start profiling of live Django web app, there are lot of links I found, but there are too old. If someone please suggest better and recommended way (latest and efficient one).
 
Sometimes the older ones still work today
 

« first day (3852 days earlier)      last day (1096 days later) »