« first day (881 days earlier)      last day (4292 days later) » 
00:00 - 18:0018:00 - 00:00

00:03
that issue was 3 years old and the reason I used the proprietary broadcom driver. unfortunately, it now crashes the latest kernel, so went back to b43 and just had to fix it (although it took 20 mins too long for SO).
c'est la vie, as the french would say.
00:49
<script type="text/python"></script>One of these days
Anyone here know to use python with TideSDK? tidesdk.org
 
2 hours later…
02:57
Plug for a question that sort of slipped through the cracks:
1
Q: Using python ElementTree's itertree function and writing modified tree to output file

user1456830I need to parse a very large (~40GB) XML file, remove certain elements from it, and write the result to a new xml file. I've been trying to use iterparse from python's ElementTree, but I'm confused about how to modify the tree and then write the resulting tree into a new XML file. I've read the d...

 
6 hours later…
09:09
hello
wsup
I ate goat cheese yesterday
not good idea
running internals?
noup, just smelling like goat.
09:49
would you remove the first paragraph in this stackoverflow.com/q/2001190/321731?
 
3 hours later…
13:07
Hey seberg
still, sounds like the wrong solution
very wrong :)
Maybe
1. you can use the out argument to ufuncs
I've thought about that too ...
So here's how my code works -- I have a datafile class which wraps around a reader instance.
2. why do you have these huge arrays if you do not use them?
they come from a file?
13:08
Yep
then... maybe you should memorymap those
the reader is responsible for actually getting the data out of the file
or, resize them at least before you do anything else... when refcount actually is 1
different readers can be used for different files
e.g. a reader for hdf5, a reader for our hand-rolled compression algo, etc.
Yeah, I have a question about what refcount actually is.
number of references pointing to the actual object, heh
13:11
a = b = c = np.arange(50)
3 references, one object
That has multiple references, but it doesn't seem dangerous at all to resize a
In python, if I have a list: ` a = b = c = [1 ,2, 3, 4]`
yeah well, it is a very special case
in that case you can also write
then I do an inplace operation on a, a.append(5)
temp = np.arange(50); temp.resize(); a = b = c = temp; del temp
13:12
I expect b and c to change too
and you should write it like that... don't do that refcheck=False unless you know very well, why you do it, and that it is safe you do it in all cases
b and c are the same object
exactly
So the base reference counting isn't sufficient.
but say you have; a = b = c = np.arange(50); a = a.view(); and you are already busted
If there was a "check to make sure nothing has views of this" option to resize, I'd use it in a heartbeat
oh, it is sufficient, it just can be stricter then necessary
mgilson, that would require you to check who holds the reference, which is impossible
its just a bad idea
13:14
I know that it would be very difficult to implement
actually, it is impossible
and probably not worth it
Not with weakrefs
even with weakrefs I am pretty certain you cannot do it
when I view is created, it could register a weakref with the actual object.
not generally... for a specific task, sure, but generally no
13:16
or register a weakref with the object which owns the data
mgilson, a C realloc may invalidate the buffer of any view
which means, you cannot safely realloc, unless you first tell everyone holding a reference to yourself to copy the data to be certain it is OK
which basically means, just copy it in the first place... because it is hard to imagine to be worth the trouble to try to delay that with some dark magic
I understand that it invalidates views, but it doesn't invalidate other references to the same array
That's my point
yes, but how do you know you don't have any views around?
thats big time playing with fire, assuming you know that... change anything anywhere and booom
By doing this in the reader -- All before the user gets access to the array
Perhaps it is a problem ... but I don't know a better way around it
why resize?
why the trouble?
if you need a small chunk, copying that chunk should hardly get you into huge trouble
13:20
I don't want to copy because I don't know what attributes the reader will add to the array
I mean the worst thing happens, before doing anything else you double the usage of one file
A copy would lose those attributes and possibly break the reader
mgilson, the reader should know how to finalize a copy
Why?
That's the responsibility of the Data
to allow to do copies, etc. ;)
well, the Data object yes
13:21
The reader only needs to return a working Data object from the file.
so if you slice something out of the data object, then copy it... then you can throw away the original data... no problem
That's it's only responsibility
-- and writing I suppose.
The reader/writer are a single object (Perhaps that's the problem)
not sure
anyway, still I am not convinced you need to safe a single small copy
No, a single copy isn't a problem
then what is?
13:23
It's putting that copy into __array_wrap__ or __array_prepare__ that I'm opposed to
why?
Because that's an additional copy for every ufunc
whats what array_finalize is for...
no its not
How not?
how is it?
array_wrap just creates views
13:25
if I call a + b + c, the parser does (a+b)+c. A temporary array is allocated for the result of a+b and for the final result.
and how does resizing change that?
resizing doesn't change that. But if I do put a copying operation in __array_wrap__ then I get an additional copy for each +
I didn't say you should put a copy into the array_wrap
That's what I'm wondering about.
Where should the copy go?
but, if you have a huge array, and know you only need a small part of it, well then copy that small part out
so that you then can delete the big array
13:28
That's what I'm trying to do ...
more or less
you don't need to resize for that
or if you know you only need the first half of the large array, then resize it before you take any views or have more then one reference
This is something that I thought about in the beginning ...
add __setattr__ to update a _dict and then have my copy operation make sure that everything in _dict gets transfered to the view. I suppose that would work.
if your reader has its own array subclass, it should copy the _dict for you...
@seberg -- Yeah, that's what I'm doing now...
The problem is that I don't want _dict to be copied exactly on everything.
e.g. it needs to be modified with copied to a view
but only some of it should pass through when using a ufunc
This is a bit more of a design problem than I had originally anticipated ...
well you can get more control with it through changing array_wrap?
13:47
Yeah, that's what I'm doing.
Does numpy.copy preserve order?
if you ask it to
ndarray.copy(order='A') or something like that?
oh, actually its the default
'K'
'A' means fortran order if fortran order and not C-order
but ndarray.copy method defaults to 'C'...
Yeah, but the problem here is that order = 'K' turns my ndarray into a C_CONTIGUOUS array
so?
13:55
All of the F_CONTIGUOUS and C_CONTIGUOUS stuff really frustrates me here too.
don't even get me started...
When I pass this copied routine to Fortran, f2py will transpose it.
mgilson, there is only one reason for that being possible... and that is you have a (N,1) or similar shaped array
'K' keeps the order, if it was F contiguous, it will be F contiguous
if it was neither, well, then in some cases 'K' might guess the wrong one, when actually both should be True
if you know you want fortran order, then why not force fortran order?
13:58
a = np.arange(50).reshape((5,10),order='F')
b = a[2:4,3:5]
b.copy()
print b.flags
I said b.copy('K') didn't I?
You said 'K' was the deault
no, I said 'C' was for the metho
+d
Or maybe I misunderstood
K is for the function
14:00
Ahh, bravo
That works out properly
it doesn't in some corner cases, but I will not rant about that
and you should not worry about that
shouldn't worry about it?
corner cases are all I've been thinking about lately
... too big to fit into memory ...
thats not something I mean :)
it can fail in some cases where numpy refuses to say that it is both C and F contiguous
its not like fixing numpy is hard, its fixing the third party code making bad assumptions ;)
anyway, doesn't matter... I hope you are not still trying to figure out a safe way to use refcheck=False and resizing ;P
I'm trying to figure out a sane way to copy my array (which really feels more difficult than it should be).
risking invalidting views is a bad idea
14:05
Maybe ...
I still feel like in numpy you need to be careful whether your working with a view or an array anyway
only if you assign to views though
If you need to know the difference for things like __setitem__ anyway ...
resising, you can do a termonuclear explosion even for readonly arrays
Yeah, I guess that's true. It does add an extra layer of unsafe-ness
ndarray.copy(self,order='K') will give me a subclass, not an ndarray I assume?
yes, and it calls array_finalize
but np.copy does not
14:09
And, when using a ufunc, __array_finalize__ gets called on the output array (assuming it wasn't given as input) before or after __array_wrap__ on the highest priority array?
not sure, depending on how array_wrap is written, maybe array_finalize is not necessarily called
(if you don't know, I can write a simple script to test)
My __array_wrap__(self,obj,context=None) simply tries to transfer the meta-data from self to obj
array_wrap usually needs to view the to be wrapped array, and I guess that then calls the finalize
if you only do metadata transfer, you don't need to define your own array_wrap
array_finalize will do
But, when I transfer that stuff over, I want to make sure that it gets written to obj._dict which gets created in obj.__array_finalize__
or, maybe not sorry
anyway I think you can find examples for that
14:14
The examples aren't as illuminating as I'd like, but I can try to unravel it.
isn't there exactly an example for a _data attribute?
There's an example for a info attribute.
sounds like what you got
but I haven't seen one for a _data dict which gets updated with __setattr__
whats the difference?
except that it needs to default to existing
14:16
Just *when*` __array_finalize__` gets called.
I need to make sure that __array_finalize__ has been called before I try to object.whatever = something
no you don't
numpy makes sure
Alright, I think I've got it now ...
Thanks for all the discussion
I hope I haven't been too draining
15:01
Hey all
@mgilson: any word from the big G yet?
Nothing official
I emailed my recruiter yesterday
It takes them such a bloody long time sometimes.
He said "things are progressing, but not confirmed yet"
One engineer not yet turning in his evaluation and everything is stalled.
and that i should "think seriously about any concerns I'd have joining Google and moving to Silicon Valley"
so that's more promising than the "No, sorry, we just forgot to give you a call" that I was expecting when I sent the email
15:08
@mgilson: Why should \b be included in the remove string?
\b is a word boundary anchor; the code should just work (provided there are no regex meta characters in his word list), the post is missing detail as to what doesn't work.
Yeah ... that's what I get for working late last night.
:)
I'm trying to get my code in a place where I can hand it over should the need present itself :-P
hrm
indeed, the code does not work.
my head is not operating today in any case.
so I am missing something here and the darn grey matter in my head is just not firing up to point out what it is.
Hello all, I was wondering advantages of overriding __init__ using method in this (stackoverflow.com/questions/15435705/redefine-init#15435705) question opposed to sublcassing and overriding?
Just allows you to dynamically change the __init__ method?
Oh raw strings. That's why you use string formatting rather than concatenation
indeed, that's the part my brain should have been firing the warning signals.
I am seriously considering a nap.
15:22
hello pythonistas :)
hello
hello, happy friday
lol....same to you
Spring Break's about to end; this calls for a dang it moment!
hey some people have confidence with interable object in a class?
post an example? i read good article on creating iterator a couple weeks ago nbviewer.ipython.org/urls/raw.github.com/wardi/…
15:30
He's probably referring to his recent post on SO.
0
Q: error in a python class with an not iterable object

Giannii have a the following error message Traceback (most recent call last): File "C:\Python27\lib\site-packages\IPython\core\interactiveshell.py", line 2721, in run_code exec code_obj in self.user_global_ns, self.user_ns File "<ipython-input-257-84ae6ec7b6f6>", line 1, in <module>...

I'd say, try out NPE's solution and see if things are still not working.
That was my first instinct too; perhaps the API is expecting a sequence and you just passed it a single object instead.
So who here knows enough about matplotlib?
Not that I am really inclined to keep on fixing the next problem the OP finds every time you solve one.
is there a solution? — wiso 10 mins ago
@dm03514 i forgot to put "self."
it's easy but i spent 45 minutes to run and re-run the code
The Friday programming disease :D
I spent ten minutes earlier today debugging a line that turned out to have an additional parenthesis.
Which is especially grievous because my IDE has highlighting for matching parenthesis pairs.
15:42
i just turn in PyCharm
today
I like IDE clean
but I miss open declaration of method
user559633
16:27
i want a way to get an RSS feed of @MartijnPieters answers
There is one, actually.
Bottom right hand side of any user page there is a RSS icon.
On my profile page the user feed link leads to stackoverflow.com/feeds/user/100297
user559633
Oh. Amazing. I like it because I compare what I would submit to an answered question to what you've submitted.
What, my first revision or the final one? :-P
I do edit my posts a lot..
user559633
The original revision. It's like Martijn unplugged
17:00
I finally got around to experimenting with collections.Counter today. It's very useful!
It definitely is!
Guess I can throw away the homespun class I used that does the same thing.
It's nice for detecting anagrams, slightly more efficiently than sorting both words.
Exactly :-)
Great little tool to know about when you are being asked in an interview to find all anagrams in a given file...
17:20
Today's one-liner sideshow horror: takes a list of words and groups them by anagrams.
reduce(lambda a, b: {k:a.get(k,[]) + b.get(k,[]) for k in set(a.keys()) | set(b.keys())}, [{"".join(sorted(w)): [w]} for w in words], {}).values()
Sample output:
[['sheet'], ['cater', 'crate', 'react', 'trace'], ['able', 'bale'], ['beyond'], ['list', 'silt', 'slit'], ['boat'], ['cat'], ['acre', 'care', 'race'], ['lawn'], ['binary', 'brainy']]
I wanted to use Counter instead of sorted, per the previous discussion, but I needed a hashable key :-(
This would be a lot simpler if there were a relative of dict that made addition easy, ex. {4:[8]} + {4:[15]} yields {4:[8,15]}
@Poik double major?
Yes.
Well, dual degree, but it's about the same.
I have respect for any student having a double major, especially in any engineering and Math/Physics/Chemistry
17:37
@Kevin: You could use dict views for those set intersections
@Kevin: a.viewkeys() | b.viewkeys()
Interesting @MartijnPieters, thanks :-)
oh, I hate that. A tease accept-unaccept.
you win some, you loose some.
hey some people know how to save "in an elegant way" a text file with different Delimiter?

line = [FID] + accuracy.data
line = " ".join([str(e) for e in line])+ "\n"
f.write(line)
I feel guilty doing an accept-unaccept. So I usually wait a few hours to do the first accept in the first place. But then I'm exchanging one form of aggravation for another. "Why isn't he accepting my obviously correct answer?"
in these example i can save with "space"
17:47
@Kneel-Before-ZOD Thanks. Really, I'm getting a dual degree because I'm stubborn, much less any other admirable quality. I suppose stubbornness helps out every so often, though. :)
@Gianni, wouldn't you just do myDelimiter.join( to save with whatever delimiter you want?
never used
myDelimiter being short hand for any string, ex. "|".join(, "\n".join(, etc
i always used " ".join, "\t".join
yes i will update my code
thanks Kevin
No problem, but I think the knowledge was within you all along :-)
@MartijnPieters I don't think I've ever seen this "timeline" page before. Interesting.
17:53
it's kind of deprecated.
You need to know about it (just add /timeline), IIRC SE wants to improve and update it still.
@kevi
sorry if i desturb
if i use how you suggest: ex: ";" i got:
line = sep.join([str(e) for e in line])+ "\n"
'F;I;D; ;c;e;n;t;r;o;i;d;X; ;c;e;n;t;r;o;i;d;Y; ;a;r;e;a; ;p;e;r;i;m;e;t;e;r; ;s;e;g;m;e;n;t;s; ;r;a;_;o;r;..............etc etc
oops
00:00 - 18:0018:00 - 00:00

« first day (881 days earlier)      last day (4292 days later) »