« first day (1739 days earlier)      last day (3434 days later) » 

DSM
DSM
17:00
How did you make the matrices in the first place if you didn't do a loop?
Hehehe....
I did it by hand, I wasn't sure if this was good or not- I thought I might need them later on in the code enough that it would be worthwhile.
DSM
DSM
Urf. No, start from your basis matrices and build from that.
You mean build in the code?
So just kron() when I need them?
DSM
DSM
It's fine to materialize them, but you shouldn't be typing 16 lines which look like kb14 = np.array(stuff) or whatever.
Well I mean, I used the computer to generate them to some extent (just copy and pasted the output of the kron() function), but I agree this was suboptimal.
But uh, I think I get the idea.
Thanks everyone~
17:05
Do we have a canonical Q&A somewhere about package structure?
I couldn't find anything in my (brief) search on sopython/canon
Don't think we've got one on there, no.
If someone feels like writing one, stackoverflow.com/q/31544794/3058609 this question is a pretty good one to build off
I don't have the time -- in fact I'm walking out the door now. RBRB all!
I never got the hang of packages.
DSM
DSM
Rhubarb for @AdamSmith!
I'm trying to get the hang of packages. The package is trying to hang me.
DSM
DSM
17:09
Aww. Well, we can't let that happen or you'll speak badly of us to your Ruby friends. Anything in particular we can help with?
Now that I'm old, my brain is too brittle and chalky to take in new information. I guess I'll just write single-directory projects forever.
@DSM Speak badly of my Python friends? Never! -- Let me pound on it some more and see if it gives. Code has a yield strength, right?
DSM
DSM
Eventually it breaks, which is sort of the same thing. :-)
Question on 3D: how can you segment a model into "parts", but load it like a single model?
17:15
Binary space partitioning?
"package" means something I upload or install from pypi, right? And "module" means, roughly, a namespace. And a package has a module (always? usually?) within which its classes and functions are contained. Are those the correct meanings of those words?
"module" is a single py file, and "package" is a directory with __init__.py in it. Then there's the parallel concept of "package" as an installable thing, even though it could just be a single module.
I don't think that a package necessarily has a namespace, you could probably export a single function, it just wouldn't be good practice, no?
So a package is a kind of module.
My brain keeps saying, "But in Ruby..." and I keep screaming at it to shut up, because I can't learn Python with it yacking at me all the time about Ruby.
17:23
Yeah, I have to turn of my reptile brain when I switch to Java.
@WayneConrad it's okay, I do the same with javascript
What even is he upset about.
So, if I say "from foo import Bar", foo could be a foo.py just sitting there in the same directory, or it could be a directory named 'foo' which contains an __init__.py which either defines bar() right there, or imports bar() into the foo module from (for example) foo/bar.py. Is that right?
DSM
DSM
17:27
@Kevin: DVs, my guess.
If he remembers SO not having downvotes, then the problem is with his memory :-)
@Kevin He asked a bad question, and got mad that people realized it was bad.
Woohoo, I got package (meaning #2) uploaded to testpypi.
Speaking of being upset, I remember this :D
Snark aside, I'm surprised he didn't get an answer, since "can I run .pyc files or bytecode directly?" seems pretty objectively answerable.
I don't know what the answer is myself, but surely it's somewhere in {yes, no}
DSM
DSM
17:34
Some questions have the answer "no", cf. the gifs from before. Doesn't make them bad questions.
Some questions don't even make sense in a Python context, but if they make sense in some context, they can still be valid questions, IMHO.
I wish people couldnt delete their own questions unless they have enough rep
even if they are dumb questions ... there was one that a pythontutor.com visualization answered perfectly ... but OP realized the answer from a comment and just deleted his question
With wxPython, is there a way to iterate over a datespan object? Eg, if I create a datespan of 6/1/2015 to 7/5/2015, can I get a list of each date (inclusive) in the span?
17:39
I really wanted to put the pythontutor link in the comments
@JoranBeasley WRT the "house robber"?
lol @MorganThrapp I didnt even know there was a datespan
@BhargavRao yup
@JoranBeasley Hmmmm. :/ I have two date controls on my form, and I want to do something for each day in that range.
Is there a better way?
DSM
DSM
I don't use wx, but does datespan really contain start and end information or is it simply a duration?
while now < end:
now += timedelta(days=1)
17:42
@DSM I'm not sure.
DSM
DSM
How did you make the DateSpan object in the first place?
date_span = start_date.DiffAsDateSpan(end_date)
[start_date + timedelta(days=i) for i in range(date_span.GetTotalDays())]
DSM
DSM
It looks to me like DateSpan only contains duration info and so by itself can't give you what you want.
Some variant of what Joran is suggesting should work, though.
@JoranBeasley I could do something like that. I'd have to do .GetTotalDays*-1 because for some reason it gives the number of days as a negative.
17:45
ummm whoa you got a problem i think
It also doesn't take months in to account.
that should not be giving a negative ... unless its actually negative
maybe you need to do end_date.Diff(start)
that would be end - start
Eg, 6/5/2015.DiffAsDateSpan(7/6/2015).GetTotalDays() returns -1.
Yeah, maybe.
start - end
DSM
DSM
This class is very strange.
17:46
Very, very strange.
its all built around c/c++
you should probably just convert it to normal python datetimes that would be easier
DSM
DSM
Its getTotalX methods aren't what I would expect them to be.
@JoranBeasley How? I was looking for a method to do that, but I couldn't find it.
@DSM Or what the docs seemed to say they were.
those date classes are all written basically because wx is C/c++ normally ... and you need a date/time class in c/c++
I guess I could do .getTicks() and feed it to a datetime.datetime.
17:49
pydate = wx.calendar._wxdate2pydate(wxtime)
is how I convert it
Oh, I'm using a DatePickerCtrl, but I'll give that a shot.
if you are getting a wx.DateTime it should work
oh I think you need import wx.calendar as just import wx doesnt seem to include calendar
dumb question but... what is the difference between a class and a module? Just that a module is already instanced?
a module is typically a folder with an __init__.py and several classes
I don't seem to have wx.calendar. :/ I'm using the 3.x fork (Project Phoenix)
17:52
did you try import wx.calendar?
if you do import wx;wx.calendar it wont find it
but if you import wx.calendar it will
ImportError: No module named 'wx.calendar'
(I think im not using 3 )
ahhh lame
that looks like its the source
ok, thoughts about this stuff that's under development? I have a ui directory/package with code that is not unittested for the most part. Then I have core config objects that are unittested. The core objects have methods that import from the ui and return urls. The dependency goes in the wrong direction, but it's convention to create urls in the ui package. Thoughts?
hmmhm
damnit I have yet another difficult problem...
Thanks. :)
17:54
need to calculate the frequency of some words
well all the words do not even fit in RAM
collections.Counter
the distinct words
I think nltk also is probably what you want :P
17:55
I really want the 1 million top words and their freq
nope, no need for nltk in this
you need to offload it from ram to a database
if all the words wont fit in ram
would be much slower
divide and conquer?
tradeoffs
ok coffee break
I put ulimit or that process, it would die at about 15 % of the data...
dunno if python is also leaking memory somehow :d
17:57
probably not. any list comps you can convert to gen exp's?
no this is just: read 1 line from file, json decode, re.findall with \W+
and counter.update with new words
I lowercased the words already but doesn't help
maybe I need to hash them and store the hashes instead and then rerun only seeing the top hashes :D
hash should use less memory...
hmm 30 million distinct tokens by now... maybe this is enough to see some statistics
maybe I'd need to do so that at 15 million I'd prune the lowest 10 million hmmh
sqlite db with table of words and count?
no
I do not have a year to do this :D
18:01
buy some more RAM?
ec2
not really
I think you could do 10%, write to db, do the next 10%, update...
data is 7 gigs so moving it out of there would take some seconds too :D
Oh, now I've made pip angry. I can't even uninstall my malformed package. Haha! Ho boy.
that could be worthwhile... but then I think it is ok if I just prune too small values
18:02
My file names may be prefixed with a number. How can I sort them according to that number?
you might prune the wrong ones...
I wonder if a trie would cut down on memory expenditure there.
@Kevin that could
but also slow things A LOT :d
Yeah :-/
18:06
You don't have space, so you have to make the time tradeoff, I think.
I am only interested in minimum 1/1million frequencies
Or spend more money on computing. It's the old triangle.
@AaronHall Reminds me of a dependency problem KevinScript had, where module A wanted to call a method in module B, but B wasn't fully initialized at the time A wanted to start importing.
yep, circularity
I guess it is enough to prune after each 10 % of data
remove those with frequency of 1 :D
18:09
You cannot guarantee that you get correct counts then.
I ended up making module A fully encapsulated in a class, with an init method like def __init__(self, method_from_b), so the main module could create the A instance at its leisure and pass in the B method when it was ready.
I'm not terribly fond of the approach but it worked
For example if 'derp' has 1 count in the first 9 iterations but then enough to make the cut in the last, you'll be off by 9.
Yeah, I don't want to monkey patch the object, but I thought about it...
@Ffisegydd true...
DSM
DSM
You could do an out-of-core sort first, so you'd know that all "derp"s were contiguous.
18:11
guess we'd have to live with that :D
I wouldn't attempt it in more type strict languages, but it was pretty painless here
The answer then would be to take your words that make the final cut and go through it all again just looking for those words.
hmm is gnu sort out of core :D
ah with postgres I'd do it this way:
load all the tokens in the db, then sort (postgresql has good out of core merge sort)
maybe I subclass the object in the UI stuff with the required methods. no can't do that
then iterate over the resultset counting
18:12
the objects are already instantiated by the time they get to the ui stuff, so they'd have to be monkeypatched with the method...
That sounds like a decent method.
DSM
DSM
@Recognize: use the key parameter to sort/sorted, and pass a function which finds the digits at the start and turns them into an integer (or, say, 0 if there aren't any.)
I still think using a DB to be your counter is the way to go...
Is What misspellings / typos are supported in Python? accurate? It claims that assret can be used in place of assert.
@Recognize search for python natural sort on stackoverflow main site :D
18:16
Not sure if he means it can be used as a replacement everywhere, or just in the context of the names of methods of the Mock class.
DSM
DSM
That is insane.
@RecognizeEvilasWaste stackoverflow.com/questions/2545532/… accepted answer looks ok
@Kevin he is a core dev
Still not clear on whether assret True would parse or not
18:23
There's been quite the kerfuffle about that on the python-dev mailing list. Ethan is being somewhat disingenuous in the service of snark.
I still don't understand what's going on well enough to explain it properly, but basically it's to do with not letting misspelled assert_foo() methods on mock objects pass silently when they should raise an exception.
@AnttiHaapala did you find a solution?
are you running 32 bit python?
Hey, is anyone good with salt modules here?
Particularly writing salt modules.
@MoAli I've been known to salt my fries.
what do you mean ... a salt is simply a string you append to content before hashing it
@MorganThrapp I wish I could throw some fry up my broken salt moduls.
18:32
hashlib.md5(password+"asalt")
def salt(s): return "Hugbees! " + s
Sorry, I meant to ask if anyone's good with writing saltstack modules.
@Kevin Hugbees!
I'm having trouble getting my module to work.
Hugbees!
18:33
o i c ...
All of my salt stack knowledge is biblical in nature
This is what I've got so far. https://gist.github.com/AkhterAli/a62ce524abb482e4aba1

However when I try to execute it, it returns 'checkhighstate' __virtual__ returned False
Yours may be biblical but mine is almost nonexistent, first time writing a module :)
That post is quite interesting from a moderation perspective ... he's been careful to make it perfectly legitimate within the norms of SO, but in the context of python-dev it's basically an act of aggression.
Ive never heard of of saltstack before just now
All I know is, when a city is getting smited, don't turn around to watch.
As usual, the Lonely Island gives life-saving tips.
18:38
I wish there was a saltstack channel in stackoverflow.
Huh. If you set something to False, it will evaluate as False. Could've saved myself a lot of debugging. facepalm.
@ZeroPiraeus A question should be judged on its own merits, so it doesn't seem like a dilemma to me.
I still don't know if assret True is syntactical or not, so the question's merits aren't that great
@Kevin it isn't :-)
lol @Kevin and his roll your own parsers
18:48
@JoranBeasley 64bit
even though I answered the same
basically
@WayneConrad Yeah, there's certainly scope for confusion with the current behaviour, so motive aside I think the existence of the Q/A is a net benefit from SO's perspective.
@AnttiHaapala so did you come up with a solution?
that didnt use a database?
If it's not clear that this isn't about the keyword from the answer, it could stand to be edited though ...
I prune the smallest counts after a slice of data
18:51
is that safe?
it seems like you could get very wrong answers
@JoranBeasley I hate and fear regex, so I'm quick to jump on the "implement a state machine from the ground up" approach
well enough...
if its not a normal distribution
hehe @Kevin I take it you were one of my upvoters for my pretty garbage state machine
:P
@JoranBeasley I am only interested in top frequencies that are in order of >= 1 / 1000000
Yeah, I'm assuming that your code produces the desired output, in which case it is worthy
18:53
so if i prune those whose counts in 10 % was less than 10th of that it is ok
Hey @Ethan :-)
heya @Ethan
Greetings! The "provocative" comment was correct, so I removed the mention of the reversal patch from my answer.
Ok, so if you scroll back you'll notice that there's a little confusion over whether this has to do with the assert keyword.
but if you prune out the smallish chunks earlier ... and then later they are more frequent you wouldnt be able to count all the ones you discarded earlier
18:59
I think that's basically down to the first sentence, so thought maybe that could be modified to make clear that it isn't.
consider the following
You causing trouble again Zero? I see why you decided to retire now... Bloody rebel rouser.
ahh let me come up with an example here it will take a few minutes
(and, given that this has gotten – let's say lively ;-) – over on python-dev, maybe somewhere less public than a comment thread would be a better place)
@Ffisegydd I'm a maverick ;-P
Risk-taker, even.
ehh its gonna be too painful to come up with an example but i dont think that that solution will guarantee the correct answer @AnttiHaapala
19:02
[edit complete] What do you think now?
looks good to me
Works for me ... have a couple of upvotes :-)
cbg
Also, since I accused you of being disingenuous ^ up there ^ I should point out that I basically agree with you regarding the "feature" :-)
That revision resolves the trouble I had with the original post. Thanks.
19:06
Goin out for food - rbrb
No worries. That happens every time I decide to look for more information - even when I don't have provocative parts. ;)
@Ethan Maybe instead of laying out the answer as some big revelation that it’s not the assert keyword, you could make it clear from the very beginning that you are referring to unittest.mock?
Like “As of Python 3.5, there is now a safety against misspelling in unittest.mock. […]”
Yeah, once things get as heated as they have on python-dev re: this, it can be difficult to read even the most carefully neutral post by a participant in the debate without projecting subtext onto it. On reflection I think that's whatr happened with my initial reaction.
Anyway, comments deleted in the interest of not fanning flames.
I don't care at all about mock either way, so I should be elected as the highly paid arbitrator.
Thanks for the help, everyone.
19:12
I have my own blindfold and scales, if that sweetens the pot any.
@ZeroPiraeus @EthanFurman sounds like the mock itself is broken
From what I've understood, mock is at least a little ungainly, yeah. Haven't thought things through enough to know whther that's avoidable.
@ZeroPiraeus it is the java idea copied to python world, yet again, and then mutilated beyond recognition
@ZeroPiraeus in java mocks, there are no methods being added to the damned mock objects at all
I don't feel like the question is really useful. Python doesn't "support misspellings", it explicitly doesn't support the misspelling or the correct spelling.
@Antti Mocking makes sense for unit testing, even in Python.
19:16
@poke I am not talking about mocking, I am talking about the asswards approach that python decided to incorporate in the stdlib :d
And it's not "supporting it", it's merely one added check in one place. It is in no way suggesting that using the word "assret" is a good or useful idea in your code.
The argument against is that it's an instance of "Do What I Mean" ... assret_ is specifically given the same treatment as assert_ in a way it otherwise wouldn't be.
the problem is that you're assumed to be calling assert_xxx on the mock object
which is just braindamaged
the java approach is to do:
Something(mock).assert_xxx()
uhm, okay, yeah, that’s weird
(I haven’t used unittest.mock in Python yet)
19:19
I would have expected some paired object which you can call the asserts on, so your mock object itself is free of everything test-related.
so in the first example if you do 'assert_called_whit'
you get tests passed
Like in the .NET Moq; you can create a mock, and then you have mock.Object for the actual object which implements the type you wanted to mock.
@poke ofc, that is the sensible approach, but it is "not very pythonic so we went with this instead"
@poke it originates from voidspace.org.uk/python/mock/mock.html
mock = unittest.Mock()
with mock as obj:
    obj.foo()
mock.assert_called('foo')
^ something like that would be cool.
yeah, something like that though it would be unwieldy with lots of mocks
19:22
Hello pythonistas, quick question - does PySpark work in IronPython (such that it could be integrated into a larger .NET application)?
I doubt it
Sure, why not? Did you try it and encounter a problem?
Actually ignore that, now I think about it.
… now I want to create a mocking library in Python…
but it would help if I had more than basic experience with unittests in Python… >_>
Mocking library: looks like a unittesting library, but actually just throws insults about your code.
5
19:25
@davidism No, I'm writing the other part of the program and just want to know if its extensible such that the PySpark parts can be part of the same solution.
"Mock is very easy to use and is designed for use with unittest. Mock is based on the ‘action -> assertion’ pattern instead of ‘record -> replay’ used by many mocking frameworks."
Note that I have no issues with magic -- I am the creator/maintainer for Enum, after all. ;) Unfortunately I had a bad reaction when I first saw assret and that didn't help. :(
DSM
DSM
@JGrindal: a quick skim of the code suggests that PySpark attempts to import modules not present in my IronPython. :-/
@poke that one is in the standard library
so now everyone will be using a broken mock lib in standard library... instead of yours :d
I know
19:27
@DSM This makes me sad.
instead no one is interested in parsing or outputting iso 8601 timestamps in stdlib
Why are those assert_X actually methods on the mock? Why aren’t they module functions?
assert_called_with(mock.method, 1, 2, 3)
@EthanFurman My initial reaction was also "no f—ing way" (also yay Enum) ... it does seem that, as @Antti says, the root problem here is that unittest.mock is just badly designed in the first place. I also thought some people were far too defensive on python-dev, but water/bridge etc.
because they assert on the object state (yeah that would be good too)
Lots of built-in functions also work on the object state..
DSM
DSM
19:29
@JGrindal: please don't take that as certain -- I don't use IronPython much (mostly to prove that something isn't compatible across implementations, TBH). But I'll admit it doesn't look promising.
What if the object I want to mock has assert_something methods that I need to mock? I have to activate unsafe now :(
So stupid
I can’t mock mocks!
mmhmm :P maybe we all should join the thread there :D
@poke I'm pretty sure we're doing that right now.
lol
why oh why this rsa library suddenly pops into python3wos
DSM
DSM
19:32
So there's a set consolidation question whose only quirk is that the data's given in a pandas dataframe. Write a new answer, or link to one of the gazillion consolidation questions?
Hahaha yeah, a bunch of people suddenly piling in on the mailing list is exactly what the cause of harmony over there needs right now ;-)
python-dev is just a _ _ _ _ _ _ _ _ _ _
Yeah, I think so too Zero. Let’s go.
We are, after all, experts at not pissing people off in here ;-)
Any one used bokeh with pandas? :)
19:37
"I suppose it's too late to change this so these aren't methods... ISTM
the problem is this collision of usage on attribute lookup - on the
one hand, it's conjuring mock method calls, and on the other, it's
implementing methods of its own."
DSM
DSM
@Jon: not really. I don't like the way bokeh looks at all compared to seaborn, so I've given it a pass.
@DSM Is there a way to make seaborn render a heatmap as an html table instead of a 800px by 10000px image though :)
DSM
DSM
Ummm..
I basically want a heatmap with varying background colours - but don't need it as an image :)
DSM
DSM
19:39
@Jon: what is that I don't even
Sent you an example of something
A commonly used mocking library for Ruby can make mocks that won't respond to any method that isn't implemented on the thing being mocked. This prevents method name typos from being silently swallowed by the mock. Does such a thing exist in the Python mocking world?
@WayneConrad NO!
Rhubarb people
@AnttiHaapala Someone wrote my thoughts down, nice!
Rhubarb @thefourtheye
19:41
@WayneConrad ofc it does exist, in the other sensible alternate implementation that is now being run over by this one in the stdlib :P
“So -1 from me.” – I have no idea what they are actually +1 and -1ing on… the reversal of that assert thing or what?
DSM
DSM
I don't use this stuff enough to have an informed opinion, but to a clueless outsider's eye it seems like this is a solution to an entirely self-inflicted problem.
I believe you can specify the object to mock when creating the Mock -- then it's AttributeErrors all around if you ask for something that doesn't exist on the spec'ced object (I think).
the problem is that the very instant I looked at this mock lib I was like "W00t??"
but it makes the "design that is different from all the other mocking libraries" a "virtue"...
@EthanFurman That's how we do it in Ruby, too. Unfortunately, the good mock call (the strict one) is a different function name, and not the one that everyone has learned to use.
19:48
because the author, who speaks in 1st person singular, then says that he does not like patterns in others
What a hack! Giving an NPC a train hat rather than building an actual train...
@DSM I swear it's an X Y problem. I just wish I knew Python mocking well enough to be able to pound on the table and declare it with authoritay.
@DSM yeah... it's about 2/3k rows and 30ish columns
user559633
@Ffisegydd it's amazing
you use create_autospec to prevent other attribute access, and in fact lists avoiding typos as one of the main goals
19:50
@WayneConrad @EthanFurman is right ;)
user559633
When attending a talk, do you want to hear details about the speaker or would you prefer he just goes into the talk?
though reading the docs, it again looks a bit troublesome
IIUC then you'd need to instantiate an object to use as the spec :d
@DSM there's also vincent... but that adds yet another bit of middleware
@tristan I want to know if the speaker is a subject matter expert on what he's talking about. "Hi, I'm Guido, and I invented Python." I don't want to know things that don't lend authority to the talk.
19:52
@tristan I like a very brief into. 30-90 seconds.
@tristan depends on how long the talk is. If it's an hour then 30s intro is good because it gives you some history.
But if it's only 10 minutes then just say "I'M TRISTAN AND I FREAKING LOVE PYTHON!"
You should just open with "I assume all of you know who I am, and if you don't get out."
DSM
DSM
@Jon: hmm. TBH, I don't see a way around doing it manually, for some degree of manually. (Someone, yp I think, was experimenting with doing this natively in pandas but the API was turning out to be a headache and so he gave up).
Probably 30s would be worthwhile, especially at a conference.
user559633
@Ffisegydd Haha, it's 25 minutes.
user559633
19:54
Thanks @ZeroPiraeus, @WayneConrad
@tristan and remember to end your presentation with "for further discussion, join the room 6 in stackoverflow chat"
No thanks for me? Well f*** you buddy.
user559633
@davidism Yeah, this is great except the room would just empty out. people that who know who i am that would already not attend
@AnttiHaapala it depends on what you're mocking: you can mock the class or an instance (and I think there's a way to fake having an instance without actually instantiating)
user559633
@Ffisegydd I WAS LINING IT UP MUCKA
19:55
I'm going to go cry on tumblr.
@davidism yes it talks about it but also then no :D
also if you mock using a class as a spec, you cannot access any attributes that would be set by __init__ on the instance :d
The general sentiment seems to be "it can probably do it, but it's a giant mess"
user559633
(anyway, the sentence was going to write was yeah, it's a conference, thank you for cleaning up what i meant @Ffisegydd)
Yeah, their advice in that case is "use class.var = None"
@davidism That's a shame, because that's the Y I was imagining when I said it sounds like an X Y problem.
@poke there are more cruft in the mock objects as attrs :d
like mock_calls
why oh why
meeh

« first day (1739 days earlier)      last day (3434 days later) »