« first day (2340 days earlier)      last day (2601 days later) » 

12:26 AM
hey have any of you tried codefights?
 
 
6 hours later…
6:34 AM
cbg all!
 
Cabbage
 
Is there some easy way to read in from source files an existing bit of C code and wrap it using python-cffi?
 
Sorry, I have no idea. I've seen cffi mentioned here a couple of times, but I had no idea what it was until a few minutes ago. It does look interesting and useful!
What you ask sounds reasonable, so I expect there is some easy way to do it. Hopefully, someone who does know cffi will respond to your question later. This room is generally fairly quiet this time of day but it should start getting busier soon-ish.
 
Basically I've wrote a library that wraps mmap with an Array-like interface, I'm trying to use it on windows but there's no native implementation of mmap on windows. That's led me to want to use this C library: github.com/witwall/mman-win32 for windows support.
Instead of having to make a complicated installation I'd really strongly prefer if I could wrap this with CFFI.
 
6:56 AM
@ZeroPiraeus I'd love to know the OP's reaction when they discover that their self-deleted question is now undeleted and has 4 upvotes. :)
 
wat?
there is mmap in python stdlib
@shuttle87 ^
in which ways does this not work?
 
The standard mmap module has a string-like interface. I guess that a more general array-like interface could be handy. Of course, you could build an array-like interface on top of the string-like interface, but I assume that'd be less efficient than a direct implementation.
Or maybe I don't know what I'm talking about and I should just shut up and not reveal my ignorance. :)
FWIW, here's the CPython source of mmapmodule.c
Hopefully, Java lovers won't think I'm Java-bashing:
Static methods aren't very common in Python, and a Python class that only has static methods seems rather strange to me - Python isn't Java, and slavishly copying Java patterns in Python tends to produce ugly, bloated code. Why not just make them into normal functions defined at the module level? You can put them into their own module to group them together. — PM 2Ring 6 mins ago
 
@PM2Ring that would be a good bet
the mmap does buffer interface.
you can use it with arrays
or if this is serious, then one would use numpy.memmap
*sorry, memoryviews
shit memoryviews are badly documented
@PM2Ring note that this wasn't in Python 2.6 :D
it was added in Python 3 and backported into 2.7
 
7:16 AM
@AnttiHaapala The Python 2 docs say "Memory-mapped file objects behave like both strings and like file objects. Unlike normal string objects, however, these are mutable. ", the Python 3 docs say "Memory-mapped file objects behave like both bytearray and like file objects.". I assume that shuttle87 wants a convenient way to treat the buffer like an array module array .
 
@PM2Ring ^
 
@AnttiHaapala Yeah, I don't think I've ever heard of memoryview.
 
it is everything that they'd want to do. And if it is not enough then perhaps they should be using numpy instead...
not really productive to reinvent the wheel beyond that :D
 
Hmmm. That memoryview looks useful...
 
the point is that, they can only do worse with any mmap library...
and less portable
and everything else is just a misunderstanding :D
 
7:24 AM
cbg
 
recbg
@GrizzledSquirrel you've got 2 possible outcomes. Either you accept that this should really be a module, or we'd close this as "primarily opinion-based" - "Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise." — Antti Haapala 13 secs ago
 
7:37 AM
@AnttiHaapala dude how do you know all this?
It's like every frickin Python topic you have a crazy amount of knowledge about
It's very frustrating for us know-nothings
Or this know-nothing
 
7:55 AM
I've read the documentation
 
It's cool you can retain that much knowledge from reading
 
I made some changes to my office's library book about Django
"Web Development Done Right" -> "Web Development Done Wrong"
 
@RobertGrant I don't
I just remember what to google again
 
Oh I see
Well that makes me feel a bit better
 
8:09 AM
I've stored some of my tests from 7-9th grades..
when I read my exam papers from 9th grade from biology or geography I am like... "well, I didn't know that... hey but I did know that back then... wtf."
 
8:25 AM
Yeah I've kept almost everything like that because I'm a hoarder
I should also see what I used to know :-)
 
you'd be so ashamed to admit that you've forgotten all that...
"No, I don't remember what secondary functions does a human liver has, but hey I know which flags the Python mmap constructor accepts on Winblows"
 
9:00 AM
This OP has confused "irritation" with "iteration"... or maybe not. :D stackoverflow.com/questions/42759502/…
 
user6845426
cbg o/
 
@AnttiHaapala yeah I'm looking forward to Rob Junior getting older so I can revisit things when he does homework :-)
 
9:20 AM
Morning cbg
@RobertGrant I cursed you silently while I endured Beastmaster with Miniwithnail at the weekend
We've agreed it might be a thing he can watch when i'm doing something else. Anything else.
 
Yeah it's bad, right
 
So bad.
 
Tell what I've been enjoying: Enterprise
Which is not amazing, and they're way too keen to have the men strip off their tops and the women to be in their bras, but apart from that it's actually not bad
 
The one with Scott Bakula?
 
Yeah
 
9:23 AM
Interesting.
Tell you what I've not been enjoying: Javascript.
 
Yeah me too :-)
 
Screw that, and its stupid how-does-this-even-work API calling and CORS headers and assets
 
Oh wait, not? I have been enjoying it.
 
Cabbage!
 
But I'm only learning some stuff, not actually doing things
CORS is alright, isn't it?
(Having never used it)
 
9:24 AM
I was trying to learn some stuff by doing, which is how I've always approached Python/Java.
Dunno, I'm sure I'm missing something obvious, but it seems inordinately difficult to test Fetch requests from your local machine, for example. gave up and moved to Axios for it.
 
Yeah I'm doing some research (was it you who recommended that React course?) before attempting to propose we create a new product (and I run it, and thus get pulled off the consulting rat race a bit)
 
I'm swinging round to front-end learning for reasons (first being that I've never done it at all), and finding it very frustrating.
It was.
 
@Withnail fetch sends OPTIONS first and queues a GET, so if the OPTIONS fails bad things may happen
That's my very mild understanding
 
I'll finish up that course today I think
Want to move to react components for slickness in some of our new builds.
 
Yeah I want to base said product on react
So I need a pretty good understanding of how to build a standard way of doing things that can also flex
That's why I'm happy that course looks at Redux, so I can get a better grip on that
 
9:29 AM
I kinda generally get React, that's after a couple of afternoons playing round with it
Redux seems kerrr-azzzy though
 
Yeah I definitely get the idea, but it's the more advanced stuff I want to know about
 
I get that it's powerful, but there just seems to be so much redundancy and repetition to do even very simple things.
 
Yeah I agree
We could build an Eclipse plugin to help visualise how props flow down through components, that sort of thing
 
Quite like the idea of the Actions / Reducers links
 
Just to make that stuff easier
 
9:30 AM
although reducers felt like quite an obvious place to put an API call, for me
Looked at the docs and it was all NO DO NOT DO THIS EVER
sigh
 
Aren't reducers meant to operate on the Big Local Data Store?
And then separate stuff can move data from a remote server to the BLDS?
(I'm also imagining a way to put BLDS data into local storage for offline first)
 
Morning cabbage
 
Morning.
 
@IntrepidBrit wow, long time
 
(is BLDS an official term? It should be.)
 
9:32 AM
Remember when I had high hopes of creating a product to sell, @IntrepidBrit? Seems like a lifetime ago.
 
I thought that was local first.
 
My son's lifetime.
 
@RobertGrant Aye - I remember. When's it going to be finished? Still need it ;)
 
It's not moved :-) I did some Backbone experimenting with it and didn't like it, and ran out of spare time.
My orthoganal reason for learning React is to probably do some nice client-sidey stuff in that project as well
My company could also totally use its capability, which I'm noticing more and more
 
I've also got some clients that could use it, and some people I would like to be clients too ;)
 
9:36 AM
Maybe we should revitalise the discussion around it
I'm definitely more able to take on additional stuff these days now we've stopped moving continent/having a baby/working hundreds of extra hours/moving house
 
Definitely mate. I think your slack has dropped off the bottom of my slack list in my client. Will check it regularly again if you're up for it
 
Cool
Oh wow yeah, I'm not even connected to it currently
@Withnail I'm also interested in the GraphQL stuff, where it can just query the exact fields it needs out of a bigger API
That seems pretty awesome
Might not be worth doing for most things, but if you're shoving around relatively large amounts of data then it means you don't have to hand-craft queries as much
 
Interesting, will have a look at that
I was mostly going to be DRF based python backend for datamunging and react for UI, but always interested in new solutions.
 
cbg
 
Hey Andy.
 
9:57 AM
@Withnail yeah there is some cool stuff happening
Some of what RobCorp does is integration, and there's always a tension between APIs having big enough data models to make them reusable and maintainable, but also being fine-grained enough that they don't do loads of unnecessary querying. GraphQL solves that, if I've understood it correctly.
 
Hm, that might be this morning's reading then
I've spent the last few weeks despairing at the quality of our supplier APIs
 
Yeah there are many terrible APIs
 
literally got an answer of 'sorry, i don't know what a webhook is' when asking our account manager if they could implement something
 
Sadly REST hasn't helped, because now we don't even have formal interface definitions (or not until recently)
(I use REST in the not-really-REST sense, i.e. HTTP+JSON+verbs)
 
And another AM said "Oh, but why do you want to do this? Your {{team}} can just go in and do that through the web interface."
 
10:01 AM
cbg(Andy)
 
i don't want them to do it, they're human and very fallible, I want to do it automatically, with error checking, every time a new order comes in! this is not, I feel a ridiculous use case.
I'm at best a mildly competent programmer. I'm fairly good at Business Process Schtuff around that. This seems to make me demand things that suppliers find totally ridiculous and 'I've never seen anyone ask for that! WTF"
headdesk
 
Yeah there is a big problem with managers thinking that existing manual processes have no cost, and automation is an expense.
 
I spent whole my life avoiding C. But my senior project just stroke me with C
 
That sounds very euphemistic.
3
 
It's scary to think how many office workers waste vast amounts of time manually doing stuff that could easily be automated, like manually transferring stuff from Excel spreadsheets into other software. I can understand doing it as a one-off or rare thing, but to spend hours on it every week is insane.
 
10:12 AM
I have the alternative of writing a network simulator by hand using my favorite langauge or use ns-3 with C
 
@PM2Ring There's literally a team of people at my company do that.
That's what this bastard system was meant to replace. We've just ended up with them munging stuff between a different set of systems.
 
@PM2Ring My brother works in such office. He says that the guys who wrote dumb VB scripts for Excel are consider to be geniuses.
 
^^ and that.
 
I know it happens a lot. I assume business practices in Australia are fairly typical. Several years ago I was involved in marketing top-end business software. Part of my work involved surveying the existing practices of businesses who could benefit from such software. A large majority of companies that weren't already running integrated systems (IOW, potential clients) mostly ran their businesses on a disorganised pile of Excel and Word documents.
 
10:27 AM
I might have said this before:
 
@Withnail change management is hard. Go to the CIO and tell him you're doing to digitally transform his/her business and you need the mandate and budget and it might start happening :)
 
A friend of mine works for a Fortune500 company who make devices that put ink on paper... their entire MIS/Data Warehouse system is built on a complicated system of spreadsheets that pull together progressively higher level reports (he maintains this system.) Those spreadsheets sit atop, at last count, 9 Access databases. Why 9 databases? Because Access has (had) a practical 2Gb limit on size, so every time it got full they just opened up a new one and pointed the spreadsheets at it.
 
Sounds saucy.
 
We're getting there. I'm getting to expand the team, for one. It seems the throwaway comment I made when told how much we were paying supplier X for shitty service Y about "..or for half that amount you could hire two developers and we could just build it in house and have control over it." found some traction. :D
 
Yeah it's amazing how much people will spend on avoiding automation without realising
 
10:33 AM
sigh I better go do the automation that I'm avoiding, tbh.
rbrb for now
 
Just automate the business process automation process and you're done, sheeple!
 
:D
 
(Good to know I'm not the only one who has these problems)
 
If a company like YouTube managed data like many companies do, it'd probably take a week for uploaded videos to become available, and they'd have to employ most of the world's population just to achieve that speed. :)
 
Yeah, indeed :)
"But I shipped you the ISO on a DVD FOUR DAYS AGO - I didn't get an MBA from Harvard to wait more than three days for an upload!"
"Wait, wait, let me look up the shipping SLAs"
 
11:30 AM
return.cbg()
Soooo...
The API from this morning. It only lets me process individual items - no lists of items.
It can handle about 4-5/second. Great, except I have 11k items to process, for just one order.
And 800 odd orders to process, at least the first time.
So now i'm just going to brute force it, by processing the jobs async and bashing their server from 10 worker clients, assuming that they can process that many requests.
\o/ waiting on a supplier's team implementing a ticket to absorb lists, only takes... 4 weeks.
 
@PM2Ring Yes its just a wrapper around mmap to make the API easier to use. So I wrote this library: github.com/JaggedVerge/mmap_backed_array to do that. The main reason for doing this was to avoid needing to have a dependency on numpy, but I'm very open to reconsidering that.
@AnttiHaapala By this: "the point is that, they can only do worse with any mmap library..." are you suggesting it's worth introducing a dependency on numpy?
 
11:45 AM
@shuttle87 Take a look at Antti's suggestion of memoryview to make sure you aren't re-inventing the wheel.
 
@PM2Ring I suspect I've already reinvented the wheel with that library... I've looked at the memoryview docs before but maybe I missed something. The main requirement is being able to dump the array to file and load from file efficiently. If there's something with an Array like interface that does that I'd be willing to use it.
 
Anyone able to tell me, using a pandas dataframe, if I lookup an item with its iloc, how can I get its ix?
 
In [146]: df = pd.DataFrame({'a':[1,3,4],'b':[10,11,12]},index=[-2,-3,-4])

In [147]: df
Out[147]:
    a   b
-2  1  10
-3  3  11
-4  4  12

In [148]: df.iloc[1]
Out[148]:
a     3
b    11
Name: -3, dtype: int64

In [149]: df.iloc[1].name
Out[149]: -3
like that ^ ?
cbg
 
it's like Andras was summoned by the mention of Pandas
 
except I don't really do pandas, I just created a dummy dataframe and looked at it:P
 
12:00 PM
"name" ... Thanks! Why can I not find that anywhere in the docs? Am I blind? pandas.pydata.org/pandas-docs/stable/generated/…
 
In [151]: type(df.iloc[1])
Out[151]: pandas.core.series.Series
once you index into it it's no longer a dataframe
and a Series can't store its original index any other way, apparently
 
@RichardDunn happens to us all.
 
lol:D
 
K, I just found the relevant page. As you say, once you index it's not a dataframe, it is, unsurprisingly, an Index... :S
Still think it's a conspiracy though. -.-
 
12:04 PM
Ooh, 6 more points and I hit 5k. THE BIG LEAGUES, PEOPLE!
 
should I burst his bubble?
 
SO chat users look for a decent answer or question to upvote and get him to 5k.
SO users give up after a while and quietly go back to what they were doing.
 
Damnit I clicked that
I'm going to serially downvote!
 
@RobertGrant I'd hate to be the person to unleash you on the world with newly found superpowers of...reviewing tag wiki edits
 
angry_actions(RobertGrant)
 
12:07 PM
That's fine - I have too many upvotes from my answer to "What is SHTML?"
I feel bad about that
 
hehe
My most viewed/upvoted is a 'how to handle db locks in sqlite3 in django' one :(
seems to get a weird amount of traffic at the start of an academic year
 
That's quite useful, though
 
Not really... the answer's basically 'Turn it off and back on again' :D
 
solid advice
 
That's the same as my What Is SHTML answer
 
12:10 PM
what's SHTML anyways?
 
shitty HTML
it's what people write by hand in text editors
 
"shitty html from 2009"
 
@shuttle87 the memoryview is the array-like interface
 
:D
 
@khajvah Well, do I have AN ANSWER FOR YOU!
48
Q: What is SHTML

balaweblogRecently I came to know about SHTML. And also I have look at site having extensions as .shtml. What is the purpose of SHTML in what ways it differ from HTML and DHTML

 
12:12 PM
daaaaamn
 
I don't know the earlier discussion @shuttle87 but there's also a numpy.memmap that will map to an array
this might or might not be entirely irrelevant to the discussion
 
Obviously I'm now researching a better answer to my most-voted answer. rolls eyes
 
@AndrasDeak I suspect it's highly relevant
I'm just hesitating slightly to introduce that dependency if its not needed
 
sure
if you're not using numpy, it's probably best avoiding it if there are reasonable other options
 
@RobertGrant That’s exactly what I did btw.
Didn’t know where to put SHTML when you mentioned it, so I googled, found your answer, was able to put the pieces back together and upvoted your answer… 🙄
 
12:16 PM
Woohoo! Thanks
 
and in 2009 that was a perfectly fine answer
 
It still is
 
well, but the question would be crap
 
Gonna need a lot more people wondering about SHTML for me to access mod tools
 
@shuttle87
import mmap, os

f = open('array.dat', 'wb+')
os.ftruncate(f.fileno(), 10 * 4)
mm = mmap.mmap(f.fileno(), 0, prot=mmap.PROT_READ|mmap.PROT_WRITE)
view = memoryview(mm)
array = view.cast('i')
print(array[:].tolist())
array[0] = 42
print(array[:].tolist())
 
12:18 PM
@RobertGrant Maybe start a soshtml.com and build up a SHTML canon?
 
Good idea!
High performance serving of web traffic through static files
Combined with a marketing campaign designed to descredit all modern techniques such as in-memory caching
 
@AnttiHaapala interesting
 
High performance web interfaces by embracing server side rendering to the max.
 
@shuttle87 you can also cast it to a n-dimensional array...
 
12:20 PM
Why have realtime APIs when you can just dump a file once a day and serve that?
(This is not always a bad idea)
 
@AnttiHaapala thanks, the docs are surprisingly lacking on this
 
(Unless you have access to a cache)
 
@RobertGrant (technically, that’s what a cache does)
(dumping stuff in a file once a day and serving that)
 
@shuttle87 hmm though prot doesn't work on widnows :P
 
basically windows is a pain in the ass with this
 
12:22 PM
@poke true :)
 
last I checekd
 
I meant an in-memory cache, but you're right
 
ah but READ|WRITE is the default...
so I guess if you just use mm = mmap.mmap(f.fileno(), 0) it works the same
 
no mcve stackoverflow.com/questions/42763065/… I tried to be a nice guy...
 
It worries me that if a wizened mentor figure appeared and told me that magic was real and I was the Chosen One and we were destined to embark on an epic quest, I'd probably have a moderate breakdown about the fact that everything I know about society and the natural world was based on false or woefully incomplete information, which would really spoil what might otherwise be a good time.
Is this what it means to be an adult? When your reaction goes from "I'm ready to go to Narnia whenever" to "how will this impact my stock portfolio?"?
 
12:34 PM
I remember the shock when I became ostensibly grown up and had that WAIT WHAT YOU'RE ALL JUST PRETENDING TO KNOW WHAT YOU'RE DOING moment.
 
The scary ones are the ones who don't realise that they're pretending.
 
I know everyone's pretending. But they're better at pretending than I am, so there are still tiers of Adultness that I can look up towards
 
OTOH, of course, you're allowed to pretend that you're not pretending.
 
Certainly. You wouldn't want, say, your brain surgeon, to drop the pretense.
(More generally, you don't want your brain surgeon to drop anything.)
 
I dunno, even as a physicist I'd be fine with the world as I know it having an incredible hidden world underneath. My issue would be embarking on an epic quest. Can't we do it across the internet?
 
12:48 PM
The weirdest thing, I think, is that we literally have no clue what anything is
 
define "anything" :D
 
Like at one point they were all, "Everything is made up of earth, wind, water, and fire"
 
I guess it depends on precisely how different reality is from what I thought it was. If it's just "you know the four fundamental forces? There's also a fifth one generated by the human pineal gland which causes telekinesis", that's not so bad. If it's "science doesn't work", I'm going to have a hard time going to my job the next day and pretending it matters
 
then they were like, "oh no the smallest things we have are atoms - totally indivisible"
then we were all, "Surprise! Atoms are actually electrons and neutrons!"
 
and protons
 
12:50 PM
But we don't actually know what any of that stuff is
 
Ohh, I see a plot twist coming!
 
we don't even know what it looks like
 
The severity of my reaction is approximately proportional to how many textbooks we're going to burn now that they're useless
 
you people and your constant need to see things:P
 
Like, what shape is an electron? We use circles/spheres to represent it, but nobody actually knows
 
12:51 PM
@WayneWerner Then they were like “You know what? There are actually quarks, even smaller!”
 
Hello I have a problem in my code. I have a method where inside I use the set() function but it seems like it is considered a variable. Indeed I get a UnboundLocalError pastebin.com/2FUMBjuL . I have no clue on how the problem could be solved.
 
Joe
can some man use labview to send a sinuswave from computer and then show it in an oscilloscope by using a coaxial cable?
 
@WayneWerner it's literally the most point-like object we know of
 
I think "looks like" and "shape" lose any coherent meaning at that scale
 
@Joe I KNOW SOME OF THOSE WORDS
 
Joe
12:52 PM
@khajvah can man do that?
 
@newbie I suspect you're doing set = later in that function.
 
Oh the joy of physics
 
Choose a different variable name and the problem should go away
This happens because assignment to a name anywhere in a function, indicates to Python that the name points to a local variable, and this indication applies retroactively all the way to the beginning of the scope
 
FWIW the standard model is way too ugly, we're dearly hoping that something better comes along
so far no luck
 
I’m not sure, but I’m kind of hoping for string theory
 
12:55 PM
@Kevin you were right, thank you.
 
user6845426
Hello all :D
 
@Kevin Maybe. But maybe that's why we get things like half-life - because the electron is more (or less?) cube shaped and more sphere shaped.
Or maybe elephant shaped. We literally have no clue.
 
@WayneWerner Cube..?
 
cute electrons are panda shaped
 
heh
@poke Why not? Seems as likely as anything else :P
 
12:56 PM
I'd hate to burst your bubble but most stuff are spherical due to high symmetry
 
guys, what's the deployment process in your companies
 
@WayneWerner I would have been fine with sphere and wave, but cube seems odd
 
the pointy edges would break off too easily in conductors
 
@khajvah We're trying to get a little more into automated deployments, but currently it's ssh to the server, hg pull && hg up
 
@WayneWerner Ok, I'll concede that. Perhaps I should have said "lose their conventional meaning at that scale". An electron is sphere shaped, but not necessarily in the sense of "there is a readily distinguishable barrier between points in space delineated by a constant distance from some central point"
 
12:57 PM
@AndrasDeak Yeah, it would scratch the cables from the inside too much
 
@poke salt crystals naturally form cubes. Why not subatomic particles?
 
@WayneWerner the thing is, they don't trust me with ssh for some odd reason
so I have to find a senior dev
 
@poke microscopic origin of resistivity \o/
 
and they are normally very busy so there is a queue
this resulted in me waiting for them while being hammered by the management
 
@WayneWerner Na+ and Cl- ions form a cubic structure in the crystal, which leads to a large-scale cubic symmetry. But the ions themselves in there are spherical
You can put together something big from elementary particles and it can be any shape, even elephant-shaped. Like, an elephant.
3
 
12:59 PM
*chuckle*
Elephants: The most elephant-shaped thing yet.
 
cbg
 
cbg
 
@khajvah Well, my preference would be either something more along the lines of GitHub where when things are merged to master because all the tests pass and someone has :+1:'d the PR, or maybe something slightly less automatic, i.e. someone has to push a button that says "deploy the most recent code"
 

« first day (2340 days earlier)      last day (2601 days later) »