« first day (2038 days earlier)      last day (2911 days later) » 

1:22 AM
@tristan in the unlikely case you're interested, plan B with a dedicated new computer seems feasible from ~300 USD
 
1:33 AM
Hello?
I have a question
If I wrote a 10,000 line program for centOS 7 which essentially runs commands, parses the output and shoves it in a database... what's the best way to now make this code run on CentOS 6, ubuntu and potentially other flavors... I can't figure this out
 
is the program written in python?:P
 
yes sir
 
actually, I have to go now (past 3 AM here)
 
=(
got a source I can read?
 
but I'd ask things like "have you tried it on other systems?", "does it work?" and "how does it not work?"
 
1:38 AM
some commands are specific to cent 7
 
and "have you tried figuring out how to solve those ways in which it doesn't work?"
 
I can solve them...
if this version... run this command and parse this way
but I'm scared it isnt the best option and may become unmanageable
 
I don't have any experience with this issue. That said, you either have to check the OS version in some way, and use corresponding functions, or just try a centos 6 version, check the output if it failed, try the next one, etc.
I don't know how the first one can be done, but that should be the simplest
for instance, answers here look reasonable
try them out, see if you can distinguish
then you can just select OS-specific commands at the beginning, possibly set them in a dispatch dict
then you do the same things using the corresponding commands
(of course you might have to tailor the parsing bits too)
and with this, good luck and good night:)
 
exactly the only solution I can come up with too
I'm hoping there is a better way
 
how?
 
1:43 AM
but thanks and night
I've been considering:
1. switching everything to kernal level commands
2. pulling apart the code behind the command itselff and comming up with versions that are platfform independant
 
is there some reason you have to use OS functions instead of pure python?
 
Dealing with a lot of hardware commands...
 
both 1. and 2. don't seem to be python-related
you might have better luck at some unix/linux-related SE, possibly
 
they aren't... might have to switch to C++
 
Which, outside of re-writing commands, can just mean looking for libraries which support what your trying to do
i.e. am I trying to re-invent the wheel
 
1:45 AM
but yeah, depending on what you're doing, pythonifying your work is another option
 
Yeah... I'm using psutil, subprocess, network, hard drive data commands... zfs commands...
basically, we are writing a NAS appliance
 
bash it is!:D
anyway, bye for reals
 
user559633
@AndrasDeak i'm actually interested insofar that i want it to work out for you
 
how to write platform independent applications/code in python is a really broad question, as shown here, but if you run into specific problems you should ask about that
 
1:58 AM
Okay, thanks... it sure sounds like the only way to do this is the first way that came to mind... put switching statement in the code to do things differently depending on which OS the person is running. It still scares me though.
 
user559633
at some point, you have to deal with differences in the OS. either by OS hooks or by a switch statement
 
user559633
you could do something like i did in this: https://github.com/tristanfisher/easyos/blob/master/easyos/easyos.py
wherein the interface remains the same to the coder, but you're handling the differences behind the scenes
 
I have written a lot of platform independent code and a lot of platform dependent code (mostly C# in that regard). Making platform independent code is usually possible but it takes more initial work and will require a few hooks, specialized classes (subroutines, tables, dicts), or at least a few if/else or switch statements somewhere
 
Thanks... I'm really getting the impression from JGreen that I know what needs to be done... it just still scares me, lol, the code is already getting to be too much for one person to handle
Just to show you guys what the code does
 
I haven't seen anyone using Darwin in years - I always thought it had a great mascot
 
2:03 AM
here is the final product bithoarder.com
 
user559633
so focus on one OS/command line implementation and just leave the hooks with a raise NotImplementedError('todo')
 
user559633
@user3656612 good to see that you got the marketing handled before the product :P
 
platypuses are cool
 
LOL... we have a potential bbig box client that loves it... but wants it on ubuntu
another that loves it but wants it on centOD 6 instead of 7
lol... pain in the butt
 
user559633
dang. i need to get myself a sales guy/gal
 
2:05 AM
Its just two of us
 
user559633
And yet, you have a potential client for something you've yet to build
 
user559633
I'm not mocking you -- well done
 
good and bad
thanks
I really appreciate that
 
user559633
Yeah, of course :)
 
This has actually been good... didn't get the answer I wanted... got the answer I already had, but was scared of... which confirms the biggest fear is not real... that being, that the first answer, the only answer I can come up with, is a really bad way to do it
 
user559633
2:08 AM
At some point you need to deal with OS differences -- it's just a matter of at what level.
 
So, for example... I'll be modifying my python to say...
 
<-- builds stuff for academic and government systems......trust me these are minor platform problems
 
user559633
If you're in a rush to deliver, stay in Python as long as you can and let the VM work its magic for you
 
iff centOS 7... use this command, parse this way... if cent os 6, use this command and parse that way... of course, there will be some commonality so hopefully the code will only grow by 10% or so to get it done
 
like last week this was said: "we need this to support Windows XP, 7, 10, three versions of Linux, and BSD (we're not sure which one or what that actually is but figure you do). Also, can you make it agile?"
 
2:11 AM
lol
sure, $3,000,000 please
 
user559633
@user3656612 focus on availability of the command(s)/hook(s) you need, not what the OS reports itself to be.
 
understood
might be able to install centos commands on ubuntu in some cases
 
user559633
you said: "Yeah... I'm using psutil, subprocess, network, hard drive data commands... zfs commands..." what are some of the examples of what you need? things like ifconfig/ipconfig? smartctl stuff?
 
yes
all that
 
user559633
i'd imagine there are existing python libraries for most common commands and if you decide to open source some of the component modules (e.g. python-zfs), you'll probably even get some contributors
 
2:16 AM
my partner, the none coder, has been going thru my code... subprocess is causing an issue on ubuntu
We are considering that too.
python-zfs sucks, tried it
 
user559633
just a suggestion, not a recommendation of a specific library
 
thanks
 
user559633
you might get pretty far on building a table of OSes with a proc filesystem, then building little utility scripts to parse out what you need
 
I've thought about other ways... like
instead of parsing the info out of the show zfs structures command
sorry, I haven't touched it in months, moving, and forgot the command
bbut anyway...
instead of parsing command output, simply mmodifying that command to put the output in my DB
 
user559633
you could maybe strace some of the utilities and see what they're reading from.
 
user559633
2:19 AM
e.g. netstat just reads from /proc (would save you from converting parsed/visual-intended output back to a structured form)
 
cool, that may help indeed
My partner is the linux wiz, I'm the computer engineer
 
user559633
i'd be interested to hear how it works out for you
 
How can I friend you or whatever?
 
user559633
i'm in here pretty often -- this is my social network (i don't twitter/facebook/etc)
 
Okay, I love this room already... so it's bookmarked.
Alright, I'm off to make my code multi-linux compatible! thanks again sincerely!
 
2:22 AM
tristan is a product of the Python room. He is the consciousness which spawned from that randomness that is this place
 
lol
 
user559633
Of course! If you get stuck on a thing, come back with a code example and someone will probably help out
 
user559633
Head's up that it's varying levels of more-strict during the week when people are at work/it's more busy in here
 
I am now a part of this room, I will be back!
ok
 
user559633
@user3656612 cool, i'd say set an avatar and pick a name so people can start getting familiar with you
 
user559633
2:23 AM
@JGreenwell not sure if the "randomness" is chicken or egg
 
I have a better account, picked the wrong gmail account to log in with... not so easy to log out and back in these days
 
user559633
@user3656612 stackoverflow.com/users/edit/3656612 you should be able to pick a name and av on this page
 
test
still wrong account, erg
 
user559633
sometimes it takes 15-30 minutes for changes from stackoverflow.com to propagate to chat.stackoverflow.com
 
test
ok... I'll be back... user name should gunslingor and I'll set an avitar
 
user559633
2:28 AM
cool, good luck
 
user559633
i made the mistake of starting to customize my nginx config. that was 2 hours ago
 
@tristan I often wonder when you get in a memeish mood
 
user559633
@JGreenwell usually waiting for a build/config to finish or just feeling silly
 
understandable, I'm waiting for the first part of my NLP code to finish right now. Yay! new course evaluations :)...:\...:)...:\
 
user559633
oh, that's awesome. i can't wait to get back into NLP. i'm working on my deployment pipeline/caching/automated infrastructure stuff
 
user559633
2:40 AM
i.e. "stuff that's good to have sorted, but not very interesting to work on"
 
I'm always excited to get into NLP and then run the gamete between pulling my hair out in frustration, or just despair at the education level of some, to talking anyone and everyone's ear off (who is foolish enough to ask me about my research) in pure excitement for some new finding
Then I start the cycle again :)
which is probably why my wife has gotten in the habit of interrupting (or saving) anyone who comes over to our house and is about to ask me: "what are you working on?"
 
DSM
@tristan: you should hire a devops specialist! ;-)
Evening cabbage, all.
 
user559633
@DSM oof.
 
evening DSM, and congrats on the hire
 
user559633
so close to done. pretty soon i'll have a web interface with a nametag on it that says "devops donald" on it
 
DSM
2:50 AM
@JGreenwell: melons!
@tristan: so when all the current work is done, your reward is.. writing more code?
 
user559633
@DSM yeah. i mean, i'm complaining, but i actually like to work.
 
user559633
[already said this; tired]
 
DSM
I like problem solving. But once I've figured out the algorithm actually getting around to the implementation can sometimes just be a nuisance, which is why I tend to write tools which write tools which generate code and so on..
 
user559633
one of my mentors told me to optimize for shipping/public facing, so i'm building tools so i can push every five minutes if i want
 
user559633
You figure out the algorithm independent of the implementation? I typically only figure out the specifics of the algorithm through trial/error
 
DSM
3:02 AM
Mostly. I tend to think in terms of data flow, in a very language-independent fashion. My inner monologue is something like "okay, so I'll need this.. and coupled to that.. so over here.. and then there.. okay, plus that here".
 
user559633
interesting. i try to separate a problem into discrete functions, then try to line up the intermediary structures, but always need to code it out to really have an algorithm
 
5:35 AM
well put a linux apache2 mysql python stack together... not sure what do to do with that but I know how to make one now... lol
 
 
2 hours later…
7:12 AM
cabbage
 
cbg
 
7:53 AM
cabbage
Here's a horror story from the xkcd coding forum about bug dependencies. I shudder to think how much production code around the world is infested with such bug dependencies. I suspect they'd be rather common in older code bases that were created in the bad old days before modern testing regimes.
 
8:49 AM
cbg
 
@PM2Ring Doubt it - our stack is still not great for unit testing, partly because of the nature of what it does, and partly just because it isn't. I'm sure others are like it.
 
@RobertGrant read this as: "our stack is still not great for unit testing because it is a messy spaghetti with meatballs, and actually is a single unit to begin with"
and then I recognize that almost all code I write is such a mess anw.
because: "we do not have time/money to write unittests/do TDD right now."
 
Nah it's super-decoupled, but some parts of it don't have easily integrated unit testing
 
sounds like php :d
@PM2Ring but what is that BS in that thread about mmapping being worse than asyncio
ah now I understood :d
it means: "If I can know the access pattern beforehand, I can do an async request for all the pages"
likewise one can use madvise with MADV_WILLNEED with mmap
 
@RobertGrant I'm not claiming that all modern production code is developed under ideal conditions with perfect test suites. Just that there's still a lot of old code out there that pre-dates modern testing philosophy, so it's more likely to incorporate such bug dependencies.
 
9:02 AM
Yeah fair enough :)
 
ppl like to write messy code, because it is easier.
and faster
or...
 
@AnttiHaapala Well, it might seem faster... until you count all the time fixing up the unexpected crap. :)
 
exactly. I had this demotivational poster from despair inc:
Had it hanging on the wall at the office 10 years ago,
wonder what the American company execs thought when they visited our office :D
hmm gotta order more posters from despair inc
 
BTW, I'm not claiming I follow best practice re: testing. OTOH, I mostly just code small stuff for my own personal use. But I still test stuff as I go, to make sure each function does what it should, even if I don't actually write formal testing suites.
 
Probably "That's true! Thankfully we all got promoted before anyone found out."
 
If code was written & tested properly in the first place a significant number of the world's maintenance programmers would be out of a job. :)
 
@PM2Ring despair inc has a poster for that too: despair.com/products/adaptation
 
I think @Kevin might like the Acquisition poster for his office. :)
 
Nice.
 
9:14 AM
"When you wish upon a falling star, your dreams can come true. Unless it's really a meteorite hurtling to the Earth which will destroy all life. Then you're pretty much hosed no matter what you wish for. Unless it's death by meteor."
 
@AnttiHaapala That saying goes back to Mark Twain.
 
9:44 AM
intermittent morning cabbage
 
TIL there's an annoying Tkinter bug: stackoverflow.com/questions/5886280/… . The bug doesn't affect Windows systems, only *nix systems, which is a bit weird considering it's a Tcl/Tk thing.
rbrb
 
@AndrasDeak cbg
@PM2Ring rbrb
 
10:13 AM
Hm, think I got something working. Quicker than expected.
Cautious woohoo.
 
@tristan well in that case... :P My new vague plan is to buy a compact box such as this one, space is important. I'd buy the cheapest motherboard that doesn't have an integrated cpu (those don't come with fans, only heat sinks), a dual-core intel celeron cpu, 4 giggles of ram and 2x1TB hdds.
This is way more expensive than my original plans, but this configuration seems such that it can be used with a future LED TV we don't own yet (and it can be upgraded in bits if needed)
 
@PM2Ring "Not found attributed to Twain until 2010"
 
I have a ~11-year-old half-dead pc lying around, but I'd probably need a new motherboard for that one too, and it's huge compared to this compact thingy
so I find it more plausible to put together a new pc from scratch
(I also don't trust second-hand stuff here, but that's a regional/personal silly thing)
 
Yeah you have to upgrade quite frequently if you want to carry PC parts forward into a new PC
 
@Robert background is "I want some kind of external storage for my photos and shit"
 
10:19 AM
That's a good idea
I've outsourced photo storage to amazon
Hoping they won't steal my photos
 
actually, it started with "I plan to use an RPi with that", but the kind gents (Antti and tristan) quickly convinced me of my folly:P
and inadvertently, my intellectual inferiority:D
 
Yeah I was going to use an RPi for some sort of media centre, but the cloud is so much simpler :) Netflix etc
 
so my plan B is to have a PC for this
we in the East don't trust no "clouds":P
 
Yeah in SA you can't do that either, but it's fine in the UK I think
Who knows.
 
so, as the guys quickly shot down my original idea, I'm eager to hear their input on my plan B
I'd rather feel stupid and do it right, than do it my own way and feel equally stupid after screwing it up
 
10:24 AM
Yeah I'm the same. Cautious with my time.
 
@AnttiHaapala Interesting. I'm pretty sure I heard that line attributed to Twain well before 2010, but I could be mistaken...
 
@PM2Ring obligatory lincoln:
 
Also,
 
10:41 AM
@Andras What OS are you planning on using for this? A dedicated NAS distro? Or something more general-purpose, eg Alpine, or Puppy?
 
@PM2Ring umm... as a first guess I assumed Debian because that's what I know, but I'm open for suggestions:)
Yes, I haven't even checked if I can do any raid things easily with debian
(I just want to mirror 2 hard drives)
 
@AndrasDeak Well, it's always a good plan to stick with what you know. OTOH, a compact distro will take up much less room and have a smaller RAM footprint.
 
I'm aware of the existence of freeNAS btw, the small research group where the missus is working is about to build their new file server with that OS
(and since chemists are trying to make it work, they are failing:D)
 
Ok. I've never used any of those Distros I've linked to, apart from Puppy. They were just some suggestions.
 
(sorry for that last edit:P)
@PM2Ring puppy sounds nice;) puppies are great
but yeah, I'd have to choose whatever fits on a USB stick, so something lightweight
 
10:52 AM
I'm quite fond of Puppy - I've been using it as a portable distro for at least 10 years. In order to make it so tiny it does have a few quirks in the way it's organised, but if you're familiar with any Linux flavour it's not too hard to get used to.
 
I've been an end user of ubuntu for 10+ years, so with a reasonable amount of googling I should figure out whatever's not too obscure
You know, transition of quantity into quality: being an end-user for a long time makes me a guru for a little while, right?
 
A core feature of Puppy's portability strategy is the extensive use of layered filesystems, so you can, for example, have the core of your system on CD, with updates on another layer kept on a USB.
 
nice:)
rhubarb for now, I'll be back in the evening (so possibly good night, @PM2Ring;)
 
Originally, Puppy just had its own packaging system and repo, but several years ago it was adapted so it can easily use other repos. So one major Puppy flavour uses Ubuntu repos, the other uses Slackware. So if you choose the Ubuntu flavour you can easily install anything available in the Ubuntu repos.
See ya, Andras.
 
I'll have Puppy opened in a tab -> there, thank you:)
 
10:58 AM
My pleasure! I'm sure you'll love it. It's cute, but surprisingly powerful for its size. And fast.
 
I liked it
Still not found a use for such a compact distro, but was fun to set up in a VM
Perhaps some sort of VM army
 
Wonder if Puppy was named after me :p
 
Nope.
 
Well - I'm going to believe it is so what ya going to do about that huh!? :p
 
11:18 AM
I forgot about github.com/nerdcop
 
11:32 AM
sunday cbg
 
A snapshot of Martijn's Interview hamsterguy.com/comic/hiring-python-ninja
9
 
11:49 AM
:D
 
LMAO
 
hahaha
 
12:32 PM
@BhargavRao No, if that were Martijn, the python would be bleeding, not him.
 
oh gawd - where to even start stackoverflow.com/questions/37238326/… :(
 
@PM2Ring Perhaps the blood is from the python itself. Perhaps the interviewer had got confused... Only Martijn can resolve my queries.
 
12:49 PM
@JonClements It's now got an answer that's just as bad, if not worse. It's a classic "blind leading the blind" scenario.
 
1:04 PM
How to send binary data in a json in requests ?
 
@JonClements accelerant and an open flame
 
Cracked it :)
 
@PM2Ring I'm surprised nobody's mentioned the abysmal indentation situation
which is why "it doesn't display what it should"
and that awful answer doesn't fix it either argh
 
@Ffisegydd I took pity on the poor kid and did his homework for him posted some code and a bit of explanation. Hopefully, he'll learn something...
 
(I read the transcript from mobile, then the question, and I had to know if the mobile version ate the indentation)
And if I could just ask why did you use three " " " — Mohammed Omer 4 mins ago
there's your problem @PM2Ring ^
 
1:32 PM
@AndrasDeak Good point. I'll update my answer to mention that.
 
1:55 PM
@AndrasDeak Done. And I linked him to the official Python 3 tutorial. Twice. Let's hope he takes the hint.
Greetings, JRS.
@idjaw @JohanLarsson Last night I "discovered" an amazing young blues / soul singer Maggie Koerner. Here she is with her band performing Judgement Day. The sound quality isn't fantastic, and the band itself is good but not spectacular, but Maggie is incredible, IMHO.
 
user559633
2:34 PM
day two of all day work cbg
 
cbg
 
3:31 PM
i use a dictionary to store and manage threads, is there a better way? maybe any good library ?
threads need to run forever
 
Wow - haven't heard this one in a while: youtube.com/watch?v=y1oN7S4tZkc
 
3:50 PM
Here's some Aussie country music for you, @Jon: the beautiful and poignant Cootamundra Wattle by John Williamson. I might have posted this one previously, but it's so good it deserves another run.
 
> Here's some Aussie country music for you, Jon: the beautiful and poignant Cootamundra Wattle by John
:|
 
@PM2Ring think that's already in my bookmarks :)
just chilling out to American Country first
I love this one: youtube.com/…
I like the theme and lyrics :)
heya JRS
 
4:19 PM
@JonClements Oh good. :) I did listen to the previous David Ball track you linked above, but right now I'm watching stuff from last month's Wanee festival. Current track: Dreams by Gregg Allman, with Warren Haynes.
 
@PM2Ring thanks!
 
@JohanLarsson My pleasure, as always. I first found Maggie performing with Warren Haynes, doing I'd Rather Go Blind
 
(and don't listen if you're in a sad mood: but I also love - youtube.com/watch?v=nTUvH7GpFVQ)
 
@JonClements Bookmarked
 
 
3 hours later…
user1440897
7:00 PM
Hi guys, I made a small script (<70 lines long) that processes old backups. It seems a bit big considering what it does; is there a way to optimize it further?
 
7:18 PM
rhubarb
 
7:33 PM
another silly question: I am trying to look up all the different things that you can import into bottle that comes with that framework work... like the framework syntax... what is that called?
 
An X/Y problem.
 
seems to be about it lol
 
8:09 PM
cbg
@JeffreyLin processes how
@JeffreyLin use python3.4+, pathlib and its glob
@JeffreyLin also your dates are in ISO 8601 format
so you can calculate the cutoff date
then remove all old backups with name less than < name
and you can glob 20[0-9][0-9]-[0-1][0-9]-[0-3][0-9] directly
 
user1440897
You can compare names?
 
Bleh fever
Good timing when the go live is this week
 
user1440897
@AnttiHaapala I don't think I would need to glob, I'm assuming that the script is in a directory with the backups (the backups are folders) and nothing else.
 
user1440897
8:37 PM
> Path.rmdir()
> Remove this directory. The directory must be empty.

How do I replicate shutil.rmtree and delete non-empty directories?
 
8:48 PM
shutil.rmtree(str(path))
 
user1440897
My main concern was this part; it seems like there's a better way to do it:

def getCutoff( keep ):
cutoff = now - timedelta(days=keep)

if cutoff.month < 10:
cmonth = "0" + str(cutoff.month)
else:
cmonth = cutoff.month

if cutoff.day < 10:
cday = "0" + str(cutoff.day)
else:
cday = cutoff.day

cutoff = PurePath(str(cutoff.year) + '-' + str(cmonth) + '-' + str(cday))
return cutoff
 
like I said,
strftime('%Y-%m-%d')
or did i :D
which is also the default format,
in sane pythons
>>> import datetime
>>> str(datetime.date.today() - datetime.timedelta(days=30))
'2016-04-15'
 
user1440897
Oh, there's no need for the hacky part.
 
also use Path instead of PurePath methinks
Path is the local Path type whatever it is for your operating system.
and Paths seem to be comparable as well, nice
@JeffreyLin I guess this has shaved quite a many lines already :D
but: I still recommend you do the datepattern matching, or at least glob with ????-??-?? before you've removed all of your precious data :d
 
user1440897
I did implement the globbing you mentioned earlier
 
user1440897
8:57 PM
Strange. I'm importing using from datetime import datetime, timedelta, but cutoff = date.today() - timedelta(days=keep) returns name 'date' is not defined.
 
user1440897
Nevermind, I had to import date from datetime as well.
 
:P
you don't need to import datetime
@tristan have you ever recovered anything from glacier?
 
user1440897
The thing that gets me every time is that there's a datetime within datetime.
 
user1440897
9:14 PM
Thanks for the help @AnttiHaapala
 
Blimey, glacier's pricing is pretty amazing
 
user559633
@AnttiHaapala yes, why?
 
10:04 PM
@PM2Ring thanks for sharing. She has a great voice! fantastic track.
cbg guys
 

« first day (2038 days earlier)      last day (2911 days later) »