« first day (2918 days earlier)      last day (2252 days later) » 
01:00 - 17:0017:00 - 00:00

wim
wim
17:03
aha, I was surprised by this:
>>> class A:
...     pass
...
>>> class B:
...     def __lt__(self, other):
...         return True
...
>>> A() > A()
TypeError: '>' not supported between instances of 'A' and 'A'
>>> B() > B()
True
but now I understand. B inherits an implementation of __gt__ from object. kinda subtle.
Makes sense.
wim
wim
guess can add that to the list of stuff that doesn't work with old-style classes
on 2.7:
Now there's a name I've not heard in a long time
wim
wim
>>> class B:
...     def __lt__(self, other):
...         return True
...
>>> B() > B()
True
>>> B() < B()
True
WTH, it same even if inherit from object. Now I'm confused again.
order by id?
17:08
When a dunder behaves in a surprising fashion, I usually chalk it up to the -~- magic of the class system -~- regardless of what it's inheriting from
wim
wim
@AndrasDeak object.__gt__ must be new in 3 point something
ah
TypeError: '>' not supported between instances of 'object' and 'object'
so what exactly does it inherit?
>>> object().__gt__(object())
NotImplemented
Wouldn't that mean that it being defined doesn't matter?
is there an __radd__ analog for __gt__? Does it defer to __lt__ the other way around?
>>> B() > B()
True
>>> B().__gt__(B())
NotImplemented
I guess it has to
>>> class B:
...     def __init__(self, id):
...         self.id = id
...     def __lt__(self, other):
...         print(f'I belong to {self.id}.')
...         return True
...
>>> B(1) > B(2)
I belong to 2.
True
I didn't realize that a missing dunder behaved differently from one that returned NotImplemented...
> There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does); rather, __lt__() and __gt__() are each other’s reflection, __le__() and __ge__() are each other’s reflection, and __eq__() and __ne__() are their own reflection.
I've had that page open for a few few weeks, I should read it all over
The dunder documents contain many arcane secrets, yes
17:26
@Kevin sorted just creates a list and calls its .sort method.
That's pretty much what I expected.
But, yknow. Constant Vigilance and all.
Yep.
Hi guys, I need a lil help: dpaste.com/2W8S9JZ
I manually created a string variable of data I am getting from system
I would like to skip ports that have no cable in it (will not have vendor/part#) etc
Example: skip ports 3,4,5,7,12-14 as they have no additional data?
ports = []
for section in data.split("Port"):
    d = {}
    for line in section.strip().split("\n")[1:]:
        k,v = line.split(":")
        d[k] = v.strip()
    ports.append(d)

print("Total ports:", len(ports))
ports_with_vendor = [port for port in ports if "SFP/SFP+ Vendor" in port]
print("Ports with a vendor:", len(ports_with_vendor))
But this is a lot of work to convert a string to a list of dicts. If you're manually creating it anyway, then consider just manually creating a list of dicts in the first place
17:41
thanks that will help!!!
I am trying to display same full data minus ports with no data
The one login I am experimenting is Port is not followed by "SFP/SFP+ Vendor:"
skip it or maybe simple is Port is followed by another Port
user6718998
18:08
pm2ring on ?\
@Thewise why?
@Thewise you exasperated Andras earlier, which takes more than a bit of effort, so I'm curious why you are looking for another chat member
user6718998
@roganjosh yes, but since I am new to this, I didn't know how to explain things to andras, meanwhile I tried understanding that thing.. but still im stuck. Pm2ring I know he knows this stuff as well, thought maybe he could help.
I want to find good questions
18:18
You can ask a question without singling out a single person to ask it to
@Thewise if you can't explain something in a way that Andras understands then it's almost certain that the rest of us won't understand either
user6718998
well I think he understood in the end but lost his patience
There was a reason for that
Hi
user6718998
I am aware. So then I should just leave this chat and stop looking for help ?
18:20
But, as Kevin just said, if you have a question then don't target a user and ask in the room
I need help in to parse json in python
@Thewise I didn't say that
I have json in text file
The json module is pretty good at parsing json.
user6718998
so then can I ask it again ? (properly this time)
18:22
@Kevin openacademic.ai/oag you can check this
format is there
I'm not a room mod but, if you think you have a coherent problem, then yes (IMO)
i need to convert that json data in csv file
The csv module is pretty good at writing csv files.
@sunil what does that have to do with JSON?
data is json format
18:24
Not from the link you gave us
what i am doing
Uh oh, the indentation got lost in your message. Consult sopython.com/wiki/… for a description of common code formatting problems.
Ok I am sorry
18:30
Uh, what's the difference between the urls https://github.com/Aran-Fey/SE-duplicate-manager/raw/master/SE_duplicate_manage‌​r.user.js and https://raw.githubusercontent.com/Aran-Fey/SE-duplicate-manager/master/SE_dupli‌​cate_manager.user.js? Does it matter which one I use for my script's auto-update?
both are the same 404 for me, so they should be equally good
@AndrasDeak ahha :P it is the chat and its invisible space
can somebody please help me to parse that json and write in csv file
if they seem the same to you I'd go with the one that has github.com in its name; the other one sounds like an implementation detail
@Aran-Fey I wget --save-headers and they seem to be pretty identical in all respects
the only things different in headers were some tags + fastly ids + dates and that'sit
18:34
@sunil So, what's wrong with the code you have now? It looks like it's doing an alright job of reading json data and writing csv data.
user6718998
So if anyone would be concerned or knows or worked with an HPC before and the mpi4py module, here is my question. I have an aglorithm that generates a bitmap. My task is to implement multiprocessing with MPI. So by using 4 cores, each one should take care of generating 1/4th of the image then. I get what size and rank represent, but im not able to figure out the rest. The code looks like: pastebin.com/5hzTxTvz If anyone would like to help, please tag me and have a great evening.
user6718998
The script will be called from a separate job file.
I guess it's a little weird that you open data.csv in w mode twice. Opening it the second time will erase everything you wrote the first time.
Okay, I'll revert the update url... My commit history is quite the mess
Or, hmm, I'm not actually sure what happens if you open a file twice without closing it. Maybe just don't open it the second time, you don't need the file variable at that point anyway
18:37
@Kevin csv is not generating in proper format , i want it each column and row will have its value. columns are 'title', 'lang', 'year', 'authors', 'publisher', 'url', 'doi', 'id', 'keywords', 'fos'
@Thewise I don't really want to get involved again, but "I get what size and rank represent" vs "im not able to figure out the rest" in light of your code not having changed at all since you last showed it to me and I told you what you should generally be doing sounds a bit weird.
Did you try reading into how parallelization works? Distributing computations across parallel workers? Or just sit down and think about how you could actually distribute the specific work to be done across separate nodes working in unison?
Yikes, I tried to download the mag_papers_0.zip file and it's like twelve gigabytes. I definitely can't try to replicate this problem unless you have a smaller MCVE
@Kevin can you please look at json format from openacademic.ai/oag
Here's a starter if you really want to learn: write a simple program that computes the square of np.arange(100) using 4 nodes. Each node should work with one fourth of the array and square it. Each should keep an array of zeros and only fill the part they are working with; then collect (reduce) the parts onto one of the nodes.
Right, I was just on that page, and all of those files are huge.
18:41
yup
@Kevin sounds like MCVE time
i have downloaded all
now i have each text file up 1.8gb
Ok, great, we're on the same page. So go ahead and create a test file that's under 1 kilobyte, and which replicates your problem, and I'll be happy to try it out.
i need take each json obj from file (at least from one file) and write it to csv
yes i took some part from file and then started writing code
If you're going to say "actually you can just copy-paste the JSON from the 'for example:' section, that will show the problem just fine" at any point, now would be a great time
18:44
sorry i am not getting you
Never mind, I just tried it and it didn't work.
ok thanks lot
do you have any link that will have similar json data in Question and we can parse it ? can we convert this json from text file to .json file anfd then we can use paddas ?
pandas have simple way to convert json file to csv
I don't know what the json data looks like because I can't download a 16 gb file.
a worthy application of your crystal ball
@sunil So you just need to create a test file that's under 1 kilobyte that has the same form as your real data. That way we can write code that processes the small file, and if it creates a CSV of the desired form then we know it will work correctly on your real data.
18:51
can i post some data from file that i have created ?
{"title": "System and Method for Maskless Direct Write Lithography", "lang": "en", "year": 2015, "references": ["354c172f-d877-4e60-a7eb-c1b1cf03ce4d", "76cf1064-b2b2-4245-940b-4e25dab9d41d"], "abstract": "A system and method for maskless direct write lithography are disclosed. The method includes receiving a plurality of pixels that represent an integrated circuit (IC) layout; identifying a first subset of the pixels that are suitable for a first compression method; and identifying a second subset of the pixels that are suitable for a second compression method. The method further includes
sorry for this
We conventionally use pastebin for sharing large data.
@kevin can you see some column values are missing
Yeah, I see that.
I'd probably do something like pastebin.com/b9034ct0
One problem with your original code is that you were calling writerow with a single string, which puts each individual character of that string in its own cell
yup your code doing good
i am still reading csv file
can you please wait
thanks in advance
@kevin keys = input[1].keys() may not have all keys that will be column in csv
am i right
??
keys will only be the keys that are present in the second row of the file, yeah
If there are other keys that aren't in the second row, they won't show up in the csv
One possible way to get all the keys is keys = sorted({key for row in input for key in row.keys()})
(Possibly there is a shorter way to write that using set.intersection, but whatever)
Maintaining the keys' original order while also filling in gaps because no one row has all the keys, is moderately difficult
Doubly so for older versions of Python that doesn't preserve insertion order of dict items
19:11
"Maintaining the keys' original order while also filling in gaps because no one row has all the keys, is moderately difficult" yup i was thinking same
but still you help and its lot
let me see what i can do more
thanks lot
can you see this question
What about it
but we have text file with json and not .json file
it is about missing and different keys in json to csv
file extensions don't mean anything. A txt file with json in it is parsed exactly the same as a json file with json in it.
@sunil you've already accepted an answer to that question
19:23
its not my question
(Although actually what we have is not json, but newline-delimited json, aka JSON Lines)
yes
Not that that matters, I expect, since by the time we call open("data.csv","w"), input contains a list of dictionaries, which is exactly the kind of data that the question is operating on. It doesn't matter whether it originally came from json or json Lines
19:54
@Kevin
df=pd.read_json("test.txt",lines=True)
print(df)
df.to_csv('results.csv')
i am getting csv file
does this solve the issue
?
can you please try it
and let me know if possible
poor pandas
I have seen the line x=04 (for python 2) .
What could it mean?
In python 3 it gives error
do you have access to python 2?
20:58
Hm. When I assumed this, further calculations went wrong and produced wrong results.
Aw, thank you for link!
>>> type(04)
<type 'int'>
>>> type(4)
<type 'int'>
hold on, these are octal literals apparently
2
Q: leading zeros in python

user2171504Python seems to be able to accept leading zeros for any number except 08 or 09. For instance, a = 04 works in the interpreter but a = 08 returns SyntaxError: invalid token I'm using python 2.7.3 on OSX, and others have been able to duplicate the error. What gives?

Sorry, I didn't realize this. I was wondering why it was worth changing its behaviour in python 3...
3.0 release notes: "Octal literals are no longer of the form 0720; use 0o720 instead."
I can certainly see why they may have decided that the original notation was easy to confuse ;)
21:03
it is still decimal integer 4, is it?
specifically for 4, yes. And 0b07 == 7, but 0b08 (python 2 08) is an error
My error should be else where.
Thank you again!
if that's the only octal literal then yes
no problem :)
or should I say не за что :D
21:24
Think PM2Ring's answer to this question is a good dupe target for stackoverflow.com/questions/52787057/… ?
22:02
I newly upgraded my Monte Carlo code with a spin dynamics alternative, and it seems to be working. I'm so glad I originally wrote it to be modular
22:14
when i first read up on iter i was like this is pointless now I see it is usefull
 
1 hour later…
23:43
@user3483203 Not exactly, since the OP wants to create an inverted dict. FWIW, I'd forgotten about that answer. And it's Python 2, although it's easy enough to adapt to Py 3. FWIW, here's a more recent answer that parses nested dict + lists, it links to a few other related answers.
I wrote those related answers rather than simply hammering, because the questions were sufficiently different IMHO, and because it can be hard to wrap your head around recursive generators, especially if you aren't comfortable with both recursion and generators.
@user3483203 Ajax's answer on the new question is ok, except he uses a mutable default in both functions, so I left the OP a warning comment. And a link to my stuff, which I forgot to put in my 1st reply to you. :oops:
@vash_the_stampede Well, most of the time you use iter it's invisible. Whenever you make a for loop, iter gets called on the iterable you want to loop over.
01:00 - 17:0017:00 - 00:00

« first day (2918 days earlier)      last day (2252 days later) »