last day (15 days later) » 

4:32 PM
Hi
 
Hello arjun
 
So the two dataframes are really big, 132k rows in one and 467k in another. And the number of keys I'm comparing is 11
So it's really not possible for me to manually check if there are matches
 
I guess your column Seller and Buyer in your example contain duplicate entries, right?
 
Yeah, they do
Should have mentioned that in the post
 
Yep that's why isin not working here, we have to come up with alternate solution.
 
4:36 PM
Makes sense
 
So you just want to compare keys, for example 11 columns from both dataframes, right?
if they contain exactly same value we have to delete them, correct?
 
give me a min..
 
With merge() I can find the columns that match exactly, but since it merges the two dfs I'm not sure of how I can use it to remove the rows
 
@ArjunSrinivasan merge is good idea but i think its not necessary here and inefficient for large dataframes.
 
4:42 PM
Hmm okay
 
Which pandas version are you using, because for me its working correctly even with duplicate entries.
 
5:04 PM
Sorry, I'm using version 0.25.1 (2019-08-22). I just started on this and didn't realize myself
I don't have the necessary permissions to update to the latest version
Will have to request upgrades if the solution doesn't work in this version
 
Hmm.Possible to request upgrade? Because 0.25 seems too old..
 
I fixed the issue! There was a problem with something I was doing before applying set_index. Now isin is working without errors. Thanks a lot!
 
Great! Dont forget to accept the answer and upvote too :).
 
For sure
 
5:19 PM
Thanks if possible Can you please delete the comment as the problem is solved.
 

last day (15 days later) »