last day (15 days later) » 

3:24 PM
1
A: Automate the process of comparing the values of 2 csv files if value matches read the second csv into the DataFrame

ChrisYou can iterate your tx_id and append the dataframes to list: import pandas as pd dfs = [] for tx in dframe1['tx_id']: dfs.append(pd.read_csv('%s.csv' % tx)) This only works if it's executed in the same directory as the csv files. Otherwise: import os import pandas dfs = [] for tx in d...

 
This is working @Chris. But can I have these in separate dataframes , like creating a separate dataframe for each grouped by tx_id.
 
@sagarkhanna I'm not sure what you mean by in separate dataframes. dfs contains separate dataframes. Do you want to access the dataframes by their names?
 
Yes. Earlier what I used to do is , I used to import the csv as df = pd.read_csv('csv_name.csv') and then I have to unstack it and use value_counts on it. I will edit my question and add what I need the result to look like.
 
@sagarkhanna I think its better if you post what you want to do and thus get from each dataframe in terms of code you would have applied individually.
 
Ok @Chris , I will edit my question.
I have edited my question.
 
3:24 PM
@sagarkhanna I've editted mine too :)
 
I have applied your code @Chris but I am getting the error that 'list indices must be integers , not str'
 
Which one did you run?
 
The first one.
I had all the csv in the same directory so I didn't use import os
 
I don't see a list there. Can you show me the entire traceback?
 
Sure
TypeError Traceback (most recent call last)
<ipython-input-38-890057a5c493> in <module>()
1 dfs = []
----> 2 for tx in dframe1['tx_id']:
3 dfs.append(pd.read_csv('%s.csv' % tx))
4 dfs.append(df.groupby('rule_id')['request_id'].value_counts().unstack().fillna(0))

TypeError: list indices must be integers, not str
 
3:27 PM
So it's saying dframe is a list not a dataframe (which is what i expected)
Have you changed your dframe1?
From your question, it seemed like it is a DataFrame.
 
Ok I'll check
Yes I think it is changed. It is a list now
 
So from your very original DataFrame, you can either make a new dframe1 that is described in your question.
Or you can make a list: OriginalDataFrame['tx_id'].unique() and iterate over this
 
Ok. I created a new dataframe and now it's working. But it is showing the unstacked data of only the dataframe which I have unstacked previously. Do I need to unstack the data everytime and then use append with different dataframes?
 
Yes. you can add unstack() to df inside the for loop to avoid doing so manually
 
3:45 PM
Can you provide me the code for that ?
 
You mean under the for tx in dframe1['tx_id']?
 
Yes. Because according to your code unstack() part is already inside the for loop
 
I'm getting a bit confused. IIUC, so far you made dframe1 which contains filenames
Then you are iterating it whether as a list or Series to use pd.read_csv(), right?
 
Yes.
This is the code you provided:
dfs = []
for tx in dft['tx_id']:
dfs.append(pd.read_csv('%s.csv' % tx))
dfs.append(df.groupby('rule_id')['request_id'].value_counts().unstack().fillna(0))
So in this unstack() is inside the for loop. So what do I need to do so that it unstacks it automatically
 
Take a look at the updated answer, after the editted part.
for tx in dft['tx_id']:
df = pd.read_csv('%s.csv' % tx)
and you do your `value_counts().unstack().fillna(0)
I'm not sure what you mean by automatically but once you put them in the for loop, every csv you read will be appended after getting unstacked
 
3:56 PM
Yes , I get it. Sorry I was using your previous code. I just added the unstack part on it. My apologies.
 
Not a problem. I think that solved all the issues :)
 
Yes it does. Thankyou so much.
 
I'll close this chat shortly. If you have any other inquiries, please leave comments :)
Take care!
 
4:22 PM
Will do :)
 

last day (15 days later) »