last day (14 days later) » 

3:27 PM
0
Q: Combining CSV files and splitting the column into 2 columns using R

VenkyI have 40 CSV files with only 1 column each. I want to combine all 40 files data into 1 CSV file with 2 columns. Data format is like this : I want to split this column by space and combine all 40 CSV files into 1 file. I want to preserve the number format as well. I tried below code but Num...

 
Thanks @Tim . Number formatting could be related to Excel but not sure about the splitting into 3 columns. The splitting is done by R. I verified the input data, it doesn't have any extra spaces. I think my program is doing something wrong while splitting.
 
Could you show us the R output instead of the excel screenshot?
 
@Xizam updated my question with R output. Thanks.
 
Try splitting by "\t"?
 
tried. "\t" is not splitting at all.
 
3:27 PM
Can you give me an example of the "merged" data so I can give it a try?
 
Thanks. You want the output csv file
 
No. I want the contents of the merged variable.
 
2.38651E-01 1.79016E+00
2
2.38834E-01 1.49658E+00
3
2.39016E-01 1.20340E+00
4
2.39199E-01 9.10339E-01
5
2.39381E-01 6.17828E-01
6
2.39564E-01 3.25867E-01
7
2.39747E-01 3.44543E-02
8
2.39930E-01 -2.56653E-01
9
2.40113E-01 -5.47607E-01
10
2.40296E-01 -8.37952E-01
11
2.40479E-01 -1.12781E+00
12
2.40662E-01 -1.41721E+00
13
2.40846E-01 -1.70618E+00
14
2.41029E-01 -1.99506E+00
15
2.41213E-01 -2.28342E+00
16
2.41396E-01 -2.57120E+00
17
2.41580E-01 -2.85852E+00
18
2.41763E-01 -3.14545E+00
19
2.41947E-01 -3.43225E+00
Is this fine for you?
 
Preferably as a file that I can load in R. So I know I am working on exactly the same data.
 
Any suggestion how i can export "merged" variable into a file?
 
3:31 PM
write.table(merged, file="file.name", quote=FALSE, row.names=FALSE, col.names=TRUE)
 
I think we cannot send any files except images here
 
dropbox link, wetransfer link
 
so, when i open that file in R it already autosplits it in two columns
without using the merge
try writing it with col.names=FALSE and sep="\t"
 
Ok. Then there is some problem in the code where i create "merged1"
 
3:41 PM
?
 
I created this file as u said..col.names=FALSE and sep="\t"
 
try opening it in excel
i dont think you did col.names=FALSE, since it still says V1 at the top of the first col
 
When i open that text file in excel.. data appears only in one column.. not splitted.
 
ah! but that is easier to fix
try writing as file1.csv
and if sep="\t" does not work, use sep=","
since .csv stands for comma seperated values
 
didn't work.
with both "," and "\t"
 
3:49 PM
wth is that crap at the top
 
ha ha.. weird.
it generated 3.5MB file with some crap on top and all the data in 1 column
 
I'm guessing the appearance of the stuff at the top messes it up
do write.csv?
* write.csv()
 
write.csv(format(merged1, scientific = F), "export1.csv", row.names=FALSE, na="NA")
ok.. with merged object
i will do that now
 
why format()?
just write.csv(merged, "export1.csv", col.names=FALSE, row.names=FALSE)
 
I just tried
with some foramtting and all that
didn't work though
same out put.. with some crap on top
so merged object seems to be corrupt
 
3:59 PM
alright
what exactly did you just try?
try without any formatting and anything special, just the basic output
 
write.csv(merged, "export2.csv", row.names=FALSE, col.names=FALSE)
i tried this.
 
ok, and do you still the the weird info at the top of your file in excel?
 
yes
 
Is file1.txt (your dropbox-link above) the initial desired format? Are the V1 and V2 labels meaningful?
If `file1.txt` is correct and labels are not meaningful, try this:
`head(strsplit(trimws(readr::read_delim("file1.txt", comment = "V", delim = "|", col_names=F)$X1), "[[:space:]]+"))`
 
if the data in the file1.txt is in 2 columns then that is the desired output.. I am not worried about the column names..
 
4:09 PM
not column names ... the first row is a single "V2", and row 6802 has a single "V1"
 
file1.txt is created from "merged" variable. I cannot use the file1.txt in the code. as it is just for testing.
I should be able to split the data in "merged" variable into 2 columns..
 
Have you tried something like strsplit(as.string(merged[,1]), " ") ?
 
I'm a little confused, sorry. Is the "merged" variable merely a data.frame with one column, and all entries look like "2.39564E-01 3.25867E-01" with no leading spaces and a space in the middle?
 
yes.. exactly..
I want to split the data in "merged" into 2 columns delimited by space. and push that data to 1 csv file
 
@Xizam: if the number of spaces is possibly variable, would strsplit(..., split = "[[:space:]]+") work?
 
4:13 PM
But it will have only 1 space in the middle always.
 
the `merged` variable I'm testing with looks like:
c("2.38651E-01 1.79016E+00", "2.38834E-01 1.49658E+00", "2.39016E-01 1.20340E+00",
"2.39199E-01 9.10339E-01", "2.39381E-01 6.17828E-01", "2.39564E-01 3.25867E-01"
)
 
Yes.. but take the values which have negative values..
 
I have to go cook. Good luck!
 
Thanks Xizam
2.39930E-01 -2.56653E-01
2.40113E-01 -5.47607E-01
2.40296E-01 -8.37952E-01
2.40479E-01 -1.12781E+00
2.40662E-01 -1.41721E+00
2.40846E-01 -1.70618E+00
2.41029E-01 -1.99506E+00
2.41213E-01 -2.28342E+00
2.41396E-01 -2.57120E+00
LIke these.
 
"take the values which have negative values..", are you doing something special with negative versus positive?
 
4:17 PM
nopes
You can see the 4 lines code which i have added in the question.. that's the only thing i am doing
I observed that split is working weirdly for negative values
it is creating another column for negative values
which is added as a screenshot in the question
 
"positive/negative" is precisely why you need " +" or "[[:space:]]+"
when the second column is positive, there are two spaces
using "[[:space:]]+" allows for tabs as well, but is probably not necessary if the file format is well-known, so " +" (that's "space plus")
data <- do.call("rbind", strsplit(as.character(trimws(merged$V1))," +",fixed=FALSE))
 
Then why would split= "\t" not work?
 
that's still only a single character
 
really? I thought \t was whitespace
 
you have TWO SPACES on some rows, "\t" is only a SINGLE TAB (which does nothing)
no
(sorry, not yelling, using all caps for emphasis)
:-)
did using " +" work?
 
4:22 PM
alright, this really explains the problem
post as answer?
 
glad it worked
 
Yup it worked..
Is there any suggestion on retaining the same number format in excel as well.
now the format is 2.40E-01
in excel.. i want it to be the orignial format.
 
format cells in excel
and set number of decimal values
 
Cool.. Thanks Xizam.
 
I understand ...
You can either (1) keep them as character strings (not convert to numeric), in which case you are betting on Excel to do the right thing (which is not always reliable); or (2) use formatting in Excel to show them the way you want.
 
4:28 PM
Got it.. Thank you so much guys..
 
Actually, they are already char in R ...
 
You really saved my time..
 
in excel, what is the format you want?
i.e., how many decimal places showing, etc
 
5
same as in input..
2.39930E-01 -2.56653E-01
 
that's excel formatting: right-click on the column (or cells), go to "Format Cells", confirm that category is "Scientific", and set "decimal places" to 5.
 
4:30 PM
Yea .. did that .. it's working
 
gth
 
:-)
 

last day (14 days later) »