last day (15 days later) » 

2:53 PM
0
Q: The current statement uses 7 columns and there are 6 supplied

Nicolas Charvoz KurzawaI'm having an issue with my python csv parser and I don't know where the mistake happens : So this is my python : # -*-coding:Utf-8 -* ...

 
I don't know why someone voted to close it, I have IMHO well documented my question, the disired behaviour is i don't want this error .. I want my 7 columns to be filled
My parser works fine for everything, I just created a new CSV and this happens
 
add a print len(vals) before executing the sql-insert.
 
Yeah my line has only 6 columns. I don't detect a mistake, is it possible, if the text is too long that python/sqlite reject it ?
 
i would rather guess the error is in the elif statement. Try reseting the quote there: quote=False, the default size-limit for text is over 1Gb afaik.
 
I used the same csv, removing the text and it works fine. So maybe My text has too many \n or something like that ..
 
2:53 PM
sizelimit of varchar/text: sqlite.org/faq.html#q9
As your last lst only has 6 items it stands to reson, that the error is in your mysplit function.
 
So you're telling me that my text isn't the problem ?
 
If you mean the length of your text? no!
you should look at vals[5] for the case of len(vals) == 6
Maybe you see the reason why obviously two records get merged as one
 
How long is the line? Your best bet would be to walk through mysplit by hand with that line, if it isn't initially obvious.
 
forget vals[5] just print the whole list if the list-length is lower 7
 
Sorry, so what should I do ?
 
3:09 PM
after vals = mysplit(entry.strip()) check for len(vals) < 7: print vals
and look for the item which should be split into two and figure out why it isn't
 
Maybe print entry also.
 
@wwii: yeah might be unavoidable
 
So my third line has only 6 columns
"6","Bergamote","Citrus bergamis L.","bergamote.png","L'huile essentielle de bergamote est obtenue par expression à froid du péricarpe frais du fruit du Citrus aurantium L.ssp bergamia Risso et Poit.","L'huile essentielle de bergamote intervient dans le traitement de l'acné qui ne tolère pas d'huile grasse.
De façon générale, elle agit sur les dermatoses, eczéma, prurit et plaies. Comme la plupart des huiles de citrus, l'huile essentielle de bergamote a des vertus apaisanteet tout à la fois tonifiantes, soit qu'elle aide à la digestion, soit qu'elle donne de l'énergie au système nerveux.En
As you can see I have a "\n" after "...huile grasse."
The problem is obviously here
 
don't you als have a \n after "photosensibilisante" ?
 
Yeah But the first one is after huile grasse, and it stops there
The enrty is : "6","Bergamote","Citrus bergamis L.","bergamote.png","L'huile essentielle de bergamote est obtenue par expression à froid du péricarpe frais du fruit du Citrus aurantium L.ssp bergamia Risso et Poit.","L'huile essentielle de bergamote intervient dans le traitement de l'acné qui ne tolère pas d'huile grasse.

6
 
3:17 PM
Do you control/make the data?
 
Yes
It's just copy/paste frome a word file
But I need those \n ( I want to display the text in a UITextView in an iOs app)
 
the 7th entr should be 0 ?
entr=entry
 
When you generate the data can you identify the newline characters that you want to keep?
 
3:21 PM
Then it's quite simple your error is in: data = open("info_max.csv", "r").readlines()[1:]
readlines splits at newlines
 
Oh yes .. Dumbass ..
 
Haha it was so simple
 
i will post that as my answer! Agreed? ;-)
 
Ca nyou tell me which function I should use ?
Becaue I don't see those \n in Text Wrangler for example .. So I don't know why they are here ?
And how should I keep them ? Because I need redlines to read every lines of my csv
 
3:24 PM
Why don't you use the csvreader-module as proposed by Hrabal? It's ind the stdlibs
Is every line the same, meaning are the colum-types for every entry the same?
 
So every line starts and ends with a number?
 
Yes
1 or 0 for the last
number
 
quoted or unquoted?
 
3:32 PM
is the distinction between 0 and 1 important?
 
then:
import re
rawdata = open(...).read()
entries = re.split('"0"|"1"', rawdata)
 
entries is data for me isn't it ?
 
or instead of entries data
yes ;-)
 
Ok i'm trying this
the read()[1:] work ?
 
3:39 PM
and you got to remove the 7th column of your database, because we use it to split the remaining 6
 
Oh so I won't have my 0 and 1 ?
 
yes
if you need them, we must use a match or find instead of re.split
 
Oh they are important sorry, i didn't understand your question
 
read()[:1] wont work
 
or maybe I could just create an 8th column with the word END in this , so I could use it to split ?
 
3:43 PM
Yes
 
so I did this, now how cqn I skip the first line ?
Just delete it from my csv ?
 
no
rawdata.split("\n", 1)[1:]
removes all before the first newline
data = rawdata.split("\n", 1)[1:]
data = re.split('"END"', data)
 
rawdata = open("info_max_try.csv", "r").read()[1:]
data = rawdata.split("\n", 1)[1:]
data = re.split('"END"', rawdata)
I got this now ok ?
 
just remove the [1:] from read/(
it just removes the first char
 
oh ok, so now it does'nt remove my first line
because i got this : invalid literal for int() with base 10: 'id_huiles'
 
3:54 PM
The csv module will parse the offending line correctly. @DonQuestion, you should also offer that as a solution in your answer. @NicolasCharvozKurzawa, you should update your question, adding the offending line and explaining that there are newline characters in some of the fields.
 
@wwii: your right
 
I apologies if it is rude to break in on the discussion
 
no its helpful, otherwise i would surely forget ;-)
 
Yeah right I'm updating thanks ;)
 
@NicolasCharvozKurzawa: we now got the data separated in lines, but you still need to split it into columns
 
3:56 PM
And what about my laste error Don ?
it takes my first line even with the rawdata.split
 
i guess int(vals[0]) doesn't contain a number
 
yes because it takes my first line so it tries to convert "id_huiles" into an integer
 
whats vals[0] after you mysplit?
 
it's "id_huiles"
 
print vals[0] returns id_hullies?
ahh yes
its you headerline
shouldn't happen after you do a rawdata.split("\n", 1)[1:]
because this should remove your headerline from your data
does it work now?
 
4:39 PM
Yes
Sorry I was on my way back home
No still the same
 

  last day (15 days later) »