last day (15 days later) » 

4:55 PM
3
A: How to train large Dataset for classification

runDOSrunBefore speeding up the training I'd personally make sure that you actually need to. While not a direct answer to your question, I'll try to provide a different angle which you might or might not be missing (hard to tell from your initial post). Take e.g. superbly's implementation as a baseline. ...

 
What is the accuracy if you ignore test tweets with label 2?
Thank you, I will try splitting 80/20% for train/test and will inform you .
 
If I do that it goes up from 0.36 to 0.5 (test size 369, train 50k, 3 features, SVM, class 0 and 4 are split 50/50). Using a training size of 6k it's still 0.5 indicating the problem I talked about. You should also definitely "test" with your training data to see at which point you reach 100% or the error converges - stop training at exactly that point as any more training will produce the same or worse results.
 
how did you select this 3 features..? I thought all unique words will be features
could you please tell me what will be accuracy if you split training Dataset 80/20% into training set and test set?
 
I just randomly took superbly's 3 features to have a baseline. All unique words can be features but it depends on what exactly you're trying to accomplish and thus which algorithm you choose to train. Using scikit you can use bag of words and sparse representation: scikit-learn.org/dev/modules/…
 
This is new Dataset link goo.gl/VRttrT . I used 40k training and 10k testing tweets. and also 2k features. But accuracy is 0.50. This is low i think
 
4:55 PM
Yes, 0.5 is random and as that quite bad. You need to find out where the error on the training set converges. The test accuracy might very well go from 0 to 1 and from 1 back to 0.5-0. Unless you test the parameters extensively it's hard to tell if your 0.5 is "on the way up to 1.0" or "way down from 1.0". If it's on the "way down", you're training too much.
 
How can I check this? "on the way up to 1.0 or way down from 1.0 ? I have no idea how to do this? How can i prevent when i'm starting way down from 1
hellow?
 
5:19 PM
I checked varying size of dataset. All most all gives 0.50 accuracy.. I think you are right.. I need your help
 
During Training your accuracy on the training set will eventually reach 100% (or a value close to it). At that point your error is quite low and will not reduce much, no matter how many more examples you present. You need to kinda figure out after how much training this happens (the most basic start would be to just test with the training set, check your accuracy, retrain with less samples, recheck accuracy and adjust your sample size)
 
I get it. but.. But any tutorial, or any thing to learn the process how can i check the accuracy of training?
 
Assuming you want to continue using scikit, you'd need something like this: scikit-learn.org/stable/auto_examples/…
as you see in the 1st picture, your optimum is reached at some point and then it gets worse
Or see the code in section "Detecting over-fitting": astro.washington.edu/users/vanderplas/Astr599/notebooks/…
This is also quite useful I think if you want to use SVM: scikit-learn.org/stable/modules/learning_curve.html
 
I need to use SVM. I have already completed naive bayes in C++ with accuracy 0.81 . Now I am trying to do it with SVM. Actually I will apply this in my native language
 
That sounds good! I think language/library doesn't matter. The things you need for this are available in pretty much any lib.
 
5:34 PM
yes.. Thats why i'm trying to understand the whole things I need in python for English. I will convert to my native language. If I need your help further, I will knock you. Thank you very much.
 
5:55 PM
I can't take all unique words as features. I've 8gb RAM. And its giving me memory error.. How can I select appropriate features in this case?
 
6:12 PM
You're welcome! Try only using the most frequent words/nouns (unigram, I think I linked it somewhere before) or/and see if you can use sparse matrix/vector representations
 

last day (15 days later) »