last day (14 days later) » 

7:47 PM
0
Q: How to use categorical (factor) predictors with different levels in train and test sets

adatumI have a data set with a categorical predictor variable that has X levels in the training set, and Y levels in the test set. The actual values of the variable are meaningless; the reason I want to include them is that they serve to group related rows, which I think is useful information to includ...

 
The model already knows they are the same observation, I think it shouldn't be necessary, specially if the variable has to many unique values. I am sure that if you graph variable importance it will tell you that it is a very low importance variable. Test that to convince yourself of it :)
 
The test/future data can be preprocessed before making predictions on them, so that the unseen level can be used to group the new data. Other examples might be grouping voltage/current measurements from individual physical devices together (to account for systematic errors), or grouping sales or inventory data for individual stores (to account for demographics), etc. Having an "Other" category wouldn't be useful, as the only purpose of the variable is to cluster related data together.
@DerekCorcoran The ratio of categories to data is low, so there are many data points per category. The reason I want to include this is because there is another predictor variable whose values give information relative to a given category, but are meaningless in isolation or averaged over the whole data set.
 
@adatum tell me if this is enough to put it in an aswer: using the function createDataPartition in the caret package ensures that you have equal representation of factor variables in your training and test data.
 
@DerekCorcoran Thanks, I know that caret samples using stratification. The issue here is that the factor levels of future data will be different. So, the training data and built model should not depend on knowing all possible levels (or even care about the level labels).
 
@adatum got it, shall think about it.
 
8:02 PM
I didn't know StackOverflow has chat. Sorry for not including the actual data/motivation for my question. It's an active ML competition, which is why my competitive side wanted to keep my strategy private for now. But the learning opportunity is more interesting so I'll share the specific data info and strategy if you'd like.
 
8:14 PM
Hey adatum
 
Hey Derek
 
I was thinking that there are some models like lmer that in the predict have the feature allow.new.levels = TRUE
 
hmm. I'm not familiar with lmer. I'll look it up.
 
yeah I don't know if caret has it
 
seems it does: lme4 package
 
8:23 PM
hope it helps good luck
 
thanks for mentioning it
would you like a quick description of the motivating data/problem?
 

last day (14 days later) »