Python 3.x 测试和训练数据集具有不同数量的特征

我试图在一些训练和测试数据上训练svm模型。如果我把测试数据和训练数据结合起来，这个程序运行得很好，但如果我把它们分开，测试模型的准确性，它会说

Traceback (most recent call last):
  File "/home/PycharmProjects/analysis.py", line 160, in <module>
  main()
  File "/home/PycharmProjects/analysis.py", line 156, in main
  learn_model(tf_idf_train,target,tf_idf_test)
  File "/home/PycharmProjects/analysis.py", line 113, in learn_model
  predicted = classifier.predict(data_test)
  File "/home/.local/lib/python3.4/site-packages/sklearn/svm/base.py", line 573, in predict
  y = super(BaseSVC, self).predict(X)
  File "/home/.local/lib/python3.4/site-packages/sklearn/svm/base.py", line 310, in predict
  X = self._validate_for_predict(X)
  File "/home/.local/lib/python3.4/site-packages/sklearn/svm/base.py", line 479, in _validate_for_predict
  (n_features, self.shape_fit_[1]))
  ValueError: X.shape[1] = 19137 should be equal to 4888, the number of features at training time

---

如何解决这个问题

列车和试验部件必须使用相同的矢量器和变压器；此外，矢量器不应该适合测试数据。所以不是

count_vectorizer = CountVectorizer(binary='true')
dataTrain = count_vectorizer.fit_transform(dataTrain)
tfidf_train_data = TfidfTransformer(use_idf=True).fit_transform(dataTrain)

count_vectorizer = CountVectorizer()
testData = count_vectorizer.fit_transform(testData)
tfidf_test_data = TfidfTransformer(use_idf=True).fit_transform(testData)

使用类似以下内容：

count_vectorizer = CountVectorizer(binary=True)
tfidf_transformer = TfidfTransformer(use_idf=True) 
dataTrain = count_vectorizer.fit_transform(dataTrain)
tfidf_train_data = transformer.fit_transform(dataTrain)

testData = count_vectorizer.transform(testData)
tfidf_test_data = tfidf_transformer.transform(testData)

您还可以使用使其更美观：

from sklearn.pipeline import make_pipeline
pipe = make_pipeline(
    CountVectorizer(binary=True),
    TfidfTransformer(use_idf=True),
)
tfidf_train_data = pipe.fit_transform(dataTrain) 
tfidf_test_data = pipe.transform(testData)

甚至可以在单个矢量器对象中使用CountVectorizer和TfidfTransformer组合：

from sklearn.feature_extraction.text import TfidfVectorizer
vec = TfidfVectorizer(binary=True, use_idf=True)
tfidf_train_data = vec.fit_transform(dataTrain) 
tfidf_test_data = vec.transform(testData)

from sklearn.feature_extraction.text import TfidfVectorizer
vec = TfidfVectorizer(binary=True, use_idf=True)
tfidf_train_data = vec.fit_transform(dataTrain) 
tfidf_test_data = vec.transform(testData)