Apache spark ';输入验证失败';Spark-MLlib多类logistic回归中的误差

Apache spark ';输入验证失败';Spark-MLlib多类logistic回归中的误差,apache-spark,machine-learning,pyspark,apache-spark-mllib,logistic-regression,Apache Spark,Machine Learning,Pyspark,Apache Spark Mllib,Logistic Regression,我有一个5000行401列的训练集,其中第1列是标签,其余400列是特征。我正在尝试使用pyspark mllib进行多类逻辑回归。请在下面找到我的代码。我必须承认,这不是一个经过优化/编写良好的代码,因为我仍然是python/pyspark领域的新手 tset=sio.loadmat('ex3data1.mat')#从mat文件加载训练集 X=tset['X']#读取X,y值 y=tset['y'] 打印(X.shape)#有效! 打印(y形) sp= SparkSession.builder

我有一个5000行401列的训练集,其中第1列是标签,其余400列是特征。我正在尝试使用pyspark mllib进行多类逻辑回归。请在下面找到我的代码。我必须承认,这不是一个经过优化/编写良好的代码,因为我仍然是python/pyspark领域的新手

tset=sio.loadmat('ex3data1.mat')#从mat文件加载训练集
X=tset['X']#读取X,y值
y=tset['y']
打印(X.shape)#有效!
打印(y形)
sp=
SparkSession.builder.master(“本地”).appName(“多分类器”).getOrCreate()
sc=sp.sparkContext
XY=np。连接((y,X),轴=1)#5000x401,其中第一列为标签。
打印(XY[0:2])
上面打印的样本输出。请注意,我只打印第一行

[[1.00000000e+01.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
8.56059680e-06 1.94035948e-06-7.37438725e-04-8.13403799e-03
-1.86104473e-02-1.87412865e-02-1.87572508e-02-1.90963542e-03
-1.64039011e-02-3.78191381e-03 3.30347316e-04 1.27655229e-05
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00.00000000e+00 1.16421569e-04
1.20052179e-04-1.40444581e-02-2.84542484e-02 8.03826593e-02
2.66540339e-01 2.73853746e-01 2.78729541e-01 2.74293607e-01
2.24676403e-01 2.77562977e-02-7.06315478e-03 2.34715414e-04
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
0.00000000e+000.00000000e+001.2833523E-17-3.26286765e-04
-1.38651604e-02 8.15651552e-02 3.82800381e-01 8.57849775e-01
1.00109761e+00 9.69710638e-01 9.30928598e-01 1.00383757e+00
9.64157356e-01 4.49256553e-01-5.60408259e-03-3.78319036e-03
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
5.10620915e-06 4.36410675e-04-3.95509940e-03-2.68537241e-02
1.00755014e-01 6.42031710e-01 1.03136838e+00 8.50968614e-01
5.43122379e-01 3.42599738e-01 2.68918777e-01 6.68374643e-01
1.01256958e+00 9.03795598e-01 1.04481574e-01-1.66424973e-02
0.00000000e+00.00000000e+00.00000000e+00.00000000e+00.00000000e+00
2.59875260e-05-3.10606987e-03 7.52456076e-03 1.77539831e-01
7.92890120e-01 9.65626503e-01 4.63166079e-01 6.91720680e-02
-3.64100526e-03-4.12180405e-02-5.01900656e-02 1.56102907e-01
9.01762651e-01 1.04748346e+00 1.51055252e-01-2.16044665e-02
0.00000000e+000.00000000e+000.00000000e+005.87012352e-05
-6.40931373e-04-3.23305249e-02 2.78203465e-01 9.36720163e-01
1.04320956e+00 5.98003217e-01-3.59409041e-03-2.16751770e-02
-4.81021923e-03 6.16566793e-05-1.23773318e-02 1.55477482e-01
9.14867477e-01 9.20401348e-01 1.09173902e-01-1.71058007e-02
0.00000000 E+00 0.00000000 E+00 1.56250000e-04-4.27724104e-04
-2.51466503e-02 1.30532561e-01 7.81664862e-01 1.02836583e+00
7.57137601e-01 2.84667194e-01 4.86865128e-03-3.18688725e-03
0.00000000e+008.36492601e-04-3.70751123e-02 4.52644165e-01
1.03180133e+00 5.39028101e-01-2.4374261E-03-4.80290033e-03
0.00000000e+000.00000000e+00-7.03635621e-04-1.2726443E-02
1.61706648e-01 7.79865383e-01 1.03676705e+00 8.04490400e-01
1.60586724e-01-1.38173339e-02 2.14879493e-03-2.12622549e-04
2.04248366e-04-6.85907627e-03 4.31712963e-04 7.20680947e-01
8.48136063e-01 1.51383408e-01-2.28404366e-02 1.98971950e-04
0.00000000e+00.00000000e+00-9.40410539e-03 3.7452005E-02
6.94389110e-01 1.02844844e+00 1.01648066e+00 8.80488426e-01
3.92123945e-01-1.74122413e-02-1.20098039e-04 5.55215142e-05
-2.23907271e-03-2.76068376e-02 3.68645493e-01 9.36411169e-01
4.59006723e-01-4.24701797e-02 1.17356610e-03 1.88929739e-05
0.00000000e+000.00000000e+00-1.93511951e-02 1.2999794E-01
9.79821705e-01 9.41862388e-01 7.75147704e-01 8.73632241e-01
2.12778350e-01-1.72353349e-02 0.00000000 E+00 1.09937426e-03
-2.61793751e-02 1.22872879e-01 8.30812662e-01 7.26501773e-01
5.24441863e-02-6.18971913e-03 0.00000000e+00.00000000e+00
0.00000000e+00.00000000e+00-9.36563862e-03 3.68349741e-02
6.9907929E-01 1.00293583e+00 6.05704402e-01 3.2729224E-01
-3.22099249e-02-4.83053002e-02-4.34069138e-02-5.75151144e-02
9.55674190e-02 7.26512627e-01 6.95366966e-01 1.47114481e-01
-1.20048679e-02-3.02798203e-04 0.00000000e+00.00000000e+00
0.00000000e+000.00000000e+00-6.76572712e-04-6.51415556e-03
1.17339359e-01 4.21948410e-01 9.93210937e-01 8.82013974e-01
7.45758734e-01 7.23874268e-01 7.23341725e-01 7.20020340e-01
8.45324959e-01 8.31859739e-01 6.88831870e-