如何使用spark和java在mysql中插入模型

如何使用spark和java在mysql中插入模型,java,mysql,apache-spark,apache-spark-sql,Java,Mysql,Apache Spark,Apache Spark Sql,我是apache spark的新手,我想在java中使用spark将模型类插入mysql数据库 我试过使用下面的代码。但它不能正常工作 型号: public class User implements Serializable{ private static final long serialVersionUID = 1L; private int id; private String name; private String email; private String city; priva

我是apache spark的新手,我想在java中使用spark将模型类插入mysql数据库

我试过使用下面的代码。但它不能正常工作

型号:

public class User implements Serializable{

private static final long serialVersionUID = 1L;
private int id;
private String name;
private String email;
private String city;
private String country;
private String ip;

private static StructType structType = DataTypes.createStructType(new StructField[] {
        DataTypes.createStructField("id", DataTypes.IntegerType, false),
        DataTypes.createStructField("name", DataTypes.StringType, true),
        DataTypes.createStructField("email", DataTypes.StringType, true),
        DataTypes.createStructField("city", DataTypes.StringType, true),
        DataTypes.createStructField("country", DataTypes.StringType, true),
        DataTypes.createStructField("ip", DataTypes.StringType, true)
});

//getters & setters
public class MyApp {

private static final String MYSQL_USERNAME = "root";
private static final String MYSQL_PWD = "";
private static final String MYSQL_CONNECTION_URL = "jdbc:mysql://localhost:3306/sample?user=" + MYSQL_USERNAME + "&password=" + MYSQL_PWD;

private static final JavaSparkContext sc =
        new JavaSparkContext(new SparkConf().setAppName("SparkSaveToDb").setMaster("local[*]"));

private static final SQLContext sqlContext = new SQLContext(sc);

public static void main(String[] args) {
    //Sample data-frame loaded from a JSON file
    try{

        List<User> list = new ArrayList<User>();

        User us = new User();
        us.setId(1);
        us.setName("Ravi");
        us.setEmail("abc.xyz0@gmail.com");
        us.setCity("hyderabad");
        us.setCountry("India");
        us.setIp("127.0.0.1");
        list.add(us);

        //Creating RDD
        JavaRDD<User> personsRDD = sc.parallelize(list);
        DataFrame userDf = sqlContext.createDataFrame(personsRDD, User.class);

        //JdbcUtils.saveTable(userDf, MYSQL_CONNECTION_URL, "users", prop);
        userDf.write().mode(SaveMode.Append).jdbc(MYSQL_CONNECTION_URL, "users", new java.util.Properties());

    }catch(Exception e){
        System.out.println(e);
        System.exit(0);
    }
}
MyApp:

public class User implements Serializable{

private static final long serialVersionUID = 1L;
private int id;
private String name;
private String email;
private String city;
private String country;
private String ip;

private static StructType structType = DataTypes.createStructType(new StructField[] {
        DataTypes.createStructField("id", DataTypes.IntegerType, false),
        DataTypes.createStructField("name", DataTypes.StringType, true),
        DataTypes.createStructField("email", DataTypes.StringType, true),
        DataTypes.createStructField("city", DataTypes.StringType, true),
        DataTypes.createStructField("country", DataTypes.StringType, true),
        DataTypes.createStructField("ip", DataTypes.StringType, true)
});

//getters & setters
public class MyApp {

private static final String MYSQL_USERNAME = "root";
private static final String MYSQL_PWD = "";
private static final String MYSQL_CONNECTION_URL = "jdbc:mysql://localhost:3306/sample?user=" + MYSQL_USERNAME + "&password=" + MYSQL_PWD;

private static final JavaSparkContext sc =
        new JavaSparkContext(new SparkConf().setAppName("SparkSaveToDb").setMaster("local[*]"));

private static final SQLContext sqlContext = new SQLContext(sc);

public static void main(String[] args) {
    //Sample data-frame loaded from a JSON file
    try{

        List<User> list = new ArrayList<User>();

        User us = new User();
        us.setId(1);
        us.setName("Ravi");
        us.setEmail("abc.xyz0@gmail.com");
        us.setCity("hyderabad");
        us.setCountry("India");
        us.setIp("127.0.0.1");
        list.add(us);

        //Creating RDD
        JavaRDD<User> personsRDD = sc.parallelize(list);
        DataFrame userDf = sqlContext.createDataFrame(personsRDD, User.class);

        //JdbcUtils.saveTable(userDf, MYSQL_CONNECTION_URL, "users", prop);
        userDf.write().mode(SaveMode.Append).jdbc(MYSQL_CONNECTION_URL, "users", new java.util.Properties());

    }catch(Exception e){
        System.out.println(e);
        System.exit(0);
    }
}

任何人都可以让我知道如何解决这个问题,并插入到mysql数据库。提前感谢

首先

如果您的表实现了
自动增量
,如果您的
id
如下所示:

id int AUTO_INCREMENT
Insert into mytable (id, ...) values ('hyderabad', ...);
那么您就不需要设置id
us.setId(1),这可能会产生问题,考虑到你已经使用了这个“代码> ID=1 < /代码>,所以当你把<代码>我们。SETID(1);<代码>这是一个冲突,所以不要使用
us.setId(1)

否则,您应该首先检查此id是否不存在

第1行“id”列的整数值“hyderabad”不正确

此错误意味着您将
'hydrabad'
字符串设置为integer类型的id,因此您正在进行如下操作:

id int AUTO_INCREMENT
Insert into mytable (id, ...) values ('hyderabad', ...);
如果这是您的桌子:

create table mytable(
id int,
....
); 
然后,您的查询将如下所示:

id int AUTO_INCREMENT
Insert into mytable (id, ...) values ('hyderabad', ...);

这是不正确的,所以请检查您的插入查询,也许您只是更改了属性的顺序。

谢谢您的回复。是的,'id'是'AUTO_INCREMENT',但即使我注释'us.sedId(1)'我也会得到错误'java.sql.SQLException:不正确的整数值:第1行'id'列的'hydrabad',并且该表是新创建的表。