Java Spring云数据流忽略Spring批处理应用程序配置的数据源

Java Spring云数据流忽略Spring批处理应用程序配置的数据源,java,spring-boot,spring-cloud-dataflow,Java,Spring Boot,Spring Cloud Dataflow,我正在设置Spring云数据流的一个实例。我已运行以下命令: 1. Run skipper server: java -jar spring-cloud-skipper-server-2.0.3.RELEASE.jar & 2. Run Dataflow server: java -jar spring-cloud-dataflow-server-2.1.2.RELEASE.jar \ --spring.datasource.url=jdbc:postgresql://10.13

我正在设置Spring云数据流的一个实例。我已运行以下命令:

1. Run skipper server: java -jar spring-cloud-skipper-server-2.0.3.RELEASE.jar &
2. Run Dataflow server: java -jar spring-cloud-dataflow-server-2.1.2.RELEASE.jar \
    --spring.datasource.url=jdbc:postgresql://10.136.66.44:8080/springclouddataflow \
    --spring.datasource.username=springclouddataflow \
    --spring.datasource.password=123456 \
    --spring.datasource.driver-class-name=org.postgresql.Driver \
    --server.port=80 &
在第二步中,我使用的是postgres数据库,而不是默认的h2

我已经使用spring批处理开发了一个spring引导作业,并部署在这个平台表单中。该作业使用两个数据源:Spring的springclouddataflow和任务元数据,以及我的业务逻辑的billrun。当我在本地运行应用程序时,它会像预期的那样将元数据保存在springclouddataflow中,并将我的业务数据保存在billrun中。 问题是当我试图在Spring云数据流中执行de-job时。plataform忽略我配置的业务逻辑数据库,只使用springclouddataflow数据库,该数据库应该只存储元数据

我在政府部门找过了。它解释了如何使用不同的数据库进行元数据存储,以及如何在应用程序中配置多个数据库。我按照指示做了,但没有成功

应用程序属性

logging.level.org.springframework.cloud.task=debug
spring.datasource.initialization-mode=always
spring.batch.initialize-schema=always
spring.application.name=Bill Run
spring.datasource.jdbc-url=jdbc:postgresql://10.136.66.44:8080/springclouddataflow?useSSL=false
spring.datasource.username=springclouddataflow
spring.datasource.password=123456
spring.datasource.driver-class-name=org.postgresql.Driver
app.datasource.jdbc-url=jdbc:postgresql://10.136.66.44:8080/billrun?useSSL=false
app.datasource.username=springclouddataflow
app.datasource.password=123456
app.datasource.driver-class-name=org.postgresql.Driver

数据源配置

@Configuration
public class DatasourceConfiguration {
    @Bean(name = "appDatasource")
    @ConfigurationProperties(prefix = "app.datasource")
    public DataSource sourceDataSource() {
        return DataSourceBuilder.create().build();
    }

    @Bean
    @Primary
    @ConfigurationProperties(prefix = "spring.datasource")
    public DataSource springDataSource() {
        return DataSourceBuilder.create().build();
    }

    @Bean
    public TaskConfigurer taskConfigurer() {
        return new DefaultTaskConfigurer(springDataSource());
    }
}
@Configuration
@EnableTask
@EnableBatchProcessing
public class BillingConfiguration {
    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Value("${usage.file.name:classpath:usageinfo.json}")
    private Resource usageResource;

    @Bean
    public Job job1(ItemReader<Usage> reader, ItemProcessor<Usage, Bill> itemProcessor, ItemWriter<Bill> writer) {
        Step step = stepBuilderFactory.get("BillProcessing").<Usage, Bill>chunk(1).reader(reader)
                .processor(itemProcessor).writer(writer).build();

        return jobBuilderFactory.get("BillJob").incrementer(new RunIdIncrementer()).start(step).build();
    }

    @Bean
    public JsonItemReader<Usage> jsonItemReader() {

        ObjectMapper objectMapper = new ObjectMapper();
        JacksonJsonObjectReader<Usage> jsonObjectReader = new JacksonJsonObjectReader<>(Usage.class);
        jsonObjectReader.setMapper(objectMapper);

        return new JsonItemReaderBuilder<Usage>().jsonObjectReader(jsonObjectReader).resource(usageResource)
                .name("UsageJsonItemReader").build();
    }

    @Bean
    public ItemWriter<Bill> jdbcBillWriter(@Qualifier("appDatasource") DataSource dataSource) {
        JdbcBatchItemWriter<Bill> writer = new JdbcBatchItemWriterBuilder<Bill>().beanMapped().dataSource(dataSource)
                .sql("INSERT INTO BILL_STATEMENTS (id, first_name, "
                        + "last_name, minutes, data_usage,bill_amount) VALUES "
                        + "(:id, :firstName, :lastName, :minutes, :dataUsage, " + ":billAmount)")
                .build();
        return writer;
    }

    @Bean
    ItemProcessor<Usage, Bill> billProcessor() {
        return new BillProcessor();
    }
}
计费配置

@Configuration
public class DatasourceConfiguration {
    @Bean(name = "appDatasource")
    @ConfigurationProperties(prefix = "app.datasource")
    public DataSource sourceDataSource() {
        return DataSourceBuilder.create().build();
    }

    @Bean
    @Primary
    @ConfigurationProperties(prefix = "spring.datasource")
    public DataSource springDataSource() {
        return DataSourceBuilder.create().build();
    }

    @Bean
    public TaskConfigurer taskConfigurer() {
        return new DefaultTaskConfigurer(springDataSource());
    }
}
@Configuration
@EnableTask
@EnableBatchProcessing
public class BillingConfiguration {
    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Value("${usage.file.name:classpath:usageinfo.json}")
    private Resource usageResource;

    @Bean
    public Job job1(ItemReader<Usage> reader, ItemProcessor<Usage, Bill> itemProcessor, ItemWriter<Bill> writer) {
        Step step = stepBuilderFactory.get("BillProcessing").<Usage, Bill>chunk(1).reader(reader)
                .processor(itemProcessor).writer(writer).build();

        return jobBuilderFactory.get("BillJob").incrementer(new RunIdIncrementer()).start(step).build();
    }

    @Bean
    public JsonItemReader<Usage> jsonItemReader() {

        ObjectMapper objectMapper = new ObjectMapper();
        JacksonJsonObjectReader<Usage> jsonObjectReader = new JacksonJsonObjectReader<>(Usage.class);
        jsonObjectReader.setMapper(objectMapper);

        return new JsonItemReaderBuilder<Usage>().jsonObjectReader(jsonObjectReader).resource(usageResource)
                .name("UsageJsonItemReader").build();
    }

    @Bean
    public ItemWriter<Bill> jdbcBillWriter(@Qualifier("appDatasource") DataSource dataSource) {
        JdbcBatchItemWriter<Bill> writer = new JdbcBatchItemWriterBuilder<Bill>().beanMapped().dataSource(dataSource)
                .sql("INSERT INTO BILL_STATEMENTS (id, first_name, "
                        + "last_name, minutes, data_usage,bill_amount) VALUES "
                        + "(:id, :firstName, :lastName, :minutes, :dataUsage, " + ":billAmount)")
                .build();
        return writer;
    }

    @Bean
    ItemProcessor<Usage, Bill> billProcessor() {
        return new BillProcessor();
    }
}
@配置
@使能任务
@启用批处理
公共类计费配置{
@自动连线
公共建筑商建筑商工厂;
@自动连线
公共StepBuilderFactory StepBuilderFactory;
@值(${usage.file.name:classpath:usageinfo.json}”)
私人资源使用来源;
@豆子
公共作业作业1(ItemReader阅读器、ItemProcessor ItemProcessor、ItemWriter写入程序){
Step Step=stepBuilderFactory.get(“BillProcessing”).chunk(1).reader(reader)
.processor(itemProcessor).writer(writer.build();
返回jobBuilderFactory.get(“BillJob”).incrementer(新的RunIdIncrementer()).start(步骤).build();
}
@豆子
公共JSONIMREADER JSONIMREADER(){
ObjectMapper ObjectMapper=新的ObjectMapper();
JacksonJsonObjectReader jsonObjectReader=新的JacksonJsonObjectReader(Usage.class);
setMapper(objectMapper);
返回新的JSONIMReaderBuilder().jsonObjectReader(jsonObjectReader).resource(usageResource)
.name(“usagejsonitmreader”).build();
}
@豆子
公共ItemWriter jdbcBillWriter(@Qualifier(“appDatasource”)数据源数据源){
JdbcBatchItemWriter writer=new JdbcBatchItemWriterBuilder().beanMapped().dataSource(数据源)
.sql(“插入账单对账单(id,名字,”
+姓氏、分钟数、数据(使用情况、账单金额)值
+(:id,:firstName,:lastName,:minutes,:dataUsage,“+”:billmount)”)
.build();
返回作者;
}
@豆子
ItemProcessor billProcessor(){
返回新的BillProcessor();
}
}
我已尝试将数据库属性作为参数传递给任务:


当我查看数据源时,springclouddataflow中只有持久化的数据。如何告诉spring cloud data flow使用我的应用程序数据源(billrun)?

看起来您正在定制spring cloud data flow服务器,以使用我认为不需要的应用程序数据源

您可以按照上面的说明启动SCDF服务器:

1. Run skipper server: java -jar spring-cloud-skipper-server-2.0.3.RELEASE.jar &
2. Run Dataflow server: java -jar spring-cloud-dataflow-server-2.1.2.RELEASE.jar \
    --spring.datasource.url=jdbc:postgresql://10.136.66.44:8080/springclouddataflow \
    --spring.datasource.username=springclouddataflow \
    --spring.datasource.password=123456 \
    --spring.datasource.driver-class-name=org.postgresql.Driver \
    --server.port=80 &
并且,让Spring批处理应用程序将其数据源属性作为Spring引导属性传递,而不是像上面那样使用自定义数据源配置


您可以使用类似的方法找到Spring Batch application development guide(Spring批处理应用程序开发指南)

只要您尝试在应用程序中添加动态数据源(如下所示),然后根据需要自动连接动态数据源

@配置
公共类数据源配置{
@Bean(name=“testingDataSource”)
@ConfigurationProperties(前缀=“testing.datasource”)
公共数据源testDataSource(){
返回DataSourceBuilder.create().build();
}
@Bean(name=“testingJdbcTemplate”)
公共JdbcTemplate testJdbcTemplate(@Qualifier(“testingDataSource”)数据源dsMySQL){
返回新的JdbcTemplate(dsMySQL);
}
}
测试: 数据源: driverClassName:'com.mysql.cj.jdbc.Driver' jdbc url:'jdbc:mysql://localhost/dbName' 用户名:“uname”
密码:“passworcd”

我需要这个“定制”,因为我必须在spring云数据库中存储元数据,在应用程序数据库中存储业务数据。文档中建议的方法为元数据和业务逻辑配置单个数据库。如何为spring批处理任务传递第二个数据库?我曾尝试将参数作为参数传递给任务(spring boot配置),但它不起作用。我还做了另一个测试,将数据库属性作为参数传递,结果成功了!非常感谢你的帮助!