Fatal编程技术网
  • cluster-computing/
  • Cluster computing Pentaho数据集成如何在点菜集群上使用厨房运行作业？

Cluster computing Pentaho数据集成如何在点菜集群上使用厨房运行作业？

cluster-computing pentaho

Cluster computing Pentaho数据集成如何在点菜集群上使用厨房运行作业？,cluster-computing,pentaho,kettle,pentaho-spoon,Cluster Computing,Pentaho,Kettle,Pentaho Spoon,我已经建立了一个carte集群（1个主集群和2个从集群），并用spoon在carte集群上运行了一个作业。但当我使用kitchen命令或CarteHTTP访问运行时，它是独立运行的（只在主节点中运行） 配置中是否遗漏了任何内容？或者它不支持集群模式 以下是我所尝试的： 我的配置： 与“环境类型——本地”一起运行 主输出： 2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/

我已经建立了一个carte集群（1个主集群和2个从集群），并用spoon在carte集群上运行了一个作业。但当我使用kitchen命令或CarteHTTP访问运行时，它是独立运行的（只在主节点中运行）

配置中是否遗漏了任何内容？或者它不支持集群模式

以下是我所尝试的：

  • 我的配置：

  • 与“环境类型——本地”一起运行

    主输出：

    2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/repositories.xml
Tue Nov 28 04:47:09 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017/11/28 04:47:10 - sortcluster111 (master) - Dispatching started for transformation [sortcluster111 (master)]
Tue Nov 28 04:47:10 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Nov 28 04:47:10 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017/11/28 04:47:10 - output.0 - Connected to database [102] (commit=1000)
2017/11/28 04:47:10 - input.0 - Finished reading query, closing connection.
2017/11/28 04:47:10 - input.0 - Finished processing (I=47, O=0, R=0, W=47, U=0, E=0)
2017/11/28 04:47:10 - input.0 - Server socket accepted for port [40001], reading from server Dynamic slave [kettleslave02:8083]
2017/11/28 04:47:10 - input.0 - Server socket accepted for port [40000], reading from server Dynamic slave [kettleslave01:8082]
2017/11/28 04:47:10 - output.0 - Finished processing (I=47, O=47, R=0, W=47, U=0, E=0)      

    2017/11/28 04:10:19 - trans1 - Starting entry [sorttrans]
2017/11/28 04:10:19 - sorttrans - Loading transformation from repository [sortcluster111] in directory [/]
2017/11/28 04:10:19 - sorttrans - Using run configuration [cluster config]
2017/11/28 04:10:19 - sorttrans - Using legacy execution engine
2017/11/28 04:10:19 - sortcluster111 - Dispatching started for transformation [sortcluster111]
Tue Nov 28 04:10:19 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be         established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL         by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Nov 28 04:10:19 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be         established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL         by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017/11/28 04:10:19 - output.0 - Connected to database [102] (commit=1000)
2017/11/28 04:10:19 - input.0 - Finished reading query, closing connection.
2017/11/28 04:10:19 - input.0 - Finished processing (I=47, O=0, R=0, W=47, U=0, E=0)
2017/11/28 04:10:19 - sort.0 - Finished processing (I=0, O=0, R=47, W=47, U=0, E=0)
2017/11/28 04:10:19 - output.0 - Finished processing (I=0, O=47, R=47, W=47, U=0, E=0)
2017/11/28 04:10:19 - trans1 - Starting entry [finish]
2017/11/28 04:10:19 - trans1 - Finished job entry [finish] (result=[true])
2017/11/28 04:10:19 - trans1 - Finished job entry [sorttrans] (result=[true])
2017/11/28 04:10:19 - trans1 - Finished job entry [SQL] (result=[true])
2017/11/28 04:10:19 - trans1 - Job execution finished
2017/11/28 04:10:19 - Kitchen - Finished!
2017/11/28 04:10:19 - Kitchen - Start=2017/11/28 04:10:00.586, Stop=2017/11/28 04:10:19.739
2017/11/28 04:10:19 - Kitchen - Processing ended after 19 seconds.
    slave01输出：

    2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/repositories.xml
Tue Nov 28 04:47:09 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must  be        established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to  explicitly disable SSL        by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017/11/28 04:47:10 - sortcluster111 (cluster1:Dynamic slave [kettleslave01:8082]) - Dispatching started for transformation [sortcluster111 (cluster1:Dynamic slave [kettleslave01:8082])]
2017/11/28 04:47:10 - sort.0 - Server socket accepted for port [40000], reading from server kettlemaster01
2017/11/28 04:47:10 - sort.0 - Finished processing (I=24, O=0, R=0, W=24, U=0, E=0)
    slave02输出：

    2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/repositories.xml
2017/11/28 04:47:09 - General - Unable to connect to the repository with name 'Mysqlrep'
2017/11/28 04:47:10 - sortcluster111 (cluster1:Dynamic slave [kettleslave02:8083]) - Dispatching started for transformation [sortcluster111 (cluster1:Dynamic slave [kettleslave02:8083])]
2017/11/28 04:47:10 - sort.0 - Server socket accepted for port [40000], reading from server kettlemaster01
2017/11/28 04:47:10 - sort.0 - Finished processing (I=23, O=0, R=0, W=23, U=0, E=0)
  • 与厨房一起运行：

    kitchen.sh -rep=Mysqlrep -user=admin -pass=admin -job trans1
    主输出：

    2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/repositories.xml
Tue Nov 28 04:47:09 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017/11/28 04:47:10 - sortcluster111 (master) - Dispatching started for transformation [sortcluster111 (master)]
Tue Nov 28 04:47:10 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Nov 28 04:47:10 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017/11/28 04:47:10 - output.0 - Connected to database [102] (commit=1000)
2017/11/28 04:47:10 - input.0 - Finished reading query, closing connection.
2017/11/28 04:47:10 - input.0 - Finished processing (I=47, O=0, R=0, W=47, U=0, E=0)
2017/11/28 04:47:10 - input.0 - Server socket accepted for port [40001], reading from server Dynamic slave [kettleslave02:8083]
2017/11/28 04:47:10 - input.0 - Server socket accepted for port [40000], reading from server Dynamic slave [kettleslave01:8082]
2017/11/28 04:47:10 - output.0 - Finished processing (I=47, O=47, R=0, W=47, U=0, E=0)      

    2017/11/28 04:10:19 - trans1 - Starting entry [sorttrans]
2017/11/28 04:10:19 - sorttrans - Loading transformation from repository [sortcluster111] in directory [/]
2017/11/28 04:10:19 - sorttrans - Using run configuration [cluster config]
2017/11/28 04:10:19 - sorttrans - Using legacy execution engine
2017/11/28 04:10:19 - sortcluster111 - Dispatching started for transformation [sortcluster111]
Tue Nov 28 04:10:19 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be         established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL         by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Nov 28 04:10:19 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be         established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL         by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2017/11/28 04:10:19 - output.0 - Connected to database [102] (commit=1000)
2017/11/28 04:10:19 - input.0 - Finished reading query, closing connection.
2017/11/28 04:10:19 - input.0 - Finished processing (I=47, O=0, R=0, W=47, U=0, E=0)
2017/11/28 04:10:19 - sort.0 - Finished processing (I=0, O=0, R=47, W=47, U=0, E=0)
2017/11/28 04:10:19 - output.0 - Finished processing (I=0, O=47, R=47, W=47, U=0, E=0)
2017/11/28 04:10:19 - trans1 - Starting entry [finish]
2017/11/28 04:10:19 - trans1 - Finished job entry [finish] (result=[true])
2017/11/28 04:10:19 - trans1 - Finished job entry [sorttrans] (result=[true])
2017/11/28 04:10:19 - trans1 - Finished job entry [SQL] (result=[true])
2017/11/28 04:10:19 - trans1 - Job execution finished
2017/11/28 04:10:19 - Kitchen - Finished!
2017/11/28 04:10:19 - Kitchen - Start=2017/11/28 04:10:00.586, Stop=2017/11/28 04:10:19.739
2017/11/28 04:10:19 - Kitchen - Processing ended after 19 seconds.
    从机中没有输出

    • 问候

    John

    首先，在远程服务器上启动点菜：

    • /carte.sh主机名端口
    • 签入浏览器：
      hostname:port/kettle/status/
      以空状态页响应
    第二，定义勺子中的点菜：

    • 在勺子视图（左面板）：单击
      Slave service
      r，然后单击
      new
    • 服务器名称=点菜
    • hostname=以前定义的主机名
    • 端口=以前定义的端口
    • web应用程序名称=空白（重要！）
    • 用户名=群集
    • 密码=群集 是主控=是
    第三，配置勺子运行：

    • 在spoon视图（左面板）：单击
      运行配置
      ，然后单击
      新建
    • 名称：carte/servername
    • 引擎：Pentaho（默认）
    • 从属服务器：是
    • 位置：carte（第二步中定义的服务器名称）
    • 将资源发送到此服务器：是（除非您）
    第四，运行转换/作业

    • 运行配置下拉框允许您在
      Pentaho local
      或
      carte/hostname
      或第三步中定义的任何配置之间进行选择
    • 如果禁用了运行选项面板，请下拉运行工具或顶部菜单/操作/运行选项或F8。并检查运行时始终显示对话框，以后您将需要它
    第五

    • 在浏览器中：
      hostname:port/kettle/status/
    • 玩得开心
        • 新版本的pdi中存在一个错误： 选项“在群集模式下运行此转换？”不存在，因此要修复此问题，您需要打开作业xml文件，删除属性Run\u configuration，并从要在群集模式下运行的转换条目中将cluster设置为Y。
        希望这有帮助。

        谢谢您的回复。但是我在我的勺子里找不到
        运行配置中的
        从服务器
        ，我只得到
        本地
        和
        远程
        ，我选择了
        远程
        ，
        位置
        是
        集群
        。我可以在集群中运行我的作业，但当我使用
        kitchen
        或
        hostname:port/kettle/executeJob
        时它不工作。我的PDI版本是7.1.0.0