Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/python-3.x/16.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Amazon emr 通过emr启动配置进行presto配置_Amazon Emr_Presto - Fatal编程技术网

Amazon emr 通过emr启动配置进行presto配置

Amazon emr 通过emr启动配置进行presto配置,amazon-emr,presto,Amazon Emr,Presto,我正试图通过我们的EMR启动配置JSON部署presto om EMR。我已经决定了本文建议的配置属性。我在启动配置中添加了以下presto属性 { "Classification": "presto-connector-hive", "Properties": { "hive.metastore.glue.datacatalog.enabled": "true", "hive.table-statistics-enabled": "true"

我正试图通过我们的EMR启动配置JSON部署presto om EMR。我已经决定了本文建议的配置属性。我在启动配置中添加了以下presto属性

{
    "Classification": "presto-connector-hive",
    "Properties": {
        "hive.metastore.glue.datacatalog.enabled": "true",
        "hive.table-statistics-enabled": "true"
    },
    "Configurations": []
},
{
    "Classification": "presto-config",
    "Properties": {
        "query.max-memory": "150G",
        "query.max-memory-per-node": "20G",
        "query.max-total-memory-per-node": "30G",
        "memory.heap-headroom-per-node": "10G",
        "query.initial-hash-partitions": "15"
    },
    "Configurations": []
}   
已创建EMR群集,但由于以下错误,presto失败

1) Explicit bindings are required and com.facebook.presto.memory.LowMemoryKiller is not explicitly bound.
  while locating com.facebook.presto.memory.LowMemoryKiller
    for parameter 7 at com.facebook.presto.memory.ClusterMemoryManager.<init>(ClusterMemoryManager.java:123)
  at com.facebook.presto.server.CoordinatorModule.setup(CoordinatorModule.java:189) (via modules: com.facebook.presto.server.ServerMainModule -> com.facebook.presto.server.CoordinatorModule)

2) Error: Could not coerce value '150G' to io.airlift.units.DataSize (property 'query.max-memory') in order to call [public com.facebook.presto.memory.MemoryManagerConfig com.facebook.presto.memory.MemoryManagerConfig.setMaxQueryMemory(io.airlift.units.DataSize)]

3) Error: Could not coerce value '20G' to io.airlift.units.DataSize (property 'query.max-memory-per-node') in order to call [public com.facebook.presto.memory.NodeMemoryConfig com.facebook.presto.memory.NodeMemoryConfig.setMaxQueryMemoryPerNode(io.airlift.units.DataSize)]

4) Configuration property 'memory.heap-headroom-per-node' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

5) Configuration property 'query.max-memory' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

6) Configuration property 'query.max-memory-per-node' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

7) Configuration property 'query.max-total-memory-per-node' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

7 errors
    at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:466)
    at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:155)
    at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:107)
    at com.google.inject.Guice.createInjector(Guice.java:96)
    at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:241)
    at com.facebook.presto.server.PrestoServer.run(PrestoServer.java:114)
    at com.facebook.presto.server.PrestoServer.main(PrestoServer.java:66)
安装程序失败,因为

  • 设置数据大小配置属性时,需要使用“GB”(而不是“G”)作为单位
  • 您的版本(0.194)不支持您正在设置的某些属性(
    memory.heap headloom per node
    query.max total memory per node
  • 安装程序失败,因为

  • 设置数据大小配置属性时,需要使用“GB”(而不是“G”)作为单位
  • 您的版本(0.194)不支持您正在设置的某些属性(
    memory.heap headloom per node
    query.max total memory per node

  • “150G”不正确。它应该是“150GB”(“B”在末尾)。其他与内存相关的属性也一样。谢谢,但我仍然面临错误4和7这是哪个Presto版本?(
    memory.heap headloom per node
    不是那么旧的属性)我使用的是Presto 0.194版本没有
    内存。heap headloom per node
    query.max total memory per node
    属性。“150G”不正确。它应该是“150GB”(“B”在末尾)。其他与内存相关的属性也一样。谢谢,但我仍然面临错误4和7这是哪个Presto版本?(
    memory.heap headloom per node
    不是那么旧的属性)我使用的是Presto 0.194版本没有
    内存。heap headloom per node
    query.max total memory per node
    属性。
    coordinator=true
    node-scheduler.include-coordinator=false
    discovery.uri=X.X.X.X:YYYY
    http-server.threads.max=500
    discovery-server.enabled=true
    sink.max-buffer-size=1GB
    query.max-memory=150G
    query.max-memory-per-node=20G
    query.max-history=40
    query.min-expire-age=30m
    http-server.http.port=8889
    http-server.log.path=/var/log/presto/http-request.log
    http-server.log.max-size=67108864B
    http-server.log.max-history=5
    log.max-size=268435456B
    log.max-history=5
    query.initial-hash-partitions = 15
    memory.heap-headroom-per-node = 10G
    query.max-total-memory-per-node = 30G