Hive 普雷斯托+；运行presto查询时打开配置单元拆分时出错’；s

我们有基于HDP版本2.6.4和ambari的Hadoop集群

注意-Hadoop集群包括

hiveserver2和hivemetastore

我们正在运行presto查询，用于搜索

HDFS

文件系统中许多分区上的数据

这个问题很奇怪，因为每个新的查询都会在不同的

snappy.parquet

这是Presto dashboard中查询日志的典型示例

关于-io.prestosql.spi.PrestoException：打开配置单元拆分时出错

知道这个查询失败的原因吗

这是我们可以通过在ambari中更新配置单元参数来解决的问题吗

---

或者调整presto参数？

堆栈跟踪中是否有以下内容？@Piotr Findeisen与堆栈跟踪不太相似，您建议检查什么？

io.prestosql.spi.PrestoException: Error opening Hive split hdfs://hdfsha/POP/GGFR/eyes_data/eyes_data_daily/seen=1/lop_mode=DC/krtfg=202007/day=20200715/part-00157-91a34c06-45b9-40b7-a882-8716fdcf92be.c000.snappy.parquet (offset=0, length=16679): Could not obtain block: BP-2021402966-34.4.23.10-1523182226447:blk_1245415177_171674861 file=/POP/GGFR/eyes_data/eyes_data_daily/seen=1/lop_mode=DC/krtfg=202007/day=20200715/part-00157-91a34c06-45b9-40b7-a882-8716fdcf92be.c000.snappy.parquet
               at io.prestosql.plugin.hive.parquet.ParquetPageSourceFactory.createParquetPageSource(ParquetPageSourceFactory.java:230)
               at io.prestosql.plugin.hive.parquet.ParquetPageSourceFactory.createPageSource(ParquetPageSourceFactory.java:120)
               at io.prestosql.plugin.hive.HivePageSourceProvider.createHivePageSource(HivePageSourceProvider.java:164)
               at io.prestosql.plugin.hive.HivePageSourceProvider.createPageSource(HivePageSourceProvider.java:98)
               at io.prestosql.spi.connector.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:45)
               at io.prestosql.split.PageSourceManager.createPageSource(PageSourceManager.java:56)
               at io.prestosql.operator.ScanFilterAndProjectOperator$SplitToPages.process(ScanFilterAndProjectOperator.java:234)
               at io.prestosql.operator.ScanFilterAndProjectOperator$SplitToPages.process(ScanFilterAndProjectOperator.java:169)
               at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:320)
               at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:373)
               at io.prestosql.operator.WorkProcessorPipelineSourceOperator.getOutput(WorkProcessorPipelineSourceOperator.java:380)
               at io.prestosql.operator.Driver.processInternal(Driver.java:379)
               at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
               at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
               at io.prestosql.operator.Driver.processFor(Driver.java:276)
               at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1075)
               at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
               at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
               at io.prestosql.$gen.Presto_317____20200727_140355_1.run(Unknown Source)
               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
               at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-2021402966-34.4.23.10-1523182226447:blk_1245415177_171674861 file=/POP/GGFR/eyes_data/eyes_data_daily/seen=1/lop_mode=DC/krtfg=202007/day=20200715/part-00157-91a34c06-45b9-40b7-a882-8716fdcf92be.c000.snappy.parquet
               at org.apache.hadoop.hdfs.DFSInputStream.refetchLocations(DFSInputStream.java:879)
               at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:862)
               at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:841)
               at org.apache.hadoop.hdfs.DFSInputStream.fetchBlockByteRange(DFSInputStream.java:997)
               at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1360)
               at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1324)
               at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:121)
               at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:111)
               at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:120)
               at io.prestosql.parquet.reader.MetadataReader.readFully(MetadataReader.java:313)
               at io.prestosql.parquet.reader.MetadataReader.readFooter(MetadataReader.java:92)
               at io.prestosql.plugin.hive.parquet.ParquetPageSourceFactory.createParquetPageSource(ParquetPageSourceFactory.java:161)