Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/redis/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 8执行器在使用ExecutorService读取'100k'文件路径时抛出RejectedExecutionException_Java_<img Src="//i.stack.imgur.com/RUiNP.png" Height="16" Width="18" Alt="" Class="sponsor Tag Img">elasticsearch_Java 8_Executorservice_Elastic Stack - Fatal编程技术网 elasticsearch,java-8,executorservice,elastic-stack,Java,elasticsearch,Java 8,Executorservice,Elastic Stack" /> elasticsearch,java-8,executorservice,elastic-stack,Java,elasticsearch,Java 8,Executorservice,Elastic Stack" />

Java 8执行器在使用ExecutorService读取'100k'文件路径时抛出RejectedExecutionException

Java 8执行器在使用ExecutorService读取'100k'文件路径时抛出RejectedExecutionException,java,elasticsearch,java-8,executorservice,elastic-stack,Java,elasticsearch,Java 8,Executorservice,Elastic Stack,我正在从ElasticSearch索引中获取一个100k文件路径,实际文件在本地驱动器中可用。根据文件路径,我必须将这些文件转换为base64 我正在使用scrollApi获取100k文件路径,并将其添加到Arraylist中。收集完所有文件路径后,我想将这些文件转换为base64 public class DocumentIndex { private final static String INDEX = "documents_local"; private final

我正在从ElasticSearch索引中获取一个
100k
文件路径,实际文件在本地驱动器中可用。根据文件路径,我必须将这些文件转换为base64

我正在使用
scrollApi
获取
100k
文件路径,并将其添加到Arraylist中。收集完所有文件路径后,我想将这些文件转换为base64

public class DocumentIndex {

    private final static String INDEX = "documents_local";  
    private final static String ATTACHMENT = "document_suggestion";
    private final static String TYPE = "doc";
    private static final Logger logger = Logger.getLogger(Thread.currentThread().getStackTrace()[0].getClassName());
    private static final int BUFFER_SIZE = 3 * 1024;

     public static void main(String[] args) throws IOException {
            ExecutorService executor = Executors.newFixedThreadPool(5);
            List<String> filePathList = new ArrayList<String>();


            RestHighLevelClient restHighLevelClient = null;
            RestHighLevelClient restHighLevelClient2 = null;
            Document doc=new Document();

            logger.info("Started Indexing the Document.....");


            //Fetching Id, FilePath & FileName from Document Index. 
            SearchRequest searchRequest = new SearchRequest(INDEX); 
            searchRequest.types(TYPE);
            final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(60L)); //part of Scroll API

            searchRequest.scroll(scroll); //part of Scroll API
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            QueryBuilder qb = QueryBuilders.matchAllQuery();

            searchSourceBuilder.query(qb);
            searchRequest.source(searchSourceBuilder);

            SearchResponse searchResponse = SearchEngineClient.getInstance3().search(searchRequest);
            String scrollId = searchResponse.getScrollId(); //part of Scroll API
            SearchHit[] searchHits = searchResponse.getHits().getHits();
            long totalHits=searchResponse.getHits().totalHits;
            logger.info("Total Hits --->"+totalHits);

            //part of Scroll API -- Starts
            while (searchHits != null && searchHits.length > 0) { 
                SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId); 
                scrollRequest.scroll(scroll);
                searchResponse = SearchEngineClient.getInstance3().searchScroll(scrollRequest);
                scrollId = searchResponse.getScrollId();
                searchHits = searchResponse.getHits().getHits();

                Map<String, Object> jsonMap ;
                for (SearchHit hit : searchHits) {

                    StringBuilder result = null;
                    String encodedfile = null;
                    File file=null;

                    Map<String, Object> sourceAsMap = hit.getSourceAsMap();


                    if(sourceAsMap != null) {  
                        doc.setId((int) sourceAsMap.get("id"));
                        doc.setApp_language(String.valueOf(sourceAsMap.get("app_language")));
                        doc.setFilename(String.valueOf(sourceAsMap.get("filename")));
                        doc.setPath(String.valueOf(sourceAsMap.get("path")));
                    }
                    if(doc.getPath()!= null && doc.getFilename() != null) {
                    filePathList.add(doc.getPath().concat(doc.getFilename()));
                    }
                }

                logger.info("File Path List size --->"+filePathList.size());


                    for (int i = 0; i < filePathList.size(); i++) {
                        Runnable worker = new WorkerThread(filePathList.get(i));
                        executor.execute(worker);
                      }
                    executor.shutdown();
                    while (!executor.isTerminated()) {
                    }
                    System.out.println("Finished all threads");
                        }
            }
}


public class WorkerThread implements Runnable {

 private String command;
 private static final Logger logger = Logger.getLogger(Thread.currentThread().getStackTrace()[0].getClassName());

    public WorkerThread(String s){
        this.command=s;
    }

    @Override
    public void run() {

        logger.info("File Path --->"+command);
        System.out.println(Thread.currentThread().getName()+" Start. Command = "+command);
        processCommand();
        System.out.println(Thread.currentThread().getName()+" End.");
    }

    private void processCommand() {
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    @Override
    public String toString(){
        return this.command;
    }

}
为此,我想为此进程创建一个工作线程(以加快进程)

请在下面找到我的代码。最初,我从弹性搜索索引读取文件路径,并将此文件路径传递给我的工作线程以读取路径(对于转换为base64,未完全实现)


当您在
ExecutorService
上调用
shutdown
后试图提交任务时,就会发生这种情况,因为您已经声明/创建了executor service的对象
ExecutorService executor=Executors.newFixedThreadPool(5)
while
循环
while(searchHits!=null&&searchHits.length>0){
之外。在
while
循环的第二次迭代中,它将尝试将任务提交到
执行器服务
,您已经在第一次迭代中对其调用了shutdown


要解决此问题,请声明
ExecutorService executor=Executors.newFixedThreadPool(5)
在while循环内或
关闭
while循环结束时

当您在
ExecutorService
上调用
关闭
后试图提交任务时会发生这种情况,因为您已经声明/创建了executor service的对象
ExecutorService executor=Executors.newFixedThreadPool(5) 
while
循环
while(searchHits!=null&&searchHits.length>0){
之外。在
while
循环的第二次迭代中,它将尝试将任务提交给
执行器服务
,您在第一次迭代时已对其调用了shutdown

要解决此问题,请声明
ExecutorService executor=Executors.newFixedThreadPool(5);
在while循环内,或
while循环结束时关闭它

Exception in thread "main" java.util.concurrent.RejectedExecutionException: Task D:\data\Files\doc753_v1_fr-FR.pdf rejected from java.util.concurrent.ThreadPoolExecutor@35dab4eb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 10]
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
    at com.jci.vb2.SearchEngineUtility.DocumentIndex.main(DocumentIndex.java:104)