Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/350.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 使用CallableTask/Futures和ObjectMapper时多线程代码的性能问题_Java_Multithreading_Jackson_Future - Fatal编程技术网

Java 使用CallableTask/Futures和ObjectMapper时多线程代码的性能问题

Java 使用CallableTask/Futures和ObjectMapper时多线程代码的性能问题,java,multithreading,jackson,future,Java,Multithreading,Jackson,Future,我正在从事一个基于REST服务的项目,其中我有两个组件,如下所述- 客户端,它将为服务组件生成必要的URL 然后服务(REST服务)组件将使用这些URL的从数据库中获取数据 一般来说,URL将如下所示- http://host.qa.ebay.com:8080/deservice/DEService/get/USERID=9012/PROFILE.ACCOUNT,个人资料。广告,个人资料。人口统计,个人资料。财务 上面URL的意思是-对于USERID-9012请从数据库中为这些列提供数据- [P

我正在从事一个基于REST服务的项目,其中我有两个组件,如下所述-

  • 客户端,它将为服务组件生成必要的
    URL
  • 然后服务(REST服务)组件将使用这些
    URL的
    从数据库中获取数据
  • 一般来说,URL将如下所示-

    http://host.qa.ebay.com:8080/deservice/DEService/get/USERID=9012/PROFILE.ACCOUNT,个人资料。广告,个人资料。人口统计,个人资料。财务

    上面URL的意思是-对于
    USERID-9012
    请从数据库中为这些列提供数据-

    [PROFILE.ACCOUNT、PROFILE.advisting、PROFILE.DEMOGRAPHIC、PROFILE.FINANCIAL]

    目前,我正在客户机组件端进行基准测试。我发现下面的方法需要花费大量的
    时间(95个百分点)
    大约
    ~15ms

    下面的方法将接受两个参数-

    列出密钥-密钥中的示例数据的USERID=9012

    列出reqAttrNames-reqAttrNames的示例数据将为-

    [PROFILE.ACCOUNT、PROFILE.advisting、PROFILE.DEMOGRAPHIC、PROFILE.FINANCIAL]

    下面是代码-

    public DEResponse getDEAttributes(List<DEKey> keys, List<String> reqAttrNames) {
    
        DEResponse response = null;
        try {
            String url = buildGetUrl(keys,reqAttrNames);
    
            if(url!=null){
                List<CallableTask<DEResponse>> tasks = new ArrayList<CallableTask<DEResponse>>();
                CallableTask<DEResponse> task = new DEResponseTask(url); 
                tasks.add(task);
    
                // STEP 2: Execute worker threads for all the generated urls
                List<LoggingFuture<DEResponse>> futures = null;
                try {
                    long waitTimeout = getWaitTimeout(keys);
                    futures = executor.executeAll(tasks, null, waitTimeout, TimeUnit.MILLISECONDS);
    
                    // STEP 3: Consolidate results of the executed worker threads
                    if(futures!=null && futures.size()>0){
                        LoggingFuture<DEResponse> future = futures.get(0);
                        response = future.get();
                    }
                } catch (InterruptedException e1) {
                    logger.log(LogLevel.ERROR,"Transport:getDEAttributes Request timed-out :",e1);
                }
            }else{
                //
            }
        }  catch(Throwable th) {
    
        }
    
        return response;
    }
    
    这个多线程代码的编写方式有问题吗?如果是,我如何才能使其高效

    executeAll的签名
    executor的方法
    与我的公司一样,他们有自己的executor,它将实现Sun executor类-

    /**
         * Executes the given tasks, returning a list of futures holding their 
         * status and results when all complete or the timeout expires, whichever
         * happens first.  <tt>Future.isDone()</tt> is <tt>true</tt> for each
         * element of the returned list.  Upon return, tasks that have not completed
         * are cancelled.  Note that a <i>completed</i> task could have terminated
         * either normally or by throwing an exception.  The results of this method
         * are undefined if the given collection is modified while this operation is
         * in progress.  This is entirely analogous to
         * <tt>ExecutorService.invokeAll()</tt> except for a couple of important
         * differences.  First, it cancels but does not <b>interrupt</b> any 
         * unfinished tasks, unlike <tt>ExecutorService.invokeAll()</tt> which
         * cancels and interrupts unfinished tasks.  This results in a better 
         * adherence to the specified timeout value, as interrupting threads may
         * have unexpected delays depending on the nature of the tasks.  Also, all 
         * eBay-specific features apply when the tasks are submitted with this 
         * method.
         * 
         * @param tasks the collection of tasks
         * @param timeout the maximum time to wait
         * @param unit the time unit of the timeout argument
         * @return a list of futures representing the tasks, in the same sequential
         * order as produced by the iterator for the given task list.  If the 
         * operation did not time out, each task will have completed.  If it did
         * time out, some of these tasks will not have completed.
         * @throws InterruptedException if interrupted while waiting, in which case
         * unfinished tasks are cancelled
         */
        public <V> List<LoggingFuture<V>> executeAll(Collection<? extends CallableTask<V>> tasks, 
                                                     Options options, 
                                                     long timeout, TimeUnit unit)
                throws InterruptedException {
            return executeAll(tasks, options, timeout, unit, false);
        }
    

    除了使用复杂的非标准执行器之外,我看不到任何会导致性能下降的因素。我知道在使用哪种执行器的问题上您没有任何选择,但出于好奇,我会尝试用
    ThreadPoolExecutor
    替换它,看看这是否有什么不同,如果你注意到一个重大的改进,那么就用你工作中的能力来解释它——在我的工作中,我们发现由另一个部门编写的加密库完全是垃圾(我们80-90%的CPU时间都花在了代码上),并成功地游说他们重写它

    编辑:

    公共类聚合器实现可运行{
    私有静态ConcurrentLinkedQueue=新ConcurrentLinkedQueue();
    私有静态ArrayList聚合=新建ArrayList();
    公开静态无效要约(未来){
    报价(未来);
    }
    公共静态ArrayList getAggregation(){
    收益聚合;
    }
    公开募捐{
    while(!queue.isEmpty()){//请确保在此循环开始之前添加了所有的未来;更好的是,如果您知道有多少工作线程,那么请记录聚合器中有多少个未来,并在aggregator.size()=[预期的未来数]时退出此循环
    add(queue.poll().get());
    }
    }
    }
    public void getdeAttribute(列表键、列表名称){
    试一试{
    如果(url!=null){
    试一试{
    futures=executor.executeAll(任务,null,等待超时,时间单位为毫秒);
    if(futures!=null&&futures.size()>0){
    聚合器.offer(futures.get(0));
    }
    }
    }
    }
    }
    
    如果我正确阅读了您的代码,那么有一个明显的性能问题。这:

    public class DEResponseTask  extends BaseNamedTask implements CallableTask<DEResponse> {
        private final ObjectMapper m_mapper = new ObjectMapper();
    
    公共类DEResponseTask扩展BaseNamedTask实现CallableTask{
    私有最终ObjectMapper m_mapper=新ObjectMapper();
    
    每个任务调用一次,创建
    ObjectMapper
    实例的成本非常高

    有很多方法可以解决此问题,但您可能希望:

  • 使m_映射器引用为静态(仅创建一次)--映射器在配置后可以安全共享,或者
  • 传入共享
    ObjectMapper
    (共享是安全的)

  • 这样做会大大提高JSON的处理效率。

    如果任务超过一定的运行时间,执行者是否是唯一可以终止任务的点?@Perception是的,我相信是的。这就是为什么我们在那里有超时的原因,这样我们就可以在占用大量时间后立即超时。谢谢Zim Zam的建议。是的,我会试试这个当然。我也更新了我的问题,没有更多的细节。也许你可以得到更多的想法。谢谢。
    future.get()
    正在阻塞;假设这个方法在它自己的线程中,你可以用
    while(!future.isDone()){thread.sleep(500);}future.get()替换它
    这样你就不用再等待了。谢谢Zim Zam。是的,它会在它自己的线程中。我用你的建议更新了我的问题。你能看一下吗,我在正确的地方做了正确的更改吗?谢谢。我又错过了一个要添加的单词。我再次添加了。再看一看,让我知道它是否好看。很抱歉。但是通过这种方式,什么我将获得的好处?看起来是正确的。好处是在未来之前。get()可能是自旋等待,这意味着它将继续消耗CPU资源,同时等待未来的完成-如果有很多线程这样做,那么它可能会对性能产生负面影响。这样,您只需在半秒的时间间隔内使用CPU,并且您可以让任意多的线程睡眠等待,而无需阻塞是的,我怎么能忘记呢。我只需要添加静态对象对吗?类似这样的东西?
    private static final ObjectMapper m\u mapper=new ObjectMapper();
    right?是的,这是最简单的方法。
    if(futures!=null && futures.size()>0){
                        LoggingFuture<DEResponse> future = futures.get(0);
                        //response = future.get();//replace this with below code-
    
                        while(!future.isDone()) {
                            Thread.sleep(500);
                        } 
    
                        response = future.get();
                    }
    
    public class Aggregator implements Runnable {
        private static ConcurrentLinkedQueue<Future<DEResponse>> queue = new ConcurrentLinkedQueue<>();
        private static ArrayList<DEResponse> aggregation = new ArrayList<>();
    
        public static void offer(Future<DEResponse> future) {
            queue.offer(future);
        }
    
        public static ArrayList<DEResponse> getAggregation() {
            return aggregation;
        }
    
        public void run() {
            while(!queue.isEmpty()) { // make sure that all of the futures are added before this loop starts; better still, if you know how many worker threads there are then keep a count of how many futures are in your aggregator and quit this loop when aggregator.size() == [expected number of futures]
                aggregation.add(queue.poll().get());
            }
        }
    }
    
    public void getDEAttributes(List<DEKey> keys, List<String> reqAttrNames) {
        try {
            if(url!=null){
                try {
                    futures = executor.executeAll(tasks, null, waitTimeout, TimeUnit.MILLISECONDS);
                    if(futures!=null && futures.size()>0){
                        Aggregator.offer(futures.get(0));
                    }
                }
            }
        }
    }
    
    public class DEResponseTask  extends BaseNamedTask implements CallableTask<DEResponse> {
        private final ObjectMapper m_mapper = new ObjectMapper();