Java 在大多数情况下，是什么让Jsoup比HttpURLConnection和HttpClient更快
我想比较一下标题中提到的三种实现的性能，我编写了一个小JAVA程序来帮助我做到这一点。主方法包含三个测试块，每个测试块如下所示：Java 在大多数情况下，是什么让Jsoup比HttpURLConnection和HttpClient更快,java,optimization,jsoup,httpclient,httpurlconnection,Java,Optimization,Jsoup,Httpclient,Httpurlconnection,我想比较一下标题中提到的三种实现的性能，我编写了一个小JAVA程序来帮助我做到这一点。主方法包含三个测试块，每个测试块如下所示： nb=0; time=0; for (int i = 0; i < 7; i++) { double v = methodX(url); if(v>0){ nb++; time+=v; }
nb=0; time=0;
for (int i = 0; i < 7; i++) {
double v = methodX(url);
if(v>0){
nb++;
time+=v;
}
}
if(nb==0) nb=1;
System.out.println("HttpClient : "+(time/ ((double) nb))+". Tries "+nb+"/7");
事实上，我分析了Jsoup的源代码，它表明它使用了带有BufferedInputStream的HttpURLConnection，我尝试以HttpURLConnection的方式使用这两种连接，但正如您所看到的，相同的结果是，区别很明显，Jsoup显然比HttpURLConnection快，并且它使用HttpURLConnection
提前感谢，您的基准测试没有意义 我为这三个库编写了一个微基准，结果是没有显著差异
Benchmark Mode Cnt Score Error Units
HttpBenchmark.httpClientGoogle avgt 2 151.162 ms/op
HttpBenchmark.httpClientStackoverflow avgt 2 151.086 ms/op
HttpBenchmark.httpUrlConnectionGoogle avgt 2 235.869 ms/op
HttpBenchmark.httpUrlConnectionStackoverflow avgt 2 145.162 ms/op
HttpBenchmark.jsoupGoogle avgt 2 391.162 ms/op
HttpBenchmark.jsoupStackoverflow avgt 2 188.059 ms/op
@Warmup(iterations = 1, time = 3, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 2, time = 5, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Threads(1)
public class HttpBenchmark {
private static final String GOOGLE = "https://google.com/";
private static final String STACKOVERFLOW = "https://stackoverflow.com";
private final CloseableHttpClient httpClient = HttpClientBuilder.create().build();
@Benchmark
public void httpClientGoogle() throws Exception {
httpClient(GOOGLE);
}
@Benchmark
public void httpClientStackoverflow() throws Exception {
httpClient(STACKOVERFLOW);
}
@Benchmark
public void httpUrlConnectionGoogle() throws Exception {
httpUrlConnection(GOOGLE);
}
@Benchmark
public void httpUrlConnectionStackoverflow() throws Exception {
httpUrlConnection(STACKOVERFLOW);
}
@Benchmark
public void jsoupGoogle() throws Exception {
jsoup(GOOGLE);
}
@Benchmark
public void jsoupStackoverflow() throws Exception {
jsoup(STACKOVERFLOW);
}
private void httpClient(final String url) throws Exception {
final CloseableHttpResponse response = httpClient.execute(new HttpGet(url));
final BasicResponseHandler basicResponseHandler = new BasicResponseHandler();
basicResponseHandler.handleResponse(response);
response.close();
}
private void httpUrlConnection(final String url) throws Exception {
final HttpURLConnection httpURLConnection = (HttpURLConnection) new URL(url).openConnection();
httpURLConnection.addRequestProperty("Accept-Encoding", "gzip");
try (final BufferedInputStream r = new BufferedInputStream(httpURLConnection.getInputStream())) {
final byte[] tmp = new byte[1024 * 32];
int read;
while (true) {
read = r.read(tmp);
if (read == -1) {
break;
}
}
}
}
private void jsoup(final String url) throws Exception {
Jsoup.connect(url).execute().parse();
}
}
你的基准没有意义 我为这三个库编写了一个微基准，结果是没有显著差异
Benchmark Mode Cnt Score Error Units
HttpBenchmark.httpClientGoogle avgt 2 151.162 ms/op
HttpBenchmark.httpClientStackoverflow avgt 2 151.086 ms/op
HttpBenchmark.httpUrlConnectionGoogle avgt 2 235.869 ms/op
HttpBenchmark.httpUrlConnectionStackoverflow avgt 2 145.162 ms/op
HttpBenchmark.jsoupGoogle avgt 2 391.162 ms/op
HttpBenchmark.jsoupStackoverflow avgt 2 188.059 ms/op
@Warmup(iterations = 1, time = 3, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 2, time = 5, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Threads(1)
public class HttpBenchmark {
private static final String GOOGLE = "https://google.com/";
private static final String STACKOVERFLOW = "https://stackoverflow.com";
private final CloseableHttpClient httpClient = HttpClientBuilder.create().build();
@Benchmark
public void httpClientGoogle() throws Exception {
httpClient(GOOGLE);
}
@Benchmark
public void httpClientStackoverflow() throws Exception {
httpClient(STACKOVERFLOW);
}
@Benchmark
public void httpUrlConnectionGoogle() throws Exception {
httpUrlConnection(GOOGLE);
}
@Benchmark
public void httpUrlConnectionStackoverflow() throws Exception {
httpUrlConnection(STACKOVERFLOW);
}
@Benchmark
public void jsoupGoogle() throws Exception {
jsoup(GOOGLE);
}
@Benchmark
public void jsoupStackoverflow() throws Exception {
jsoup(STACKOVERFLOW);
}
private void httpClient(final String url) throws Exception {
final CloseableHttpResponse response = httpClient.execute(new HttpGet(url));
final BasicResponseHandler basicResponseHandler = new BasicResponseHandler();
basicResponseHandler.handleResponse(response);
response.close();
}
private void httpUrlConnection(final String url) throws Exception {
final HttpURLConnection httpURLConnection = (HttpURLConnection) new URL(url).openConnection();
httpURLConnection.addRequestProperty("Accept-Encoding", "gzip");
try (final BufferedInputStream r = new BufferedInputStream(httpURLConnection.getInputStream())) {
final byte[] tmp = new byte[1024 * 32];
int read;
while (true) {
read = r.read(tmp);
if (read == -1) {
break;
}
}
}
}
private void jsoup(final String url) throws Exception {
Jsoup.connect(url).execute().parse();
}
}
没有什么代码的速度仅与网络和服务器响应以及任何中间缓存的速度相同。客户的选择不会影响这些事情。重新检查您的数据并检查您的假设。事实上，我分析了Jsoup的来源，它表明它使用了带有BufferedInputStream的HttpURLConnection，我尝试以HttpURLConnection的方式使用这两种连接，但正如您所看到的，结果相同，区别很明显，Jsoup显然比HttpURLConnection快，它使用HttpURLConnection！很抱歉，您的基准没有意义。Java JIT、GC、内存分配等方面的一些因素。。。不考虑。当您将顺序更改为Jsoup、HttpClient、HttpUrlConnection时，结果可能不同。尝试像JHM这样的微基准框架，并尝试更多尝试。代码的速度仅与网络和服务器响应以及任何中间缓存的速度相同。客户的选择不会影响这些事情。重新检查您的数据并检查您的假设。事实上，我分析了Jsoup的来源，它表明它使用了带有BufferedInputStream的HttpURLConnection，我尝试以HttpURLConnection的方式使用这两种连接，但正如您所看到的，结果相同，区别很明显，Jsoup显然比HttpURLConnection快，它使用HttpURLConnection！很抱歉，您的基准没有意义。Java JIT、GC、内存分配等方面的一些因素。。。不考虑。当您将顺序更改为Jsoup、HttpClient、HttpUrlConnection时，结果可能不同。尝试像JHM这样的微基准框架和更多的尝试。你能告诉我你用什么来执行这些测试吗？我想用itI做更多的测试我发现它是JMH，我会做更多的测试，感谢基准测试的想法太棒了，如果你能接受我的回答就太棒了即使通过测试更多的案例我发现相同的结果，Jsoup比HUC对我测试的大多数网站都快，我想这是由于Jsoup对HUC做了一些优化，例如，连接持久性原因在测试时，许多请求被发送到我在HUC测试中未使用的同一个网站，但不确定JSoup是否正在使用它。您能告诉我您用于执行这些测试的内容吗？我想用itI做更多的测试我发现它是JMH，我会做更多的测试，感谢基准测试的想法太棒了，如果你能接受我的回答就太棒了即使通过测试更多的案例我发现相同的结果，Jsoup比HUC对我测试的大多数网站都快，我想这是由于Jsoup对HUC做了一些优化，例如，连接持久化原因在测试时，许多请求被发送到我在HUC测试中没有使用的同一个网站，但不确定JSoup是否正在使用它
HttpUrlConnection : 112.57142857142857. Tries 7/7
HttpClient : 194.85714285714286. Tries 7/7
Jsoup : 67.42857142857143. Tries 7/7
HttpUrlConnection : 439.2857142857143. Tries 7/7
HttpClient : 283.42857142857144. Tries 7/7
Jsoup : 144.71428571428572. Tries 7/7
Benchmark Mode Cnt Score Error Units
HttpBenchmark.httpClientGoogle avgt 2 151.162 ms/op
HttpBenchmark.httpClientStackoverflow avgt 2 151.086 ms/op
HttpBenchmark.httpUrlConnectionGoogle avgt 2 235.869 ms/op
HttpBenchmark.httpUrlConnectionStackoverflow avgt 2 145.162 ms/op
HttpBenchmark.jsoupGoogle avgt 2 391.162 ms/op
HttpBenchmark.jsoupStackoverflow avgt 2 188.059 ms/op
@Warmup(iterations = 1, time = 3, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 2, time = 5, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Threads(1)
public class HttpBenchmark {
private static final String GOOGLE = "https://google.com/";
private static final String STACKOVERFLOW = "https://stackoverflow.com";
private final CloseableHttpClient httpClient = HttpClientBuilder.create().build();
@Benchmark
public void httpClientGoogle() throws Exception {
httpClient(GOOGLE);
}
@Benchmark
public void httpClientStackoverflow() throws Exception {
httpClient(STACKOVERFLOW);
}
@Benchmark
public void httpUrlConnectionGoogle() throws Exception {
httpUrlConnection(GOOGLE);
}
@Benchmark
public void httpUrlConnectionStackoverflow() throws Exception {
httpUrlConnection(STACKOVERFLOW);
}
@Benchmark
public void jsoupGoogle() throws Exception {
jsoup(GOOGLE);
}
@Benchmark
public void jsoupStackoverflow() throws Exception {
jsoup(STACKOVERFLOW);
}
private void httpClient(final String url) throws Exception {
final CloseableHttpResponse response = httpClient.execute(new HttpGet(url));
final BasicResponseHandler basicResponseHandler = new BasicResponseHandler();
basicResponseHandler.handleResponse(response);
response.close();
}
private void httpUrlConnection(final String url) throws Exception {
final HttpURLConnection httpURLConnection = (HttpURLConnection) new URL(url).openConnection();
httpURLConnection.addRequestProperty("Accept-Encoding", "gzip");
try (final BufferedInputStream r = new BufferedInputStream(httpURLConnection.getInputStream())) {
final byte[] tmp = new byte[1024 * 32];
int read;
while (true) {
read = r.read(tmp);
if (read == -1) {
break;
}
}
}
}
private void jsoup(final String url) throws Exception {
Jsoup.connect(url).execute().parse();
}
}