Java 尝试连接网页时出现HtmlUnit错误

我今天刚刚听说了这个

HtmlUnit

的事情，当时我正试图找出如何转储网站的源代码。我试图做的是使用一个脚本从一个网站上转储源代码，但是当我运行它时，会得到一个很长的红色列表

这是我使用的代码：

public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

    String url = "http://www.runelocus.com/forums/member.php?102785&tab=aboutme#aboutme";
    WebClient client = new WebClient(BrowserVersion.FIREFOX_3_6);
    HtmlPage page = client.getPage(url);
    System.out.println(page.getWebResponse().getContentAsString());
}

这是我得到的错误：

Exception in thread "main" org.apache.http.client.ClientProtocolException
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:822)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:776)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:152)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1439)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1358)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:307)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:358)
at HTMLDumping.htmlunittest.main(htmlunittest.java:18)
Caused by: org.apache.http.ProtocolException: Invalid header: blcc_proxy
at org.apache.http.impl.io.AbstractMessageParser.parseHeaders(AbstractMessageParser.java:224)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)

---

如果有人知道如何解决这个问题，请提出建议。接受所有反馈。

使用：System.out.println（page.asText（））

您是打算使用

HTMLUnit

进行测试，还是试图

scrape

网站？如果你只是想

scrape

网站，那么这是一个更好的选择。