Java 在Htmlunit中处理AJAX请求

Java 在Htmlunit中处理AJAX请求,java,ajax,htmlunit,Java,Ajax,Htmlunit,我有一个程序,用来在点击按钮后从网页上刮取源代码。我无法抓取正确的页面,因为我相信正在发送一个AJAX请求,我不会等待响应发生。我的代码当前为: public class Htmlunitscraper { private static String s = "http://cpdocket.cp.cuyahogacounty.us/SheriffSearch/results.aspx?q=searchType%3dSaleDate%26searchString%3d10%2f21%2f

我有一个程序,用来在点击按钮后从网页上刮取源代码。我无法抓取正确的页面,因为我相信正在发送一个AJAX请求,我不会等待响应发生。我的代码当前为:

public class Htmlunitscraper { 

  private static String s = "http://cpdocket.cp.cuyahogacounty.us/SheriffSearch/results.aspx?q=searchType%3dSaleDate%26searchString%3d10%2f21%2f2013%26foreclosureType%3d%27NONT%27%2c+%27PAR%27%2c+%27COMM%27%2c+%27TXLN%27";

  public static String scrapeWebsite() throws IOException {

    java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF); 
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");

    final WebClient webClient = new WebClient();
    final HtmlPage page = webClient.getPage(s);
    final HtmlForm form = page.getForms().get(2);
    final HtmlSubmitInput button = form.getInputByValue(">");
    final HtmlPage page2 = button.click();
    String originalHtml = page2.refresh().getWebResponse().getContentAsString();
    return originalHtml;
  }
}

在提到这一点之后,我相信要解决这个问题,我可以实现方法“webClient.waitForBackgroundJavaScript(10000)”。唯一的问题是我不明白如何做到这一点,因为每次单击按钮时,我都会创建一个HtmlPage对象,而不是WebClient对象。我如何结合此方法来解决问题?

我将尝试设置的解决方案

webClient.setAjaxController(new NicelyResynchronizingAjaxController());
这将导致所有ajax调用都是同步的

或者,您是否尝试在您的解决方案中在您获得页面后调用“webClient.waitForBackgroundJavaScript(10000)”呢

大概是这样的:

final HtmlPage page2 = button.click();
webClient.waitForBackgroundJavaScript(10000)
String originalHtml = page2.asXml();
return originalHtml;

请同时使用htmlunit 2.13

对我来说,将htmlunit 2.15与NicelyResynchronizingAjaxController配合使用,会有所帮助,而且

webClient.getOptions().setThrowExceptionOnScriptError(false);
我的完整设置是

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setCssEnabled(false);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());