Java HtmlUnit xpath getElement
我正在尝试创建一个特定网站的网页内容爬虫 很快,我的XPath无法找到更改页面的链接(使用javascript),这会导致NullPointExecution。我尝试过以各种方式修改XPath,但没有任何效果 另外,在脚本运行之后,我是否需要运行任何方法来获取新页面Java HtmlUnit xpath getElement,java,html,xpath,htmlunit,Java,Html,Xpath,Htmlunit,我正在尝试创建一个特定网站的网页内容爬虫 很快,我的XPath无法找到更改页面的链接(使用javascript),这会导致NullPointExecution。我尝试过以各种方式修改XPath,但没有任何效果 另外,在脚本运行之后,我是否需要运行任何方法来获取新页面 > package gimasys.webService; import java.io.IOException; import java.net.MalformedURLException; import com.gar
> package gimasys.webService;
import java.io.IOException;
import java.net.MalformedURLException;
import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.ThreadedRefreshHandler;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlAnchor;
import com.gargoylesoftware.htmlunit.html.HtmlButton;
import com.gargoylesoftware.htmlunit.html.HtmlLink;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class Crawlv1000 {
/**
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
final WebCrawler wc = new WebCrawler();
final PageCrawler pc = new PageCrawler();
final WebClient webClient = new WebClient(BrowserVersion.CHROME_16);
webClient.setRefreshHandler(new ThreadedRefreshHandler()); // This is to allow handling the page operation using threads else an exception will pop up
try {
HtmlPage page = webClient.getPage("http://v1000.vn/bang-xep-hang?ref=bang-xep-hang-1000-doanh-nghiep-dong-thue-thu-nhap-nhieu-nhat-2012");
HtmlAnchor link = page.getFirstByXPath("//a[@href='javascript:loadRankingTable(3)']");
link.click();
System.out.println(page.getTextContent());
} catch (FailingHttpStatusCodeException | IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
/*
wc.crawl("http://v1000.vn/bang-xep-hang?ref=bang-xep-hang-1000-doanh-nghiep-dong-thue-thu-nhap-nhieu-nhat-2012");
for (String url:wc.urlList)
{
pc.crawl(url);
}
*/
}
}
谢谢,
明阮你犯的很小的错误,亲爱的,分号错误
HtmlAnchor link = page.getFirstByXPath("//a[@href='javascript:loadRankingTable(3);']");
link.click();