Java 如何在提交从数据库中提供数据的搜索表单后解析Html
数据来自一个数据库，所以如何用这些数据解析HTML，在我的代码中，我用cookies直接连接到我的目标页面，然后连接到设置搜索表单，那么如何用搜索结果进行解析。它来自后端。下面的代码首先包含我的服务连接代码，然后是搜索表单响应给出的HTML，最后是在提交API post方法的搜索表单后返回的浏览器网络中显示的HTMLJava 如何在提交从数据库中提供数据的搜索表单后解析Html,java,spring-boot,web-scraping,web-crawler,jsoup,Java,Spring Boot,Web Scraping,Web Crawler,Jsoup,数据来自一个数据库，所以如何用这些数据解析HTML，在我的代码中，我用cookies直接连接到我的目标页面，然后连接到设置搜索表单，那么如何用搜索结果进行解析。它来自后端。下面的代码首先包含我的服务连接代码，然后是搜索表单响应给出的HTML，最后是在提交API post方法的搜索表单后返回的浏览器网络中显示的HTML Connection.Response redirectResponse = Jsoup.connect(internalConstant.getSearc
Connection.Response redirectResponse = Jsoup.connect(internalConstant.getSearchResultURL())
.userAgent(USER_AGENT)
.sslSocketFactory(utilService.socketFactory())
.cookies(coky)
.method(Method.GET)
.execute();
//search file
String yestarday = utilService.getYesterdayDateString();
logger.info("yestarday date: " + yestarday);
Document responseDocument = redirectResponse.parse();
FormElement searchForm = (FormElement) responseDocument.select("form[id=searchDisputesForm]").first();
checkElement("form element", searchForm);
FormElement form = (FormElement) searchForm;
Element searchField = form.select("input[name=DateFrom]").first();
checkElement("date from: ", searchField);
searchField.val(yestarday);
Element searchField1 = form.select("input[name=DateTo]").first();
checkElement("Date to: ", searchField1);
searchField1.val(yestarday);
Connection.Response searchResponse = form.submit()
.cookies(coky)
.sslSocketFactory(utilService.socketFactory())
.userAgent(USER_AGENT)
.method(Method.POST)
.ignoreHttpErrors(true)
.execute();
//and here is what gives me:
2020-10-15 11:37:38.193 INFO 3360 --- [http-nio-8095-exec-1] c.d.e.serviceImpl.UploaderTemplateImpl : searchResponse :<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>404 - File or directory not found.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;}
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header">
<h1>Server Error</h1>
</div>
<div id="content">
<div class="content-container">
<fieldset>
<h2>404 - File or directory not found.</h2>
<h3>The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.</h3>
</fieldset>
</div>
</div>
</body>
</html>
//but the response in the network shows what I want:
<form action="/123Mobile/Portal/Dispute/SearchDisputes" class="form-horizontal" enctype="multipart/form-data" id="searchDisputesForm" method="post"><input name="__RequestVerificationToken" type="hidden" value="ClhG-V26p7Aj3vi6W8tarCwHk_enagGT4mDVv2AZsU4MHATOBTbEQHuFmylooB5qvxF29aF1-wSvZZ26ijlcZJn5p_OSshL3KeqcEbAaYg01" />
<div class='col-md-12'>
<div class='panel panel-grey equal-height-column' id=''>
<div class='panel-heading'>
<h3 class='panel-title'>
<i class='fa fa-tasks'></i>
Search criteria
</h3>
</div>
<div class='panel-body'>
<div class='form-group'> <label class="col-md-2 control-label" for="DateFrom">Date From</label>
<div class='col-md-4 col-md-offset-0' id=''>
<input class="form-control datepicker" data-val="true" data-val-date="The field Date From must be a date." id="DateFrom" name="DateFrom" placeholder="" type="text" value="2020-10-14" />
<span class="field-validation-valid text-danger" data-valmsg-for="DateFrom" data-valmsg-replace="true"></span>
</div><label class="col-md-2 control-label" for="DateTo">Date To</label>
<div class='col-md-4 col-md-offset-0' id=''>
<input class="form-control datepicker" data-val="true" data-val-date="The field Date To must be a date." id="DateTo" name="DateTo" placeholder="" type="text" value="2020-10-14" />
<span class="field-validation-valid text-danger" data-valmsg-for="DateTo" data-valmsg-replace="true"></span>
</div></div>
<div class='col-md-12'>
<div class=' col-md-6'>
</div>
<button type='button' class='btn btn-success col-md-2 col-md-offset-1 ' id='ResetBtn' name='ResetBtn' onclick=''>
<span class='glyphicon glyphicon-refresh' aria-hidden='true'></span> Reset
</button>
<button type='submit' class='btn btn-primary col-md-2 col-md-offset-1 ' id='SearchBtn' name='SearchBtn' formaction='/123Mobile/Portal/Reports/SearchTransactionReport' onclick=''>
<span class='glyphicon glyphicon-search' aria-hidden='true'></span> Search
</button></div>
</div>
</div>
</div>
<div class='col-md-12'>
<div class='panel panel-grey equal-height-column' id=''>
<div class='panel-heading'>
<h3 class='panel-title'>
<i class='fa fa-tasks'></i>
Search results
</h3>
</div>
<div class='panel-body'>
<div style='' class='table-responsive'><table class='table table-hover table-striped'><thead><tr><th style='width:15%;'>File Name</th><th style='width:10%;'>Scheme</th><th style='width:15%;'>Proccessing Date</th><th style='width:8%;'></th><th style='width:8%;'></th></tr> </thead><tbody><tr><td style='-moz-word-break: break-all; -o-word-break: break-all; word-break: break-word;-ms-word-break: break-all;overflow-wrap: break-word;word-wrap: break-word; width: 15%;' class=''>Transaction-FIBWallet-20201014</td><td style='-moz-word-break: break-all; -o-word-break: break-all; word-break: break-word;-ms-word-break: break-all;overflow-wrap: break-word;word-wrap: break-word; width: 10%;' class=''>FIBWallet</td><td style='-moz-word-break: break-all; -o-word-break: break-all; word-break: break-word;-ms-word-break: break-all;overflow-wrap: break-word;word-wrap: break-word; width: 15%;' class=''>2020-10-14</td><td style='width: 8%;' class=''><a href="/123Mobile/Portal/Reports/DownloadTransactionReport/40072">Download as Text </a></td><td style='width: 8%;' class=''><a href="/123Mobile/Portal/Reports/DownloadTransactionReport/40073">Download as Excel </a></td></tr></tbody></table></div>
</div>
</div>
</div>
<div class='col-md-12'> <div class="pagination-container"><ul class="pagination"><li class="disabled PagedList-skipToFirst"><a>««</a></li><li class="disabled PagedList-skipToPrevious"><a rel="prev">«</a></li><li class="active"><a>1</a></li><li class="disabled PagedList-skipToNext"><a rel="next">»</a></li><li class="disabled PagedList-skipToLast"><a>»»</a></li></ul></div></div>
</form>
Connection.Response redirectResponse=Jsoup.connect（internalConstant.getSearchResultURL（））
.userAgent（USER\u AGENT）
.sslSocketFactory（utilService.socketFactory（））
.饼干（可口的）
.method（method.GET）
.execute（）；
//搜索文件
String yestarday=utilService.getYesterdayDateString（）；
logger.info（“yestarday日期：+yestarday”）；
Document responseDocument=redirectResponse.parse（）；
FormElement searchForm=（FormElement）responseDocument.select（“form[id=searchDisputesForm]”）。first（）；
checkElement（“表单元素”，搜索表单）；
FormElement form=（FormElement）searchForm；
元素searchField=form.select（“输入[name=DateFrom]”）。first（）；
checkElement（“日期开始：”，搜索字段）；
searchField.val（yestarday）；
元素searchField1=form.select（“输入[name=DateTo]”）。first（）；
checkElement（“日期：”，searchField1）；
searchField1.val（yestarday）；
Connection.Response searchResponse=form.submit（）
.饼干（可口的）
.sslSocketFactory（utilService.socketFactory（））
.userAgent（USER\u AGENT）
.method（method.POST）
.ignoreHttpErrors（真）
.execute（）；
//以下是我的感受：
2020-10-15 11:37:38.193信息3360---[http-nio-8095-exec-1]c.d.e.serviceImpl.UploaderTemplateImpl:searchResponse:
404-找不到文件或目录。
服务器错误
404-找不到文件或目录。
您正在查找的资源可能已被删除、名称已更改或暂时不可用。
//但网络中的响应显示了我想要的：
搜索条件
从
日期
重置
搜寻
搜索结果
文件名ChemeProcessing Date Transaction-FIBWallet-20201014FIBWallet2020-10-14
- ««««
-
404-找不到文件或目录。
感兴趣的典型标头涉及某种会话cookie，可能是一个referer标头。但是，如果这两个单独无法满足您的需求，您需要开始准确地复制请求。。。您需要获取的形式中可能有CSRF令牌（在这种情况下，初始GET请求对于获取有效令牌以便在POST中发回至关重要）。那么，问题出在哪里？您正在使用Jsoup进行解析，发生了什么？你能帮我吗