C# 如何将cookie传递给HtmlAgilityPack或WebClient?
我使用此代码登录:C# 如何将cookie传递给HtmlAgilityPack或WebClient?,c#,cookies,httpwebrequest,webclient,html-agility-pack,C#,Cookies,Httpwebrequest,Webclient,Html Agility Pack,我使用此代码登录: CookieCollection cookies = new CookieCollection(); HttpWebRequest request = (HttpWebRequest)WebRequest.Create("example.com"); request.CookieContainer = new CookieContainer(); request.CookieContainer.Add(cookies); HttpWebResponse response =
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("example.com");
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string getUrl = "example.com";
string postData = String.Format("my parameters");
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies);
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream();
newStream.Write(byteArray, 0, byteArray.Length);
newStream.Close();
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream(), Encoding.GetEncoding("windows-1251")))
{
doc.LoadHtml(sr.ReadToEnd());
webBrowser1.DocumentText = doc.DocumentNode.OuterHtml;
}
然后我想使用HtmlWeb(HtmlAgilityPack)或Webclient将HTML解析为HtmlDocument(HtmlAgilityPack)
我的问题是,当我使用:
WebClient wc = new WebClient();
webBrowser1.DocumentText = wc.DownloadString(site);
或
登录消失了,所以我想我必须以某种方式传递cookies。。有什么建议吗?这里有一些建议: 但是,继续使用
HttpWebRequest
并在CookieContainer
中设置cookie可能更容易:
// Create a HttpWebRequest
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(getUrl);
// Create the cookie container and add a cookie
request.CookieContainer = new CookieContainer();
// Add all the cookies
foreach (Cookie cookie in response.Cookies)
{
request.CookieContainer.Add(cookie);
}
第二件事是,您不需要再次下载该网站,因为您已经从web响应中获得了该网站,并将其保存在此处:
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream(), Encoding.GetEncoding("windows-1251")))
{
webBrowser1.DocumentText = doc.DocumentNode.OuterHtml;
}
您应该能够获取HTML并使用HTML Agility Pack解析它:
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(webBrowser1.DocumentText);
这样就可以了……:) 检查
下面是一个您正在寻找的示例(语法没有100%测试,我只是修改了一些我通常使用的类):
以下是您如何使用它:
var client = new MyWebClient();
HtmlDocument doc = client.GetPage("http://somepage.com");
//This request will be sent with the cookies obtained from the page
doc = client.GetPage("http://somepage.com/another-page");
注意:如果您还想使用
POST
方法,只需使用POST
逻辑创建一个类似于GetPage
的方法,重构类,等等。尝试在本地缓存先前响应中的cookie,并按如下方式重新发送每个web请求:
private CookieCollection cookieCollection;
...
parserObject = new HtmlWeb
{
AutoDetectEncoding = true,
PreRequest = request =>
{
if (cookieCollection != null)
cookieCollection.Cast<Cookie>()
.ForEach(cookie => request.CookieContainer.Add(cookie));
return true;
},
PostResponse = (request, response) => { cookieCollection = response.Cookies; }
};
私有CookieCollection CookieCollection;
...
parserObject=新的HtmlWeb
{
自动检测编码=真,
预请求=请求=>
{
if(cookieCollection!=null)
cookieCollection.Cast()
.ForEach(cookie=>request.CookieContainer.Add(cookie));
返回true;
},
PostResponse=(请求,响应)=>{cookieCollection=response.Cookies;}
};
@ShahroozJefri这不是一个应答检查,我登录到该网站,但我想在该网站的其他地方导航。事实上,我在这个网站上做了一个搜索。你必须在每次请求中提供cookies。如果您没有为每个请求提供cookie,那么它将假定您已注销(大多数登录信息都包含在cookie中)代码>函数,你能帮我制作getHTML(url)代码>函数导致上述代码不完整。@Loclip OK,login()
不是C#函数,也不是getHTML
,所以在这方面我帮不了你多少忙。我向您展示的代码旨在帮助您了解如何在HttpWebRequest
中包含cookie(这应该是您从页面请求HTML内容所需的全部内容)。那么,根据你在问题中展示的代码和我在回答中展示的代码,问题到底在哪里?你必须向我提供一些有意义的信息,仅仅说“我试图完成你的代码,但它仍然要求我登录”并不能告诉我太多。
var client = new MyWebClient();
HtmlDocument doc = client.GetPage("http://somepage.com");
//This request will be sent with the cookies obtained from the page
doc = client.GetPage("http://somepage.com/another-page");
private CookieCollection cookieCollection;
...
parserObject = new HtmlWeb
{
AutoDetectEncoding = true,
PreRequest = request =>
{
if (cookieCollection != null)
cookieCollection.Cast<Cookie>()
.ForEach(cookie => request.CookieContainer.Add(cookie));
return true;
},
PostResponse = (request, response) => { cookieCollection = response.Cookies; }
};