C# 从.Net代码模拟web浏览器引擎

C# 从.Net代码模拟web浏览器引擎,c#,.net,browser,simulate,C#,.net,Browser,Simulate,我希望能够从我的代码中模拟一个浏览器引擎,从特定链接检索网页,并解析数据,问题是该页面包含一些Ajax调用,在任何脚本运行之前,我只能得到源代码 有没有一种简单的方法可以在测试结束后从代码中获取最终页面 我目前正在使用以下代码: //create the constructor with post type and few data MyWebRequest myRequest = new MyWebRequest("https://www.kayak.com/flights/LHR-AMS/2

我希望能够从我的代码中模拟一个浏览器引擎,从特定链接检索网页,并解析数据,问题是该页面包含一些Ajax调用,在任何脚本运行之前,我只能得到源代码

有没有一种简单的方法可以在测试结束后从代码中获取最终页面

我目前正在使用以下代码:

//create the constructor with post type and few data
MyWebRequest myRequest = new MyWebRequest("https://www.kayak.com/flights/LHR-AMS/2014-09-10/2014-10-15");
//show the response string on the console screen.
Response.Write(myRequest.GetResponse());



public class MyWebRequest
{
    private WebRequest request;
    private Stream dataStream;
    private string status;

    public String Status
    {
        get
        {
            return status;
        }
        set
        {
            status = value;
        }
    }

    public MyWebRequest(string url)
    {
        // Create a request using a URL that can receive a post.

        request = WebRequest.Create(url);
    }

    public MyWebRequest(string url, string method)
        : this(url)
    {

        if (method.Equals("GET") || method.Equals("POST"))
        {
            // Set the Method property of the request to POST.
            request.Method = method;
        }
        else
        {
            throw new Exception("Invalid Method Type");
        }
    }

    public MyWebRequest(string url, string method, string data)
        : this(url, method)
    {

        // Create POST data and convert it to a byte array.
        string postData = data;
        byte[] byteArray = Encoding.UTF8.GetBytes(postData);

        // Set the ContentType property of the WebRequest.
        request.ContentType = "application/x-www-form-urlencoded";

        // Set the ContentLength property of the WebRequest.
        request.ContentLength = byteArray.Length;

        // Get the request stream.
        dataStream = request.GetRequestStream();

        // Write the data to the request stream.
        dataStream.Write(byteArray, 0, byteArray.Length);

        // Close the Stream object.
        dataStream.Close();

    }

    public string GetResponse()
    {
        // Get the original response.
        WebResponse response = request.GetResponse();

        this.Status = ((HttpWebResponse)response).StatusDescription;

        // Get the stream containing all content returned by the requested server.
        dataStream = response.GetResponseStream();

        // Open the stream using a StreamReader for easy access.
        StreamReader reader = new StreamReader(dataStream);

        // Read the content fully up to the end.
        string responseFromServer = reader.ReadToEnd();

        // Clean up the streams.
        reader.Close();
        dataStream.Close();
        response.Close();

        return responseFromServer;
    }

问题是您需要自己执行JS代码。在你的位置上,我会稍微修改一下,使用WebBrowser控件,用IE引擎打开页面,甚至执行JS。过一段时间你就可以获取最终的HTML代码了。谢谢,我也考虑过朝这个方向走,我会试试并发布结果。