C# 并行for循环和httpclient死锁并引发异常

C# 并行for循环和httpclient死锁并引发异常,c#,asynchronous,task-parallel-library,dotnet-httpclient,parallel.foreach,C#,Asynchronous,Task Parallel Library,Dotnet Httpclient,Parallel.foreach,我不明白为什么我的parallels.for循环在一个httpclient调用中不断爆炸。 该代码适用于大约10-15个请求,然后将挂起很长时间,并在System.AggregateException上出错 我尝试了多种不同的变体,包括webclient。 请考虑以下事项: class Program { static void Main(string[] args) { Parallel.For(0, 250, i => {

我不明白为什么我的parallels.for循环在一个httpclient调用中不断爆炸。 该代码适用于大约10-15个请求,然后将挂起很长时间,并在System.AggregateException上出错 我尝试了多种不同的变体,包括webclient。 请考虑以下事项:

class Program
{
    static void Main(string[] args)
    {
        Parallel.For(0, 250, i =>
        {

            var task = GetPages.CallHttp();
            task.Wait();
            var content = task.Result;
            TestObject obj = content.ToTestObject(); //take the string and find some values in it. 
            Console.WriteLine(obj.H1Tag);
        });
    }
}

public static class GetPages
{
    private static readonly HttpClient client = new HttpClient() {Timeout = new TimeSpan(0,5,0)};

   public static async Task<string> CallHttp()
    {

        client.DefaultRequestHeaders.UserAgent.ParseAdd("customAgent/1.0");
        string astr = await client.GetStringAsync("the url I am testing").ConfigureAwait(false);
        return astr;
    }
}

public static class StringExtensions
{
    private static readonly object objLock = new object();
    public static TestObject ToTestObject(this string content)
    {
        lock (objLock)
        {
            var obj = new TestObject();
// creates a bunch of properties inspecting the html string
            var result = new HtmlExtractions(content); 
            obj.PageTitle = result.PageTitle;
            obj.H1Tag = result.H1Tag;
           ...
            return obj;
        }
    }

}


public class HtmlExtractions
{

    internal HtmlDocument doc;

    public HtmlExtractions(string contentToRead)
    {
        doc = new HtmlDocument();
        doc.LoadHtml(contentToRead);
    }

    public string PageTitle => doc.DocumentNode.Descendants("title").FirstOrDefault()?.InnerHtml.Replace("&amp;", "&").Trim();
...
}
异步任务(释放线程)和
并行任务。For
(强制使用多个线程)往往不能很好地混合使用
.Wait()
.Result
在用于异步任务时都是阻塞调用和邀请死锁。尝试使用
Task重新编写
Main
方法。当所有
并避免阻塞调用时:

var tasks = Enumerable.Range(0, 250).Select(i => GetPages.CallHttp());
var contents = await Task.WhenAll(tasks);
foreach (var content in contents)
{
    TestObject obj = content.ToTestObject(); //take the string and find some values in it. 
    Console.WriteLine(obj.H1Tag);
}

查看
System.aggregateeexception
InnerExceptions
属性。此外,不需要使用
lock
语句。据我所知,没有并发读写操作。内部异常在文章的底部。我应该说得更清楚些。一项任务被取消。InnerException:Id=50,Status=cancelled,Method=“{null}”,Result=“{Not Not computed}”关于锁,我同意,但此时正在黑暗中射箭。@PeterBons知道可能是什么问题吗?我在没有锁的情况下尝试了相同的代码,但遇到了相同的问题执行250次,在我的测试中,这会生成一个异常。您应该重构它,使它只执行一次,而不是
CallHttp
的一部分。我想这个集合不是线程安全的。没有任何类型的真正运气。在底部更新了有问题的代码。谢谢你的建议。首先创建集合,然后进行迭代是有意义的。
    static async void RunPagesAsync()
    {
        Console.WriteLine("getting contents");
        var tasks = Enumerable.Range(0, 5).Select(i => GetPages.CallHttp());
        var contents = await Task.WhenAll(tasks);

        Console.WriteLine("Got Contents..continuing");
        foreach (var content in contents)
        {
            TestObject obj = content.ToTestObject(); //take the string and find some values in it. 
            Console.WriteLine(obj.H1Tag);
        }
        Console.WriteLine("completed");
    }

    static void Main(string[] args)
    {

      //Task.Run(() => RunPagesAsync()); //doesn't work. 
      //  RunPagesAsync(); //just hangs after what looks like 2 itterations

      //  var tasks = Enumerable.Range(0, 5).Select(i => GetPages.CallHttp());
      //  var contents = await Task.WhenAll(tasks); //won't compile under synch Main dues to await 
      //  foreach (var content in contents)
      //  {
      //      TestObject obj = content.ToTestObject(); //take the string and find some values in it. 
      //      Console.WriteLine(obj.H1Tag);
      //  }

        var tasks1 = Enumerable.Range(0, 5).Select(i => GetPages.CallHttp());
        var contents1 =  Task.WhenAll(tasks1); 
        contents1.Wait();
        foreach (var content in contents1.Result)
        {
            TestObject obj = content.ToTestObject(); //take the string and find some values in it. 
            Console.WriteLine(obj.H1Tag);
        }
var tasks = Enumerable.Range(0, 250).Select(i => GetPages.CallHttp());
var contents = await Task.WhenAll(tasks);
foreach (var content in contents)
{
    TestObject obj = content.ToTestObject(); //take the string and find some values in it. 
    Console.WriteLine(obj.H1Tag);
}