C# 多线程创建FTP请求的线程池超时
我正在尝试创建一个FTP web请求集合以下载一个文件集合 在一个线程中正常工作,但我现在尝试使用多个线程,但遇到超时异常。我想我错过了一些很简单但似乎无法解决的事情 以下是代码:C# 多线程创建FTP请求的线程池超时,c#,asp.net,multithreading,threadpool,ftpwebrequest,C#,Asp.net,Multithreading,Threadpool,Ftpwebrequest,我正在尝试创建一个FTP web请求集合以下载一个文件集合 在一个线程中正常工作,但我现在尝试使用多个线程,但遇到超时异常。我想我错过了一些很简单但似乎无法解决的事情 以下是代码: internal static void DownloadLogFiles(IEnumerable<string> ftpFileNames, string localLogsFolder) { BotFinder.DeleteAllFilesFromDirectory(localLogsFold
internal static void DownloadLogFiles(IEnumerable<string> ftpFileNames, string localLogsFolder)
{
BotFinder.DeleteAllFilesFromDirectory(localLogsFolder);
var ftpWebRequests = new Collection<FtpWebRequest>();
// Create web request for each log filename
foreach (var ftpWebRequest in ftpFileNames.Select(filename => (FtpWebRequest) WebRequest.Create(filename)))
{
ftpWebRequest.Credentials = new NetworkCredential(BotFinderSettings.FtpUserId, BotFinderSettings.FtpPassword);
ftpWebRequest.KeepAlive = false;
ftpWebRequest.UseBinary = true;
ftpWebRequest.CachePolicy = NoCachePolicy;
ftpWebRequest.Method = WebRequestMethods.Ftp.DownloadFile;
ftpWebRequests.Add(ftpWebRequest);
}
var threadDoneEvents = new ManualResetEvent[ftpWebRequests.Count];
for (var x = 0; x < ftpWebRequests.Count; x++)
{
var ftpWebRequest = ftpWebRequests[x];
threadDoneEvents[x] = new ManualResetEvent(false);
var threadedFtpDownloader = new ThreadedFtpDownloader(ftpWebRequest, threadDoneEvents[x]);
ThreadPool.QueueUserWorkItem(threadedFtpDownloader.PerformFtpRequest, localLogsFolder);
}
WaitHandle.WaitAll(threadDoneEvents);
}
class ThreadedFtpDownloader
{
private ManualResetEvent threadDoneEvent;
private readonly FtpWebRequest ftpWebRequest;
/// <summary>
///
/// </summary>
public ThreadedFtpDownloader(FtpWebRequest ftpWebRequest, ManualResetEvent threadDoneEvent)
{
this.threadDoneEvent = threadDoneEvent;
this.ftpWebRequest = ftpWebRequest;
}
/// <summary>
///
/// </summary>
/// <param name="localLogsFolder">
///
/// </param>
internal void PerformFtpRequest(object localLogsFolder)
{
try
{
// TIMEOUT IS HAPPENING ON LINE BELOW
using (var response = ftpWebRequest.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
const int length = 1024*10;
var buffer = new Byte[length];
var bytesRead = responseStream.Read(buffer, 0, length);
var logFileToCreate = string.Format("{0}{1}{2}", localLogsFolder,
ftpWebRequest.RequestUri.Segments[3].Replace("/", "-"),
ftpWebRequest.RequestUri.Segments[4]);
using (var writeStream = new FileStream(logFileToCreate, FileMode.OpenOrCreate))
{
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer, 0, length);
}
}
}
}
threadDoneEvent.Set();
}
catch (Exception exception)
{
BotFinder.HandleExceptionAndExit(exception);
}
}
}
它为每个FTP web请求进入PerformFtpRequest方法,并调用ftpWebRequest.GetResponse(),但随后仅对前两个请求进行进一步处理。其余的请求保持活动状态,但在前两个请求完成之前,不会再继续。因此,这基本上意味着它们在开始之前等待其他请求完成时保持打开状态
我认为这个问题的解决方案要么是允许所有请求一次执行(ConnectionLimit属性在这里不起作用),要么是阻止执行调用GetResponse,直到它真正准备好使用响应为止
有什么好主意可以解决这个问题吗?目前,我所能想到的似乎都是我想要避免的黑客解决方案:)
谢谢 您应该获取请求的ServicePoint并设置ConnectionLimit
ServicePoint sp = ftpRequest.ServicePoint;
sp.ConnectionLimit = 10;
默认的ConnectionLimit为2——这就是为什么您会看到这种行为
更新:有关更详细的说明,请参阅此答案:
谢谢你,伙计。我尝试在foreach循环中添加代码,该循环正在创建FtpWebRequests,但遇到了相同的问题。这就是我应该设置连接限制的地方吗?添加了一个链接,指向包含更多详细信息的问题/答案。ServicePoint是一种直到你撞了几次头才知道的东西。
ServicePoint sp = ftpRequest.ServicePoint;
sp.ConnectionLimit = 10;