Java Apache Commons FTPClient挂起

Java Apache Commons FTPClient挂起,java,ftp,ftp-client,apache-commons-net,Java,Ftp,Ftp Client,Apache Commons Net,我们使用以下Apache Commons Net FTP代码连接到FTP服务器,轮询某些目录中的文件,如果找到文件,则将其检索到本地计算机: try { logger.trace("Attempting to connect to server..."); // Connect to server FTPClient ftpClient = new FTPClient(); ftpClient.setConnectTimeout(20000); ftpClient.connect("my-se

我们使用以下Apache Commons Net FTP代码连接到FTP服务器,轮询某些目录中的文件,如果找到文件,则将其检索到本地计算机:

try {
logger.trace("Attempting to connect to server...");

// Connect to server
FTPClient ftpClient = new FTPClient();
ftpClient.setConnectTimeout(20000);
ftpClient.connect("my-server-host-name");
ftpClient.login("myUser", "myPswd");
ftpClient.changeWorkingDirectory("/loadables/");

// Check for failed connection
if(!FTPReply.isPositiveCompletion(ftpClient.getReplyCode()))
{
    ftpClient.disconnect();
    throw new FTPConnectionClosedException("Unable to connect to FTP server.");
}

// Log success msg
logger.trace("...connection was successful.");

// Change to the loadables/ directory where we poll for files
ftpClient.changeWorkingDirectory("/loadables/");    

// Indicate we're about to poll
logger.trace("About to check loadables/ for files...");

// Poll for files.
FTPFile[] filesList = oFTP.listFiles();
for(FTPFile tmpFile : filesList)
{
    if(tmpFile.isDirectory())
        continue;

    FileOutputStream fileOut = new FileOutputStream(new File("tmp"));
    ftpClient.retrieveFile(tmpFile.getName(), fileOut);
    // ... Doing a bunch of things with output stream
    // to copy the contents of the file down to the local
    // machine. Ommitted for brevity but I assure you this
    // works (except when the WAR decides to hang).
    //
    // This was used because FTPClient doesn't appear to GET
    // whole copies of the files, only FTPFiles which seem like
    // file metadata...
}

// Indicate file fetch completed.
logger.trace("File fetch completed.");

// Disconnect and finish.
if(ftpClient.isConnected())
    ftpClient.disconnect();

logger.trace("Poll completed.");
} catch(Throwable t) {
    logger.trace("Error: " + t.getMessage());
}
我们已经安排好每分钟,每分钟运行一次。当部署到Tomcat(7.0.19)时,这段代码可以很好地加载,并且可以顺利地开始工作。然而,每一次,在某个时刻,它似乎只是挂起。我的意思是:

  • 不存在堆转储
  • Tomcat仍在运行(我可以看到它的pid并可以登录到web manager应用程序)
  • 在manager应用程序中,我可以看到我的WAR仍在运行/启动
  • catalina.out
    和我的特定于应用程序的日志没有显示任何异常被抛出的迹象
因此JVM仍然在运行。Tomcat仍在运行,我部署的战争仍在运行,但它只是挂起。有时它运行2小时,然后挂起;其他时候它会运行几天,然后挂起。但是当它挂起时,它会在读取
即将检查可加载项/文件的行…
(我在日志中看到)和读取
文件获取完成的行之间挂起。
(我没有看到)

这告诉我挂起发生在实际轮询/获取文件的过程中,这种挂起将我指向与FTPClient死锁相关的方向。这让我想知道这些问题是否相同(如果是,我很乐意删除这个问题!)。但是我不相信它们是相同的(我在日志中没有看到相同的异常)

一位同事提到,这可能是一种“被动”与“主动”的FTP方式。由于不太清楚两者之间的区别,我对FTP客户端字段
主动远程数据连接模式
被动远程数据连接模式
,等感到有点困惑,不知道他怎么认为这是一个潜在的问题

由于我在这里使用了
Throwable
s作为最后手段,因此如果出现问题,我希望在日志中看到一些内容。因此,我觉得这是一个明确的问题


有什么想法吗?不幸的是,我对FTP内部结构了解不够,无法做出确切的诊断。这可能是服务器端的东西吗?与FTP服务器相关吗?

这可能是很多事情,但是你朋友的建议是值得的

尝试
ftpClient.enterLocalPassiveMode()
查看是否有帮助


我还建议将断开连接放到
finally
块中
,这样它就不会在那里留下连接

昨天我没睡,但我想我解决了问题

您可以使用FTPClient.setBufferSize()增加缓冲区大小


我希望这段代码能对某些人有用

为了调用s.listFiles并进行传输,我必须在登录后包含以下内容,而不会“挂起”并最终失败:

s.login(username, password);
s.execPBSZ(0);
s.execPROT("P");

我在尝试从Linux计算机向IIS服务器执行listfiles时遇到了同样的问题。代码在我的开发人员工作站上运行得很好,但在服务器上运行时会挂起,特别是由于防火墙使混合变得混乱

必须按顺序执行这些操作,并要求您扩展FTPSClient 3.5

  • 连接(隐式=真,SSLContext=TLS)
  • 检查是否正确完成
  • 认证(当然)
  • execPBSZ(0)
  • 执行保护(“P”)
  • 设置布尔值以指示跳过被动IP(自定义FTPSClient类)
  • 设置保存连接IP地址(自定义FTPSClient类)
  • SetUsePSVhitIPv4(假)
  • enterLocalPassiveMode()或enterRemotePassiveMode()
  • initiateListParsing()或任何list命令 a、 )此时将执行openDataConnection,请确保保存此处使用的端口 b、 )执行PASV命令 c、 )执行_parsePassiveModeReply时,您将在此处打开具有用于连接的IP地址和保存的端口的套接字
  • 断开(始终)
  • 更多信息: 我的问题特定于Linux计算机和IIS服务器之间的防火墙。
    我的问题的根源在于,在被动模式下,在进行数据连接时用于打开套接字的IP地址与用于进行初始连接的IP地址不同。因此,由于ApacheCommonsNet3.5存在两个问题(见下文),很难弄清楚。 我的解决方案: 扩展FTPSClient,这样我就可以重写方法_parsePassiveModeReply和openDataConnection。我的parsePassiveModeReply实际上只是从应答中保存端口,因为应答指示正在使用的端口。我的openDataConnection方法是使用保存的端口和连接期间使用的原始IP

    APACHE FTPCLient 3.5的问题

  • 数据连接未超时(挂起),因此其不明显 问题是什么
  • FTPSClient类不会跳过被动IP地址。背景 “被动”将Tworkaround设置为“真”并不像我预期的那样工作,或者可能是这样 根本不跳过IP
  • 注意事项:

    • 当通过防火墙时,您必须能够访问端口范围 由IIS定义(请参阅配置Microsoft IIS防火墙)
    • 您还应确保您的计算机中有任何适当的证书 密钥库或运行时指定的证书
    • 将以下内容添加到您的类中,这非常有助于了解 正在执行FTP命令

         ftpClient.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
      
    • 检查FTP服务器日志,因为它会告诉您正在执行的操作 也可能是你遇到问题的原因。你应该经常去看电影 数据通道在执行列表之前已打开。比较测试结果 您的应用程序将显示成功的curl命令执行的操作
    • 回复代码,因为它们将指示问题发生的位置
    • 使用curl命令验证是否具有连接,如下所示 这是一个良好的开端,如果一切顺利
         ftpClient.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
      
      curl -3 ftps://[user id]:[password][ftp server ip]:990/ -1 -v --disable-epsv --ftp-skip-pasv-ip --ftp-ssl --insecure
      
      import java.io.IOException;
      import java.net.Inet6Address;
      import java.net.InetSocketAddress;
      import java.net.Socket;
      
      import javax.net.ssl.SSLContext;
      
      import org.apache.commons.net.MalformedServerReplyException;
      import org.apache.commons.net.ftp.FTPReply;
      import org.apache.commons.net.ftp.FTPSClient;
      
      /**
       * TODO Document Me!
       */
      public class PassiveFTPSClient extends FTPSClient {
          private String passiveSkipToHost;
          private int passiveSkipToPort;
          private boolean skipPassiveIP;
      
      
          /** Pattern for PASV mode responses. Groups: (n,n,n,n),(n),(n) */
          private static final java.util.regex.Pattern PARMS_PAT;    
          static {
          PARMS_PAT = java.util.regex.Pattern.compile(
                  "(\\d{1,3},\\d{1,3},\\d{1,3},\\d{1,3}),(\\d{1,3}),(\\d{1,3})");
             }
          /**
           * @param b
           * @param sslContext
           */
          public PassiveFTPSClient(boolean b, SSLContext sslContext) {
          super(b, sslContext);
          }
      
          protected void _parsePassiveModeReply(String reply) throws MalformedServerReplyException {
          if (isSkipPassiveIP()) {
              System.out.println( "================> _parsePassiveModeReply"  + getPassiveSkipToHost());
              java.util.regex.Matcher m = PARMS_PAT.matcher(reply);
              if (!m.find()) {
              throw new MalformedServerReplyException(
                  "Could not parse passive host information.\nServer Reply: " + reply);
              }
              try {
              int oct1 = Integer.parseInt(m.group(2));
              int oct2 = Integer.parseInt(m.group(3));
              passiveSkipToPort = (oct1 << 8) | oct2;
              }
              catch (NumberFormatException e) {
              throw new MalformedServerReplyException(
                  "Could not parse passive port information.\nServer Reply: " + reply);
              }            
              //do nothing
          } else {
              super._parsePassiveModeReply(reply);
          }
          }
      
          protected Socket _openDataConnection_(String command, String arg) throws IOException {
          System.out.println( "================> _openDataConnection_"  + getPassiveSkipToHost());
          System.out.println( "================> _openDataConnection_ isSkipPassiveIP: " + isSkipPassiveIP());        
          if (!isSkipPassiveIP()) {
              return super._openDataConnection_(command, arg);
          }
          System.out.println( "================> getDataConnectionMode: " + getDataConnectionMode());
          if (getDataConnectionMode() != ACTIVE_LOCAL_DATA_CONNECTION_MODE &&
              getDataConnectionMode() != PASSIVE_LOCAL_DATA_CONNECTION_MODE) {
              return null;
          }
      
          final boolean isInet6Address = getRemoteAddress() instanceof Inet6Address;
      
          Socket socket;
          if (getDataConnectionMode() == ACTIVE_LOCAL_DATA_CONNECTION_MODE) {
              return super._openDataConnection_(command, arg);
      
          }
          else
          { // We must be in PASSIVE_LOCAL_DATA_CONNECTION_MODE
      
              // Try EPSV command first on IPv6 - and IPv4 if enabled.
              // When using IPv4 with NAT it has the advantage
              // to work with more rare configurations.
              // E.g. if FTP server has a static PASV address (external network)
              // and the client is coming from another internal network.
              // In that case the data connection after PASV command would fail,
              // while EPSV would make the client succeed by taking just the port.
              boolean attemptEPSV = isUseEPSVwithIPv4() || isInet6Address;
              if (attemptEPSV && epsv() == FTPReply.ENTERING_EPSV_MODE)
              {
      
              System.out.println( "================> _parseExtendedPassiveModeReply a: ");                
              _parseExtendedPassiveModeReply(_replyLines.get(0));
              }
              else
              {
              if (isInet6Address) {
                  return null; // Must use EPSV for IPV6
              }
              // If EPSV failed on IPV4, revert to PASV
              if (pasv() != FTPReply.ENTERING_PASSIVE_MODE) {
                  return null;
              }
              System.out.println( "================> _parseExtendedPassiveModeReply b: ");
              _parsePassiveModeReply(_replyLines.get(0));
              }
              // hardcode fore testing
              //__passiveHost = "10.180.255.181";
              socket = _socketFactory_.createSocket();
              if (getReceiveDataSocketBufferSize() > 0) {
              socket.setReceiveBufferSize(getReceiveDataSocketBufferSize());
              }
              if (getSendDataSocketBufferSize()  > 0) {
              socket.setSendBufferSize(getSendDataSocketBufferSize() );
              }
              if (getPassiveLocalIPAddress() != null) {
              System.out.println( "================> socket.bind: " + getPassiveSkipToHost());
              socket.bind(new InetSocketAddress(getPassiveSkipToHost(), 0));
              }
      
              // For now, let's just use the data timeout value for waiting for
              // the data connection.  It may be desirable to let this be a
              // separately configurable value.  In any case, we really want
              // to allow preventing the accept from blocking indefinitely.
              //     if (__dataTimeout >= 0) {
              //         socket.setSoTimeout(__dataTimeout);
              //     }
      
              System.out.println( "================> socket connect: " + getPassiveSkipToHost() + ":" + passiveSkipToPort);
              socket.connect(new InetSocketAddress(getPassiveSkipToHost(), passiveSkipToPort), connectTimeout);
              if ((getRestartOffset() > 0) && !restart(getRestartOffset()))
              {
              socket.close();
              return null;
              }
      
              if (!FTPReply.isPositivePreliminary(sendCommand(command, arg)))
              {
              socket.close();
              return null;
              }
          }
      
          if (isRemoteVerificationEnabled() && !verifyRemote(socket))
          {
              socket.close();
      
              throw new IOException(
                  "Host attempting data connection " + socket.getInetAddress().getHostAddress() +
                  " is not same as server " + getRemoteAddress().getHostAddress());
          }
      
          return socket;
              }
      
          /**
          * Enable or disable passive mode NAT workaround.
          * If enabled, a site-local PASV mode reply address will be replaced with the
          * remote host address to which the PASV mode request was sent
          * (unless that is also a site local address).
          * This gets around the problem that some NAT boxes may change the
          * reply.
          *
          * The default is true, i.e. site-local replies are replaced.
          * @param enabled true to enable replacing internal IP's in passive
          * mode.
          */
          public void setSkipPassiveIP(boolean enabled) {
          super.setPassiveNatWorkaround(enabled);
          this.skipPassiveIP = enabled;
          System.out.println( "================> skipPassiveIP: " + skipPassiveIP);
          }
          /**
           * Return the skipPassiveIP.
           * @return the skipPassiveIP
           */
          public boolean isSkipPassiveIP() {
          return skipPassiveIP;
          }
          /**
           * Return the passiveSkipToHost.
           * @return the passiveSkipToHost
           */
          public String getPassiveSkipToHost() {
          return passiveSkipToHost;
          }
      
          /**
           * Set the passiveSkipToHost.
           * @param passiveSkipToHost the passiveSkipToHost to set
           */
          public void setPassiveSkipToHost(String passiveSkipToHost) {
          this.passiveSkipToHost = passiveSkipToHost;
          System.out.println( "================> setPassiveSkipToHost: " + passiveSkipToHost);
          }
      
      }