Java 如何为Spring Integration SFTP入站适配器动态定义文件过滤器模式？

我需要从不同的sftp服务器的不同目录中动态地将特定文件拉到spring引导应用程序的本地服务器目录中

我将路径和文件模式存储在postgres数据库中。我让一切都正常工作了，但我不知道如何根据spring integration sftp入站适配器的远程目录动态定义一个文件过滤器模式，这样就不会提取该特定目录中的所有xml文件

我使用RotatingServerAdvice和DelegatingSessionFactory访问动态目录和服务器

例如，对于动态文件模式过滤器，我尝试使用

.filterFunction(f -> do_some_filtering_based_on_filename(f)

我想读取该文件的远程目录，但f来自ChannelSftp.LsEntry类型，没有包含远程目录的字段。否则，我将从数据库加载配置数据，搜索路径并应用文件模式

有没有更好的方法来实现我的场景

SFTP示例：

| URL       | PATH      | FILE_PATTERN      |
|-----------|-----------|-------------------|
| 127.0.0.1 | /partner1 | test_p1_*.xml     |
| 127.0.0.2 | /partner2 | companyname_*.xml |

127.0.0.1:22

目录：root/partner1

。。。test_p1_2343545.xml

。。。test_p1_453453.xml

。。。不要拉取这个文件

127.0.0.2:22

目录：root/partner2

。。。companyname_2343545.xml

。。。companyname_453453.xml

。。。不要拉取这个文件\u 3434.xml

数据库配置示例：

| URL       | PATH      | FILE_PATTERN      |
|-----------|-----------|-------------------|
| 127.0.0.1 | /partner1 | test_p1_*.xml     |
| 127.0.0.2 | /partner2 | companyname_*.xml |

我的spring boot应用程序的适配器类，具有工作代码，但由于.patternFilter（“*.xml”）：

| URL       | PATH      | FILE_PATTERN      |
|-----------|-----------|-------------------|
| 127.0.0.1 | /partner1 | test_p1_*.xml     |
| 127.0.0.2 | /partner2 | companyname_*.xml |

import com.jcraft.jsch.ChannelSftp；
导入org.slf4j.Logger；
导入org.slf4j.LoggerFactory；
导入org.springframework.beans.factory.annotation.Value；
导入org.springframework.context.annotation.Bean；
导入org.springframework.context.annotation.Configuration；
导入org.springframework.integration.channel.DirectChannel；
导入org.springframework.integration.channel.NullChannel；
导入org.springframework.integration.dsl.IntegrationFlow；
导入org.springframework.integration.dsl.IntegrationFlows；
导入org.springframework.integration.dsl.poller；
导入org.springframework.integration.dsl.SourcePollingChannelAdapterSpec；
导入org.springframework.integration.expression.FunctionExpression；
导入org.springframework.integration.file.remote.aop.RotatingServerAdvice；
导入org.springframework.integration.file.remote.session.DelegatingSessionFactory；
导入org.springframework.integration.file.remote.session.SessionFactory；
导入org.springframework.integration.scheduling.PollerMetadata；
导入org.springframework.integration.sftp.dsl.sftp；
导入org.springframework.integration.sftp.dsl.SftpInboundChannelAdapterSpec；
导入org.springframework.integration.sftp.session.DefaultSftpSessionFactory；
导入org.springframework.messaging.MessageChannel；
导入org.springframework.stereotype.Component；
导入java.io.File；
导入java.time.Instant；
导入java.time.ZoneId；
导入java.time.format.DateTimeFormatter；
导入java.util.ArrayList；
导入java.util.LinkedHashMap；
导入java.util.List；
导入java.util.Map；
导入java.util.function.Consumer；
/**
*流动。
*/
@配置
@组成部分
公共类集成{
公共静态最终字符串时区\u UTC=“UTC”；
_FILES=“yyyymmddhhmmssss”的公共静态最终字符串时间戳\u格式；
公共静态最终字符串临时文件后缀=“.part”；
公共静态最终整数轮询器固定周期延迟=5000；
公共静态最终int MAX_MESSAGES_PER_POLL=100；
私有静态最终记录器LOG=LoggerFactory.getLogger（SFTIntegration.class）；
专用静态最终字符串通道\u中间\u阶段=“中间通道”；
/**数据库访问存储库*/
私人最终PartnerConfigRepo PartnerConfigRepo；
@值（${app.tmp dir}”）
私有字符串localTemporaryPath；
公共SFT集成（最终PartnerConfigRepo PartnerConfigRepo）{
this.partnerConfigRepo=partnerConfigRepo；
}
/**
*带有5s、100条消息、旋转服务器建议和事务的默认轮询器。
*
*@return默认轮询器。
*/
@Bean（name=PollerMetadata.DEFAULT\u POLLER）
公共民意调查机构{
回程轮询器
.fixedDelay（轮询器\固定\周期\延迟）
.advice（advice（））
.maxMessagesPerPoll（每个轮询的最大消息数）
.事务性
.get（）；
}
/**
*流动的直接通道。
*
*@returnmessagechannel
*/
@豆子
公共消息频道stockIntermediateChannel（）{
返回新的DirectChannel（）；
}
/**
*从远程目录获取文件。向文件名添加时间戳
*并将它们写入本地临时文件夹。
*
*@返回积分流
*/
@豆子
公共集成流stockInboundFlowFromSFTPServer（）{
//源定义
最终SftpInboundChannelAdapterSpec sourceSpec=Sftp.inboundAdapter（delegatingSFtpSessionFactory（））
.保留时间戳（true）
.patternFilter（“*.xml”）
//.filterFunction（f->do_some_filter_基于_文件名（f，delegatingSFtpSessionFactory（）.getSession（））进行过滤）
//.filter（新的ModifiedFilter（））
//.filterExpression（“#remoteDirectory”）
.deleteRemoteFiles（true）
.maxFetchSize（每个轮询的最大消息数）
.remoteDirectory（“/”）
.localDirectory（新文件（localTemporaryPath））
.temporaryFileSuffix（临时文件后缀）
.localFilenameExpression（新函数表达式->{
final int fileTypeSepPos=s.lastIndexOf（'.'）；
返回
日期时间格式化程序
.OF模式（时间戳\u格式\u文件）
.withZone（ZoneId.of（时区协调世界时））
.format（Instant.now（））
+ "_"
+s.substring（0，文件类型seppos）
+s.子字符串（fileTypeSepPos）；
}));
//轮询器定义
最终消费者股票边界轮询器
import com.jcraft.jsch.ChannelSftp.LsEntry;
import org.springframework.integration.file.filters.AbstractSimplePatternFileListFilter;

/**
 * Implementation of {@link AbstractSimplePatternFileListFilter} for SFTP with logic for changing the file pattern at runtime.
 */
public class MySftpPatternFileListFilter extends MyAbstractSimplePatternFileListFilter<LsEntry> {

  public MySftpPatternFileListFilter(final String pattern) {
    super(pattern);
  }

  @Override
  protected String getFilename(final LsEntry entry) {
    return (entry != null) ? entry.getFilename() : null;
  }

  @Override
  protected boolean isDirectory(final LsEntry file) {
    return file.getAttrs().isDir();
  }

  /**
   * Sets the file pattern for the file filter
   *
   * @param pattern a file pattern like "*.xml"
   */
  public void setPattern(final String pattern) {
    setPath(pattern);
  }

}

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.integration.core.MessageSource;
import org.springframework.integration.file.remote.AbstractRemoteFileStreamingMessageSource;
import org.springframework.integration.file.remote.aop.RotatingServerAdvice;
import org.springframework.integration.file.remote.session.DelegatingSessionFactory;
import org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource;
import org.springframework.util.Assert;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

/**
 * Adapted version of {@link RotatingServerAdvice.RotationPolicy} with own FileListFilter {@link MySftpPatternFileListFilter} and
 * database access for configuration.
 * <p>
 * Standard rotation policy; iterates over key/directory pairs; when the end
 * is reached, starts again at the beginning. If the fair option is true
 * the rotation occurs on every poll, regardless of result. Otherwise rotation
 * occurs when the current pair returns no message.
 */
public class MyStandardRotationPolicy implements RotatingServerAdvice.RotationPolicy {

  protected final Log logger = LogFactory.getLog(getClass());

  private final DelegatingSessionFactory<?> factory;

  private final List<RotatingServerAdvice.KeyDirectory> keyDirectories = new ArrayList<>();

  private final boolean fair;
  private final MySftpPatternFileListFilter fileListFilter;
  private final PartnerConfigRepo partnerConfigRepo;
  private volatile Iterator<RotatingServerAdvice.KeyDirectory> iterator;
  private volatile RotatingServerAdvice.KeyDirectory current;
  private volatile boolean initialized;

  public MyStandardRotationPolicy(final DelegatingSessionFactory<?> factory,
      final List<RotatingServerAdvice.KeyDirectory> keyDirectories,
      final boolean fair,
      final MySftpPatternFileListFilter fileListFilter,
      final PartnerConfigRepo partnerConfigRepo) {

    Assert.notNull(factory, "factory cannot be null");
    Assert.notNull(keyDirectories, "keyDirectories cannot be null");
    Assert.isTrue(keyDirectories.size() > 0, "At least one KeyDirectory is required");
    this.factory = factory;
    this.keyDirectories.addAll(keyDirectories);
    this.fair = fair;
    this.iterator = this.keyDirectories.iterator();
    this.fileListFilter = fileListFilter;
    this.partnerConfigRepo = partnerConfigRepo;
  }

  protected Iterator<RotatingServerAdvice.KeyDirectory> getIterator() {
    return this.iterator;
  }

  protected void setIterator(final Iterator<RotatingServerAdvice.KeyDirectory> iterator) {
    this.iterator = iterator;
  }

  protected boolean isInitialized() {
    return this.initialized;
  }

  protected void setInitialized(final boolean initialized) {
    this.initialized = initialized;
  }

  protected DelegatingSessionFactory<?> getFactory() {
    return this.factory;
  }

  protected List<RotatingServerAdvice.KeyDirectory> getKeyDirectories() {
    return this.keyDirectories;
  }

  protected boolean isFair() {
    return this.fair;
  }

  @Override
  public void beforeReceive(final MessageSource<?> source) {
    if (this.fair || !this.initialized) {
      configureSource(source);
      this.initialized = true;
    }
    if (this.logger.isTraceEnabled()) {
      this.logger.trace("Next poll is for " + this.current);
    }
    this.factory.setThreadKey(this.current.getKey());
  }

  @Override
  public void afterReceive(final boolean messageReceived, final MessageSource<?> source) {
    if (this.logger.isTraceEnabled()) {
      this.logger.trace("Poll produced "
          + (messageReceived ? "a" : "no")
          + " message");
    }
    this.factory.clearThreadKey();
    if (!this.fair && !messageReceived) {
      configureSource(source);
    }
  }

  protected void configureSource(final MessageSource<?> source) {
    Assert.isTrue(source instanceof AbstractInboundFileSynchronizingMessageSource
            || source instanceof AbstractRemoteFileStreamingMessageSource,
        "source must be an AbstractInboundFileSynchronizingMessageSource or a "
            + "AbstractRemoteFileStreamingMessageSource");
    if (!this.iterator.hasNext()) {
      this.iterator = this.keyDirectories.iterator();
    }
    this.current = this.iterator.next();

    fileListFilter.setPattern(getPatternFromDataBase(this.current.getDirectory()));

    if (source instanceof AbstractRemoteFileStreamingMessageSource) {
      ((AbstractRemoteFileStreamingMessageSource<?>) source).setRemoteDirectory(this.current.getDirectory());
    } else {
      ((AbstractInboundFileSynchronizingMessageSource<?>) source).getSynchronizer()
          .setRemoteDirectory(this.current.getDirectory());

    }
  }

  private String getPatternFromDataBase(final String directory) {
    //String Pattern;
    final List<PartnerConfigEntity> allStock = partnerConfigRepo.findByTypeAndActiveIsTrue(PartnerConfigType.STOCK);
    for (final PartnerConfigEntity s : allStock) {
      if (s.getServerPath().equals(directory)) {
        return s.getFileNamePattern();
      }
    }
    //TODO throw exception
    return "*.xml";
  }

}

/**
 * flow.
 */
@Configuration
@Component
public class SFTIntegration {

  public static final String TIMEZONE_UTC = "UTC";
  public static final String TIMESTAMP_FORMAT_OF_FILES = "yyyyMMddHHmmssSSS";
  public static final String TEMPORARY_FILE_SUFFIX = ".part";
  public static final int POLLER_FIXED_PERIOD_DELAY = 5000;
  public static final int MAX_MESSAGES_PER_POLL = 100;
  private static final Logger LOG = LoggerFactory.getLogger(SFTIntegration.class);
  private static final String CHANNEL_INTERMEDIATE_STAGE = "stockIntermediateChannel";

  /**
   * database access repository
   */
  private final PartnerConfigRepo partnerConfigRepo;

  @Value("${app.tmp-dir}")
  private String localTemporaryPath;

  public SFTIntegration(final PartnerConfigRepo partnerConfigRepo) {
    this.partnerConfigRepo = partnerConfigRepo;
  }

  @Bean
  MySftpPatternFileListFilter getFilter() {
    //initial pattern
    return new MySftpPatternFileListFilter("*.xml");
  }

  /**
   * The default poller with 5s, 100 messages, RotatingServerAdvice and transaction.
   *
   * @return default poller.
   */
  @Bean(name = PollerMetadata.DEFAULT_POLLER)
  public PollerMetadata poller() {
    return Pollers
        .fixedDelay(POLLER_FIXED_PERIOD_DELAY)
        .advice(advice())
        .maxMessagesPerPoll(MAX_MESSAGES_PER_POLL)
        .transactional()
        .get();
  }

  /**
   * The direct channel for the flow.
   *
   * @return MessageChannel
   */
  @Bean
  public MessageChannel stockIntermediateChannel() {
    return new DirectChannel();
  }

  /**
   * Get the files from a remote directory. Add a timestamp to the filename (milliseconds since midnight January 1st 1970, UTC)
   * and write them to a local temporary folder. Get the files from the local temporary folder.
   *
   * @return IntegrationFlow
   */
  @Bean
  public IntegrationFlow stockInboundFlowFromSFTPServer() {
    // Source definition
    final SftpInboundChannelAdapterSpec sourceSpec = Sftp.inboundAdapter(delegatingSFtpSessionFactory())

        .preserveTimestamp(true)
        .filter(getFilter())
        .deleteRemoteFiles(true)
        .maxFetchSize(MAX_MESSAGES_PER_POLL)
        .remoteDirectory("/")
        .localDirectory(new File(localTemporaryPath))
        .temporaryFileSuffix(TEMPORARY_FILE_SUFFIX)
        .localFilenameExpression(new FunctionExpression<String>(s -> {
          final int fileTypeSepPos = s.lastIndexOf('.');
          // use Instant.now().toEpochMilli() for utc time in milliseconds
          return
              DateTimeFormatter
                  .ofPattern(TIMESTAMP_FORMAT_OF_FILES)
                  .withZone(ZoneId.of(TIMEZONE_UTC))
                  .format(Instant.now())
                  + "_"
                  + (new SecureRandom()).nextInt(99999)
                  + "_"
                  + s.substring(0, fileTypeSepPos)
                  + s.substring(fileTypeSepPos);
        }));

    // Poller definition
    final Consumer<SourcePollingChannelAdapterSpec> stockInboundPoller = endpointConfigurer -> endpointConfigurer
        .id("stockInboundPoller")
        .autoStartup(true)
        .poller(poller());

    return IntegrationFlows
        .from(sourceSpec, stockInboundPoller)
        .transform(File.class, p -> {
          // log step
          LOG.info("flow=stockInboundFlowFromAFT, message=incoming file: " + p);
          return p;
        })
        .channel(CHANNEL_INTERMEDIATE_STAGE)
        .get();
  }

  @Bean
  public IntegrationFlow stockIntermediateStageChannel() {
    return IntegrationFlows
        .from(CHANNEL_INTERMEDIATE_STAGE)
        .transform(p -> {
          //log step
          LOG.info("flow=stockIntermediateStageChannel, message=rename file: " + p);
          return p;
        })
        //TODO
        .channel(new NullChannel())
        .get();
  }

  public DefaultSftpSessionFactory createNewSftpSessionFactory(final PartnerConfigEntity pc) {
    final DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(
        false); //TODO set true but use caching session https://docs.spring.io/spring-integration/reference/html/sftp.html
    factory.setHost(pc.getServerIp());
    factory.setPort(pc.getPort());
    factory.setUser(pc.getUsername());
    factory.setPassword(pc.getPassword());
    factory.setAllowUnknownKeys(true);
    return factory;
  }

  @Bean
  public DelegatingSessionFactory<ChannelSftp.LsEntry> delegatingSFtpSessionFactory() {
    final List<PartnerConfigEntity> partnerConnections = partnerConfigRepo
        .findByTypeAndActiveIsTrue(PartnerConfigType.STOCK);

    if (partnerConnections.isEmpty()) {
      return null;
    }

    final Map<Object, SessionFactory<ChannelSftp.LsEntry>> factories = new LinkedHashMap<>(10);

    for (final PartnerConfigEntity pc : partnerConnections) {
      // create a factory for every key containing PartnerConfigEntity.getKey() attributes (server type, url and port)
      if (factories.get(pc.getKey()) == null) {
        factories.put(pc.getKey(), createNewSftpSessionFactory(pc));
      }
    }

    // use the first SF as the default
    return new DelegatingSessionFactory<>(factories, factories.values().iterator().next());
  }

  @Bean
  public RotatingServerAdvice advice() {
    final List<PartnerConfigEntity> partnerConnections = partnerConfigRepo
        .findByTypeAndActiveIsTrue(PartnerConfigType.STOCK);
    LOG.debug("Found " + partnerConnections.size() + " server entries for type stock.");

    final List<RotatingServerAdvice.KeyDirectory> keyDirectories = new ArrayList<>();
    for (final PartnerConfigEntity pc : partnerConnections) {
      keyDirectories
          .add(new RotatingServerAdvice.KeyDirectory(pc.getKey(), pc.getServerPath()));
    }
    final RotatingServerAdvice rot = new RotatingServerAdvice(
        new MyStandardRotationPolicy(delegatingSFtpSessionFactory(), keyDirectories, true,
            getFilter(), partnerConfigRepo));
    return rot;
  }

}