Java NiFi ExecuteScript输出已使用zip4j损坏

Java NiFi ExecuteScript输出已使用zip4j损坏,java,groovy,stream,apache-nifi,zip4j,Java,Groovy,Stream,Apache Nifi,Zip4j,我使用groovy编写的executescript使用密码压缩csv。executescript处理器能够将流文件毫无问题地传递到下一个处理器,但该文件已损坏。PUTSFP处理器正在抱怨以下错误 虽然文件是通过SFTP存储的,但我无法解压缩,因为它已损坏。我的代码有什么问题?在开发自定义处理器解压加密内容时,我们成功地使用下面的代码片段使用Lingala解压受密码保护的流文件。请参考下面的代码,将逻辑复制到ExecuteScript处理器- @Grab('net.lingala.zip4j:zi

我使用groovy编写的executescript使用密码压缩csv。executescript处理器能够将流文件毫无问题地传递到下一个处理器,但该文件已损坏。PUTSFP处理器正在抱怨以下错误


虽然文件是通过SFTP存储的,但我无法解压缩,因为它已损坏。我的代码有什么问题?

在开发自定义处理器解压加密内容时,我们成功地使用下面的代码片段使用Lingala解压受密码保护的流文件。请参考下面的代码,将逻辑复制到ExecuteScript处理器-

@Grab('net.lingala.zip4j:zip4j:2.2.8')
import net.lingala.zip4j.io.outputstream.ZipOutputStream;
import net.lingala.zip4j.model.ZipParameters;
import net.lingala.zip4j.model.enums.EncryptionMethod;

import org.apache.commons.io.IOUtils

flowFile = session.get()
if(!flowFile) return

flowFile = session.write(flowFile, {inputStream, outputStream ->
    byte[] inputByteArray = IOUtils.toByteArray(inputStream)

    ByteArrayOutputStream outputByteStream = new ByteArrayOutputStream()
    ZipOutputStream outputZipStream = new ZipOutputStream(outputByteStream, "password".toCharArray())

    //init the zip parameters
    ZipParameters zipParams = new ZipParameters()
    zipParams.setEncryptFiles(true)
    zipParams.setEncryptionMethod(EncryptionMethod.AES)
    zipParams.setFileNameInZip("records.csv")

    outputZipStream.putNextEntry(zipParams)
    outputZipStream.write(inputByteArray)
    outputZipStream.closeEntry()

    outputZipStream.close()
    outputByteStream.close()

    outputStream.write(outputByteStream.toByteArray())
} as StreamCallback)

session.transfer(flowFile, REL_SUCCESS)
import net.lingala.zip4j.io.inputstream.ZipInputStream;
导入net.lingala.zip4j.model.LocalFileHeader;
@凌驾
public void解包(最终ProcessSession会话、最终FlowFile源、最终列表解包){
最终字符串fragmentId=UUID.randomUUID().toString();
session.read(源,新的InputStreamCallback(){
@凌驾
公共无效进程(最终输入流输入)引发IOException{
int fragmentCount=0;
try(final-ZipInputStream-zipIn=new-ZipInputStream(new-BufferedInputStream(in),UNPACK_PASSWORD.tocharray()){
LocalFileHeader-zipEntry;
而((zipEntry=zipIn.getNextEntry())!=null){
if(zipEntry.isDirectory()| |!fileMatchesWithEncryption(zipEntry)){
继续;
}
最终文件=新文件(zipEntry.getFileName());
最后一个字符串parentDirectory=(file.getParent()==null)?“/”:file.getParent();
最终路径absPath=file.toPath().toabsolutionPath();
最后一个字符串absPathString=absPath.getParent().toString()+“/”;
FlowFile UnpactedFile=session.create(源);
试一试{
最终映射属性=新HashMap();
attributes.put(CoreAttributes.FILENAME.key(),file.getName());
attributes.put(CoreAttributes.PATH.key(),parentDirectory);
attributes.put(CoreAttributes.ABSOLUTE_PATH.key(),absPathString);
attributes.put(CoreAttributes.MIME_TYPE.key(),八位字节流);
attributes.put(FRAGMENT\u ID,fragmentId);
attributes.put(FRAGMENT_INDEX,String.valueOf(++fragmentCount));
解包文件=session.putalAttribute(解包文件,属性);
解包文件=session.write(解包文件,新的OutputStreamCallback(){
@凌驾
公共无效进程(最终输出流输出)引发IOException{
StreamUtils.copy(zipIn,out);
}
});
}最后{
解包。添加(解包文件);
}
}
}
}
});
}

对于任何人的参考,您需要在使用to toByteArray后手动关闭inputStream:以下代码应该可以工作:

import net.lingala.zip4j.io.inputstream.ZipInputStream;
import net.lingala.zip4j.model.LocalFileHeader;

@Override
        public void unpack(final ProcessSession session, final FlowFile source, final List<FlowFile> unpacked) {
            final String fragmentId = UUID.randomUUID().toString();
            session.read(source, new InputStreamCallback() {
                @Override
                public void process(final InputStream in) throws IOException {
                    int fragmentCount = 0;

                    try (final ZipInputStream zipIn = new ZipInputStream(new BufferedInputStream(in), UNPACK_PASSWORD.toCharArray())) {
                        LocalFileHeader zipEntry;
                        while ((zipEntry = zipIn.getNextEntry()) != null) {
                            if (zipEntry.isDirectory() || !fileMatchesWithEncryption(zipEntry)) {
                                continue;
                            }
                            final File file = new File(zipEntry.getFileName());
                            final String parentDirectory = (file.getParent() == null) ? "/" : file.getParent();
                            final Path absPath = file.toPath().toAbsolutePath();
                            final String absPathString = absPath.getParent().toString() + "/";

                            FlowFile unpackedFile = session.create(source);
                            try {
                                final Map<String, String> attributes = new HashMap<>();
                                attributes.put(CoreAttributes.FILENAME.key(), file.getName());
                                attributes.put(CoreAttributes.PATH.key(), parentDirectory);
                                attributes.put(CoreAttributes.ABSOLUTE_PATH.key(), absPathString);
                                attributes.put(CoreAttributes.MIME_TYPE.key(), OCTET_STREAM);

                                attributes.put(FRAGMENT_ID, fragmentId);
                                attributes.put(FRAGMENT_INDEX, String.valueOf(++fragmentCount));

                                unpackedFile = session.putAllAttributes(unpackedFile, attributes);
                                unpackedFile = session.write(unpackedFile, new OutputStreamCallback() {
                                    @Override
                                    public void process(final OutputStream out) throws IOException {
                                        StreamUtils.copy(zipIn, out);
                                    }
                                });
                            } finally {
                                unpacked.add(unpackedFile);
                            }
                        }
                    }
                }
            });
        }

该sftp处理器的用户是否有权在目标系统上更改文件名?是的。要注意的是,通过sftp传输文件而不进行压缩可以很好地工作。我在错误消息中没有看到任何关于损坏的信息。PUTSFP不关心文件内容,即使您的zip已损坏。请提供日志文件中的完整stacktrace以查看真正的错误。
@Grab('net.lingala.zip4j:zip4j:2.2.8')
import net.lingala.zip4j.io.outputstream.ZipOutputStream;
import net.lingala.zip4j.model.ZipParameters;
import net.lingala.zip4j.model.enums.EncryptionMethod;

import org.apache.commons.io.IOUtils

flowFile = session.get()
if(!flowFile) return

flowFile = session.write(flowFile, {inputStream, outputStream ->
    byte[] inputByteArray = IOUtils.toByteArray(inputStream)
    inputStream.close() // this is what was missing in the original code

    ByteArrayOutputStream outputByteStream = new ByteArrayOutputStream()
    ZipOutputStream outputZipStream = new ZipOutputStream(outputByteStream, "password".toCharArray())

    //init the zip parameters
    ZipParameters zipParams = new ZipParameters()
    zipParams.setEncryptFiles(true)
    zipParams.setEncryptionMethod(EncryptionMethod.AES)
    zipParams.setFileNameInZip("records.csv")

    outputZipStream.putNextEntry(zipParams)
    outputZipStream.write(inputByteArray)
    outputZipStream.closeEntry()

    outputZipStream.close()
    outputByteStream.close()

    outputStream.write(outputByteStream.toByteArray())
} as StreamCallback)

session.transfer(flowFile, REL_SUCCESS)