使用java从大文件中读取数据块
我有一个10K实体的大文件(每行实体) 我想把它分成1K个实体来阅读 我试过:使用java从大文件中读取数据块,java,file,file-io,io,randomaccessfile,Java,File,File Io,Io,Randomaccessfile,我有一个10K实体的大文件(每行实体) 我想把它分成1K个实体来阅读 我试过: public List<String> getNextRequestsChunk() { List<String> requests = new ArrayList<>(); try { randomAccessFile.seek(currentSeekPosition); String line = null;
public List<String> getNextRequestsChunk() {
List<String> requests = new ArrayList<>();
try {
randomAccessFile.seek(currentSeekPosition);
String line = null;
while ((requests.size() < chunkSize) && (line = randomAccessFile.readLine()) != null)
{
currentSeekPosition += line.length();
requests.add(line);
}
} catch (IOException ex) {
ex.printStackTrace();
throw new RuntimeException(ex);
}
return requests;
}
当我为chunk#2重新运行此方法时,它不会给我预期的字符串33
,而是字符串2
(chunkSize
为2行,currentSeekPosition
=4)
如何修复此问题?添加
currentSeekPosition=randomAccessFile.getFilePointer()代码>在之后,而循环
public List<String> getNextRequestsChunk() {
List<String> requests = new ArrayList<>();
try {
randomAccessFile.seek(currentSeekPosition);
String line = null;
while ((requests.size() < chunkSize) && (line = randomAccessFile.readLine()) != null)
{
// currentSeekPosition += line.length()+1;
requests.add(line);
}
// add this
currentSeekPosition = randomAccessFile.getFilePointer();
} catch (IOException ex) {
ex.printStackTrace();
throw new RuntimeException(ex);
}
return requests;
}
public List getNextRequestsChunk(){
列表请求=新建ArrayList();
试一试{
randomAccessFile.seek(currentSeekPosition);
字符串行=null;
而((requests.size()
您的问题是readLine
方法不计算新行字符\n
如果不是第10行,它会给您带来什么?请确保currentSeekPosition没有在外面的某个地方重置。请查看我的更新。我想您需要添加:currentSeekPosition+=line.length()+1;你为什么还要和seek()混在一起?只需读取数据,让文件指针自动前进。
public List<String> getNextRequestsChunk() {
List<String> requests = new ArrayList<>();
try {
randomAccessFile.seek(currentSeekPosition);
String line = null;
while ((requests.size() < chunkSize) && (line = randomAccessFile.readLine()) != null)
{
// currentSeekPosition += line.length()+1;
requests.add(line);
}
// add this
currentSeekPosition = randomAccessFile.getFilePointer();
} catch (IOException ex) {
ex.printStackTrace();
throw new RuntimeException(ex);
}
return requests;
}