Java 如何在这个for循环中使用并行流
jsonarray对象包含50000个jsonObject。我需要处理这个jsonObject来创建一个对象,然后我想将这个对象添加到列表中。这需要花费很多时间,所以我想使用并行流来创建对象并更快地添加到列表中 下面是我想用并行循环替换的代码Java 如何在这个for循环中使用并行流,java,java-stream,Java,Java Stream,jsonarray对象包含50000个jsonObject。我需要处理这个jsonObject来创建一个对象,然后我想将这个对象添加到列表中。这需要花费很多时间,所以我想使用并行流来创建对象并更快地添加到列表中 下面是我想用并行循环替换的代码 // I want to insert object in this array List<Curriculum> curriculamList = new ArrayList<>();
// I want to insert object in this array
List<Curriculum> curriculamList = new ArrayList<>();
// This is the org.json array
JSONArray jsonarray = new JSONArray(content);
// This loop I want to replace
for (int i = 0; i < jsonarray.length(); i++) {
JSONObject jsonobject = jsonarray.getJSONObject(i);
// Start of processing
Program program = programDAO.getProgramDetails(jsonobject.getInt("programId"));
Batch batch = batchDAO.getBatchDetails(jsonobject.getInt("batchId"), program);
if(batch.getBatchId() == 0)
continue;
Major major = majorDAO.getMajorDetails(jsonobject.getInt("majorId"));
Course course = courseDAO.getCourseDetails(jsonobject.getString("courseCode"));
if(course == null)
continue;
double credits = jsonobject.getDouble("credits");
CourseType courseType = courseTypeDAO.getCourseTypeDetails(program, jsonobject.optString("type"));
int semesterCount = jsonobject.getInt("semester");
String prereqCourseString = jsonobject.getString("prereq");
Course alternateCourse = courseDAO.getCourseDetails(jsonobject.getString("alternate"));
List<Course> prereqCourseList = new ArrayList<>();
if (prereqCourseString.length() != 0) {
String[] prereqCourseSplit = prereqCourseString.split("AND");
for (String prereqCourseSplitString : prereqCourseSplit) {
prereqCourseList.add(courseDAO.getCourseDetails(prereqCourseSplitString.trim()));
}
}
List<Course> prereqChainCourseList = new ArrayList<>();
// End of processing
// This is the object
Curriculum curriculum = new Curriculum(course, credits, courseType, semesterCount, prereqCourseList, prereqChainCourseList, alternateCourse, batch, major);
// Pushing object into the list
curriculamList.add(curriculum);
}
我是java新手,如果我犯了任何错误,请原谅。Stream.ofjsonarray返回一个包含单个元素的流,即JSONArray对象,这不是您想要的
我想这就是你想要的:
List<Curriculum> curriculumList = IntStream.range(0, jsonarray.length()).parallel()
.mapToObj(i -> {
JSONObject jsonobject = jsonarray.getJSONObject(i);
// fetch the various parts...
return new Curriculum(course, credits, courseType, semesterCount, prereqCourseList, prereqChainCourseList, alternateCourse, batch, major);
}).collect(Collectors.toList());
但是,看起来您的代码正在进行多个阻塞网络调用。使用并行流并行执行阻塞网络调用通常是不可取的,因为并行流用于CPU并行化,所以使用只有少量线程的单个共享线程池。因此,您可能更喜欢以下内容:
ExecutorService executor = Executors.newFixedThreadPool(10);
// start many calls running in parallel
List<Future<Curriculum>> futures = IntStream.range(0, jsonarray.length())
.mapToObj(i -> executor.submit(() -> {
JSONObject jsonobject = jsonarray.getJSONObject(i);
// fetch the various parts...
return new Curriculum(course, credits, courseType, semesterCount, prereqCourseList, prereqChainCourseList, alternateCourse, batch, major);
})).collect(Collectors.toList());
executor.shutdown();
List<Curriculum> curriculumList = new ArrayList<>();
for (Future<Curriculum> future : futures) {
curriculumList.add(future.get());
}
ofjsonarray返回一个包含单个元素的流,即JSONArray对象,这不是您想要的
我想这就是你想要的:
List<Curriculum> curriculumList = IntStream.range(0, jsonarray.length()).parallel()
.mapToObj(i -> {
JSONObject jsonobject = jsonarray.getJSONObject(i);
// fetch the various parts...
return new Curriculum(course, credits, courseType, semesterCount, prereqCourseList, prereqChainCourseList, alternateCourse, batch, major);
}).collect(Collectors.toList());
但是,看起来您的代码正在进行多个阻塞网络调用。使用并行流并行执行阻塞网络调用通常是不可取的,因为并行流用于CPU并行化,所以使用只有少量线程的单个共享线程池。因此,您可能更喜欢以下内容:
ExecutorService executor = Executors.newFixedThreadPool(10);
// start many calls running in parallel
List<Future<Curriculum>> futures = IntStream.range(0, jsonarray.length())
.mapToObj(i -> executor.submit(() -> {
JSONObject jsonobject = jsonarray.getJSONObject(i);
// fetch the various parts...
return new Curriculum(course, credits, courseType, semesterCount, prereqCourseList, prereqChainCourseList, alternateCourse, batch, major);
})).collect(Collectors.toList());
executor.shutdown();
List<Curriculum> curriculumList = new ArrayList<>();
for (Future<Curriculum> future : futures) {
curriculumList.add(future.get());
}