Ruby on rails Rails,组合选择、分组、查找
我有一个lib任务,从一个更大、更臃肿的记录集创建一个更小、更简洁的记录集Ruby on rails Rails,组合选择、分组、查找,ruby-on-rails,rails-activerecord,ruby-on-rails-6,Ruby On Rails,Rails Activerecord,Ruby On Rails 6,我有一个lib任务,从一个更大、更臃肿的记录集创建一个更小、更简洁的记录集 膨胀的记录集~11.1M 简明记录集~170K 臃肿的模型/记录的模型类似于: Bloated.new(id: string, state: string, county: string, city: name, block_code: string, sub_block_code: string
- 膨胀的记录集~11.1M
- 简明记录集~170K
Bloated.new(id: string,
state: string,
county: string,
city: name,
block_code: string,
sub_block_code: string)
简明模型类似于,其中类别为:州、县、市、区块代码
Concise.new(id: uuid,
name: string
category: string)
从臃肿的模型中,我不关心sub_块
代码,因此我想创建一组更简洁的记录,这些记录不引用该值。因此,要创建我拥有的简明记录集:
Bloated
.select('state, county, city, block_code, count(id)')
.group(:state, :county, :city, :block_code)
.each do |r|
state = Concise.find_or_create_by(name: r.state,
category: 'state')
county = Concise.find_or_create_by(parent_id: state.id,
name: r.county,
category: 'county')
city = Concise.find_or_create_by(parent_id: county.id,
name: r.city,
category: 'city')
Concise.find_or_create_by(parent_id: city.id,
name: r.block_code,
category: 'block_code')
end
上述工作正常,但是,所有11.1M记录都使用加载到内存中。。。现在,我很难找到使用的解决方案。查找每个,以便它在较小的批次中工作。。。我需要一些帮助来决定怎么做
更新:
忘记添加find_each
错误:
PG::GroupingError: ERROR: column "bloated.id" must appear in
the GROUP BY clause or be used in an aggregate function
LINE 1: ..., "public"."bloated"."block_code" ORDER BY "public"."...