Ruby中em同步的嵌套迭代器

我正在尝试使用eventmachine和em同步编写解析器（解析邮政编码的街道和房屋）。问题是，我想要解析的网站具有嵌套结构——对于每个邮政编码，都有许多带有分页的街道页面。因此，算法非常简单：

• 每个邮政编码
  • 访问邮政编码索引页面
    • 解析索引页
    • 解析分页
    • 对于每个分页页面，请分析此页面

下面是这样一个解析器的示例（它可以工作）：

如何在同步迭代器循环中实现嵌套的EM HTTP调用

---

我尝试了不同的方法，但每次都会出现错误，例如“无法从根光纤中生成”或调用errback块。

一种解决方案是使用

光纤操作符和同步
。get
而不是
。aget
：

require "em-synchrony"
require "em-synchrony/em-http"
require "em-synchrony/fiber_iterator"

def url page = nil
  url = "http://gistflow.com/all"
  url << "?page=#{page}" if page
  url
end

EM.synchrony do
  concurrency = 2

  master_pages = [1,2,3,4]

  EM::Synchrony::FiberIterator.new(master_pages, concurrency).each do |iter|
    result = EM::HttpRequest.new(url).get
    if result
      puts "SUCC #{iter}"
      detail_pages = [1,2,3,4]       
      EM::Synchrony::FiberIterator.new(detail_pages, concurrency).each do |iter2|
        result2 = EM::HttpRequest.new(url).get
        puts "SUCC/ERR #{iter} > #{iter2}"
      end
    else
      puts "ERR #{iter}"
    end
  end

  EM.stop

end

要求“em同步”
需要“em同步/em http”
需要“em同步/光纤迭代器”
def url page=nil
url=”http://gistflow.com/all"
url谢谢你的帮助，正是我所需要的！
unless pages.empty?
  # here we need to parse all pages
  # with urls like url(page)
  # how can I do it more efficiently?
end

require "em-synchrony"
require "em-synchrony/em-http"
require "em-synchrony/fiber_iterator"

def url page = nil
  url = "http://gistflow.com/all"
  url << "?page=#{page}" if page
  url
end

EM.synchrony do
  concurrency = 2

  master_pages = [1,2,3,4]

  EM::Synchrony::FiberIterator.new(master_pages, concurrency).each do |iter|
    result = EM::HttpRequest.new(url).get
    if result
      puts "SUCC #{iter}"
      detail_pages = [1,2,3,4]       
      EM::Synchrony::FiberIterator.new(detail_pages, concurrency).each do |iter2|
        result2 = EM::HttpRequest.new(url).get
        puts "SUCC/ERR #{iter} > #{iter2}"
      end
    else
      puts "ERR #{iter}"
    end
  end

  EM.stop

end