Scrubyt使用_details方法单击链接时出现404错误
这可能与我前面的两个问题类似-请参阅和,但我正尝试使用_detail命令自动单击链接,以便为每个单独的事件刮取详细信息页面 我使用的代码是:Scrubyt使用_details方法单击链接时出现404错误,ruby,scrubyt,Ruby,Scrubyt,这可能与我前面的两个问题类似-请参阅和,但我正尝试使用_detail命令自动单击链接,以便为每个单独的事件刮取详细信息页面 我使用的代码是: require 'rubygems' require 'scrubyt' nuffield_data = Scrubyt::Extractor.define do fetch 'http://www.nuffieldtheatre.co.uk/cn/events/event_listings.php' event do title 'T
require 'rubygems'
require 'scrubyt'
nuffield_data = Scrubyt::Extractor.define do
fetch 'http://www.nuffieldtheatre.co.uk/cn/events/event_listings.php'
event do
title 'The Coast of Mayo'
link_url
event_detail do
dates "1-4 October"
times "7:30pm"
end
end
next_page "Next Page", :limit => 20
end
nuffield_data.to_xml.write($stdout,1)
有没有办法打印出使用事件详细信息试图访问的URL?这个错误似乎没有给我提供404的URL
更新:我认为该链接可能是相对链接-这会导致问题吗?你知道怎么处理吗
sudo gem install ruby-debug
This will give you access to a nice ruby debugger, start the debugger by altering your script:
require 'rubygems'
require 'ruby-debug'
Debugger.start
Debugger.settings[:autoeval] = true if Debugger.respond_to?(:settings)
require 'scrubyt'
nuffield_data = Scrubyt::Extractor.define do
fetch 'http://www.nuffieldtheatre.co.uk/cn/events/event_listings.php'
event do
title 'The Coast of Mayo'
link_url
event_detail do
dates "1-4 October"
times "7:30pm"
end
end
next_page "Next Page", :limit => 2
end
nuffield_data.to_xml.write($stdout,1)
Then find out where scrubyt is throwing an exception - in this case:
/Library/Ruby/Gems/1.8/gems/scrubyt-0.3.4/lib/scrubyt/core/navigation/fetch_action.rb:52:in `fetch'
Find the scrubyt gem on your system, and add a rescue clause to the method in question so that the end of the method looks like this:
if @@current_doc_protocol == 'file'
@@hpricot_doc = Hpricot(PreFilterDocument.br_to_newline(open(@@current_doc_url).read))
else
@@hpricot_doc = Hpricot(PreFilterDocument.br_to_newline(@@mechanize_doc.body))
store_host_name(self.get_current_doc_url) # in case we're on a new host
end
rescue
debugger
self # the self is here because debugger doesn't like being at the end of a method
end
现在再次运行脚本,当引发异常时,应该将您放入调试器中。只需在调试提示中键入以下内容,即可查看有问题的URL:
@@current_doc_url
如果要检查正在发生的情况,也可以在该方法的任何位置添加调试器语句-例如,您可能希望在该方法的第51行和第52行之间添加一条调试器语句,以检查被调用的url如何更改以及更改原因
这就是我对你之前问题的答案
祝你好运。很抱歉,我不知道为什么会是零-每次我运行它时,它都会返回一个url-self.fetch方法需要一个url,您应该能够作为局部变量doc\u url访问该url。如果返回nil,您应该在包含调试器调用的地方发布代码。我尝试访问doc\u url,但似乎也返回nil。当我有权访问我的服务器时(当天晚些时候),我会发布带有调试位的代码。我在相对链接上也遇到了同样的问题,并像这样修复了它。。。您必须将:resolve参数设置为正确的基本url
event do
title 'The Coast of Mayo'
link_url
event_detail :resolve => 'http://www.nuffieldtheatre.co.uk/cn/events' do
dates "1-4 October"
times "7:30pm"
end
end