Mechanize to access https links

Ruby
1

I would like to open and save to disk https tutorials which were
created with Flash i think.

To simplify, I login to the site with a webbrowser, so I don’t need to
access a form. Just pages with links to the tutorials.

mechanize seems the tool to use from what I found out. I tried this
code found on this list:

require ‘net/https’
require ‘uri’

class HTTPS < Net::HTTP
def initialize *a, &b
super
self.use_ssl = true
self.verify_mode = OpenSSL::SSL::VERIFY_NONE
self.set_debug_output $stderr
end
end

the url is of the form

url = ‘https://tutorials.siteofinterest.com/?page=tutorialView&tId=47’

uri = URI.parse url

HTTPS.start(uri.host, uri.port) do |https|
response = https.post(“#{uri.path}?#{uri.query}”)
puts response.body
end

-----error message —>
opening connection to tutorials.siteofinterest.com…
opened
opurl2.rb:18:in `post’: wrong number of arguments (1 for 2)
(ArgumentError)

I checked with IRB, uri seems to do its parsing, finding the host, the
path and the arguments after the question mark.

How can I find out what the second argument is that I need to put in?
or How can I find out what post Safari sends when I click on the same
link?

2

require ‘rubygems’
require ‘mechanize’

mech = WWW::Mechanize.new
page =
mech.get(‘https://tutorials.siteofinterest.com/?
page=tutorialView&tId=47’)

This code does return a page, but not the page I wanted. So there must
be something missing, a cookie or something which indicates I have
logged in.

If I try to access one of the links in ruby
mech.get(‘http://tutorials.siteofinterest.com/tutorials/Intro/Intface/
Intface.html’)
I get an error, please provide username and password
while if I paste the http address in safari, it opens the tutorial
page. How do I find out what the browser is doing to satisfy the
tutorial web page server?

3

On Mon, Jul 23, 2007 at 12:25:02AM +0900, anne001 wrote:

logged in.

If I try to access one of the links in ruby
mech.get(‘http://tutorials.siteofinterest.com/tutorials/Intro/Intface/
Intface.html’)
I get an error, please provide username and password
while if I paste the http address in safari, it opens the tutorial
page. How do I find out what the browser is doing to satisfy the
tutorial web page server?

Make sure to clear all your cookies before you do that. I recommend
getting Firefox, clearing your cookies, and using LiveHTTPHeaders.

http://livehttpheaders.mozdev.org/

Hope that helps.