Problem: open-uri blocking redirection from http to https

Ruby
1

Hello,

I was working on a small script to verify the presence of an element
through a list of URL, some of these URLs have a redirections from http
to
https, when the script crawled into them I got the following error:

/usr/lib/ruby/1.9.1/open-uri.rb:216:in `open_loop’: redirection
forbidden:
http://beta.carsdirect.com/auto-loans/finance-app →
https://beta.carsdirect.com/auto-loans/finance-app (RuntimeError)

I understand that this is intentional, as per the comments on
open-uri.rb

 # This test is intended to forbid a redirection from http://... to
 # file:///etc/passwd.
 # https to http redirect is also forbidden intentionally.
 # It avoids sending secure cookie or referer by non-secure HTTP

protocol.
# (RFC 2109 4.3.1, RFC 2965 3.3, RFC 2616 15.1.3)
# However this is ad hoc. It should be extensible/configurable.

This mentions that “https to http” redirects are forbidden
intentionally,
but redirections from “http to https” are also blocked.

Is there a way to override this security check? currently I had to
change the
following line in the library to allow “http to https” re-directions:

(/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~
uri2.scheme)

to

(/\A(?:http|ftp|https)\z/i =~ uri1.scheme && /\A(?:http|ftp|https)\z/i
=~
uri2.scheme)

Thanks,
Xavi

2

2011/2/18 Xavier Del C. [email protected]:

However this is ad hoc. It should be extensible/configurable.

This mentions that “https to http” redirects are forbidden intentionally,
but redirections from “http to https” are also blocked.

Is there a way to override this security check? currently I had to change
the
following line in the library to allow “http to https” re-directions:

(/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~ uri2.scheme)

Currently it is not configurable (as the comment says) except monkey
patching.

Maybe open-uri should have some hooks.