Ruby's HTTP Client Sucks

I implemented what's up in ruby since it seemed to have some of the best XMPP support I know of. I also got to learn the datamapper API, which is alright.

Ruby's HTTP client, however, really sucks. I have to imagine someone else has known how much it sucks, but I haven't found much talking about it, or any solutions to the problems. For a reason simple example, consider the following code:

#!/usr/bin/env ruby

require 'net/http'

u = URI.parse $*[0]
puts Net::HTTP.start(u.host, u.port) { |h| h.get u.path }

There are some URLs I simply can't get that thing to deal with. Examples: http://digg.com/ and http://bleu.west.spy.net/diggwatch/comments/dlsspy (these two fail in different ways, but work fine in browsers).

Is this really as bad as it seems to be, or am I just doing it wrong?

Update: Sun Jul 6 16:50:24 PDT 2008

In these two examples, there seems to be something wrong on the server side. digg gets very upset if you don't send a user agent and just kind of hangs for a while. That's...odd. diggwatch (a service I have running here) is emitting incorrect responses through lighttpd. It seems that lighttpd's proxy module is just...wrong (service works fine without lighttpd in the way).