How to properly format data when using TCPSocket.send()

Ruby
1

It would be greatly appreciated if someone would clue me in to what I am
doing wrong here. I am trying to send this data below to httpd(Apache)
however I am receiving a http response code of 400 (bad request).
Apache is logging “request failed: error reading the headers” in
error_log, however if I replace the \r\n with \x0d\x0a and send this
through netcat to Apache I get the proper response code I am looking
for.

http_socket = TCPSocket.new("#{i}", 80); http_socket.send(“PUT
/puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:
close\r\nContent-type: text/html\r\nContent-Length:
309\r\nSee Spot Run!See Spot
Run!See Spot Run!\r\n\r\n”, 0)

Thank you,

Nathan

2

On 12/29/06, Nathan T. [email protected] wrote:

http_socket = TCPSocket.new(“#{i}”, 80); http_socket.send("PUT
Posted via http://www.ruby-forum.com/.

Why are you putting HTML content in the header block? Try putting it
after
the \r\n\r\n that terminates the header. Also, you want to get the
content
length value correct. If that still doesn’t work, you could try POST
instead
of PUT (except you already implied that PUT works correctly on this
Apache
server with netcat).

3

On 29.12.2006 18:44, Nathan T. wrote:

/puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:
close\r\nContent-type: text/html\r\nContent-Length:
309\r\nSee Spot Run!See Spot
Run!See Spot Run!\r\n\r\n", 0)

You need “\r\n\r\n” to separate the header from the body. Your code
shows just a single pair.

Btw, why don’t you use Net::HTTP?

robert
4

On 12/29/06, Nathan T. [email protected] wrote:

Btw, why don’t you use Net::HTTP?
METHOD = ‘PUT’

I think you would do well to do some basic reading on HTTP. To
understand how headers work, you might even read RFC-822. HTTP doesn’t
use multiple TCP connections (not in the sense of relating them to
each other, at any rate), and I don’t know what you mean when you say
that it uses multiple TCP “sessions.”

5

Robert K. wrote:

On 29.12.2006 18:44, Nathan T. wrote:

/puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:
close\r\nContent-type: text/html\r\nContent-Length:
309\r\nSee Spot Run!See Spot
Run!See Spot Run!\r\n\r\n", 0)

You need “\r\n\r\n” to separate the header from the body. Your code
shows just a single pair.

Btw, why don’t you use Net::HTTP?

robert

I tried using “\r\n\r\n”, but I’m still having some problems due to the
way the HTTP protocol uses multiple TCP sessions and my lack of
understanding on how to handle that with Net::TCPSocket. Looks I will
have to use Net:HTTP. I am new to OO programming, and I don’t quite
understand this portion of the Net::HTTP::Put documentation.

Constants
METHOD = ‘PUT’
REQUEST_HAS_BODY = true
RESPONSE_HAS_BODY = true

I know that I should be passing a URI and some content to include in
the “PUT” request but I do not see any explanation of how to do that.
How do I know what parameters this class accepts?

6

Francis C. wrote:

On 12/29/06, Nathan T. [email protected] wrote:

Btw, why don’t you use Net::HTTP?
METHOD = ‘PUT’

I think you would do well to do some basic reading on HTTP. To
understand how headers work, you might even read RFC-822. HTTP doesn’t
use multiple TCP connections (not in the sense of relating them to
each other, at any rate), and I don’t know what you mean when you say
that it uses multiple TCP “sessions.”

I mean that a new TCP session is created for every HTTP request/response
pair. For example if you were to send two different GET requests you
would use two TCP sessions(3-way hand shake, and graceful close).
However, rethinking the situation this should not be a problem for what
I am trying to do. Even if the server responds with multiple response
codes it should be ok since they would be transported via the same TCP
session as the initial request sent from the client. Unfortunately the
packets generated and sent out on the wire from my code does not seem to
be waiting for the HTTP response data before it sends out a request for
a graceful close of the TCP session.

03:27:40.315919 IP 127.0.0.1.51200 > 127.0.0.1.80: P 1:257(256) ack 1
win 257 <nop,nop,timestamp 37976322 37976315>
0x0000: 4500 0134 529f 4000 4006 e922 7f00 0001
E…4R.@.@…"…
0x0010: 7f00 0001 c800 0050 5f61 2a67 5fe0 3c72
…P_ag_.<r
0x0020: 8018 0101 ff28 0000 0101 080a 0243 7902
…(…Cy.
0x0030: 0243 78fb 5055 5420 2f70 7574 7465 7374
.Cx.PUT./puttest
0x0040: 2e68 746d 6c20 4854 5450 2f31 2e30 0d0a
.html.HTTP/1.0…
0x0050: 486f 7374 3a20 3132 372e 302e 302e 310d
Host:.127.0.0.1.
0x0060: 0a41 6363 6570 743a 2069 6d61 6765 2f67
.Accept:.image/g
0x0070: 6966 2c20 696d 6167 652f 782d 7862 6974
if,.image/x-xbit
0x0080: 6d61 702c 2069 6d61 6765 2f6a 7065 672c
map,.image/jpeg,
0x0090: 2069 6d61 6765 2f70 6a70 6567 2c20 2a2f
.image/pjpeg,./
0x00a0: 2a0d 0a43 6f6e 7465 6e74 2d74 7970 653a
*…Content-type:
0x00b0: 2074 6578 742f 6874 6d6c 0d0a 436f 6e74
.text/html…Cont
0x00c0: 656e 742d 4c65 6e67 7468 3a20 3330 390d
ent-Length:.309.
0x00d0: 0a0d 0a3c 6874 6d6c 3e3c 6865 6164 3e53
…S
0x00e0: 6565 2053 706f 7420 5275 6e21 3c2f 6865
ee.Spot.Run!</he
0x00f0: 6164 3e3c 7469 746c 653e 5365 6520 5370
ad>See.Sp
0x0100: 6f74 2052 756e 213c 2f74 6974 6c65 3e3c
ot.Run!<
0x0110: 626f 6479 3e53 6565 2053 706f 7420 5275
body>See.Spot.Ru
0x0120: 6e21 3c2f 626f 6479 3e3c 2f68 746d 6c3e
n!
0x0130: 0d0a 0d0a …
03:27:40.316702 IP 127.0.0.1.80 > 127.0.0.1.51200: . ack 257 win 265
<nop,nop,timestamp 37976322 37976322>
0x0000: 4500 0034 4570 4000 4006 f751 7f00 0001
E…4Ep@.@…Q…
0x0010: 7f00 0001 0050 c800 5fe0 3c72 5f61 2b67
…P….<r_a+g
0x0020: 8010 0109 91bb 0000 0101 080a 0243 7902
…Cy.
0x0030: 0243 7902 .Cy.
03:28:17.925697 IP 127.0.0.1.51200 > 127.0.0.1.80: F 257:257(0) ack 1
win 257 <nop,nop,timestamp 38013926 37976322>
0x0000: 4500 0034 52a0 4000 4006 ea21 7f00 0001
E…4R.@.@…!..
0x0010: 7f00 0001 c800 0050 5f61 2b67 5fe0 3c72
…P_a+g.<r
0x0020: 8011 0101 fedd 0000 0101 080a 0244 0be6
…D…
0x0030: 0243 7902 .Cy.
03:28:17.932379 IP 127.0.0.1.80 > 127.0.0.1.51200: P 1:468(467) ack 258
win 265 <nop,nop,timestamp 38013932 38013926>
0x0000: 4500 0207 4571 4000 4006 f57d 7f00 0001
E…Eq@.@…}…
0x0010: 7f00 0001 0050 c800 5fe0 3c72 5f61 2b68
…P…_.<r_a+h
0x0020: 8018 0109 fffb 0000 0101 080a 0244 0bec
…D…
0x0030: 0244 0be6 4854 5450 2f31 2e31 2034 3033
.D…HTTP/1.1.403
0x0040: 2046 6f72 6269 6464 656e 0d0a 4461 7465
.Forbidden…Date
0x0050: 3a20 4672 692c 2032 3920 4465 6320 3230
:.Fri,.29.Dec.20
0x0060: 3036 2030 393a 3237 3a34 3020 474d 540d
06.09:27:40.GMT.
0x0070: 0a53 6572 7665 723a 2041 7061 6368 652f
.Server:.Apache/

See above where the client sends out a segment with the FIN bit set
before the response from the Apache server arrives. When I run this
code ruby just hangs. I suppose awaiting the HTTP response that will
never be sent up the stack.

This is the code I was attempting to read the response with.

puts http_socket.readlines; http_socket.close

This code works with two other requests I’ve setup, but not with my
attempt to send the PUT request. Perhaps there is something I am not
understanding here?

7

See above where the client sends out a segment with the FIN bit set
before the response from the Apache server arrives. When I run this
code ruby just hangs. I suppose awaiting the HTTP response that will
never be sent up the stack.

I guess a good question would be “does ruby’s TCPSocket take advantage
of a TCP half close and still received data after it sends a FIN
advising the other side that it will not be sending any more data?” I
have consulted the “TCP/IP Illustrated”, but it is apparently a decision
that is up to the application developer that does the implementation.
Perhaps my understanding is skewed and this is even a deeper question.
Does Ruby’s Net::TCPSocket class use the underlying OS’s TCP/IP stack?
I think that it would, but I am not yet skilled in the Ruby Way, or it’s
underlying code. I am still consulting RFC 2822 so my next post should
include more educated questions.

  • Nathan
8

On Dec 29, 2006, at 09:44, Nathan T. wrote:

http_socket = TCPSocket.new(“#{i}”, 80); http_socket.send(“PUT
/puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:
close\r\nContent-type: text/html\r\nContent-Length:
309\r\nSee Spot Run!See Spot
Run!See Spot Run!\r\n\r\n”, 0)

the net/http library handles this much better:

require ‘net/http’

data = “See Spot Run!See Spot Run!</
title>See Spot Run!”

uri = URI.parse ‘http://example.com/puttest.html’

Net::HTTP.start uri.host, uri.port do |http|
http.post uri.request_uri, data, ‘Content-Type’ => ‘text/html’
end

–
Eric H. - [email protected] - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

9

Nathan T. wrote:

/puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:
close\r\nContent-type: text/html\r\nContent-Length:
309\r\nSee Spot Run!See Spot
Run!See Spot Run!\r\n\r\n", 0)

Looks like you found your answer, but one tool for sockets programming,
if you aren’t already using it, is ‘nc’. It can be used as a ‘telnet’
replacement but more relevant to your case, you can make it into a web
(or any) server and see every byte your client is sending:

nc -l -k 3001

…will start your ‘server’. Then you set your client to connect to
localhost 3001, you can use nc as a client to test that as well:

echo hello | nc localhost 3001

10

I tried using “\r\n\r\n”, but I’m still having some problems due to the
way the HTTP protocol uses multiple TCP sessions and my lack of
understanding on how to handle that with Net::TCPSocket.

I just wanted to add some comments about using TCPSocket that might help
others as myself.

I found out that the problem I was having with TCPSocket was a problem
with the closing of the session. I was sending some data to craft an
HTTP SEARCH request using the format client = TCPSocket.new(host, 80);
data = "SEARCH /vroot/folder/ HTTP/1.1
Content-Type: text/xml
Content-Length: 253
Host: server.tld

<?xml version=\"1.0\"?>

<D:searchrequest xmlns:D = “DAV:”>
<D:sql>
SELECT “DAV:contentclass”, “DAV:displayname”
FROM “/vroot/folder/”
WHERE “DAV:ishidden” = false
AND “DAV:isfolder” = false
</D:sql>
</D:searchrequest>"

client.send(data, 0)
response = client.recvfrom(1024)
puts response
client.close

However this would just cause the script to hang at the command line and
the web server would never send it’s response until I sent an interrupt
to the running script. I realized through some tcpdump magic that the
web server would send the response after I killed the script so I
modified the end of the code to;

client.send(data, 0)
client.shutdown( how=1 )
response = client.recvfrom(1024)
puts response
client.close

It works… Everyone’s happy. This is just an FYI for anyone running in
to a similar problem implementing a network client, sorry for the
verboseness.

  • Nathan
11

This is so much easier with http-access2. It handles keep alive
connections, ssl, cookies, headers, pretty much everything you would
want without having to do all the hard work yourself.

Chris

12

Thank you for your example Eric. I was having problems figuring out
how to add the header values until I saw how you did it. I am up and
running with my script now. Thanks to all.