From CS 455 Spring 2013

CS455-S13: Program 3: HTTP Request Capture and HTTP Client

Assigned: Tuesday, March 26, 2013
Due: Tuesday, April 9, 2013 by 11:59pm

Description

The goal of this assignment is to allow you to demonstrate your knowledge of HTTP requests and HTTP responses. We'll be implementing an HTTP request capture program so that you can look at requests generated by your favorite web browser and a simple HTTP client so that you can build valid HTTP requests and process HTTP responses from real web servers.

Note: The two programs that you are writing (HTTP Request Capture and HTTP Client) are totally independent.

HTTP Request Capture

Use your HTTP Request Capture program with a web browser. Setup your capture program as a server on a CS Unix machine (use either atria or sirius). Use the hostname and port where you are running your capture program in the URL you type in your browser. For example, if you are running your capture program on atria port 10010, use something like http://atria:10010/my/url/ or http://atria.cs.odu.edu:10010/. The path doesn't matter because your capture program is not actually serving pages. There is no need to change any proxy settings on your browser.

• The HTTP request capture program must be named HTTPReqCap.java.
• It must accept a single command-line argument: port, which is the port that it will listen on for incoming HTTP requests.
  • As with the Programs 1 and 2, if any of the arguments are incorrect, exit after printing an error message of the form ERR - arg x, where x is the argument number.
  • The only error-checking that needs to be done on the port is to ensure it is a positive integer less than 65536.
  • Remember that only ports 10000-11000 are open for use.
• For each new HTTP request, print the client's IP address and port in the format IP:port

• Print each line in the HTTP request.
• You do not need to return anything to the client. (Note that this will typically cause the browser to display a blank page.)
• Close the socket after you have printed the entire HTTP request.
• To get the browser to request something from your server, use the hostname and port your server is running on in the URL.

• As with the Ping Server you wrote for Program 2, the HTTP request capture program should remain running until the user closes it with Ctrl-C. The program should close the client socket after each request has been received, but leave the welcoming/listening socket open.
• Since the CS servers only allow connections from on-campus, make sure that the browser you are testing with is on-campus (or you won't be able to reach your server program). You can run 'firefox &' from any of the CS Unix machines if you're working on this from home and need to test (make sure that you have X-Win32 or another XWindows server running at home first).

Hints:

• Look back in your notes to recall how HTTP requests are formatted and terminated.
• Note that readLine() in Java strips off newline characters before returning a String.
• Use the equals() method in Java to compare two Strings.

HTTP Client

Use the HTTP client to request data from a web server. Your client should be able to send requests and receive responses from any web server on the Internet. There is no need to write a server program to communicate with this client.

• The HTTP client program must be named HTTPClient.java.
• It must accept a single command-line argument: URL, which is the URL that it will request.
  • The URL will be given in the following format: http://hostname[:port][/path] (only lowercase letters will be used)
    • hostname - the web server's hostname
    • :port - an optional port
    • path - the path from the web server's main directory to the requested file, if this is not present, then the path is '/'
  • As with the Programs 1 and 2, if any of the arguments are incorrect, exit after printing an error message of the form ERR - arg x, where x is the argument number.
  • Treat any URL as valid as long as it starts with http://
• Connect to the host given in the URL at the port given in the URL (use 80, if no port is given).
  • Handle exceptions as you wish -- for example, if the host doesn't exist or the port given is not open
• Request the path/file given in the URL using HTTP/1.0.
  • Use the HEAD method so that only the HTTP response header will be returned.
  • Your request must include the Host field (with the proper entry).
  • Your request must also include the User-agent field with the entry 'ODU-CS455/555'.
  • No other request fields are needed.
• Print the HTTP request sent to the server.

• Parse the HTTP response header received from the server and print the following information:
  • Response code (number)
  • Server type
  • If the response is in the 200-level:
    • Last modified date
    • Number of bytes in the response data
  • If the response is in the 300-level:
    • The URL where the file is located
• Print the HTTP response header.

Rules

• The only networking classes allowed are the basic socket classes that we've used with the examples and Program 2. For example, java.net.URL is not allowed and urllib2 (Python) is not allowed.
• As with all projects, you are not permitted to work with anyone else (even students not in the class) - all of the coding and documentation must be your own.
• Your programs must compile and run on the CS Unix machines.
• You must write neat code and document it well. You will lose points for sloppy programs that contain little or no comments.

Testing

A large part of your program's grade will be determined by how well it handles a set of inputs. You should test your program rigorously before submitting. Because your programs will be run and tested using a script, you must format your output exactly as I have described or you will lose points.

The examples below are just examples. I will test your programs rigorously. In particular, I will test your HTTP Client on a wide range of URLs.

Example 1

java HTTPReqCap
Usage: java HTTPReqCap port

Example 2

atria> java HTTPReqCap 10003
128.82.4.118:33083
GET /my/url HTTP/1.1
Host: atria.cs.odu.edu:10003
User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20070606
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

Note that after setting up the server, the user opened a web browser to the URL http://atria.cs.odu.edu:10003/my/url

Example 3

java HTTPClient 
Usage: java HTTPClient URL

Example 4

java HTTPClient http://www.cs.odu.edu/~mweigle/files/foo.txt
HEAD /~mweigle/files/foo.txt HTTP/1.0
Host: www.cs.odu.edu
User-agent: ODU-CS455/555

200
Apache/2.2.17 (Unix) PHP/5.3.5 mod_ssl/2.2.17 OpenSSL/0.9.8q
Thu, 19 May 2011 19:23:43 GMT
92

HTTP/1.1 200 OK
Date: Thu, 21 Mar 2013 20:53:29 GMT
Server: Apache/2.2.17 (Unix) PHP/5.3.5 mod_ssl/2.2.17 OpenSSL/0.9.8q
Last-Modified: Thu, 19 May 2011 19:23:43 GMT
ETag: "5c-4a3a5f178cdd0"
Accept-Ranges: bytes
Content-Length: 92
Connection: close
Content-Type: text/plain

Example updated to include final CRLF after HTTP response. -MCW 3/31/13

Submission

You must name your source files HTTPReqCap.java and HTTPClient.java or HTTPReqCap.py and HTTPClient.py (note the capitalization). Make sure that you submit all files necessary to compile your program. But, do not submit compiled files (.class files). Do not submit compressed (.zip) files.

Directions for submitting your assignment through Blackboard