Learning how to perform HTTP POST requests

A POST request is another very common HTTP server request used by many APIs. We will be mining the University of Virginia directory search. When sending a POST request for a search query, the Lightweight Directory Access Protocol (LDAP) server replies with a web page of search results.

Getting ready

For this recipe, access to the Internet is necessary.

Install the HandsomeSoup CSS selector package, and also install the HXT library if it is not already installed:

$ cabal install HandsomeSoup
$ cabal install hxt

How to do it...

1. Import the following libraries:

   import Network.HTTP
   import Network.URI (parseURI)
   import Text.XML.HXT.Core
   import Text.HandsomeSoup
   import Data.Maybe (fromJust)

2. Define the POST request specified by the directory search website. Depending on the server, the following POST request details would be different. Refer to the following code snippet:

   myRequestURL = "http://www.virginia.edu/cgi-local/ldapweb"
   
   myRequest :: String -> Request_String
   myRequest query = Request { 
       rqURI = fromJust $ parseURI myRequestURL
     , rqMethod = POST
     , rqHeaders = [ mkHeader HdrContentType "text/html"
                   , mkHeader HdrContentLength $ show $ length body ]
     , rqBody = body
     }
     where body = "whitepages=" ++ query

3. Define and implement main to run the POST request on a query as follows:

   main :: IO ()
   main = do
     response <- simpleHTTP $ myRequest "poon"

4. Gather the HTML and parse it:

     html <- getResponseBody response
     let doc = readString [withParseHTML yes, withWarnings no] html

5. Find the table rows and print it out using the following:

     rows <- runX $ doc >>> css "td" //> getText
     print rows

Running the code will display all search results relating to "poon", such as "Poonam" or "Witherspoon".

How it works...

A POST request needs the specified URI, headers, and body. By filling out a Request data type, it can be used to establish a server request.

See also

Refer to the Understanding how to perform HTTP GET requests recipe for details on how to perform a GET request instead.