<?xml version="1.0" encoding="ISO-8859-1"?>

<document>
  
  <properties>
    <title>HttpClient exception handling guide</title>
    <author email="oleg -at- ural.ru">Oleg Kalnichevski</author>
    <revision>$Id$</revision>
  </properties>

  <body>

    <section name="Exception handling">
        <p>
            There are two main type of exceptions that the user of HttpClient may encounter 
            when executing HTTP methods:
            <ol>
                <li><strong>transport exceptions</strong></li>
                <li><strong>protocol exceptions</strong></li>
            </ol>
            Not all of these exceptions will be propagated to the user in regular HttpClient use.  
            Exceptions handled internally by HttpClient are marked below as <b>INTERNAL</b>.
        </p>
        <ul>
            <li><a href="#Transport exceptions">Transport exceptions</a></li>
            <li><a href="#Protocol exceptions">Protocol exceptions</a></li>
            <li><a href="#HTTP transport safety">HTTP transport safety</a></li>
            <li><a href="#Automatic exception recovery">Automatic exception recovery</a></li>
            <li><a href="#Custom exception handler">Custom exception handler</a></li>
        </ul>
    </section>
    <section name="Transport exceptions">
     <p>
      Transport exceptions are those caused by input/output failures such as an unreliable connection 
      or an inability to complete the execution of an HTTP method within the given time constraint 
      (socket timeout). Generally transport exceptions are non-fatal and may be recovered from by 
      retrying the failed method. However, special care must be taken when recovering from 
      exceptions in non-idempotent methods (refer to <a href="#HTTP transport safety">HTTP transport safety</a> 
      for details). 
     </p>
     <subsection name="java.io.IOException">
     <p>
      Generic transport exceptions in HttpClient are represented by the standard Java 
      java.io.IOException class or its sub classes such as java.net.SocketException and
      java.net.InterruptedIOException.
     </p>
     <p>
      In addition to standard input/output exception classes HttpClient defines several custom transport 
      exceptions that convey HttpClient specific information.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.NoHttpResponseException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.NoHttpResponseException]]></source>
     <p>
      In some circumstances, usually when under heavy load, the web server may be able to receive 
      requests but unable to process them.  A lack of sufficient resources like worker threads is a good
      example. This may cause the server to drop the connection to the client 
      without giving any response. HttpClient throws NoHttpResponseException when it encounters 
      such a condition. In most cases it is safe to retry a method that failed with 
      NoHttpResponseException.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.ConnectTimeoutException">
      <source><![CDATA[
java.io.IOException
  +- java.io.InterruptedIOException
    +- org.apache.commons.httpclient.ConnectTimeoutException]]></source>
     <p>
      This exception signals that HttpClient is unable to establish a connection with the target 
      server or proxy server within the given period of time.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.ConnectionPoolTimeoutException">
      <source><![CDATA[
java.io.IOException
  +- java.io.InterruptedIOException
    +- org.apache.commons.httpclient.ConnectTimeoutException
      +- org.apache.commons.httpclient.ConnectionPoolTimeoutException]]></source>
     <p>
      This exception can only occur when using the multithreaded connection manager. The exception 
      signals that the connection manager fails to obtain a free connection from the connection pool 
      within the given period of time.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.HttpRecoverableException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.HttpRecoverableException]]></source>
     <p>
      Deprecated and no longer thrown any of the standard HttpClient classes.
     </p>
     </subsection>
    </section>
    <section name="Protocol exceptions">
     <p>
      Protocol exceptions generally indicate logical errors caused by a mismatch between the client 
      and the server (web server or proxy server) in their interpretation of the HTTP specification. 
      Usually protocol exceptions cannot be recovered from without making adjustments to either 
      the client request or the server. Some aspects of the HTTP specification allow for different, 
      at times conflicting, interpretations. HttpClient can be configured to support different degrees 
      of HTTP specification compliance varying from very lenient to very strict. 
     </p>
     <subsection name="org.apache.commons.httpclient.HttpException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException]]></source>
     <p>
      HttpException represents an abstract logical error in HttpClient. Generally this kind of exception
      cannot be automatically recovered from.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.ProtocolException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException]]></source>
     <p>
      ProtocolException signals a violation of the HTTP specification. It is important to note that HTTP 
      proxies and HTTP servers can have different level of HTTP specification compliance. It may be 
      possible to recover from some HTTP protocol exceptions by configuring HttpClient to be more 
      lenient about non-fatal protocol violations.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.auth.MalformedChallengeException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException
      +- org.apache.commons.httpclient.auth.MalformedChallengeException]]></source>
     <p><b>INTERNAL</b></p>
     <p>
      MalformedChallengeException signals that an authentication challenge is in some way invalid or 
      illegal in the given authentication context.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.auth.AuthenticationException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException
      +- org.apache.commons.httpclient.auth.AuthenticationException]]></source>
     <p><b>INTERNAL</b></p>
     <p>
      AuthenticationException signals a failure in the authentication process. Usually authentication 
      exceptions are handled internally when executing HTTP methods and are not propagated to the 
      caller.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.auth.AuthChallengeException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException
      +- org.apache.commons.httpclient.auth.AuthenticationException
        +- org.apache.commons.httpclient.auth.AuthChallengeException]]></source>
     <p><b>INTERNAL</b></p>
     <p>
      AuthenticationException is thrown when HttpClient is unable to respond to any of the authentication 
      challenges sent by the server.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.auth.CredentialsNotAvailableException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException
      +- org.apache.commons.httpclient.auth.AuthenticationException
        +- org.apache.commons.httpclient.auth.CredentialsNotAvailableException]]></source>
     <p><b>INTERNAL</b></p>
     <p>
      CredentialsNotAvailableException indicates that credentials required to respond to the authentication
      challenge are not available.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.auth.InvalidCredentialsException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException
      +- org.apache.commons.httpclient.auth.AuthenticationException
        +- org.apache.commons.httpclient.auth.InvalidCredentialsException]]></source>
     <p><b>INTERNAL</b></p>
     <p>
      InvalidCredentialsException indicates that the credentials used to respond to the authentication
      challenge have been rejected by the server.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.cookie.MalformedCookieException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException
      +- org.apache.commons.httpclient.cookie.MalformedCookieException]]></source>
     <p><b>INTERNAL</b></p>
     <p>
      MalformedCookieException signals that the cookie is in some way invalid or illegal in the given 
      HTTP session context.
     </p>
     <p>
      There are several cookie specifications that are often incompatible. Thus the validity of 
      a cookie is established within a context of a specific cookie specification used to parse 
      and validate the cookie header(s) sent by the server. If the application needs to process cookies
      differently from the commonly used cookie specifications, it may choose to provide a 
      custom cookie policy or extend the existing one.  Please see <a href="cookies.html">cookies</a> 
      for more details.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.RedirectException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.ProtocolException
      +- org.apache.commons.httpclient.RedirectException]]></source>
     <p>
      RedirectException signals violation of the HTTP specification caused by an invalid 
      redirect response. If the application that uses HttpClient needs to be more lenient
      about redirect responses, it may choose to disable automatic redirect processing and implement
      a custom redirect strategy.
     </p>
     </subsection>
     <subsection name="org.apache.commons.httpclient.URIException">
      <source><![CDATA[
java.io.IOException
  +- org.apache.commons.httpclient.HttpException
    +- org.apache.commons.httpclient.URIException]]></source>
     <p>
      URIException is thrown when the request URI violates the URI specification.
     </p>
     </subsection>
   </section>
   <section name="HTTP transport safety">
     <p>
     It is important to understand that the HTTP protocol is not well suited for all types of applications. 
     HTTP is a simple request/response oriented protocol which was initially designed to support static 
     or dynamically generated content retrieval. It has never been intended to support transactional 
     operations. For instance, the HTTP server will consider its part of the contract fulfilled if it 
     succeeds in receiving and processing the request, generating a response and sending a status code back
     to the client. The server will make no attempts to roll back the transaction if the client fails to 
     receive the response in its entirety due to a read timeout, a request cancellation or a system crash. 
     If the client decides to retry the same request, the server will inevitably end up executing the same 
     transaction more than once. In some cases this may lead to application data corruption or inconsistent 
     application state.
     </p>
     <p>
      Even though HTTP has never been designed to support transactional processing, it can still be used
      as a transport protocol for mission critical applications provided certain conditions are met. To 
      ensure HTTP transport layer safety the system must ensure the idempotency of HTTP methods on the 
      application layer.
     </p>
     <subsection name="Idempotent methods">
       <p>HTTP/1.1 specification defines idempotent method as</p>
        <blockquote> 
         <p>
          Methods can also have the property of "idempotence" in that (aside from error or expiration 
          issues) the side-effects of N > 0 identical requests is the same as for a single request. 
         </p>
        </blockquote>
       <p>
         In other words the application ought to ensure that it is prepared to deal with the
         implications of multiple execution of the same method. This can be achieved, for instance,
         by providing a unique transaction id and by other means of avoiding execution of the same 
         logical operation.
        </p>
       <p>
         Please note that this problem is not specific to HttpClient. Browser based applications
         are subject to exactly the same issues related to HTTP methods non-idempotency.
        </p>
     </subsection>
   </section>
    <section name="Automatic exception recovery">
     <p>
     By default HttpClient attempts to automatically recover from exceptions. The default 
     auto-recovery mechanism is limited to just a few exceptions that are known to be safe.
     </p>
     <p>
     HttpClient will make no attempt to recover from any logical or HTTP protocol error (those derived
     from HttpException class).
     </p>
     <p>
     HttpClient will automatically retry up to 5 times those methods that fail with a transport exception 
     while the HTTP request is still being transmitted to the target server (i.e. the request has 
     not been fully transmitted to the server).
     </p>
     <p>
     HttpClient will automatically retry up to 3 times those methods that have been fully transmitted to 
     the server, but the server failed to respond with an HTTP status code (the server simply drops the 
     connection without sending anything back). In this case it is assumed that the request has not been 
     processed by the server and the application state has not changed. If this assumption may not hold 
     true for the web server your application is targeting it is highly recommended to provide a custom 
     exception handler.
     </p>
   </section>
    <section name="Custom exception handler">
     <p>
     In order to enable a custom exception recovery mechanism one should provide an implementation
     of the <a href="apidocs/org/apache/commons/httpclient/HttpMethodRetryHandler.html">
     HttpMethodRetryHandler</a> interface.
     </p>
      <source><![CDATA[
HttpClient client = new HttpClient();

HttpMethodRetryHandler myretryhandler = new HttpMethodRetryHandler() {
    public boolean retryMethod(
        final HttpMethod method, 
        final IOException exception, 
        int executionCount) {
        if (executionCount >= 5) {
            // Do not retry if over max retry count
            return false;
        }
        if (exception instanceof NoHttpResponseException) {
            // Retry if the server dropped connection on us
            return true;
        }
        if (!method.isRequestSent()) {
            // Retry if the request has not been sent fully or
            // if it's OK to retry methods that have been sent
            return true;
        }
        // otherwise do not retry
        return false;
    }
};
        
GetMethod httpget = new GetMethod("http://www.whatever.com/");
httpget.getParams().
    setParameter(HttpMethodParams.RETRY_HANDLER, myretryhandler);
try {
    client.executeMethod(httpget);
    System.out.println(httpget.getStatusLine().toString());
} finally {
    httpget.releaseConnection();
}]]></source>
   </section>
  </body>
</document>
