Reportcenter Query API

From ControlTier

Version

0.3

Last Modified Date

3/8/2008

Author

Greg Schueler <greg@controltier.com>

Summary

This document specifies the Reports Remote Query API for ControlTier 3.0rc2.

Reports can be viewed from the main page of the Workbench GUI after selecting aproject, and are messages generated by both the ControlTier Server and theControlTier Client.

The reports can also be queried by use of REST-ful API for report entriesmatching a set of query parameters. All of the fields of the reports canbe queried, and the query response will be returned in a simple XML documentformat.

Report Fields

The report tool uses Lucene to index and search the reports, and Antdepo uses Log4j to send them to the Reports tool of ControlTier Server.

The Lucene index uses a particular set of fields for each report:

These fields are present in every Report entry, however not all fields are usedin each entry. Depending on the "actionType" and "itemType", some fields will have different meanings as well.

Note: The following fields are stored in "keyword" form in the lucene index, meaning they are not textually tokenized to allow full searching ability. They can be searched for the exact value, but wildcard operators cannot be used.

• project
• maprefUri
• actionType
• itemType
• commandName
• controller

Report Entry Item Types and Action Types

Each report entry is generated by one of two events: a change to the Workbenchdata model due to user action, or a command execution in the CTL/Jobcenter environment. The type of the report entry can be determined by the itemType field.

For command execution reports, the itemType field will be"commandExec". For data model changes, the itemType field will be one of:"object","type","pattern",or "project", depending on the resource that was modified in the Workbench data model.

Further, the actionType field specifies what kind of action occurred to generate the report entry: For model changes, actionType is either "create","update" or "delete". For command executions, actionType is either "succeed" or"fail".

Query API

The format of the request and response are described below.

Request

The query is sent in a REST-ful manner using a HTTP GET request with URL parameters to specify the field query values, and retrieving the result set as adocument in the HTTP response.

Note: Before performing the query request, your HTTP user-agent must be authenticated to the ControlTier server. See: #Authentication.

The query URL is based on the location of your ControlTier CtlCenter server installation:

http://<host>:<port>/ctlcenter/reports/query?<params>

Where <host> and <port> are as described in this table:

The query fields and query values are specified as simple HTTP query parameters at the end of the URL. The names of the query parameters match the field names for Report entries, as described above. (See: #Report Fields.)

For each query, the following field query parameters are required:

In addition, at least one field in addition to project is required. This is to prevent the result set from being too large.

In addition to all of the Fields names, the following additional parameters are allowed in the query.

The sortBy parameter specifies that the results are sorted by that field. The sortOrder specifies how the sorting is ordered: ascending (incrementing) or descending (decrementing).

The reports entries can be filtered by date by specifying the "date", "dateStart" or "dateEnd" parameters. The value for these query parameters is a Unix epoch time in milliseconds. If you specify "date", then only entries matching that date exactly will be returned. "dateStart" filters the results to all entries at and after that timestamp, and "dateEnd" includes all entries at or before that timestamp.

The request parameter names and values must be formatted properly according to the HTTP spec. This means that characters like space (" ") and other characters that are special in the HTTP request and URL must be properly URL encoded. Normally a HTTP-access code library will do this encoding for you.

Wildcards

Wildcard operators can be used in the queries, for most of the fields. (Except the Keyword fields as described above.)

These operators are:

• "*": star, matches any sequence of zero or more characters
• "?": question-mark. matches a single character.

Thus you can search for any value starting with the string "blue", by using the query "blue*". Or, you can search for any three-letter word that starts with B and ends with D by searcing for "B?D".

Note: Wildcard characters cannot be used as the first character of the query.

Query Example

For example, suppose you wanted to query the following field values:

And your ControlTier Server exist on host ctierserv1 and port 8080.

They query would be constructed as so:

http://ctierserv1:8080/ctlcenter/reports/query?project=MyProject1&resourceName=testObject&resourceType=MyType&itemType=commandExec&actionType=succeed&reportId=20070313-*

XML Response Format

The Response to the query comes in the form of a simple XML document, with the following structure (See: #Result XML DTD) :

<reports-query-result>
  <reports count="N">
    <report-entry>
      <!-- report-entry contents -->
    </report-entry>
  </reports>
 </reports-query-result>

The <reports> element will contain the number of <report-entry> sub-elements int the "count" attribute.

The contents of the <report-entry> element are elements for each of the Fields described above. Some fields may or may not be present, depending on whether the report contained values for those fields, and the itemType and actionType of the report. These elements will always be present, however: date, author, project, itemType, actionType, action, message. The itemType and actionType fields are present as attributes of the <report-entry> element as well. The date element will also have an attribute named "time" that specifies the unix epoch timestamp in milliseconds.

The MIME type of the result will be: "text/xml;charset=UTF-8".

Example

<reports-query-result>
 <reports count="2">
    <report-entry itemType="commandExec" actionType="succeed">
      <date time="1174677709274">Mar 23, 2007 12:21:49 PM</date>
      <author>greg</author>
      <project>Test1</project>
      <itemType>commandExec</itemType>
      <resourceType>ProjectBuilder</resourceType>
      <resourceName>test</resourceName>
      <controller>Deployment</controller>
      <commandName>Install</commandName>
      <maprefUri>http://www.opendepo.org/ct/Test1/Objects#obj_83</maprefUri>
      <nodename>gozer</nodename>
      <reportId>LY_ctier_v1</reportId>
      <actionType>succeed</actionType>
      <action>Command Execution</action>
      <message>
        command completed successfully. Execution time: 0.722 sec
      </message>
    </report-entry>
 
    <report-entry itemType="object" actionType="update">
      <date time="1174677709257">Mar 23, 2007 12:21:49 PM</date>
      <author>default</author>
      <project>Test1</project>
      <itemType>object</itemType>
      <resourceType>ProjectBuilder</resourceType>
      <resourceName>test</resourceName>
      <maprefUri>http://www.opendepo.org/ct/Test1/Objects#obj_83</maprefUri>
      <actionType>update</actionType>
      <action>Update Object Referrers</action>
      <message>Modified parents</message>
    </report-entry>
 
  </reports>
</reports-query-result>

Errors

Successful requests will result in a 200 OK HTTP response. If there is an error with authentication, or an error processing the HTTP request, the web server will reply with a non-200 error code in the HTTP response.

If there is an error processing the query, or an error occurs during the query execution, then the <reports-query-result> element will have an attribute named "error" with a value of "true", and an element named <errors> will contain any error messages :

<reports-query-result error="true">
  <errors>
    <error>Error message</error>
  </errors>
</reports-query-result>

Authentication

Authentication is required to query the Reports API. A valid ControlTier Server username and password are required.

The authentication must occur prior to sending of the Query Request, and the Query Request must include the correct HTTP cookies to maintain the authentication that was previously carried out.

The ControlTier Client ("Commander") contains a set of Java classes that can perform this authentication using the Apache Commons HttpClient library. To use the Commander classes to authenticate, see the section #Authentication Through Commander.

Performing Authentication ControlTier server runs on the Tomcat 4.x servlet container application, and the servlet specification defines three ways that authentication can be performed. Of these, ControlTier Server uses the FORM-BASED authentication method.

This is the sequence for performing FORM-BASED authentication to the ControlTier Server:

1. Submit a HTTP-GET request for a URL (the Original URL) that requires authentication from the server.
   1. Instead of presenting the requested page, the server will redirect the HTTP client to the Logon URL, a page which includes a logon form.
   2. The server will also set a Cookie in the response containing a Session ID.
   3. Submit the Logon Request (see below)
      1. If the username and password are correct, the server will redirect the client back to the Original URL.
         1. Further authenticated requests can now be made to the server, as long as the correct Session ID is included as a Cookie.
         2. If the username and password are invalid, the server will redirect to the Logon Error URL, a page where the logon form is included again.

This authentication can be done programmatically, and is most easily done by use of a HTTP code library that can act as a User-Agent and correctly maintain the Cookies between requests.

Authentication In Detail

Definitions:

To perform Authentication do the following:

1. Send a HTTP-GET request for the START_URL to the server.
   1. The response should be one of:
      1. 200 (OK): then authentication has already occurred, and nothing more needs to be done.
      2. A Redirect Response (see below).
      3. If it is not one of the 300 status codes, then it is an error.
      4. The response header field "Location" should contain a URL X.
      5. X should be equivalent to LOGON_URL.
      6. The response should contain a Cookie named JSESSIONID. This Cookie must be stored and sent in subsequent requests to maintain the user session.
      7. Send a HTTP-GET request for X to the server.
         1. If the response STATUS code is not 200 (OK) it is an error.
      8. Send the Logon Request to the server using the USERNAME and PASSWORD values
         1. The response should be a Redirect Response.
         2. The "Location" header should contain a URL Y.
         3. Y should be equivalent to the START_URL. If so, authentication has been completed.
         4. If Y is equivalent to LOGON_ERROR_URL then the user authentication failed, and the username or password are incorrect.
         5. Any other response is an error.

After completing these steps, the Cookie which specifies the user session will allow further requests to ControlTier server to succeed without authentication.

Authentication Through Commander

The ControlTier Client (Commander) contains Java code which can perform the authentication steps described above, using the Apache Commons HttpClient library (3.x).

Library Dependencies:

• apache-commons-httpclient-3.0.jar
• apache-commons-codec-1.3.jar
• commander-3.0rc2.jar
• log4j-1.2.9.jar
• apache-commons-logging-1.0.4.jar

The Java class that is used is:

com.controltier.commander.utils.DefaultFormAuthenticator

You can create a new instance of this class by using the constructor:

DefaultFormAuthenticator(String username, String password);

You can then authenticate your HttpClient instance with the following method:

/**
* Authenticate the client http state so that the workbench requests can be made.
* @param baseURL base URL of the Workbench server, used to determine the server's protocol, host, and port.
* @param client HttpClient instance
* @return true if authentication succeeded.
* @throws HttpClientException
*/
public boolean authenticate(URL baseURL, HttpClient client) throws com.controltier.commander.utils.HttpClientException;

The Apache Commons HttpClient library's HttpClient (org.apache.commons.httpclient.HttpClient) class maintains inter-request state via a class named HttpState (org.apache.commons.httpclient.HttpState). By re-using the HttpClient instance, or at least the HttpState instance, once you authenticate to Workbench you will be able to make further requests without having to re-authenticate.

The following Java code shows an example of how to use the DefaultFormAuthenticator class:

HttpClient client = new HttpClient();
DefaultFormAuthenticator authenticator = new DefaultFormAuthenticator("user1","pass1");
              
//check that the state has a workbench session
URL reqUrl = new URL("http://ctierserv:8080/ctlcenter");
if (!authenticator.authenticate(reqUrl, getHttpClient())) {
  throw new AuthorizationFailureException("Unable to authenticate user: " + authenticator.getUsername() );
}
 
//now the HttpClient is authenticated.
 

Appendix

Redirect Response

A Redirect response has a STATUS code that should be one of:

• 302 (Moved Temporarily)
• 301 (Moved Permanently)
• 303 (See Other)
• 307 (Temporary Redirect)

It also has a Header field named "Location" that contains a URL or URL path to redirect to.

Logon Request

The Logon Request is a HTTP-POST request sent to the URL path "/j_security_check". The body of the request has the MIME type of "application/x-www-form-urlencoded", and be in the URL encoded format. The body should contain the following key/value pairs:

Result XML DTD

<xml version="1.0" encoding="UTF-8" ?>

<!ELEMENT reports-query-result (reports|errors)   >

<!ELEMENT reports (report-entry*) >
<!ATTLIST reports count CDATA #REQUIRED >

<!ELEMENT errors (error+) >

<!ELEMENT error (#PCDATA) >

<!ELEMENT report-entry (date,author,project,itemType,resourceType?,resourceName?,controller?,commandName?,maprefUri?,nodename?,actionType,action,message)   >
<!ATTLIST report-entry 
     itemType (object|type|project|pattern|commandExec) #REQUIRED 
     actionType (create|delete|update|fail|succeed) #REQUIRED >

<!ELEMENT date (#PCDATA) >
<!ATTLIST date time CDATA #REQUIRED >

<!ELEMENT author (#PCDATA) >
<!ELEMENT project (#PCDATA) >
<!ELEMENT itemType (#PCDATA) >
<!ELEMENT resourceType (#PCDATA) >
<!ELEMENT resourceName (#PCDATA) >
<!ELEMENT controller (#PCDATA) >
<!ELEMENT commandName (#PCDATA) >
<!ELEMENT maprefUri (#PCDATA) >
<!ELEMENT nodename (#PCDATA) >
<!ELEMENT actionType (#PCDATA) >
<!ELEMENT action (#PCDATA) >
<!ELEMENT message (#PCDATA) >

Full Code Example

 
/* file: TestReports.java 
 dependencies: 
  apache-commons-httpclient-3.0.jar 
  apache-commons-codec-1.3.jar 
  commander-3.0rc2.jar 
  log4j-1.2.9.jar
  apache-commons-logging-1.0.4.jar
*/
import com.controltier.commander.utils.DefaultFormAuthenticator;
import com.controltier.commander.utils.HttpClientException;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.methods.GetMethod;
 
import java.io.IOException;
import java.net.URL;
import java.net.URLEncoder;
import java.util.Iterator;
import java.util.Map;
import java.util.HashMap;
 
 
/**
  * TestReports demonstrates the Query Reports API of ControlTier Workbench.
  *
  * @author Greg Schueler <a href="mailto:greg@controltier.com">greg@controltier.com</a>
  */
public class TestReports {
       private static final String CTIER_SERVER = "localhost";
       private static final String CTIER_PORT = "8080";
       private static final String HTTP_PROTOCOL = "http";
 
       /**
         * main method
         *
         * @param args
         *
         * @throws IOException
         * @throws HttpClientException
         */
       public static void main(String[] args) throws IOException, HttpClientException {
 
               if (args.length < 2) {
                       System.err.println("usage: username password [param value ...]");
                       System.exit(2);
               }
               int c = 0;
               String username = args[c++];
               String password = args[c++];
               HashMap map = new HashMap();
               while (c + 1 < args.length) {
                       map.put(args[c++], args[c++]);
               }
               new TestReports(username, password).query(map);
       }
 
       DefaultFormAuthenticator auth;
       HttpClient client;
 
       /**
         * Constructor
         *
         * @param username controltier username
         * @param password controltier password
         */
       public TestReports(String username, String password) {
               this.client = new HttpClient();
               this.auth = new DefaultFormAuthenticator(username, password);
       }
 
       public static String QUERY_URL = HTTP_PROTOCOL + "://" + CTIER_SERVER + ":" + CTIER_PORT
                                                                         + "/ctlcenter/reports/query";
 
 
       public void query(Map params) throws HttpClientException, IOException {
               String request = constructURLQuery(QUERY_URL, params);
               auth.authenticate(new URL(request), client);
               HttpMethod method = new GetMethod(request);
 
               int resultCode;
               String resultType = null;
               String expectedContentType = "text/xml";
               StringBuffer results = new StringBuffer();
               try {
                       resultCode = client.executeMethod(method);
                       if (200 != resultCode) {
                               System.err.println("Bad Result Code: " + resultCode);
                               return;
                       }
 
                       if (null != method.getResponseHeader("Content-Type")) {
                               resultType = method.getResponseHeader("Content-Type").getValue();
                               String[] ar = resultType.split(";");
                               if (null != ar && ar.length > 1) {
                                       resultType = ar[0];
                               }
                       }
                       if (null == expectedContentType || expectedContentType.equals(resultType)) {
 
                               results.append(method.getResponseBodyAsString());
                       } else {
                               System.err.println("Content-Type was not expected: " + resultType);
                       }
               } catch (HttpException e) {
                       e.printStackTrace();
               }
               method.releaseConnection();
               System.out.println(results.toString());
       }
 
 
       /**
         * Create new URL with query parameters
         */
       private String constructURLQuery(String urlbase, Map query) {
               StringBuffer sb = new StringBuffer(urlbase);
               sb.append("?");
               for (Iterator i = query.entrySet().iterator(); i.hasNext();) {
                       Map.Entry entry = (Map.Entry) i.next();
                       String key = (String) entry.getKey();
                       Object val = entry.getValue();
                       if (null == val) {
                               val = "";
                       }
                       try {
                               sb.append(URLEncoder.encode(key, "UTF-8"))
                                       .append("=")
                                       .append(URLEncoder.encode(val.toString(), "UTF-8"));
                       } catch (java.io.UnsupportedEncodingException exc) {
                               throw new RuntimeException("URLEncoder failed on UTF-8 encoding");
                       }
                       if (i.hasNext()) {
                               sb.append("&");
                       }
               }
               return sb.toString();
       }
}