<?xml version="1.0" encoding="UTF-8"?>
<!--
  ! CCPL HEADER START
  !
  ! This work is licensed under the Creative Commons
  ! Attribution-NonCommercial-NoDerivs 3.0 Unported License.
  ! To view a copy of this license, visit
  ! http://creativecommons.org/licenses/by-nc-nd/3.0/
  ! or send a letter to Creative Commons, 444 Castro Street,
  ! Suite 900, Mountain View, California, 94041, USA.
  !
  ! You can also obtain a copy of the license at
  ! legal/CC-BY-NC-ND.txt.
  ! See the License for the specific language governing permissions
  ! and limitations under the License.
  !
  ! If applicable, add the following below this CCPL HEADER, with the fields
  ! enclosed by brackets "[]" replaced with your own identifying information:
  !      Portions Copyright [yyyy] [name of copyright owner]
  !
  ! CCPL HEADER END
  !
  !      Copyright 2011-2012 ForgeRock AS
  !    
-->
<chapter xml:id='chap-auditing'
 xmlns='http://docbook.org/ns/docbook'
 version='5.0' xml:lang='en'
 xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
 xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'
 xmlns:xlink='http://www.w3.org/1999/xlink'
 xmlns:xinclude='http://www.w3.org/2001/XInclude'>
 <title>Using Audit Logs</title>
 <indexterm>
  <primary>Audit logs</primary>
 </indexterm>

 <para>OpenIDM auditing can publish and log all relevant system activity to
 the targets you specify. Auditing can include data from reconciliation as
 a basis for reporting, to access details, to activity logs that capture
 operations on internal (managed) objects and external (system) objects.
 Auditing provides the data for all the relevant reports, including orphan
 account reports.</para>
 <!--
 For instructions on setting up an external reporting engine with OpenIDM,
 see the chapter on <link xlink:href="integrators-guide#chap-reporting"
 xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Generating
 Reports</citetitle></link>.
  -->

 <para>The auditing interface allows you to push auditing data to files and
 to the OpenIDM repository.</para>

 <section xml:id="audit-log-types">
  <title>Audit Log Types</title>

  <variablelist>
   <para>This section describes the types of audit log OpenIDM provides.</para>
   <varlistentry>
    <term>Access Log</term>
    <listitem>
     <para>OpenIDM writes messages concerning access to the REST API in this
     log.</para>
     <para>Default file: <filename>openidm/audit/access.csv</filename></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term>Activity Log</term>
    <listitem>
     <para>OpenIDM logs operations on internal (managed) and external (system)
     objects to this log type.</para>
     <para>Entries in the activity log contain identifiers both for the
     reconciliation or synchronization action that triggered the activity,
     and also for the original caller and the relationships between related
     actions.</para>
     <para>Default file: <filename>openidm/audit/activity.csv</filename></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term>Reconciliation Log</term>
    <listitem>
     <para>OpenIDM logs the results of a reconciliation run, including
     situations and the resulting actions taken to this log type. The activity
     log contains details about the actions, where log entries display parent
     activity identifiers,
     <literal>recon/<replaceable>reconID</replaceable></literal>.</para>
     <para>Default file: <filename>openidm/audit/recon.csv</filename></para>
    </listitem>
   </varlistentry>
  </variablelist>

  <para>Where an action happens the context of a higher level business
  function, the log entry points to a parent activity for the context.
  The relationships are hierarchical. For example, a synchronization operation
  could result from scheduled reconciliation for an object type. OpenIDM also
  logs the top level root activity with each entry, making it possible to
  query related activities.</para>
 </section>

 <section xml:id="audit-log-format">
  <title>Audit Log File Formats</title>                                                  

  <para>This section describes the audit log file formats to help you map
  these to the reports you generate.</para>

  <variablelist xml:id="audit-access-fields">
   <title>Access Log Fields</title>
   <para>Access messages are split into the following fields.</para>

   <varlistentry>
    <term><literal>"_id"</literal></term>
    <listitem>
     <para>UUID for the message object, such as
     <literal>"0419d364-1b3d-4e4f-b769-555c3ca098b0"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"action"</literal></term>
    <listitem>
     <para>Action requested, such as <literal>"authenticate"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"ip"</literal></term>
    <listitem>
     <para>IP address of the client. For access from the local host, this
     can appear for example as <literal>"0:0:0:0:0:0:0:1%0"</literal>.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"principal"</literal></term>
    <listitem>
     <para>Principal requesting the operation, such as
     <literal>"openidm-admin"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"roles"</literal></term>
    <listitem>
     <para>Roles associated with the principal, such as
     <literal>"[openidm-admin, openidm-authorized]"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"status"</literal></term>
    <listitem>
     <para>Result of the operation, such as <literal>"SUCCESS"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"timestamp"</literal></term>
    <listitem>
     <para>Time when OpenIDM logged the message, such as
     <literal>"2012-01-10T10:45:42"</literal></para>
    </listitem>
   </varlistentry>
  </variablelist>

  <variablelist xml:id="audit-activity-fields">
   <title>Activity Log Fields</title>
   <para>Activity messages are split into the following fields.</para>

   <varlistentry>
    <term><literal>"_id"</literal></term>
    <listitem>
     <para>UUID for the message object, such as
     <literal>"0419d364-1b3d-4e4f-b769-555c3ca098b0"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"action"</literal></term>
    <listitem>
     <para>Action performed, such as <literal>"create"</literal>. See
     the section on <link xlink:href="integrators-guide#audit-eventtypes"
     xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Event
     Types</citetitle></link> for a list.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"activityId"</literal></term>
    <listitem>
     <para>UUID for the activity corresponding to the UUID of the resource
     context</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"after"</literal></term>
    <listitem>
     <para>JSON representation of the object resulting from the activity</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"before"</literal></term>
    <listitem>
     <para>JSON representation of the object prior to the activity</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"message"</literal></term>
    <listitem>
     <para>Human readable text about the activity</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"objectId"</literal></term>
    <listitem>
     <para>Object identifier such as
     <literal>"managed/user/DDOE1"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"parentActionId"</literal></term>
    <listitem>
     <para>UUID of the action leading to the activity</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"requester"</literal></term>
    <listitem>
     <para>Principal requesting the operation</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"rev"</literal></term>
    <listitem>
     <para>Object revision number</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"rootActionId"</literal></term>
    <listitem>
     <para>UUID of the root cause for the activity. This matches a
     corresponding <literal>"rootActionId"</literal> in a reconciliation
     message.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"status"</literal></term>
    <listitem>
     <para>Result of the operation, such as <literal>"SUCCESS"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"timestamp"</literal></term>
    <listitem>
     <para>Time when OpenIDM logged the message, such as
     <literal>"2012-01-10T10:45:42"</literal></para>
    </listitem>
   </varlistentry>
  </variablelist>

  <variablelist xml:id="audit-recon-fields">
   <title>Reconciliation Log Fields</title>
   <para>Reconciliation messages are split into the following fields.</para>

   <varlistentry>
    <term><literal>"_id"</literal></term>
    <listitem>
     <para>UUID for the message object, such as
     <literal>"0419d364-1b3d-4e4f-b769-555c3ca098b0"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"action"</literal></term>
    <listitem>
     <para>Synchronization action, such as <literal>"CREATE"</literal>. See
     the section on <link xlink:href="integrators-guide#sync-actions"
     xlink:role="http://docbook.org/xlink/role/olink"
     ><citetitle>Actions</citetitle></link> for a list.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"ambiguousTargetObjectIds"</literal></term>
    <listitem>
     <para>When the situation is AMBIGUOUS or UNQUALIFIED and OpenIDM cannot
     distinguish between more than one target object, OpenIDM logs the
     identifiers of the objects in this field in comma-separated format. This
     makes it possible to figure out what was ambiguous afterwards.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"entryType"</literal></term>
    <listitem>
     <para>Kind of reconciliation log entry, such as <literal>"start"</literal>,
     or <literal>"summary"</literal>.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"message"</literal></term>
    <listitem>
     <para>Human readable text about the reconciliation action</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"reconciling"</literal></term>
    <listitem>
     <para>What OpenIDM is reconciling, 
     <literal>"source"</literal> for the first phase, <literal>"target"</literal> for the second phase</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"reconId"</literal></term>
    <listitem>
     <para>UUID for the reconciliation operation, which is the same for all
     entries pertaining to the reconciliation run</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"rootActionId"</literal></term>
    <listitem>
     <para>UUID of the root cause for the activity. This matches a
     corresponding <literal>"rootActionId"</literal> in an activity
     message.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"situation"</literal></term>
    <listitem>
     <para>The situation encountered. See
     the section describing <link xlink:href="integrators-guide#sync-situations"
     xlink:role="http://docbook.org/xlink/role/olink">sychronization
     situations</link> for a list.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"sourceObjectId"</literal></term>
    <listitem>
     <para>UUID for the source object.</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"status"</literal></term>
    <listitem>
     <para>Result of the operation, such as <literal>"SUCCESS"</literal></para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"targetObjectId"</literal></term>
    <listitem>
     <para>UUID for the target object</para>
    </listitem>
   </varlistentry>
   <varlistentry>
    <term><literal>"timestamp"</literal></term>
    <listitem>
     <para>Time when OpenIDM logged the message, such as
     <literal>"2012-01-10T10:45:42"</literal>.</para>
    </listitem>
   </varlistentry>
  </variablelist>
 </section>

 <section xml:id="audit-configuration">
  <title>Audit Configuration</title>

  <para>OpenIDM exposes the audit logging configuration under
  <literal>http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/openidm/config/audit</literal>
  for the REST API, and in the file <filename>conf/audit.json</filename> where
  you installed OpenIDM. The default <filename>conf/audit.json</filename> file
  contains the following object.</para>

  <programlisting language="javascript">
{
    "eventTypes": {
        "activity": {
            "filter": {
                "actions": [
                    "create",
                    "update",
                    "delete",
                    "patch",
                    "action"
                ]
            }
        },
        "recon": {}
    },
    "logTo": [
        {
            "logType": "csv",
            "location": "audit",
            "recordDelimiter": ";"
        },
        {
            "logType": "repository"
        }
    ]
}</programlisting>

  <section xml:id="audit-eventtypes">
   <title>Event Types</title>

   <para>The <literal>eventTypes</literal> configuration specifies what events
   OpenIDM writes to audit logs. OpenIDM supports two
   <literal>eventTypes</literal>: <literal>activity</literal> for the activity
   log, and <literal>recon</literal> for the reconciliation log. The filter
   for actions under activity logging shows the actions on managed or system
   objects for which OpenIDM writes to the activity log.</para>

   <variablelist>
    <para>The <literal>filter</literal> <literal>actions</literal> list lets
    you configure under what conditions OpenIDM writes to the activity
    log.</para>
    <varlistentry>
     <term><literal>read</literal></term>
     <listitem>
      <para>When an object is read by using its identifier</para>
     </listitem>
    </varlistentry>
    <varlistentry>
     <term><literal>create</literal></term>
     <listitem>
      <para>When an object is created.</para>
     </listitem>
    </varlistentry>
    <varlistentry>
     <term><literal>update</literal></term>
     <listitem>
      <para>When an object is updated.</para>
     </listitem>
    </varlistentry>
    <varlistentry>
     <term><literal>delete</literal></term>
     <listitem>
      <para>When an object is deleted.</para>
     </listitem>
    </varlistentry>
    <varlistentry>
     <term><literal>patch</literal></term>
     <listitem>
      <para>When an object is partially modified.</para>
     </listitem>
    </varlistentry>
    <varlistentry>
     <term><literal>query</literal></term>
     <listitem>
      <para>When a query is performed on an object.</para>
     </listitem>
    </varlistentry>
    <varlistentry>
     <term><literal>action</literal></term>
     <listitem>
      <para>When an action is performed on an object.</para>
     </listitem>
    </varlistentry>
   </variablelist>

   <variablelist>
    <para>The <literal>logTo</literal> list lets you define where OpenIDM
    writes the log.</para>
    <varlistentry>
     <term><literal>csv</literal></term>
     <listitem>
      <para>Write to a comma-separated variable format file in the location
      specified relative to the directory where you installed OpenIDM.</para>
      <para>Audit file names are fixed, <filename>access.csv</filename>,
      <filename>activity.csv</filename>, and
      <filename>recon.csv</filename>.</para>
     </listitem>
    </varlistentry>
    <varlistentry>
     <term><literal>repository</literal></term>
     <listitem>
      <para>Write to the OpenIDM database repository.</para>
      <para>OpenIDM stores entries under the
      <literal>/openidm/repo/audit/</literal> context, such entries appear as
      <literal>audit/access/<replaceable>_id</replaceable></literal>,
      <literal>audit/activity/<replaceable>_id</replaceable></literal>, and
      <literal>audit/recon/<replaceable>_id</replaceable></literal>, where
      the <replaceable>_id</replaceable> is the UUID of the entry such as
      <literal>0419d364-1b3d-4e4f-b769-555c3ca098b0</literal>.</para>
      <para>In the OrientDB repository, OpenIDM stores records in
      <literal>audit_activity</literal>, <literal>audit_activity</literal>,
      and <literal>audit_recon</literal> tables.</para>
      <para>In a JDBC repository, OpenIDM stores records in
      <literal>auditaccess</literal>, <literal>auditactivity</literal>, and
      <literal>auditrecon</literal> tables.</para>
     </listitem>
    </varlistentry>
   </variablelist>
  </section>
 </section>

 <section xml:id="audit-reports">
  <title>Generating Reports</title>

  <para>When generating reports from audit logs, you can correlate information
  from activity and reconciliation logs by matching the
  <literal>"rootActionId"</literal> on entries in both logs.</para>
  
  <para>The following MySQL query shows a join of the audit activity and
  audit reconciliation tables using root action ID values.</para>

  <screen width="95"><?dbfo pgwide="1"?>
mysql&gt; select distinct auditrecon.activity,auditrecon.sourceobjectid,
 auditrecon.targetobjectid,auditactivity.activitydate,auditrecon.status
 from auditactivity inner join auditrecon
 <emphasis role="strong">auditactivity.rootactionid=auditrecon.rootactionid</emphasis>
 where auditrecon.activity is not null group by auditrecon.sourceobjectid;
+----------+--------------------------+----------------------+---------------------+---------+
| activity | sourceobjectid           | targetobjectid       | activitydate        | status  |
+----------+--------------------------+----------------------+---------------------+---------+
| CREATE   | system/xmlfile/account/1 | managed/user/juser   | 2012-01-17T07:59:12 | SUCCESS |
| CREATE   | system/xmlfile/account/2 | managed/user/ajensen | 2012-01-17T07:59:12 | SUCCESS |
| CREATE   | system/xmlfile/account/3 | managed/user/bjensen | 2012-01-17T07:59:12 | SUCCESS |
+----------+--------------------------+----------------------+---------------------+---------+
3 rows in set (0.00 sec)</screen>
 </section>
</chapter>

