Synchronization The central function of OpenIDM is synchronizing identity data from different resources. This chapter explains what you must know to get started configuring OpenIDM's flexible synchronization mechanism, and illustrates the concepts with examples.

Synchronization Direct (push) Synchronization happens either when OpenIDM receives a change directly, or when OpenIDM discovers a change on an external resource. For direct changes to OpenIDM, OpenIDM immediately pushes updates to all external resources configured to receive the updates. A direct change can originate not only as a write request through the REST interface, but also as an update resulting from reconciliation with another resource. OpenIDM discovers changes on external resources through reconciliation, and through LiveSync. Reconciliation Reconciliation In identity management, reconciliation is the process of bidirectional synchronization of objects between different data stores. Reconciliation applies chiefly to user objects, though OpenIDM can reconcile any objects, including groups and roles. To perform reconciliation, OpenIDM analyzes both source and target systems to uncover the differences that it must reconcile. Reconciliation can therefore be a heavyweight process. When working with large data sets, finding all changes can be more work than processing the changes. Reconciliation is, however, thorough. It recognizes system error conditions and catches changes that might be missed by the more lightweight LiveSync mechanism. Reconciliation therefore serves as the basis for compliance and reporting functionality. LiveSync LiveSync LiveSync performs the same job as reconciliation. LiveSync relies on a change log on the external resource to determine which objects have changed. LiveSync is intended to react quickly to changes as they happen. LiveSync is however a best effort mechanism that in some cases can miss changes. Furthermore, not all resources provide the change log mechanism that LiveSync requires. The change long provides OpenIDM with a list of objects changed since the last request such that OpenIDM does not need to scan all objects for changes. OpenDJ provides an external change log. Active Directory also provides a change log. In all cases, OpenIDM relies on mappings that you configure to determine what to synchronize and how to carry out synchronization. LiveSync relies on the set of mappings that you configure once per OpenIDM server, whereas reconciliation also allows you to configure specific mappings for a particular reconciliation job. You must trigger OpenIDM to poll for changes on external resources, usually by scheduling reconciliation or LiveSync as described in the chapter on Scheduling Synchronization. Alternatively, you can start reconciliation through the REST interface. $ curl --header "X-OpenIDM-Username: openidm-admin" --header "X-OpenIDM-Password: openidm-admin" --request POST "http://localhost:8080/openidm/sync?_action=recon&mapping=systemLdapAccounts_managedUser"

Objects Managed objects Identity management software tends to favor either a meta-directory data model, where all data are mirrored in a central repository, or a virtual data model, where only a minimum set of attributes are stored centrally, and most are loaded on demand from the external resources on which they are stored. The meta-directory model offers fast access at the risk of getting out-of-date data. The virtual model guarantees fresh data, but pays for that guarantee in terms of performance. OpenIDM leaves the data model choice with you. You determine the right trade offs for a particular deployment. OpenIDM does not hard code any particular schema or set of attributes stored in the repository. Instead, you define how external system objects map onto managed objects, and OpenIDM dynamically updates the repository to store the managed object attributes that you configure. You can, for example, choose to follow the data model defined in the Simple Cloud Identity Management (SCIM) specification. The following object represents a SCIM user. { "userName": "james1", "familyName": "Berg", "givenName": "James", "email": [ "james1@example.com" ], "description": "Created by OpenIDM REST.", "password": "asdfkj23", "displayName": "James Berg", "phoneNumber": "12345", "employeeNumber": "12345", "userType": "Contractor", "title": "Vice President", "active": true } Concerning attribute names there is one restriction which is the use of "-" characters. Attribute names like given-name must be avoided since they can cause problems when used in java script!

Data flow for synchronization involves three types of configuration files, two of which you typically edit, and also a links table that OpenIDM maintains in its repository, as well as scripts needed to check objects and manipulate attributes. The two types of configuration files you edit are the connector configuration files, with one file per external resource, and the synchronization mappings file, with one file per OpenIDM instance. Connector configuration files Synchronization Connectors Connector configuration files are described in the chapter on Connecting to External Resources. Connector configuration files are named openidm/conf/provisioner.resource-name.json, where resource-name reflects the connector technology and external resource, such as openicf-xml. An excerpt from an example connector configuration follows. The example shows the name for the connector and two attributes of an account object type. In the attribute mapping definitions, the attribute name is mapped from the nativeName, the attribute name used on the external resource, to the attribute name used in OpenIDM. Thus the example shows that sn from LDAP is mapped to lastName in OpenIDM. The homePhone attribute can have multiple values. { "name": "MyLDAP", "objectTypes": { "account": { "lastName": { "type": "string", "required": true, "nativeName": "sn", "nativeType": "string" }, "homePhone": { "type": "array", "items": { "type": "string", "nativeType": "string" }, "nativeName": "homePhone", "nativeType": "string" } } } } In order for OpenIDM to access external resource objects and attributes, the object and its attributes must match the connector configuration. Also, the connector file strictly maps external resource objects to OpenIDM objects. To construct attributes and to manipulate their values, you use the synchronization mappings file. Synchronization mappings file Synchronization Mappings Mappings The synchronization mappings file is openidm/conf/sync.json. The synchronization mappings represent the core configuration for OpenIDM synchronization. The sync.json file describes a set of mappings. Each mapping specifies how attributes from source objects correspond to attributes on target objects. The source and target indicate the direction for the data flow, so you must define a mapping for each data flow. For example, if you want data flows from an LDAP server to the repository and also from the repository to the LDAP server, you must define two separate mappings. You identify external resource sources and targets as system/name/object-type, where name is the name used in the connector configuration file, and object-type is the object defined in the connector configuration file list of object types. For objects in OpenIDM's internal repository, you use managed/object-type, where object-type is defined in openidm/conf/managed.json. The name for the mapping by convention is set to a string of the form source_target, as shown in the following example. { "mappings": [ { "name": "systemLdapAccounts_managedUser", "source": "system/MyLDAP/account", "target": "managed/user", "properties": [ { "target": "familyName", "source": "lastName" }, { "target": "homePhone", "source": "homePhone" }, { "target": "phoneExtension", "default": "0047" }, { "target": "mail", "comment": "Set mail if non-empty.", "source": "email", "condition": { "type": "text/javascript", "source": "(source.email != null)" } }, { "target": "displayName", "source": ""; "transform": { "type": "text/javascript", "source": "(source.lastName +', ' + source.firstName;)" } } ] } ] } In this example, the source is the external resource, MyLDAP, and the target is OpenIDM's repository, specifically the managed user objects. The properties reflect OpenIDM attribute names. For example, the mapping has the attribute lastName defined in the MyLDAP connector configuration file mapped to familyName in the OpenIDM managed user object. Notice that the attribute names come from the connector configuration, rather than the external resource itself. Synchronization Creating attributes You can create attributes on the target as part of the mapping. In the example, OpenIDM creates a phoneExtension attribute with a default value of 0047. Synchronization Conditions You can also set up conditions under which OpenIDM maps attributes as shown for the email attribute in the example. By default, OpenIDM synchronizes all attributes. In the example, the mail attribute is set only if the script for the condition returns true. Synchronization Transforming attributes OpenIDM lets you transform attributes as well. In the example, the displayName attribute is set using a combination of the lastName and firstName attribute values from the source. For transformations, the source property is optional. However, the source object is only available when you specify the source property. Therefore, in order to use source.lastName and source.firstName to calculate the displayName, the example specifies "source" : "". To add a flow from the repository to MyLDAP, you would define a mapping with source managed/user and target system/MyLDAP/account, named for example managedUser_systemLdapAccounts. OpenIDM name space and object paths The image shows paths to objects in the OpenIDM name space. OpenIDM stores managed objects in the repository, and exposes them under /openidm/managed. System objects on external resources are exposed under /openidm/system. Synchronization Filtering By default, OpenIDM synchronizes all objects matching those defined in the connector configuration for the resource. Many connectors let you limit the scope of objects the connector accesses. For example, the LDAP connector lets you specify base DNs and LDAP filters so you need not access every entry in the directory. Yet, OpenIDM also lets you filter what is considered a valid source or valid target for synchronization by using JavaScript code. To apply these filters, use the validSource, and validTarget properties in your mapping. validSource A script that determines if a source object is valid to be mapped. The script yields a boolean value: true indicates the source object is valid; false can be used to defer mapping until some condition is met. In the root scope, the source object is provided in the "source" property. If the script is not specified, then all source objects are considered valid. { "validTarget": { "type": "text/javascript", "source": "target.employeeType == 'internal'" } } validTarget A script used during reconciliation that determines if a target object is valid to be mapped. The script yields a boolean value: true indicates the target object is valid; false indicates that the target object should not be included in reconciliation. In the root scope, the source object is provided in the "target" property. If the script is not specified, then all target objects are considered valid for mapping. { "validSource": { "type": "text/javascript", "source": "source.ldapPassword != null" } } During synchronization, your scripts always have access to a source object and a target object. Examples already shown in this section use source.attributeName to retrieve attributes from the source objects. Your scripts can also write to target attributes using target.attributeName syntax. { "onUpdate": { "type": "text/javascript", "source": "if ((source.email != null) {target.mail = source.email;}" } } See the Scripting Reference appendix for more on scripting.

Synchronization Encryption OpenIDM supports reversible encryption of attributes values for managed objects. Attribute values to encrypt include passwords (if passwords are not already encrypted on the external resource), and also authentication questions, credit card numbers, and social security numbers. You configure encryption in the managed object configuration. The file to edit is openidm/conf/managed.json. The following example shows a managed object configuration that encrypts and decrypts securityAnswer, ssn, and password attributes using the default symmetric key, and additional scripts for extra passwords. { "objects": [ { "name": "user", "properties": [ { "name": "securityAnswer", "encryption": { "key": "openidm-sym-default" } }, { "name": "ssn", "encryption": { "key": "openidm-sym-default" } }, { "name": "password", "encryption": { "key": "openidm-sym-default" } } ], "onStore": { "type": "text/javascript", "file": "script/encryptExtraPassword.js" }, "onRetrieve": { "type": "text/javascript", "file": "script/decryptExtraPassword.js" } } ] } Do not use the default symmetric key, openidm-sym-default, in production. See the chapter on Securing & Hardening OpenIDM for instructions on adding your own symmetric key.

Synchronization Creating attributes OpenIDM lets you construct and manipulate attributes using scripts triggered when an object is created (onCreate), updated (onUpdate), or deleted (onDelete), or when a link is created (onLink), or removed (onUnlink). The following example derives a DN for an LDAP entry when the entry is created in the internal repository. { "onCreate": { "type": "text/javascript", "source": "target.dn = 'uid=' + source.uid + ',ou=people,dc=example,dc=com'" } }

Synchronization Reusing links When two mappings exist to sync the same objects bidirectionally, you can use the links property in one mapping to have OpenIDM use the same internally managed link for both mappings. Otherwise, if no links property is specified, OpenIDM maintains a link for each mapping. The following excerpt shows two mappings, one from MyLDAP accounts to managed users, and another from managed users to MyLDAP accounts. In the second mapping, the link property tells OpenIDM to reuse the links created in the first mapping, rather than create new links. { "mappings": [ { "name": "systemMyLDAPAccounts_managedUser", "source": "system/MyLDAP/account", "target": "managed/user" }, { "name": "managedUser_systemMyLDAPAccounts", "source": "managed/user", "target": "system/MyLDAP/account", "links": "systemMyLDAPAccounts_managedUser" } ] }

Synchronization Situations Synchronization Actions During synchronization, OpenIDM categorizes objects by situation. Situations are characterized by whether an object exists on a source or target system, whether OpenIDM has registered a link between the source object and the target object, and whether the object is considered valid. OpenIDM takes action depending on the situation. You can define actions for particular situations in the policies section of a synchronization mapping, as shown in the following excerpt. { "policies": [ { "situation": "CONFIRMED", "action": "UPDATE" }, { "situation": "FOUND", "action": "IGNORE" }, { "situation": "ABSENT", "action": "CREATE" }, { "situation": "AMBIGUOUS", "action": "IGNORE" }, { "situation": "MISSING", "action": "IGNORE" }, { "situation": "UNQUALIFIED", "action": "IGNORE" }, { "situation": "UNASSIGNED", "action": "IGNORE" } ] } If you do not define a policy for a particular situation, OpenIDM takes the default action for the situation. The situations and their corresponding actions are described in the sections below.

OpenIDM performs synchronization action in two phases. First, OpenIDM performs source reconciliation, where OpenIDM accounts for source objects and associated links based on the mapping configured. Second, OpenIDM runs target reconciliation, where OpenIDM interates over the target objects not processed in the first phase. During reconciliation OpenIDM builds three lists, assigning values to the objects to reconcile. All valid objects from the source OpenIDM assigns valid source objects qualifies=1. Invalid objects, including those not found in the source system, and those filtered out by the script specified in the validSource property get qualifies=0. All records from the appropriate link table Object with corresponding links in the link table of the repository get link=1. Objects without corresponding links get link=0. All valid objects on the target system Object found in the target system get target=1. Objects not found in the target system get target=0. Based on the values assigned to objects during source reconciliation, OpenIDM assigns situations, listed here with their default actions. "CONFIRMED" (qualifies=1, link=1, target=1) The mapping qualifies for a target object, and a link to a target object exists. Detected during change events and reconciliation. Default action: "UPDATE". "FOUND" (qualifies=1, link=0, target=1) The mapping qualifies for a target object, there is no link to a target object, and there is a single correlated target object to link. Detected during change events and reconciliation. Default action: "UPDATE". "ABSENT" (qualifies=1, link=0, target=0) The mapping qualifies for a target object, there is no link to a target object, and there is no correlated target object to link. Detected during change events and reconciliation. Default action: "CREATE". "AMBIGUOUS" (qualifies=1, link=0, target>1) The mapping qualifies for a target object, there is no link to a target object, but there is more than one correlated target object to link. Detected during change events and reconciliation. Default action: "EXCEPTION". "MISSING" (qualifies=1, link=1, target=0) The mapping is qualified for a target object, and there is a qualified link to a target object, but the target object is missing. Only detected during reconciliation and synchronous operations. Default action: "EXCEPTION". "UNQUALIFIED" (qualifies=0, link=1) The mapping is not qualified for a target object and there is a link to an existing target object. Detected during change events and reconciliation. Default action: "DELETE". Target synchronization assigns the same situation when (qualifies=0, link=0 or source=0). "UNASSIGNED" (qualifies=0, link=0, target=1) (not in use) A target object exists for which there is no link. Only detected during reconciliation. Default action: "EXCEPTION". Based on the values assigned to objects during target reconciliation, OpenIDM assigns situations, listed here with their default actions. "CONFIRMED" (qualifies=1, link=1, source=1) The mapping qualifies for a target object, and a link to a source object exists. Detected during only reconciliation. Default action: "UPDATE". "UNQUALIFIED" (qualifies=0, link=0 or source=0) The mapping is not qualified for a target object, and there is a link to an existing target object. Detected during change events and reconciliation. Default action: "DELETE". The following sections reiterate in detail how OpenIDM assigns situations during each of the two synchronization phases.

OpenIDM starts reconciliation and LiveSync by reading a list of objects from the resource. For reconciliation, the list includes all objects available through the connector. For LiveSync, the list contains only changed objects. The connector can filter objects out of the list, too. You can filter objects out of the list by using the validSource property. OpenIDM then iterates over the list, checking each entry against the validSource filter, classifying objects according to their situations as described in . OpenIDM uses the list of links for the current mapping to classify objects. Finally, OpenIDM executes the action configured for the situation. The following table shows how OpenIDM assigns the appropriate situation during source reconciliation, depending on whether a valid source exists (Source Qualifies), whether a link with the appropriate type exists in the repository (Link Exists), and how many target objects are found either based on links or correlation results. Source Qualifies? Link Exists? Target Objects FoundIf no link exists for the source object, then OpenIDM executes a correlation query. Situation Yes No Yes No 0 1 > 1   X   X X     SOURCE_IGNORED   X   X   X   UNQUALIFIED   X   X     X UNQUALIFIED   X X   X     UNQUALIFIED   X X     X   UNQUALIFIED   X X       X UNQUALIFIED X     X X     ABSENT X     X   X   FOUND X     X     X AMBIGUOUS X   X   X     MISSING X   X     X   CONFIRMED

During source reconciliation, OpenIDM cannot detect situations where no source object exists, such as the UNASSIGNED situation. When no source object exists, OpenIDM detects the situation during the second reconciliation phase, target reconciliation. During target reconciliation, OpenIDM iterates over all target objects that do not have a representation on the source, checking each object against the validTarget filter, determining the appropriate situation, and executing the action configured for the situation. The following table shows how OpenIDM assigns the appropriate situation during target reconciliation, depending on whether a valid target exists (Target Qualifies), whether a link with an appropriate type exists in the repository (Link Exists), whether a source object exists (Source Exists), and whether the source object qualifies (Source Qualifies). Not all situations assigned during source reconciliation are assigned during target reconciliation. Target Qualifies? Link Exists? Source Exists? Source Qualifies? Situation Yes No Yes No Yes No Yes No   X             TARGET_IGNORED X     X   X     UNASSIGNED X   X   X   X   CONFIRMED X   X   X     X UNQUALIFIED X   X     X     SOURCE_MISSING

Once OpenIDM has assigned a situation to an object, OpenIDM takes the actions configured in the mapping. If no action is configured, then OpenIDM takes the default action for the situation. OpenIDM supports the following actions. "CREATE" Create and link a target object. "UPDATE" Link and update a target object. "DELETE" Delete and unlink the target object. "LINK" Link the correlated target object. "UNLINK" Unlink the linked target object. "EXCEPTION" Flag the link situation as an exception. "IGNORE" Do not change the link or target object state.

Synchronization Correlation queries Correlation queries Every time OpenIDM creates an object through synchronization, it creates a link between the source and target objects. OpenIDM then uses the link to determine the object's situation during later synchronization operations. Initial, bulk synchronization operations can involve correlating many objects that exist both on source and target systems. In this case, OpenIDM uses correlation queries to find target objects that already exist, and that correspond to source objects. For the target objects that match a correlation query, OpenIDM needs only to create a link, rather than a new target object. Correlation queries run against target resources. The query syntax therefore depends on the target system, and is either specific to the data store underlying the OpenIDM repository, or to OpenICF query capabilities.

Queries on managed objects in the repository must defined in the configuration file for the repository, which is either openidm/conf/repo.orientdb.json, or openidm/conf/repo.jdbc.json. The following example shows a correlation query defined in openidm/conf/repo.orientdb.json. "for-userName" : "SELECT * FROM ${_resource} WHERE userName = '${uid}'" The following correlation query example shows the JavaScript to call the query defined for OrientDB. The _query-id property value matches the name of the query specified in openidm/conf/repo.orientdb.json, for-userName. The source.name value replaces ${uid} in the query. OpenIDM replaces ${_resource} in the query with the name of table that holds managed objects. { "correlationQuery": { "type": "text/javascript", "source": "var query = {'_query-id' : 'for-userName', 'uid' : source.name}; query;" } } The query can return zero or more objects, so the situation OpenIDM assigns to the source object depends on the number of target objects returned. With a JDBC-based repository, the query defined in openidm/conf/repo.jdbc.json is more complex due to how the tables are indexed. The correlation query you define in openidm/conf/sync.json is the same, however.

Correlation queries on system objects access the connector. The connector then executes the query on the external resource. Your correlation query JavaScript must return a map holding a generic query with the following elements. A condition such as "Equals" The naming attribute to compare on the system object. In the example that follows, the naming attribute is uid. The value from the source object to use in the search filter. You set this as the value of the value property, which takes an array. In the example that follows, the value to use in the search filter is source.userName. varmap={"query": {"Equals": {"field": "uid", "values": [ source.userName ]}}}; map;

Mappings Hooks for scripting shows how to trigger scripts when objects are created and updated. Other situations require you to trigger scripts in response to other synchronization actions. For example, you might not want OpenIDM to delete a managed user directly when an external account is deleted, but instead unlink the objects and deactivate the user in another resource. (Alternatively, you might delete the object in OpenIDM but nevertheless execute a script.) The following example shows a more advanced mapping configuration. { "mappings": [ { "name": "systemLdapAccount_managedUser", "source": "system/ldap/account", "target": "managed/user", "validSource": { "type": "text/javascript", "file": "jscript/isValid.js" }, "correlationQuery": { "type": "text/javascript", "file": "jscript/ldapCorrelationQuery.js" }, "properties": [ { "source": "uid", "transform": { "type": "text/javascript", "source": "source.toLowerCase()" }, "target": "userName" }, { "source": "", "transform": { "type": "text/javascript", "source": "if (source.myGivenName) {source.myGivenName;} else {source.givenName;}" }, "target": "givenName" }, { "source": "", "transform": { "type": "text/javascript", "source": "if (source.mySn) {source.mySn;} else {source.sn;}" }, "target": "familyName" }, { "source": "cn", "target": "fullname" }, { "comment": "Multi-valued in LDAP, single-valued in AD. Retrieve first non-empty value.", "source": "title", "transform": { "type": "text/javascript", "file": "jscript/getFirstNonEmpty.js" }, "target": "title" }, { "condition": { "type": "text/javascript", "source": "var clearObj = openidm.decrypt(object); ((clearObj.password != null) && (clearObj.ldapPassword != clearObj.password))" }, "transform": { "type": "text/javascript", "source": "source.password" }, "target": "__PASSWORD__" } ], "onCreate": { "type": "text/javascript", "source": "target.ldapPassword = null; target.adPassword = null; target.password = null; target.ldapStatus = 'New Account'" }, "onUpdate": { "type": "text/javascript", "source": "target.ldapStatus = 'OLD'" }, "onUnlink": { "type": "text/javascript", "file": "jscript/triggerAdDisable.js" }, "policies": [ { "situation": "CONFIRMED", "action": "UPDATE" }, { "situation": "FOUND", "action": "UPDATE" }, { "situation": "ABSENT", "action": "CREATE" }, { "situation": "AMBIGUOUS", "action": "EXCEPTION" }, { "situation": "MISSING", "action": "EXCEPTION" }, { "situation": "UNQUALIFIED", "action": "UNLINK" }, { "situation": "UNASSIGNED", "action": "EXCEPTION" } ] } ] } Here is the complete list of properties you can use as hooks in mapping configurations to call scripts. Triggered by Situation onCreate, onUpdate, onDelete, onLink, onUnlink Object Filter vaildSource, validTarget Correlating Objects correlationQuery Triggered on Reconciliation result Scripts Inside Properties condition, transform Your scripts can get data from any connected system at any time by using the openidm.read(id) function, where id is the identifier of the object to read. The following example reads a managed user object from the repository. repoUser = openidm.read("managed/user/ddoe); The following example reads an account from an external LDAP resource. externalAccount = openidm.read("system/ldap/account/ddoe");

Mappings Scheduled reconciliation Mappings for synchronization are usually stored in openidm/conf/sync.json for reconciliation, LiveSync, and for pushing changes made to managed objects to external resources. You can, however, provide alternative mappings for scheduled reconciliation by adding the mapping to the scheduler configuration instead of referencing the sync.json mapping. { "enabled": true, "type": "cron", "schedule": "0 08 16 * * ?", "invokeService": "sync", "invokeContext": { "action": "reconcile", mapping": { "name": "CSV_XML", "source": "system/Ldap/account", "target": "managed/user", "properties": [ { "source": "firstname", "target": "firstname" }, ... ], "policies": [...] } } }