One of the common uses for the MIM Portal is to define query filter-based-membership groups for mastering (static) group membership in AD and/or Entra ID. In Access Panel there is also the concept of filter-based criteria groups, but the task of migrating thousands of group definitions to Identity Panel may seem daunting at first.
Each unique query-based group definition in MIM should ideally be uniquely identifiable by its query definition ... although it is reasonable to expect there to be duplicates because this is not enforceable by MIM. Consider the following typical scenario:
- there are 100 cost centers
- there are 50 store locations
- there are 3 employee status values A, B and C that constitute an "active" employee
- I have 500 MIM group definitions with filters which all look something like this:
"/Person[(CostCenter = 'xxxx') and ((employeeStatus = 'A') or (employeeStatus = 'B') or (employeeStatus = 'C'))]"
In other words, you will generally find that on closer inspection of large numbers of query-based MIM groups that the majority of them can be condensed down to a much smaller number of query "templates".
It is ONLY these templates which then need to be migrated, not the group definitions themselves.
But how do you extract all the templates represented in 1000s of MIM dynamic groups?
MIM Group Filters
MIM Group filters are stored as XML documents within the MIM Service database. These can be retrieved by querying MIM then extracting the inner text of the XML document.
For Identity Panel implementations licensed for the MIM Provider, once scanned, the MIM Portal will be present as a data silo. As with any silo, this can then be queried, targeting only the dynamic groups, and the (xpath) Filter retrieved. Additionally, by applying a RegEx replacement a generic query template can be derived from the xpath for comparison to other filter-based groups.
Report #1 (Base)
This first report definition represents the common query used to extract query-based groups:
{
"Data": [
{
"$type": "SoftwareIDM.ReportingModule.Models.Report, SoftwareIDM.ReportingModule",
"Id": "b37b069b-032a-4164-bfb8-62377deae90c",
"Name": "AAA MIM Group Filters Base",
"UserId": "8b80e9bd-2cd9-41d3-9374-f0a982698f40",
"UserProfile": true,
"Tags": null,
"Roles": [
"Admin",
"User"
],
"CacheExpiration": "01:00:00",
"Description": "Assess the breadth of group filters in use across all MIM dynamic groups",
"Parameters": [],
"DataSets": [
{
"$type": "SoftwareIDM.ReportingModule.Models.QueryData, SoftwareIDM.ReportingModule",
"Clauses": [
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Silo",
"Operation": "Eq",
"Value": "special.Identity Silo.MIM Legacy: Portal",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "ObjectType",
"Operation": "Eq",
"Value": "\"Group\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.MembershipAddWorkflow",
"Operation": "Eq",
"Value": "\"None\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.MembershipLocked",
"Operation": "Eq",
"Value": "True",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.Filter",
"Operation": "EndsWith",
"Value": "\">\"",
"ObjectValue": null
}
],
"SubQueries": [],
"MapRule": "",
"MemoRules": [],
"Aliases": null,
"Name": "MIM Groups",
"Type": "SoftwareIDM.PanelModel.Models.ObjectRecord, SoftwareIDM.PanelModel",
"Include": [
"Attributes.DisplayName",
"Attributes.AccountName",
"Attributes.Filter"
],
"Limit": null
}
],
"ReportSets": [],
"Relations": [],
"FieldProjections": [
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Name",
"RelationName": "",
"RelationSide": null,
"ValueRule": "FirstNotNull(Attributes.AccountName,Attributes.DisplayName)"
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Filter",
"RelationName": "",
"RelationSide": null,
"ValueRule": "$\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Pattern",
"RelationName": "",
"RelationSide": null,
"ValueRule": "RegexReplace($\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\", \"\\'.[^']*\\'\",\"'xxxx'\")"
}
],
"Transformations": [],
"Styles": [],
"Sort": {
"Field": "Name",
"Direction": "Ascending"
}
}
],
"Count": 1
}
However, once we have extracted all the dynamic group filters, and derived a generic xpath pattern for comparison, how do we determine which ones are just variations on the same thing? This is especially challenging because xpath filters can have extra spacing and parentheses and still be functionally equivalent.
Levenshtein Distance
The answer is by using the Levenshtein Distance formula, present in Identity Panel's rule engine as the EditDistance() function, to compare group filters and patterns.
The Levenshtein Distance is a measure of the similarity between two strings, representing the minimum number of single-character edits (insertions, deletions, or substitutions) required to change one string into the other. While it's often used for comparing individual strings, it can also be adapted for comparing and analyzing patterns within groups of strings.
In the context of Microsoft Identity Manager (MIM) or similar identity management systems, dynamic group definitions are rules or filters used to determine the membership of a group based on certain attributes or conditions. If you have two group filter values and you want to identify repeating dynamic group definitions, you might use the Levenshtein distance as follows:
Representation of Dynamic Group Definitions:
Represent each dynamic group definition as a string, where the characters represent the conditions or attributes used in the filter.
For example, if you have two dynamic group filters: "Department equals 'Sales'" and "Department equals 'Marketing'", you can represent them as "Dept=Sales" and "Dept=Marketing".
Calculating Levenshtein Distance:
Calculate the Levenshtein distance between each pair of dynamic group definitions.
The lower the distance, the more similar the dynamic group definitions are.
Setting a Threshold:
Define a threshold value for the Levenshtein distance that determines when two dynamic group definitions are considered similar enough to be considered repeating.
The threshold might be determined based on the complexity of your dynamic group definitions and how much variation you expect.
Identifying Repeating Dynamic Group Definitions:
Compare each dynamic group definition with every other definition and identify pairs or groups with a Levenshtein distance below the threshold.
These pairs or groups represent dynamic group definitions that are similar or repeating.
Review and Consolidation:
Review the identified pairs or groups and assess whether they can be consolidated into a single dynamic group definition or if there are variations that need to be considered.
Implementation and Testing:
Implement the consolidated dynamic group definitions in Access Panel and thoroughly test to ensure that they correctly capture the desired memberships.
By using Levenshtein distance to compare dynamic group definitions, you can automate the identification of similar patterns and potentially reduce redundancy in your group definitions. However, it's essential to carefully review the results and ensure that consolidation or modification aligns with your organization's requirements and policies.
Report #2
The following report definition uses the EditDistance() function to compare all rows in the base report (#1) to every other row in the same report:
{
"Data": [
{
"$type": "SoftwareIDM.ReportingModule.Models.Report, SoftwareIDM.ReportingModule",
"Id": "2969f998-7125-4b95-8a8e-cf99d011c71b",
"Name": "AAA MIM Group Filters",
"UserId": "8b80e9bd-2cd9-41d3-9374-f0a982698f40",
"UserProfile": true,
"Tags": null,
"Roles": [
"Admin",
"User"
],
"CacheExpiration": "01:00:00",
"Description": "Assess the breadth of group filters in use across all MIM dynamic groups",
"Parameters": [],
"DataSets": [
{
"$type": "SoftwareIDM.ReportingModule.Models.QueryData, SoftwareIDM.ReportingModule",
"Clauses": [
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Silo",
"Operation": "Eq",
"Value": "special.Identity Silo.MIM Legacy: Portal",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "ObjectType",
"Operation": "Eq",
"Value": "\"Group\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.MembershipAddWorkflow",
"Operation": "Eq",
"Value": "\"None\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.MembershipLocked",
"Operation": "Eq",
"Value": "True",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.Filter",
"Operation": "EndsWith",
"Value": "\">\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.Filter",
"Operation": "Contains",
"Value": "\"txtLocID =\"",
"ObjectValue": null
}
],
"SubQueries": [],
"MapRule": "",
"MemoRules": [],
"Aliases": null,
"Name": "MIM Groups",
"Type": "SoftwareIDM.PanelModel.Models.ObjectRecord, SoftwareIDM.PanelModel",
"Include": [
"Attributes.DisplayName",
"Attributes.AccountName",
"Attributes.Filter"
],
"Limit": null
},
{
"$type": "SoftwareIDM.ReportingModule.Models.QueryData, SoftwareIDM.ReportingModule",
"Clauses": [
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Silo",
"Operation": "Eq",
"Value": "special.Identity Silo.MIM Legacy: Portal",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "ObjectType",
"Operation": "Eq",
"Value": "\"Group\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.MembershipAddWorkflow",
"Operation": "Eq",
"Value": "\"None\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.MembershipLocked",
"Operation": "Eq",
"Value": "True",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.Filter",
"Operation": "EndsWith",
"Value": "\">\"",
"ObjectValue": null
},
{
"$type": "SoftwareIDM.PanelModel.Models.FilterClause, SoftwareIDM.PanelModel",
"Field": "Attributes.Filter",
"Operation": "Contains",
"Value": "\"txtLocID =\"",
"ObjectValue": null
}
],
"SubQueries": [],
"MapRule": "",
"MemoRules": [],
"Aliases": null,
"Name": "MIM Groups 2",
"Type": "SoftwareIDM.PanelModel.Models.ObjectRecord, SoftwareIDM.PanelModel",
"Include": [
"Attributes.Filter",
"Attributes.DisplayName",
"Attributes.AccountName"
],
"Limit": null
}
],
"ReportSets": [],
"Relations": [
{
"$type": "SoftwareIDM.ReportingModule.Models.Relation, SoftwareIDM.ReportingModule",
"Name": "GroupFilters",
"LeftSet": "MIM Groups",
"RightSet": "MIM Groups 2",
"LeftRule": "//$\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\"\n1",
"RightRule": "//$\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\"\n1",
"AllowLeftOuter": false,
"AllowRightOuter": false
}
],
"FieldProjections": [
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Name1",
"RelationName": "GroupFilters",
"RelationSide": "Left",
"ValueRule": "FirstNotNull(Attributes.AccountName,Attributes.DisplayName)"
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Filter1",
"RelationName": "GroupFilters",
"RelationSide": "Left",
"ValueRule": "$\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Pattern1",
"RelationName": "GroupFilters",
"RelationSide": "Left",
"ValueRule": "RegexReplace($\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\", \"\\'.[^']*\\'\",\"'xxxx'\")"
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Name2",
"RelationName": "GroupFilters",
"RelationSide": "Right",
"ValueRule": "FirstNotNull(Attributes.AccountName,Attributes.DisplayName)"
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Filter2",
"RelationName": "GroupFilters",
"RelationSide": "Right",
"ValueRule": "$\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Projection, SoftwareIDM.ReportingModule",
"Name": "Pattern2",
"RelationName": "GroupFilters",
"RelationSide": "Right",
"ValueRule": "RegexReplace($\"{If(Attributes.Filter, Map(ParseJson(ParseXml(Attributes.Filter)).Filter, Value).4)}\", \"\\'.[^']*\\'\",\"'xxxx'\")"
}
],
"Transformations": [
{
"$type": "SoftwareIDM.ReportingModule.Models.Transform, SoftwareIDM.ReportingModule",
"FilterRule": "",
"GroupingRule": "",
"TransformRules": [
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Filter1",
"Name": "",
"Operator": "None",
"TransformRule": ""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Pattern1",
"Name": "",
"Operator": "None",
"TransformRule": ""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Filter2",
"Name": "",
"Operator": "None",
"TransformRule": ""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Pattern2",
"Name": "",
"Operator": "None",
"TransformRule": ""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Name1",
"Name": "Distance",
"Operator": "None",
"TransformRule": "EditDistance(Filter1,Filter2,1000)"
},
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Name1",
"Name": "Count",
"Operator": "None",
"TransformRule": "1"
}
]
},
{
"$type": "SoftwareIDM.ReportingModule.Models.Transform, SoftwareIDM.ReportingModule",
"FilterRule": "And(Distance >= 0, Distance < 50)",
"GroupingRule": "Pattern1",
"TransformRules": [
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Pattern1",
"Name": "Pattern",
"Operator": "None",
"TransformRule": ""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Distance",
"Name": "",
"Operator": "Max",
"TransformRule": ""
},
{
"$type": "SoftwareIDM.ReportingModule.Models.TransformField, SoftwareIDM.ReportingModule",
"Field": "Count",
"Name": "",
"Operator": "Sum",
"TransformRule": ""
}
]
}
],
"Styles": [],
"Sort": {
"Field": "Pattern",
"Direction": "Ascending"
}
}
],
"Count": 1
}
Comments
0 comments
Please sign in to leave a comment.