How to remove duplicates from arrays in Power Automate?

Process automation often requires repeating specific steps multiple times. In Power Automate, loops are very handy for executing these repetitions, , allowing you to perform a task for on each element of an array (for further instructions on how to loop and access individual elements from an array, check this article).

However, Power Automate doesn’t contain a built-in action for capturing only the unique values from an array, ignoring any duplicates. In this article, we will explore a real-world scenario where a flow reads data from a SharePoint List and filters out all duplicates.

Use case overview

For this blog, we will use the following SharePoint List, which contains data related to customer invoices:

sharepoint list overview

Each customer can have multiple related invoices, which each identified by a unique number. Our flow must capture the names of all customers from this SharePoint List, filtering out any duplicates. The final result will be posted in a Teams Channel, and will look like this:

post in teams channel

Flow structure

Let’s begin by creating a new flow and retrieving all records from the SharePoint List using the Get items action. Be sure to fill in the Site Address and the corresponding List Name:

get items from sharepoint list in power automate

No filters are required for this example, but feel free to add them if you wish (for more details on how to use filter queries in Power Automate, check out this article).

Next, we need to initialize a new variable that will store the list of unique customer names. This variable should be of the array data type (to learn more about data types in Power Automate, check this post). For this example, we will name it unique_names, but you can choose a different name if you prefer:

create array variable in power automate

After creating the variable, we need to loop through all the SharePoint List elements. To achieve this, add an Apply to each action, setting the List’s value property as dynamic content:

loop through sharepoint list records

Now that we accessed all invoice data, the next step is to add each customer’s name individually to the unique_names variable, ignoring duplicates. Insert a Condition inside the Apply to each block, so we can check whether the variable already contains the customer’s name (for more details on using conditions, check this article).

In the Condition’s left input field, you must enter the following expression: contains(variables(‘unique_names’), item()[‘customer’]). Make sure to replace unique_names with the actual name of the variable you created and customer with the exact column name from which you want to capture unique values. Keep the central dropdown field as is equal to, and set the right input field to true:

condition to check if array contains item

The Condition will return true (executing the If yes block) if the customer’s name is already included in the unique_names variable, and it will return false (executing the If no block) if the customer’s name is not yet found in the unique_names variable. In this case, we must add an Append to array variable action inside the If no block, setting its value using the expression item()[‘customer’] (make sure to change it to the same column name that you are using):

add item to array variable

As no actions must are required if the customer is already included in the unique_names variable, we will leave the If yes block empty.

Finally, outside the Apply to each block, add a Compose action and use the unique_names variable as input. This allows us to visualize the final result of the unique_names variable, which will contain an array of all customers without any duplicates.

We must also include a Post message in a chat or channel action, using a join() expression to concatenate all customer names from unique_names variable into a single text, separated by commas. The expression to achieve this result is join(variables(‘unique_names’), ‘, ‘):

craft message for teams channel

After running the flow, the Compose action will display the final value of the unique_names variable, containing only the distinct values extracted from customer name column:

array variable output

Additionally, a message is posted in the Teams Channel, listing these customers in a single text, separated by commas:

output in teams channel

Conclusion

Although there is no special action for it, it’s possible to combine some tasks in Power Automate to extract unique values from arrays. This can be helpful for extracting unique values from data collections, like JSON and CSV files, or columns in SharePoint Lists or SQL tables.

By Raphael Zaneti

Power Platform Developer