39

I am trying to get duplicate objects within an array of objects. Let's say the object is like below.

values = [ { id: 10, name: 'someName1' }, { id: 10, name: 'someName2' }, { id: 11, name: 'someName3' }, { id: 12, name: 'someName4' } ]; 

Duplicate objects should return like below:

duplicate = [ { id: 10, name: 'someName1' }, { id: 10, name: 'someName2' } ]; 
2
  • 2
    And what is your question?
    – ssc-hrep3
    CommentedNov 8, 2018 at 16:30
  • Trying how? You haven't shown us any code that attempts to solve this. A web search for array duplicates should have given you a good start pointCommentedNov 8, 2018 at 16:31

8 Answers 8

69

You can use Array#reduce to make a counter lookup table based on the id key, then use Array#filter to remove any items that appeared only once in the lookup table. Time complexity is O(n).

const values = [{id: 10, name: 'someName1'}, {id: 10, name: 'someName2'}, {id: 11, name:'someName3'}, {id: 12, name: 'someName4'}]; const lookup = values.reduce((a, e) => { a[e.id] = ++a[e.id] || 0; return a; }, {}); console.log(values.filter(e => lookup[e.id]));

You can simplify this if your browser supports Object.groupBy:

const values = [{id: 10, name: 'someName1'}, {id: 10, name: 'someName2'}, {id: 11, name:'someName3'}, {id: 12, name: 'someName4'}]; const lookup = Object.groupBy(values, e => e.id); console.log(values.filter(e => lookup[e.id].length > 1));

9
  • 3
    Can you please explain your code?I confuse.You don't have any equality in your code how this work?And why a in every iteration show the same object?I need detailed analysis of your code.
    – Randall
    CommentedFeb 6, 2020 at 8:13
  • 5
    I recommend taking a look at the docs for reduce and filter if you haven't yet. I use reduce to build an object mapping of each object's id to the count of its occurrences - 1. a is the accumulator object which is passed from one callback to the next by reduce. filter uses the truthiness of lookup[e.id] to determine if an element is unique. If the lookup entry is 0 (falsey), it was only seen once and filter omits it from the result set, else it's kept.
    – ggorlen
    CommentedFeb 6, 2020 at 8:23
  • 1
    Actually I know about reduce (not an expert but know) and filter.And know how they work.But it is to hard to understanding your code.But with your explanation everything has become little bit easier :) Thank you.
    – Randall
    CommentedFeb 6, 2020 at 8:33
  • 4
    This is extremely clever code and I don't think your explanation does it justice @ggorlen ! To explain exactly what is going on inside the reduce, here goes: each item in the array is processed by the arrow function, and either creates a property in the lookup (new item), or increments the counter (we hit a duplicate). Exactly how this works is as follows: a[e.id] = ++a[e.id] || 0 will only increment a property using ++ if it already exists. If it doesn't, then it will fall through the OR (||) and create a property with value 0.
    – hevans900
    CommentedJul 15, 2021 at 15:02
  • @ggorlen Have can we compare multiple fields ID and Name?
    – Vivek
    CommentedJul 16, 2021 at 15:18
17

Let's say you have:

arr = [ { id:10, name: 'someName1' }, { id:10, name: 'someName2' }, { id:11, name: 'someName3' }, { id:12, name: 'someName4' } ] 

So, to get unique items:

unique = arr .map(e => e['id']) .map((e, i, final) => final.indexOf(e) === i && i) .filter(obj=> arr[obj]) .map(e => arr[e]); 

Then, result will be

unique = [ { id:10, name: 'someName1' }, { id:11, name: 'someName3' }, { id:12, name: 'someName4' } ] 

And, to get duplicate ids:

duplicateIds = arr .map(e => e['id']) .map((e, i, final) => final.indexOf(e) !== i && i) .filter(obj=> arr[obj]) .map(e => arr[e]["id"]) 

List of IDs will be

duplicateIds = [10] 

Thus, to get duplicates objects:

duplicate = arr.filter(obj=> dublicateIds.includes(obj.id)); 

Now you have it:

duplicate = [ { id:10, name: 'someName1' }, { id:10, name: 'someName2' } ] 

Thanks https://reactgo.com/removeduplicateobjects/

7
  • How do we find out duplicates for multiple properties?
    – Vivek
    CommentedJul 19, 2021 at 8:14
  • I am not what do you mean @Vivek ,, if you have examples for the inputs and the expected output, so I can help!
    – HMagdy
    CommentedJul 20, 2021 at 9:44
  • 2
    you can do that easily by replacing ".map(e => e['id'])" by ".map(e => e['id']+'_'+e['name'])"
    – HMagdy
    CommentedJul 22, 2021 at 11:50
  • 1
    Use of indexOf inside a map makes this a O(n^2) algorithm. Using the countBy or the groupBy method above is easier to understand and works in O(n).
    – abhaga
    CommentedOct 9, 2021 at 10:56
  • 1
    Your solution for finding unique items is exactly what we were trying to conclude and literally had to change indexOf to lastIndexOf and achieved what we needed. Thank you!
    – Koshux
    CommentedDec 21, 2022 at 17:45
9

You haven't clarified whether two objects with different ids, but the same "name" count as a duplicate. I will assume those do not count as a duplicate; in other words, only objects with the same id will count as duplicate.

let ids = {}; let dups = []; values.forEach((val)=> { if (ids[val.id]) { // we have already found this same id dups.push(val) } else { ids[val.id] = true; } }) return dups; 
    9

    With lodash you can solve this with filter and countBy for complexity of O(n):

    const data = [{ id: 10,name: 'someName1' }, { id: 10,name: 'someName2' }, { id: 11,name: 'someName3' }, { id: 12,name: 'someName4' } ] const counts = _.countBy(data, 'id') console.log(_.filter(data, x => counts[x.id] > 1))
    <script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.10/lodash.min.js"></script>

    You could do the same with ES6 like so:

    const data = [{ id: 10,name: 'someName1' }, { id: 10,name: 'someName2' }, { id: 11,name: 'someName3' }, { id: 12,name: 'someName4' } ] const countBy = (d, id) => d.reduce((r,{id},i,a) => (r[id] = a.filter(x => x.id == id).length, r),{}) const counts = countBy(data, 'id') console.log(data.filter(x => [x.id] > 1))

      4

      You can use an array to store unique elements and use filter on values to only return duplicates.

      const unique = [] const duplicates = values.filter(o => { if(unique.find(i => i.id === o.id && i.name === o.name)) { return true } unique.push(o) return false; }) 
        2

        With lodash you can use _.groupBy() to group elements by their id. Than _.filter() out groups that have less than two members, and _.flatten() the results:

        const values = [{id: 10, name: 'someName1'}, {id: 10, name: 'someName2'}, {id: 11, name:'someName3'}, {id: 12, name: 'someName4'}]; const result = _.flow([ arr => _.groupBy(arr, 'id'), // group elements by id g => _.filter(g, o => o.length > 1), // remove groups that have less than two members _.flatten // flatten the results to a single array ])(values); console.log(result);
        <script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.11/lodash.min.js"></script>

          1

          An alternative based in @ggorlen solution with new Map() as accumulator (for better performance) and without unary operator ++ (not advised by default in projects with ESLint).

          const values = [{ id: 10, name: "someName1" }, { id: 10, name: "someName2" }, { id: 11, name: "someName3" }, { id: 12, name: "someName4" },]; const lookup = values.reduce((a, e) => { a.set(e.id, (a.get(e.id) ?? 0) + 1); return a; }, new Map()); console.log(values.filter(e => lookup.get(e.id) > 1));

          0
            -1

            Try this

            function checkDuplicateInObject(propertyName, inputArray) { var seenDuplicate = false, testObject = {}; inputArray.map(function(item) { var itemPropertyName = item[propertyName]; if (itemPropertyName in testObject) { testObject[itemPropertyName].duplicate = true; item.duplicate = true; seenDuplicate = true; } else { testObject[itemPropertyName] = item; delete item.duplicate; } }); return seenDuplicate; } 

            referred from : http://www.competa.com/blog/lets-find-duplicate-property-values-in-an-array-of-objects-in-javascript/

              Start asking to get answers

              Find the answer to your question by asking.

              Ask question

              Explore related questions

              See similar questions with these tags.