Duplicate Activity Detection
Feature Detail
Description
This feature detects when a peer mentor or coordinator attempts to register an activity that appears to be a duplicate of one already recorded — either by the same user or by another coordinator reporting on their behalf. The system compares key fields such as date, duration, contact, and activity type to identify likely duplicates and presents a warning dialog before the submission is finalized. Users can confirm the registration is intentional or cancel to avoid double-counting in Bufdir reports and statistics.
Analysis
NHF explicitly identified duplicate activity registration as a problem where the same activity is recorded by multiple coordinators, inflating statistics and creating compliance risks in Bufdir reporting. Double-counting can result in grant clawback, audit findings, or reputational damage for organizations. For proxy and bulk registration workflows, the risk of duplication is especially high since coordinators may not be aware of what a peer mentor has already submitted. Detecting duplicates proactively at the point of entry prevents data quality problems before they compound, reducing the cost of correction and protecting organizational integrity in regulatory reporting contexts.
Duplicate detection runs a server-side similarity query in the Activity Deduplication Repository before the activity is saved, comparing the candidate record against existing activities within a configurable time window (default: same day) for the same peer mentor and activity type. The query uses indexed columns (user_id, activity_date, activity_type_id) for performance. Fuzzy matching on duration (within 15 minutes) and contact_id is applied as secondary signals. The Duplicate Detection Service returns a confidence score; scores above threshold trigger the Duplicate Warning Dialog. The dialog shows the conflicting record's details so users can make an informed decision. Bulk registration flows check each record individually before batch insert.
Dependencies
Definition of Done
Components (3)
User Stories (7)
As a As a user
I want I want to be able to confirm that a flagged activity is intentionally distinct and proceed with submitting it despite the duplicate warning
So that So that legitimate separate activities that share similar attributes are not blocked by the detection system, preserving the accuracy of activity records
- Given the Duplicate Warning Dialog is visible, when the user selects the confirm/proceed action, then the activity is saved to the database and the user is returned to the success state of the registration flow
- Given the user confirms submission despite the duplicate warning, when the record is saved, then an audit marker indicating the user overrode a duplicate warning is stored with the activity
- Given the user confirms an intentional duplicate, when the activity is saved, then the existing conflicting record is not modified or deleted
- +1 more
As a As a user
I want I want to be able to cancel my activity submission directly from the duplicate warning dialog
So that So that I can avoid creating a duplicate entry when the system correctly identifies a likely double-registration, protecting the integrity of Bufdir reports and organizational statistics
- Given the Duplicate Warning Dialog is visible, when the user selects the cancel action, then no new activity record is created in the database
- Given the user cancels from the duplicate warning, when the dialog closes, then the user is returned to the activity registration form with their previously entered data still populated
- Given the user cancels from the duplicate warning, when the form is shown, then the user can edit fields and re-submit, which will trigger a fresh duplicate check
- +1 more
As a As a Coordinator
I want I want each individual activity record in a bulk registration batch to be checked for duplicates before the batch is committed to the database
So that So that bulk or proxy registrations — which carry a higher risk of overlap with activities already submitted by peer mentors — do not silently introduce duplicate records into the system
- Given a coordinator submits a bulk registration batch, when the batch is processed, then each record is individually evaluated by the Duplicate Detection Service before any record is committed
- Given a bulk batch contains one or more records with confidence scores above the duplicate threshold, when detection completes, then the coordinator is shown the flagged records with details of each conflicting existing activity
- Given flagged records are shown, when the coordinator reviews them, then the coordinator can selectively exclude specific flagged records from the batch while allowing the rest to proceed
- +3 more
As a As a user
I want I want to be able to confirm that a flagged activity is intentionally distinct and proceed with submitting it despite the duplicate warning
So that So that legitimate separate activities that share similar attributes are not blocked by the detection system, preserving the accuracy of activity records
- Given the Duplicate Warning Dialog is visible, when the user selects the confirm/proceed action, then the activity is saved to the database and the user is returned to the success state of the registration flow
- Given the user confirms submission despite the duplicate warning, when the record is saved, then an audit marker indicating the user overrode a duplicate warning is stored with the activity
- Given the user confirms an intentional duplicate, when the activity is saved, then the existing conflicting record is not modified or deleted
- +1 more
As a As a user
I want I want to be able to cancel my activity submission directly from the duplicate warning dialog
So that So that I can avoid creating a duplicate entry when the system correctly identifies a likely double-registration, protecting the integrity of Bufdir reports and organizational statistics
- Given the Duplicate Warning Dialog is visible, when the user selects the cancel action, then no new activity record is created in the database
- Given the user cancels from the duplicate warning, when the dialog closes, then the user is returned to the activity registration form with their previously entered data still populated
- Given the user cancels from the duplicate warning, when the form is shown, then the user can edit fields and re-submit, which will trigger a fresh duplicate check
- +1 more
As a As a user
I want I want to be warned when an activity I am about to submit appears to be a duplicate of one already recorded for the same peer mentor
So that So that I can make an informed decision before finalizing the registration, preventing double-counting in Bufdir reports and statistical records
- Given a user is submitting an activity, when the system finds an existing activity for the same peer mentor on the same day with the same activity type, then the Duplicate Warning Dialog is shown before saving
- Given the duplicate check runs, when primary signals (user_id, activity_date, activity_type_id) match and secondary signals (duration within 15 min, contact_id) also match, then a high-confidence duplicate warning is displayed
- Given the Duplicate Warning Dialog is shown, when it renders, then it displays the conflicting existing record's date, duration, contact name, and activity type alongside the candidate record's details
- +2 more
As a As a user
I want I want to be warned when an activity I am about to submit appears to be a duplicate of one already recorded for the same peer mentor
So that So that I can make an informed decision before finalizing the registration, preventing double-counting in Bufdir reports and statistical records
- Given a user is submitting an activity, when the system finds an existing activity for the same peer mentor on the same day with the same activity type, then the Duplicate Warning Dialog is shown before saving
- Given the duplicate check runs, when primary signals (user_id, activity_date, activity_type_id) match and secondary signals (duration within 15 min, contact_id) also match, then a high-confidence duplicate warning is displayed
- Given the Duplicate Warning Dialog is shown, when it renders, then it displays the conflicting existing record's date, duration, contact name, and activity type alongside the candidate record's details
- +2 more