deduplication

Fast, secure, efficient backup program

Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.

Deduplicating archiver with compression and authenticated encryption.

Prometheus Alertmanager

Find duplicate files

A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.

Windows disk usage analyzer and cleanup tool with interactive treemaps, large-file discovery, duplicate detection, and file-system monitoring.

rustic - fast, encrypted, and deduplicated backups powered by Rust

A fast high-compression read-only file system for Linux, FreeBSD, macOS and Windows

Ephemeral Workspaces, durable shared history. LayerFS gives every agent a complete, writable filesystem Workspace over shared committed history, without copying the base. Agents work in parallel, commit what changed as deduplicated checkpoints, then branch, rewind or merge. Try it with LayerFS × DSH.

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

Extremely fast tool to remove duplicates and other lint from your filesystem

Mirror of borgmatic: Simple, configuration-driven backup software for servers and workstations

plakar is a backup solution powered by Kloset and ptar

Config driven, easy backup cli for restic.

Scalable data pre processing and curation toolkit for LLMs

Replace complex Borg Backup terminal commands with a beautiful web UI. Create, schedule, and restore backups with just a few clicks.

Scalable master data management, identity resolution, entity resolution, and deduplication using ML

A powerful and modular toolkit for record linkage and duplicate detection in Python

Fast Multimodal Semantic Deduplication & Filtering