Please consider supporting me on SubscribeStar so I can continue to bring open source software and tutorial videos to you!
Over the past weekend, I implemented a feature in jdupes that was pretty strongly desired by the user community. Version 1.16.0 has the ability to filter by file extensions, either by excluding files with certain extensions or only scanning files with certain extensions. Now you can do things like scan only the JPEG files you dumped from your phone while ignoring all of the videos, scan a software project folder’s .c and .h files for duplicates while ignoring all of the others, or find all duplicates except for XML files.
In addition, I’ve cleaned up the extended filter (-X/–extfilter) framework and created an entirely separate help text section (jdupes -X help) that explains the extfilter options and behavior in great detail.
The extended filters are also cumulative, so specifying multiple filter options works as expected; for example, “jdupes -X noext=mp3 -X noext=aac -X size+=:1M” will exclude all files from consideration that end in .mp3/.aac as well as all files that are 1MiB or larger in size.
Unfortunately, there is not currently a way to combine filters, i.e. exclude all files with a particular extension over a particular size. That may be a feature in the future, but right now, I’m trying to add some basic filter functionality that satisfies as many basic filtering use cases as possible with as little work as possible. In the case of the extension filter, it took me about 3-4 hours to code, test, and fix issues with the feature, then issue a new release. It was relatively easy to implement, and even includes the ability to scan a comma-separated list of extensions rather than requiring a separate option for every single extension you want to filter.
Other features that filter on file paths will be more difficult to implement, but are highly desired by the community, so I have plans to implement those soon. The next fix, however, will be for the problematic BTRFS/XFS dedupe support that can’t dedupe (ioctl_fideduperange) large file sets properly.
Thanks as always for your continued support. More support means more time to work on my projects! I appreciate all of your help.
2 thoughts on “jdupes 1.16.0: File Extension Filtering, And I Need Your Support”
i can’t help to offer sincere thanks for jdupes.
i just learned to cross compile it for my arm OS (very old Synology 410j running DSM5).
my arm box had only fdupes 1.40 which took at least 45 minutes to get 31% of scanning only 361 files.
in the time it took to get up to 36%, i searched how to cross compile arm, did so, then executed successfully jdupes as a test on the arm box… i tested a single directory with success then decided to trust in it and executed a recursive jdupes on a real important folder… by the time the entire task was accomplished, fdupes was only at 39% of the initial scan. :))
i’ve used jdupes for years now and have complete trust in it and appreciate what you’ve done for the world community.
Thank you for the kind words!