Unpacking Software Livestream

Join our monthly Unpacking Software livestream to hear about the latest news, chat and opinion on packaging, software deployment and lifecycle management!

Learn More

Chocolatey Product Spotlight

Join the Chocolatey Team on our regular monthly stream where we put a spotlight on the most recent Chocolatey product releases. You'll have a chance to have your questions answered in a live Ask Me Anything format.

Learn More

Chocolatey Coding Livestream

Join us for the Chocolatey Coding Livestream, where members of our team dive into the heart of open source development by coding live on various Chocolatey projects. Tune in to witness real-time coding, ask questions, and gain insights into the world of package management. Don't miss this opportunity to engage with our team and contribute to the future of Chocolatey!

Learn More

Calling All Chocolatiers! Whipping Up Windows Automation with Chocolatey Central Management

Webinar from
Wednesday, 17 January 2024

We are delighted to announce the release of Chocolatey Central Management v0.12.0, featuring seamless Deployment Plan creation, time-saving duplications, insightful Group Details, an upgraded Dashboard, bug fixes, user interface polishing, and refined documentation. As an added bonus we'll have members of our Solutions Engineering team on-hand to dive into some interesting ways you can leverage the new features available!

Watch On-Demand
Chocolatey Community Coffee Break

Join the Chocolatey Team as we discuss all things Community, what we do, how you can get involved and answer your Chocolatey questions.

Watch The Replays
Chocolatey and Intune Overview

Webinar Replay from
Wednesday, 30 March 2022

At Chocolatey Software we strive for simple, and teaching others. Let us teach you just how simple it could be to keep your 3rd party applications updated across your devices, all with Intune!

Watch On-Demand
Chocolatey For Business. In Azure. In One Click.

Livestream from
Thursday, 9 June 2022

Join James and Josh to show you how you can get the Chocolatey For Business recommended infrastructure and workflow, created, in Azure, in around 20 minutes.

Watch On-Demand
The Future of Chocolatey CLI

Livestream from
Thursday, 04 August 2022

Join Paul and Gary to hear more about the plans for the Chocolatey CLI in the not so distant future. We'll talk about some cool new features, long term asks from Customers and Community and how you can get involved!

Watch On-Demand
Hacktoberfest Tuesdays 2022

Livestreams from
October 2022

For Hacktoberfest, Chocolatey ran a livestream every Tuesday! Re-watch Cory, James, Gary, and Rain as they share knowledge on how to contribute to open-source projects such as Chocolatey CLI.

Watch On-Demand

Downloads:

2,850

Downloads of v 1.1.0.2:

50

Last Update:

19 Nov 2019

Package Maintainer(s):

Software Author(s):

  • Hermann Schinagl

Tags:

cli ntfs data-integrity checksum dedupe freeware

Dupemerge

Downloads:

2,850

Downloads of v 1.1.0.2:

50

Maintainer(s):

Software Author(s):

  • Hermann Schinagl

Dupemerge

  • 1
  • 2
  • 3

All Checks are Passing

3 Passing Tests


Validation Testing Passed


Verification Testing Passed

Details

Scan Testing Successful:

No detections found in any package files

Details
WARNING

This package was rejected on 28 Nov 2019. The reviewer Pauby has listed the following reason(s):

schinagl (maintainer) on 19 Nov 2019 18:31:44 +00:00:

User 'schinagl' (maintainer) submitted package.

schinagl (maintainer) on 19 Nov 2019 18:40:54 +00:00:

User 'schinagl' (maintainer) submitted package.

chocolatey-ops (reviewer) on 19 Nov 2019 19:12:19 +00:00:

dupemerge has passed automated validation. It may have or may still fail other checks like testing (verification).
NOTE: No required changes that the validator checks have been flagged! It is appreciated if you fix other items, but only Requirements will hold up a package version from approval. A human review could still turn up issues a computer may not easily find.

Suggestions

Suggestions are either newly introduced items that will later become Guidelines or items that are don't carry enough weight to become a Guideline. Either way they should be considered. A package version can be approved without addressing Suggestion comments.

  • The nuspec has been enhanced to allow more information related to the software. More... Please consider adding one or more of the following to the nuspec, if available:
    • docsUrl - points to the location of the wiki or docs of the software
    • mailingListUrl - points to the forum or email list group for the software
    • bugTrackerUrl - points to the location where issues and tickets can be accessed
    • projectSourceUrl - points to the location of the underlying software source
Notes

Notes typically flag things for both you and the reviewer to go over. Sometimes this is the use of things that may or may not be necessary given the constraints of what you are trying to do and/or are harder for automation to flag for other reasons. Items found in Notes might be Requirements depending on the context. A package version can be approved without addressing Note comments.

  • The package maintainer field (owners) matches the software author field (authors) in the nuspec. The reviewer will ensure that the package maintainer is also the software author. More...

chocolatey-ops (reviewer) on 20 Nov 2019 01:14:27 +00:00:

dupemerge has passed automated testing.
This is not the only check that is performed so check the package page to ensure a 'Ready' status.
Please visit https://gist.github.com/7e7c7d75c00cb824a9f7f1feec3e44d1 for details.
This is an FYI only. There is no action you need to take.

schinagl (maintainer) on 22 Nov 2019 06:09:28 +00:00:

Just wanted to note, that I uploaded the 'new' version, because the 1102 which is currently provided has a broken checksum. This because the currently provided dupemerge points to a location which was volatile on my site.

The 1102 which I provided with this request is basically the the same (well it is code signed too), but on a different non-volatile location. So it is only a choco package change.

So to make the package cleaner I would like to get this package up.
What do you think?

schinagl (maintainer) on 22 Nov 2019 18:45:17 +00:00:

Just wanted to note, that I uploaded the 'new' version, because the 1102 which is currently provided has a broken checksum. This because the currently provided dupemerge points to a location which was volatile on my site.

The 1102 which I provided with this request is basically the the same (well it is code signed too), but on a different non-volatile location. So it is only a choco package change.

So to make the package cleaner I would like to get this package up.
What do you think?

Pauby (reviewer) on 26 Nov 2019 11:49:00 +00:00:

The version you have submitted, 1.1.0.2, is an earlier version (in version numbering terms) that 1.102. When running choco upgrade it will not be downloaded as a new version.

If you are not submitting a newer version (in version numbering terms) than 1.102 then to fix that package you will need to submit a version with package fix notation and then reject this version.

Let me know what you want to do.

schinagl (maintainer) on 27 Nov 2019 21:03:02 +00:00:

ok. Dropped 1102, and uploaded 1103 to make things easy

Pauby (reviewer) on 28 Nov 2019 11:27:47 +00:00:

Do you want this version rejected?

schinagl (maintainer) on 28 Nov 2019 19:18:30 +00:00:

yes please reject, I have already submitted 1103

Pauby (reviewer) on 28 Nov 2019 20:02:30 +00:00:

Rejected at maintainers request - new version submitted.
Status Change - Changed status of package from 'submitted' to 'rejected'.

Description

Overview

Dupemerge searches for identical files on a logical drive and creates hardlinks among those files, thus saving lots of hard disk space. Most hard disks contain quite a lot of completely identical files, which consume a lot of disk space. This waste of space can be drastically reduced by using the NTFS file system hardlink functionality to link the identical files ("dupes") together.

### Technical

Dupemerge creates a cryptological hashsum for each file found below the given paths and compares those hashes to each other to find the dupes. There is no file date comparison involved in detecting dupes, only the size and content of the files.

To speed up comparison, only files with the same size get compared to each other. Furthermore the hashsums for equal sized files get calculated incrementally, which means that during the first pass only the first 4 kilobyte are hashed and compared, and during the next rounds more and more data are hashed and compared.

Due to long run time on large disks, a file which has already been hashsummed might change before all dupes to that file are found. To prevent false hardlink creation due to intermediate changes, dupemerge saves the file write time of a file when it hashsums the file and checks back if this time changed when it tries to hardlink dupes.

#### Multiple Runs

If dupemerge is run once, hardlinks among identical files are created. To save time during a second run on the same locations, dupemerge checks if a file is already a hardlink, and tries to find the other hardlinks by comparing the unique NTFS file-id. This saves a lot of time, because checksums for large files need not be created twice.

#### Transaction based Hardlinking

Before DupeMerge hardlinks file together it renames the file to a temporary name, then creates the hardlink, and afterwards deletes the temporary file. All that is done to be able to roll-back the operation if e.g the hardlinking failes.

#### TimeStamp Handling

A tupel of hardlinks for one file has always the one timestamp. This is by design of NTFS. But things are a bit confusing sometimes, because after hardlinking the same timestamp is only shown, after the hardlink was once e.g. opened/accessed. So it may happen, that immediatley after dupemerge one observes different timestamps within a tupel of hardlinks, but after such a hardlink has been opened for e.g. read, the timestamp changes to the timestamp of the whole tupel. That's also by design of NTFS.

Dupemerge has a dupe-find algorithm which is tuned to perform especially well on large server disks, where it has been tested in depth to guarantee data integrity.

### Documentation
* FAQ
* Manual


tools\chocolateyInstall.ps1
$packageName = 'dupemerge'
$url = 'https://schinagl.priv.at/nt/dupemerge/save/1102/dupemerge.zip'
$checksum = 'fce525818990ff54f384459310807fcf44bd83604aa019bee83e6a9c384b05bb'
$checksumType = 'sha256'
$url64 = 'https://schinagl.priv.at/nt/dupemerge/save/1102/dupemerge64.zip'
$checksum64 = '7a27b83f4e34ecaff90f0b9f610570f04141f39a2cdebe3aa123d0fb38467a18'
$checksumType64 = 'sha256'
$toolsDir = "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"

Install-ChocolateyZipPackage -PackageName "$packageName" `
                             -Url "$url" `
                             -UnzipLocation "$toolsDir" `
                             -Url64bit "$url64" `
                             -Checksum "$checksum" `
                             -ChecksumType "$checksumType" `
                             -Checksum64 "$checksum64" `
                             -ChecksumType64 "$checksumType64"

Log in or click on link to see number of positives.

In cases where actual malware is found, the packages are subject to removal. Software sometimes has false positives. Moderators do not necessarily validate the safety of the underlying software, only that a package retrieves software from the official distribution point and/or validate embedded software against official distribution point (where distribution rights allow redistribution).

Chocolatey Pro provides runtime protection from possible malware.

Add to Builder Version Downloads Last Updated Status
Dupemerge 1.103 669 Saturday, November 30, 2019 Approved
Dupemerge 1.102 350 Monday, January 28, 2019 Approved
Dupemerge 1.080.0.20170625 484 Thursday, September 21, 2017 Approved
Dupemerge 1.080 555 Sunday, June 18, 2017 Approved
Dupemerge 1.07 530 Sunday, April 19, 2015 Approved

Discussion for the Dupemerge Package

Ground Rules:

  • This discussion is only about Dupemerge and the Dupemerge package. If you have feedback for Chocolatey, please contact the Google Group.
  • This discussion will carry over multiple versions. If you have a comment about a particular version, please note that in your comments.
  • The maintainers of this Chocolatey Package will be notified about new comments that are posted to this Disqus thread, however, it is NOT a guarantee that you will get a response. If you do not hear back from the maintainers after posting a message below, please follow up by using the link on the left side of this page or follow this link to contact maintainers. If you still hear nothing back, please follow the package triage process.
  • Tell us what you love about the package or Dupemerge, or tell us what needs improvement.
  • Share your experiences with the package, or extra configuration or gotchas that you've found.
  • If you use a url, the comment will be flagged for moderation until you've been whitelisted. Disqus moderated comments are approved on a weekly schedule if not sooner. It could take between 1-5 days for your comment to show up.
comments powered by Disqus