How to Use Command line Find Duplicate Files on Windows PC

Many people unknowingly end up with duplicate files on their computers. These duplicate files can consume much more disk space than you might think and cause the computer to slow down. When users become aware of this issue and its negative effects, they often seek ways to clean up their computers. One effective approach is to use a specific command line to find duplicate files and remove them.

If you're familiar with this, you can choose either CMD (Command Prompt) or PowerShell on your Windows computer. In the following guide, we'll show you the steps for both options, showing you how to use these commands to efficiently remove those redundant files.

2 options of using command line to find duplicate files

In this section, we will show you how to use CMD and PowerShell to find duplicate files. However, it's important to be cautious because entering the wrong commands can potentially lead to data loss or other irreversible issues. Kindly please be careful about each command line you input.

If you're interested in exploring safer and more effective methods, you can click here to jump directly to that particular section.

Option 1. Use CMD to find duplicate files

Trying CMD to find duplicate files requires patience. This method is suitable for finding specific file types such as pictures, audio, and documents with clear and correct expansion name.

Step 1. Open CMD as an administrator. To do this, click on Start, then type "cmd" into the search bar, and choose “Run as administrator”.

Step 2. Type this command into CMD: "dir /s /b *.extension". Replace ".extension" with the type of files you want to find duplicates of. For example, if you're looking for duplicate JPG files, type "dir /s /b *.jpg".

Step 3. Press Enter. CMD will begin scanning for duplicate files, and it will display the results with duplicated files in a list.

You can also find duplicate files using the “WMIC FINDSTR” command: 

Open CMD and type "FINDSTR /?" to see all options for the "FINDSTR" command.

Type "FINDSTR /I /C:WINDOWS *.jpg" to search for all JPEG images in the WINDOWS folder.

The results will show all the JPEG images in the WINDOWS folder, with duplicates highlighted in red.

▌To remove duplicate files in a specific folder, follow these steps:

Step 1. Open the folder using File Explorer.

Step 2. Right-click on the folder you want and select "Copy as path".

Step 3. Open Command Prompt. Type "del /s /f *.duplicate" and hit Enter. This command will remove all duplicate files in the current folder and its subfolders.

For a more advanced file deletion, you can use wildcard characters with this command. Just type "rmdir /s /q DirName", and it will delete all folders in "DirName" and its subfolders.

Option 2. Utilize PowerShell to find duplicate files

Using PowerShell to find duplicate files can be much difficult and complex than CMD. Kindly please be careful about each command you input.

▌Stage 1. Use Get-FileHash cmdlet to check duplicate files

The Get-FileHash cmdlet in PowerShell is for file checksum. This hash can be used to identify a file uniquely. We will utilize the hash value to identify duplicate files in this post. The command's syntax is as follows:

Get-FileHash -Path file_path -Algorithm hashing_algorithm

You need to type a specific storage path of the files you want to search. E.g: Get-FileHash -Path 'D:\ISO\WinPE.iso' -Algorithm SHA512

If you don't specify a special way to check files, the computer will use the default method, SHA256, which is usually good enough. This method ensures files are intact, especially when downloaded from the Internet. We use a tool called Get-FileHash, combined with Get-ChildItem, to quickly check files one by one.

To find identical files, they must have the same size. Different-sized files cannot be identical. By focusing only on files with the same size, we speed up the process. You just need to adjust the file path based on the Get-FileHash result.

$srcDir = "D:\ISO Files" (Here, change the path based on your case)
Get-ChildItem -Path $srcDir -File -Recurse | Group -Property Length `
    | where { $_.Count -gt 1 } | select -ExpandProperty Group | Get-FileHash `
    | Group -Property Hash | where { $_.count -gt 1 }| foreach { $_.Group | select Path, Hash }

You can see this command that groups files that are the same size and then send those files to Get-FileHash to determine their hash.

▌Stage 2. Delete duplicate files with PowerShell

After we find duplicate files with PowerShell, we should deal with them appropriately. It is not advisable to eliminate duplicates immediately while working with essential files. You can instead relocate them to another directory, most likely on a separate disc with adequate free space. With our new command, this is simple. Simply feed the output of the above operation to the Move-Item cmdlet.

# Define source directory
$srcDir = "D:\ISO Files"  (Here, change the path based on your case)
# Define destination directory
$targetDir = "E:\DuplicateFiles\$(Get-Date -Format 'yyyyMMdd')" (This is where those target files will be stored )
# Create destination directory
if(!(Test-Path -PathType Container $targetDir)){ New-Item -ItemType Directory -Path $targetDir | Out-Null }
# Move duplicate files to a different location
Get-ChildItem -Path $srcDir -File -Recurse | group -Property Length | where { $_.Count -gt 1 } `
        | select -ExpandProperty Group | Get-FileHash | group -Property Hash `
        | where { $_.Count -gt 1 }| foreach { $_.Group | select -Skip 1 } `
        | Move-Item -Destination $targetDir -Force -Verbose

The point you need to notice is that we used -Skip 1 to leave one file in the source directory while moving other duplicates to the specified target directory. Once moved, you can manually review them later on and remove them, if necessary. If you're working on a huge source directory with millions of files, it is a good idea to avoid using the -verbose parameter with Move-Item.

If your directory contains a few files, you may wish to choose which to relocate and which to leave in the source directory manually:

# Define source directory
$srcDir = "D:\ISO Files"
# Define destination directory
$targetDir = "E:\DuplicateFiles\$(Get-Date -Format 'yyyyMMdd')"
# Create destination directory
if(!(Test-Path -PathType Container $targetDir)){ New-Item -ItemType Directory -Path $targetDir | Out-Null }
# Manually choose duplicate files to move to target directory
Get-ChildItem -Path $srcDir -File -Recurse | Group -Property Length `
    | where { $_.Count -gt 1 } | select -ExpandProperty Group | Get-FileHash `
    | Group -Property Hash | where { $_.count -gt 1 } | foreach { $_.Group | select Path, Hash } `
    | Out-GridView -Title "Select the file(s) to move to `"$targetDir`" directory." -PassThru `
    | Move-Item -Destination $targetDir -Force -Verbose

If you have backed up your files, you can also directly move the duplicate files with PowerShell:

# Define source directory
$srcDir = "D:\ISO Files"
# Permanently delete duplicate files; use with caution
Get-ChildItem -Path $srcDir -File -Recurse | group -Property Length | where { $_.Count -gt 1 } `
        | select -ExpandProperty Group | Get-FileHash | group -Property Hash `
        | where { $_.Count -gt 1 }| foreach { $_.Group | select -Skip 1 } `
        | Remove-Item -Force -Verbose

How to easily and safely find and delete duplicates with a few clicks?

Using Windows command lines to find duplicate files and remove them can be challenging for some users. More importantly, these complex commands can potentially lead to the accidental overwriting of system-related data, which can cause serious consequences.

Given the risks and complexities involved, why not consider trying an easy and safe solution to deal with the bothersome issue of duplicate files? AOMEI Partition Assistant Professional is dedicated to providing straightforward solutions for computer problems.

For example, its Duplicate Files Finder function can speedily and comprehensively scan your entire computer and list all the file types (pictures, audio, documents, etc.), and offers customization based on your needs. Once you delete important files, you can also use its Recover Data feature to get them back.

Step 1. Click the "Free up" on the main tab and select "Duplicate Files Finder".

Step 2. Here, all partitions on your computer will be displayed. You can personalize it according to your needs and click "Scan". In addition, if a partition is locked, the program will still scan it and display duplicate files on the partition. However, it is unable to delete the duplicated files on the locked drive.

Step 3. After the process is finished, all duplicate files that meet the conditions you set will be displayed. You can click the "Preview" button to see the duplicate files.

Step 4. You can select the duplicate files you want to delete. Or, click "Smart select" to automatically select duplicate files. Then, click "Move to folder".

Conclusion

This is a comprehensive guide on using command line to find duplicate files. If you are willing to explore more convenient method like relying on AOMEI Partition Assistant, you can make it effective and easy to maintain high computer performance. By the way, this powerful tool shares functions like App Mover, Disk Speed Test, Migrate OS, and so on for your computer management.