This post will share guidance on using command lines to find duplicate files. Besides, here is a facile solution to help you easily and safely find and remove duplicate files.
Many people unknowingly end up with duplicate files on their computers. These duplicate files can consume much more disk space than you might think and cause the computer to slow down. When users become aware of this issue and its negative effects, they often seek ways to clean up their computers. One effective approach is to use a specific command line to find duplicate files and remove them.
If you're familiar with this, you can choose either CMD (Command Prompt) or PowerShell on your Windows computer. In the following guide, we'll show you the steps for both options, showing you how to use these commands to efficiently remove those redundant files.
In this section, we will show you how to use CMD and PowerShell to find duplicate files. However, it's important to be cautious because entering the wrong commands can potentially lead to data loss or other irreversible issues. Kindly please be careful about each command line you input.
If you're interested in exploring safer and more effective methods, you can click here to jump directly to that particular section by using a handy tool.
The Best Windows Disk Partition Manager and PC Optimizer
Trying CMD to find duplicate files requires patience. This method is suitable for finding specific file types such as pictures, audio, and documents with clear and correct expansion name.
Step 1. Open CMD as an administrator. To do this, click on Start, then type "cmd" into the search bar, and choose “Run as administrator”.
Step 2. Type this command into CMD: "dir /s /b *.extension". Replace ".extension" with the type of files you want to find duplicates of. For example, if you're looking for duplicate JPG files, type "dir /s /b *.jpg".
Step 3. Press Enter. CMD will begin scanning for duplicate files, and it will display the results with duplicated files in a list.
Step 1. Open the folder using File Explorer.
Step 2. Right-click on the folder you want and select "Copy as path".
Step 3. Open Command Prompt. Type "del /s /f *.duplicate" and hit Enter. This command will remove all duplicate files in the current folder and its subfolders.
For a more advanced file deletion, you can use wildcard characters with this command. Just type "rmdir /s /q DirName", and it will delete all folders in "DirName" and its subfolders.
Using PowerShell to find duplicate files can be much difficult and complex than CMD. Kindly please be careful about each command you input.
The Get-FileHash cmdlet in PowerShell is for file checksum. This hash can be used to identify a file uniquely. We will utilize the hash value to identify duplicate files in this post. The command's syntax is as follows:
Get-FileHash -Path file_path -Algorithm hashing_algorithm
You need to type a specific storage path of the files you want to search. E.g: Get-FileHash -Path 'D:\ISO\WinPE.iso' -Algorithm SHA512
If you don't specify a special way to check files, the computer will use the default method, SHA256, which is usually good enough. This method ensures files are intact, especially when downloaded from the Internet. We use a tool called Get-FileHash, combined with Get-ChildItem, to quickly check files one by one.
To find identical files, they must have the same size. Different-sized files cannot be identical. By focusing only on files with the same size, we speed up the process. You just need to adjust the file path based on the Get-FileHash result.
$srcDir = "D:\ISO Files" (Here, change the path based on your case) Get-ChildItem -Path $srcDir -File -Recurse | Group -Property Length ` | where { $_.Count -gt 1 } | select -ExpandProperty Group | Get-FileHash ` | Group -Property Hash | where { $_.count -gt 1 }| foreach { $_.Group | select Path, Hash }
You can see this command that groups files that are the same size and then send those files to Get-FileHash to determine their hash.
After we find duplicate files with PowerShell, we should deal with them appropriately. It is not advisable to eliminate duplicates immediately while working with essential files. You can instead relocate them to another directory, most likely on a separate disc with adequate free space. With our new command, this is simple. Simply feed the output of the above operation to the Move-Item cmdlet.
# Define source directory $srcDir = "D:\ISO Files" (Here, change the path based on your case) # Define destination directory $targetDir = "E:\DuplicateFiles\$(Get-Date -Format 'yyyyMMdd')" (This is where those target files will be stored ) # Create destination directory if(!(Test-Path -PathType Container $targetDir)){ New-Item -ItemType Directory -Path $targetDir | Out-Null } # Move duplicate files to a different location Get-ChildItem -Path $srcDir -File -Recurse | group -Property Length | where { $_.Count -gt 1 } ` | select -ExpandProperty Group | Get-FileHash | group -Property Hash ` | where { $_.Count -gt 1 }| foreach { $_.Group | select -Skip 1 } ` | Move-Item -Destination $targetDir -Force -Verbose
The point you need to notice is that we used -Skip 1 to leave one file in the source directory while moving other duplicates to the specified target directory. Once moved, you can manually review them later on and remove them, if necessary. If you're working on a huge source directory with millions of files, it is a good idea to avoid using the -verbose parameter with Move-Item.
If your directory contains a few files, you may wish to choose which to relocate and which to leave in the source directory manually:
# Define source directory $srcDir = "D:\ISO Files" # Define destination directory $targetDir = "E:\DuplicateFiles\$(Get-Date -Format 'yyyyMMdd')" # Create destination directory if(!(Test-Path -PathType Container $targetDir)){ New-Item -ItemType Directory -Path $targetDir | Out-Null } # Manually choose duplicate files to move to target directory Get-ChildItem -Path $srcDir -File -Recurse | Group -Property Length ` | where { $_.Count -gt 1 } | select -ExpandProperty Group | Get-FileHash ` | Group -Property Hash | where { $_.count -gt 1 } | foreach { $_.Group | select Path, Hash } ` | Out-GridView -Title "Select the file(s) to move to `"$targetDir`" directory." -PassThru ` | Move-Item -Destination $targetDir -Force -Verbose
If you have backed up your files, you can also directly move the duplicate files with PowerShell:
# Define source directory $srcDir = "D:\ISO Files" # Permanently delete duplicate files; use with caution Get-ChildItem -Path $srcDir -File -Recurse | group -Property Length | where { $_.Count -gt 1 } ` | select -ExpandProperty Group | Get-FileHash | group -Property Hash ` | where { $_.Count -gt 1 }| foreach { $_.Group | select -Skip 1 } ` | Remove-Item -Force -Verbose
Using Windows command lines to find duplicate files and remove them can be challenging for some users. More importantly, these complex commands can potentially lead to the accidental overwriting of system-related data, which can cause serious consequences.
Given the risks and complexities involved, why not consider trying an easy and safe solution to deal with the bothersome issue of duplicate files? AOMEI Partition Assistant is dedicated to providing straightforward solutions for computer problems.
For example, its Duplicate Files Finder function can speedily and comprehensively scan your entire computer and list all the file types (pictures, audio, documents, etc.), and offers customization based on your needs. Once you delete important files, you can also use its Recover Data feature to get them back.
The Best Windows Disk Partition Manager and PC Optimizer
Step 1. Click the "Free up" on the main tab and select "Duplicate Files Finder".
Step 2. Here, all partitions on your computer will be displayed. You can personalize it according to your needs and click "Scan". In addition, if a partition is locked, the program will still scan it and display duplicate files on the partition. However, it is unable to delete the duplicated files on the locked drive.
Step 3. After the process is finished, all duplicate files that meet the conditions you set will be displayed. You can click the "Preview" button to see the duplicate files.
Step 4. You can select the duplicate files you want to delete. Or, click "Smart select" to automatically select duplicate files. Then, click "Move to folder".
Sometimes, apps or software can unintentionally create duplicates during updates or file syncing. Over time, these duplicates can clutter your storage, making it harder to manage files and potentially slowing down your device, and even causing 99% disk usage. The same results will also happen if your computer stores many useless large files or junk files. AOMEI Partition Assistant offers the 2 functions that can help you quickly free up space with ease.
The Best Windows Disk Partition Manager and PC Optimizer
Step 1. Install and launch AOMEI Partition Assistant, and then click "Free up" and "Delete Large Files".
Step 2. In the pop-up window, select the partitions that you want to find large files, and click "Start Scan" to scan for large files.
Step 3. When the scan is complete, all large files found are listed. Tick the files you want to delete, and then click the "Delete" button.
Note: If the file is marked as red, once you’re sure it’s unnecessary you can also delete it.
Step 4. This step will list all the files you have ticked to delete, click "Delete Now" after confirming that it is correct, then these files will be deleted successfully.
Step 1. Download and launch AOMEI Partition Assistant Standard, then sequentially select "Free up" and "PC Cleaner".
Step 2. Select "Start Scan" in the new popup window to initiate a scan of your computer for unnecessary files.
Step 3. Please be patient while the system scans for junk files. Once the scan is complete, any unnecessary data will be automatically marked for deletion. You can click "Clean All" to remove them. If there are additional unwanted files, you can manually select them by ticking the corresponding checkboxes.
Step 4. When you see a correct symbol displayed in the interface, you can click "Back" to leave the function.
This is a comprehensive guide on using command line to find duplicate files. If you are willing to explore more convenient method like relying on AOMEI Partition Assistant, you can make it effective and easy to maintain high computer performance. By the way, this powerful tool shares functions like App Mover, Disk Speed Test, Migrate OS, and so on for your computer management.