To Find the Hash Value of a File Faster

Asked 1 years ago, Updated 1 years ago, 143 views

I'm going to find the hash value of the file and use it to verify the same file.
On Linux

md5sum FILE

The prompts you for a hash value, but when it becomes a large file, it also takes time to find a hash value.
Is there any way to speed it up with hardware processing?
The goal is to speed up the hash value, so it doesn't matter how you do it.OS is expected to be CentOS 7.

Please let me know if you know.

centos hash md5sum md5

2022-09-30 21:25

2 Answers

Although there is a way to accelerate hash calculations using GPUs, FPGA, or dedicated ASICs, it is not a one-time installation.

It is not always a good idea to use hash values if you want to verify the same file.If you compare the files themselves, you may be able to read only the file size and some blocks to confirm that they are not identical, but you must always read the entire file when comparing hash values.

Using hash values for file comparisons is a valid

  • I don't have the files to compare
  • Requires repeated actions, such as comparing many files to each other

For example, そのThere is at most one hash value to calculate on the spot, and then compare it to a pre-calculated hash value. ようWith this usage, the amount of hash value calculation should not be a big problem.


2022-09-30 21:25

If you write a program that asks for hash values, you can imagine that disk I/O will be the bottleneck in terms of computational resources. I can't imagine disk I/O support that specializes in finding hash values.Therefore, I believe that the speed that the questioner is looking for will be difficult unless you change your perspective.

(For example, when writing a file, hash is calculated at the same time, etc.)


2022-09-30 21:25

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.