How to rename the file you are downloading on wget

Asked 2 years ago, Updated 2 years ago, 45 views

This is when saving files with wget http://~.

When wget in the background and saving multiple files, ls does not know which file is complete.
Therefore, I would like to put _incomplete_ at the beginning of the file name and make it the original file name after completion. Is there any good way?

linux macos

2022-09-30 10:56

2 Answers

I don't know if it meets the requirements, but most content has a timestamp, so wget-N keeps the timestamp.
This will set the timestamp after the download completes, so you can see the completion by arranging it in the order of the timestamp in ls-t.


2022-09-30 10:56

The following story is Linux OS only.

The Linux operating system provides a mechanism called inotify (monitoring filesystem events).For more information, see man7inotify, which allows you to capture the issue of open(2)/close(2) to files and add any action.

Specifically, use the inotifywait command in the inotify-tools package.The following sample script assumes that you download files to the $HOME/downloads directory.

mark_incomplete.sh

#!/bin/bash

inotifywait-q-m$HOME/downloads-e open, close --format'%e%w%f'|
while read-revent dir file
do
  [[!-f$dir$file]]&continue
  p=($(fuser"$dir$file"2>/dev/null))
  case "$event" in
    OPEN)
      [[${#p[@]}==1]]&[[$(basename"$(readlink-e/proc/${p[0]}/exe)]==wget]]&
      ln-sf "$dir$file" "${dir}_incomplete_$file";
    *CLOSE*)
      ([[${#p[@]}==0]]||
      ([[${#p[@]}>1]]&[[$(basename"$(readlink-e/proc/${p[0]}/exe)]!=wget]]) &
      rm-f "${dir}_incomplete_${file}";
  esac
done

Here is an example of how it works:

$./mark_incomplete.sh&
$ cd$HOME/downloads
$ wget http://greengenes.lbl.gov/Data/JD_Tutorial/UnAligSeq24606.txt

# Check on another device
$ ls-l$HOME/downloads
-rw -r --r -- 1 nemo 1.4K Jan 23 20:48 UnAligSeq 24606.txt
lrwxrwxrwx1nemo43Jan2320:48_incomplete_UnAligSeq24606.txt->/home/nemo/downloads/UnAligSeq24606.txt

# Download Complete
$ ls-l$HOME/downloads
-rw-r--r--1 nemo35144 Aug 11 2006 UnAligSeq24606.txt

This script creates a symbolic link instead of an mv command.The reason is that mv "$dir/$file" "$dir/_incomplete_$file" will prevent you from changing the timestamp after the download is complete.

$wget http://greengenes.lbl.gov/Data/JD_Tutorial/UnAligSeq24606.txt
                 :
utime (UnAligSeq24606.txt): No such file or directory

Now, there are some problems with this script.

  • Error if name of destination directory contains blank characters
  • Error if the name of the file to download contains a new line code (0x0a)

I want to pass the output string of the inotifywait command separated by NULL(0x00) characters, but I cannot include NULL in the format string specified in the --format option.

  • Opening a file using another command named wget creates a symbolic link file in the destination directory

inotifywait does not provide information (such as PID) about the process that caused the event, so it gets the necessary information from /proc.

Because of the above situation, even if you use it, it will be quite limited.Please take it for reference only.


2022-09-30 10:56

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.