Hoping that this is a good place to post a question about Bash scripting. My wife and I have run into a problem in PhotoPrism where it keeps tagging pictures and videos with similar names together and so the thumbnail and the video do not match. I decided that rather than try to get her iPhone to tweak its naming it's easier to just offload to a directory then rename every file to a UUID before sending to photoprism. I'm trying to write a bash script to simplify this but cannot get the internal loop to fire. The issue appears to be with the 'while IFS= read -r -d '' file; do' portion. Is anyone able to spot what the issue may be?
#! /bin/bash
echo "This script will rename all files in this directory with unique names. Continue? (Y/N)"
read proceed
if [[ "$proceed" == "Y" ]]; then
echo "Proceed"
#use uuidgen -r to generate a random UUID.
#Currently appears to be skipping the loop entirely. the find command works so issue should be after the pipe.
# Troubleshooting
#Seems like changing IFS to $IFS helped. Now however it's also pulling others., don't think this is correct.
#verified that the find statement is correct, its the parsing afterwards that's wrong.
#tried removing the $'\0' after -d as that is string null in c. went to bash friendly '' based on https://stackoverflow.com/questions/57497365/what-does-the-bash-read-d-do
#issue definitely appears to be with the while statement
find ./ -type f \( -iname \*.jpg -o -iname \*.png \) | while IFS= read -r -d '' file; do
echo "in loop"
echo "$file"
#useful post https://itsfoss.gitlab.io/post/how-to-find-and-rename-files-in-linux/
#extract the directory and filename
dir=$(dirname "$file")
base=$(basename "$file")
echo "'$dir'/'$base'"
#use UUID's to get around photoprism poor handling of matching file names and apples high collision rate
new_name="$dir/$(uuidgen -r)"
echo "Renaming ${file} to ${new_name}"
#mv "$file" "$new_name" #uncomment to actually perform the rename.
done
echo "After loop"
else
echo "Cancelling"
fi
phaedrus - 6day
You can do the entire thing as a one-liner using only find:
find ./ -type f \( -iname "*.jpg" -or -iname "*.png" \) -exec sh -c 'mv "$0" "$(uuidgen -r).${0##*.}"' {} \;
Test on my machine:
phaedrus@sys76 ~/D/test> ls -lh
total 0
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 test1.jpg
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 test1.png
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 test2.jpg
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 test2.png
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 test3.jpg
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 test3.png
phaedrus@sys76 ~/D/test> find ./ -type f \( -iname "*.jpg" -or -iname "*.png" \) -exec sh -c 'mv "$0" "$(uuidgen -r).${0##*.}"' {} \;
phaedrus@sys76 ~/D/test> ls -lh
total 0
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 062d8954-9921-42bd-ad24-0e4ed403a5db.jpg
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 111f859f-b1fe-4488-b2bc-75585320e3a3.png
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 39b9fe4e-7a05-43c9-b30a-69e9a13aa3a9.png
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 57bda91e-49e5-43fe-8318-aeeb2e3adde7.png
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 97398eb7-54aa-488f-8fbe-0b84b5e5a50d.jpg
-rw-r--r-- 1 phaedrus users 0 Dec 6 01:08 f7a13274-e2c0-4fa7-9907-c590d1280c2e.jpg
btw, Lemmy doesn't like language specifiers in the multi-line code blocks, so it's difficult to read all that in its current form since there are no tabs to know how you have it formatted. Makes it virtually impossible to troubleshoot your specific script.
edit: You beat me to it with your link on parameter expansion. I'll be reading through that tonight as well. Thanks again.
19
phaedrus - 6day
It might be instance related, I'm on PieFed, so perhaps the markdown implementation is different.
Also, I realized that the parameter expansion might not be straightforward and added the GNU docs on it, but looks like you found a post about it at the same time! Glad to hear it got you sorted out.
Bash FAQs and pitfalls are the primary sections to look at there.
5
Badabinski - 6day
Thank you for providing the easiest and most portable answer. This will handle files with special characters perfectly unlike most of the responses here which rely on a while loop (to say nothing of a for loop ).
4
phaedrus - 6day
Indeed, folks tend not to look into the docs enough to realize find is a powerful tool on its own!
I think the other answers were just adhering to the request (trying to troubleshoot the script as is), but I generally go for pragmatism despite not being what was actually requested.
1
Rentlar - 6day
Edit: I think there are better answers downthread than mine, but I hope my first comment spurned them on.
Not the most experienced bash guru at it but let me see...
does the while condition have to be within [ ] brackets?
Also I can't figure out what your condition is, it seems to have an unclosed quotation mark.
Most bash while-do-done loops I've made have a comparator like -ne for not equal or -le for less or equal to. So for example: while [ $variable -ne 5 ]; do
4
FigMcLargeHuge @sh.itjust.works - 6day
Also, I am not sure you can automatically expect the output of the find command to be assigned to the file variable. I would output the find command to a temp file, and then grab the filenames one by one from the file and then put that in a do/done loop. Eg:
find ./ -type f \( -iname \*.jpg -o -iname \*.png \) > yourtempfile.tmp
for file in `cat yourtempfile.tmp`
do
echo "in loop"
echo "$file"
....
done
Here are some results I got when it ran:
in loop
./Figs_XSR900.JPG
'.'/'Figs_XSR900.JPG'
Renaming ./Figs_XSR900.JPG to ./356bb549-d25c-4c9b-a441-f8175f963c8c
in loop
./20240625195740_411A6199.JPG
'.'/'20240625195740_411A6199.JPG'
Renaming ./20240625195740_411A6199.JPG to ./3cc9ba51-1d15-4a10-b9ee-420a5666e3e2
3
elmicha - 6day
You forgot the -print0 at the end of the find command. In the read -r -d '' you want to read NUL-separated strings, so you must tell the find command to also use NUL characters between the filenames.
4
non_burglar @lemmy.world - 6day
Your find statement is not creating a variable "file" because it's missing the first part of the for loop. This:
find ./ -type f \( -iname \*.jpg -o -iname \*.png \) | while IFS= read -r -d '' file; do
should be this:
for file in "$(find ./ -type f \( -iname \*.jpg -o -iname \*.png \))"; do
However, the above command would find all files in current and subdirectories. You can just evaluate current context much more simply. I tested the below, it seems to work.
#! /bin/bash
echo "This script will rename all files in this directory with unique names. Continue? (Y/N)"
read proceed
if [[ "$proceed" == "Y" ]]; then
echo "Proceed"
for file in *.{jpg,JPG,png,PNG}; do
echo "in loop"
echo "$file"
dir=$(dirname "$file")
base=$(basename "$file")
echo "'$dir'/'$base'"
new_name="$dir/$(uuidgen -r)"
echo "Renaming ${file} to ${new_name}"
#mv "$file" "$new_name" #uncomment to actually perform the rename.
done
echo "After loop"
else
echo "Cancelling"
fi
You could also find matching files first, evaluate if anything is found and add a condition to exit if no files are found.
Edit: who the fuck downvoted this, it literally works and the for loop was the issue.
3
ravenaspiring @sh.itjust.works - 6day
We could diagnose it for you... Or you could ask the same question of an LLM and get a more interactive answer.
Or several other free models like Qwen-coder ollama/deepseek-r1, kimi k2, Kat koder pro, etc
-24
raspberriesareyummy @lemmy.world - 6day
With all due respect, you're being an pain in the ass for contributing to stealing millions from people who need to buy RAM for new computer. Pardon my French but fuck off with that LLM bullshit already.
12
atzanteol @sh.itjust.works - 6day
Let's encourage human interaction rather than sending people away to an llm.
7
FigMcLargeHuge @sh.itjust.works - 6day
Please, I would love to see how AI would fix this...
5
atzanteol @sh.itjust.works - 6day
I'm totally in favor of people asking other people for help with these things. But here's what Claude gave.
Found the Issue!
The problem is a mismatch between your find output and what read expects:
find with a regular pipe outputs newline-separated filenames
read -r -d '' expects null-terminated input (the -d '' means "use null byte as delimiter")
Solution 1: Use -print0 with find (Recommended)
Change your find command to use -print0:
find ./ -type f \( -iname "*.jpg" -o -iname "*.png" \) -print0 | while IFS= read -r -d '' file; do
Solution 2: Remove -d '' from read
find ./ -type f \( -iname "*.jpg" -o -iname "*.png" \) | while IFS= read -r file; do
Additional Issues to Fix:
Quote your wildcards: -iname \*.jpg should be -iname "*.jpg" to prevent shell expansion
File extension preservation: Your script generates UUIDs but loses the file extension (.jpg, .png). You probably want to keep those!
Improved Script:
#! /bin/bash
echo "This script will rename all files in this directory with unique names. Continue? (Y/N)"
read proceed
if [[ "$proceed" == "Y" ]]; then
echo "Proceed"
find ./ -type f \( -iname "*.jpg" -o -iname "*.png" \) -print0 | while IFS= read -r -d '' file; do
echo "in loop"
echo "$file"
# Extract the directory and extension
dir=$(dirname "$file")
ext="${file##*.}" # Get file extension
# Generate new name with UUID but keep extension
new_name="$dir/$(uuidgen -r).$ext"
echo "Renaming ${file} to ${new_name}"
# mv "$file" "$new_name" # Uncomment to actually perform the rename
done
echo "After loop"
else
echo "Cancelling"
fi
The key changes:
Added -print0 to find
Quoted the wildcard patterns
Preserved file extensions using ${file##*.}
Try this and let me know if it works!
3
comrade_twisty @feddit.org - 6day
It will delete your drive and say sorry
2
SchwertImStein @lemmy.dbzer0.com - 6day
find ./ -type f ( -iname '*.jpg' -o -iname '*.png' ) -print0 |
while IFS= read -r -d '' file; do
echo "in loop"
echo "$file"
dir=$(dirname "$file")
base=$(basename "$file")
echo "'$dir'/'$base'"
new_name="$dir/$(uuidgen -r)"
echo "Renaming ${file} to ${new_name}"
# mv "$file" "$new_name"
done
That's what perplexity gave me after copy pasting the post.
1
TrickDacy @lemmy.world - 6day
Fucking cringe answer
3
MonkderVierte @lemmy.zip - 6day
LLM-psychosis?
3
Badabinski - 6day
Shell scripts are one of the worst possible applications of an LLM. They're trained on shit fucking GitHub scripts, and they give you shit in return.
BingBong in linux
Bash scripting question
Hello everyone,
Hoping that this is a good place to post a question about Bash scripting. My wife and I have run into a problem in PhotoPrism where it keeps tagging pictures and videos with similar names together and so the thumbnail and the video do not match. I decided that rather than try to get her iPhone to tweak its naming it's easier to just offload to a directory then rename every file to a UUID before sending to photoprism. I'm trying to write a bash script to simplify this but cannot get the internal loop to fire. The issue appears to be with the 'while IFS= read -r -d '' file; do' portion. Is anyone able to spot what the issue may be?
You can do the entire thing as a one-liner using only
find:Test on my machine:
btw, Lemmy doesn't like language specifiers in the multi-line code blocks, so it's difficult to read all that in its current form since there are no tabs to know how you have it formatted. Makes it virtually impossible to troubleshoot your specific script.
edit: further reading on the ever useful variable expansions (
${0##*.}portion of my one-liner):https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
Interesting, the code shows up correctly for me in firefox. I wonder if that's due to my instance?
This works perfectly! Thank you!!! In case anyone else finds themselves wondering about the ${0##*.} portion, I found this article to be very helpful. https://stackoverflow.com/questions/30980062/0-and-0-in-sh
edit: You beat me to it with your link on parameter expansion. I'll be reading through that tonight as well. Thanks again.
It might be instance related, I'm on PieFed, so perhaps the markdown implementation is different.
Also, I realized that the parameter expansion might not be straightforward and added the GNU docs on it, but looks like you found a post about it at the same time! Glad to hear it got you sorted out.
If you want more help with Bash in the future, this is the best resource I've found in 13 years of writing bash professionally: https://mywiki.wooledge.org/EnglishFrontPage
Bash FAQs and pitfalls are the primary sections to look at there.
Thank you for providing the easiest and most portable answer. This will handle files with special characters perfectly unlike most of the responses here which rely on a
whileloop (to say nothing of aforloop ).Indeed, folks tend not to look into the docs enough to realize
findis a powerful tool on its own!I think the other answers were just adhering to the request (trying to troubleshoot the script as is), but I generally go for pragmatism despite not being what was actually requested.
Edit: I think there are better answers downthread than mine, but I hope my first comment spurned them on.
Not the most experienced bash guru at it but let me see...
Also, I am not sure you can automatically expect the output of the find command to be assigned to the file variable. I would output the find command to a temp file, and then grab the filenames one by one from the file and then put that in a do/done loop. Eg:
Here are some results I got when it ran:
in loop
./Figs_XSR900.JPG
'.'/'Figs_XSR900.JPG'
Renaming ./Figs_XSR900.JPG to ./356bb549-d25c-4c9b-a441-f8175f963c8c
in loop
./20240625195740_411A6199.JPG
'.'/'20240625195740_411A6199.JPG'
Renaming ./20240625195740_411A6199.JPG to ./3cc9ba51-1d15-4a10-b9ee-420a5666e3e2
You forgot the
-print0at the end of the find command. In theread -r -d ''you want to read NUL-separated strings, so you must tell the find command to also use NUL characters between the filenames.Your find statement is not creating a variable "file" because it's missing the first part of the for loop. This:
find ./ -type f \( -iname \*.jpg -o -iname \*.png \) | while IFS= read -r -d '' file; doshould be this:
for file in "$(find ./ -type f \( -iname \*.jpg -o -iname \*.png \))"; doHowever, the above command would find all files in current and subdirectories. You can just evaluate current context much more simply. I tested the below, it seems to work.
You could also find matching files first, evaluate if anything is found and add a condition to exit if no files are found.
Edit: who the fuck downvoted this, it literally works and the for loop was the issue.
We could diagnose it for you... Or you could ask the same question of an LLM and get a more interactive answer.
https://chat.qwen.ai/ is capable of answering for free
Or several other free models like Qwen-coder ollama/deepseek-r1, kimi k2, Kat koder pro, etc
With all due respect, you're being an pain in the ass for contributing to stealing millions from people who need to buy RAM for new computer. Pardon my French but fuck off with that LLM bullshit already.
Let's encourage human interaction rather than sending people away to an llm.
Please, I would love to see how AI would fix this...
I'm totally in favor of people asking other people for help with these things. But here's what Claude gave.
Found the Issue!
The problem is a mismatch between your
findoutput and whatreadexpects:findwith a regular pipe outputs newline-separated filenamesread -r -d ''expects null-terminated input (the-d ''means "use null byte as delimiter")Solution 1: Use
-print0withfind(Recommended)Change your find command to use
-print0:Solution 2: Remove
-d ''from readAdditional Issues to Fix:
-iname \*.jpgshould be-iname "*.jpg"to prevent shell expansion.jpg,.png). You probably want to keep those!Improved Script:
The key changes:
-print0tofind${file##*.}Try this and let me know if it works!
It will delete your drive and say sorry
That's what perplexity gave me after copy pasting the post.
Fucking cringe answer
LLM-psychosis?
Shell scripts are one of the worst possible applications of an LLM. They're trained on shit fucking GitHub scripts, and they give you shit in return.